2e5021dd83d84a1ea9a3b546f1820fe0daf3eb90
18
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 surface-camera-build
|
2e5021dd83 |
cubelinux: the region walk's cursor actually advances
The gate caught it on its first run: the region walk answered the box correctly five times over — the seek and the membership test were right — but the batch re-served the same five records forever, because the cursor never skipped. The cause is a one-word difference between the two existing walk paths, and it was mine. `v3_enum` positions itself with `at = first + cursor` and so does not need `Batch.seen` to skip; the packed walk has no index to reposition and relies on `seen` instead, which is why it initialises `seen: 0`. My two region paths re-seek to the span's foot every batch — there is no `first + cursor` for a box, because a span holds records that are *not* returned, so the index position of the cursor-th match is not arithmetic — and I had set `seen: cursor`, which disables the skip and re-serves the span from its foot forever. The fix is `seen: 0`, and the gate now shows why the trap matters: an unaligned box [3,3,3]-[10,10,10] answers five records, and the record at (2,7,7) — outside the box but inside the span — is examined and rejected, not returned. |
||
|
surface-camera-build
|
88ce1bf2bf |
cubelinux: CUBE_OP_RANGE — a region walk that seeks on the box's key span
The seventh operation: `cube(2)` gains CUBE_OP_RANGE, a bounded walk of one space's records that lie in a box. Its argument block is its own (`cube_range_args`, versioned by `size` like the walk's), and the box travels as its six numbers for the reason a coordinate does — the key it has to become is the driver's business. The operation rests on the span that the shared file already owns. `key_span(lo, hi)` bounds every key in the box because the interleave is monotone on each axis, so a v3 image is **sought**: the fixed-stride index is binary-searched for the span's foot (`Addressed::lower_bound`) and read forward to its head, merging the log's edits exactly as a walk does. A packed v1/v2 image has no index to search, so its space is walked with the same span used only to stop early — and the contract is the same either way, so a caller is not told which path it got. The trap that shaped the code, and the reason it is written the way it is: **the span is a bound, not the set.** Keys of points outside the box fall inside it (Z-order amplification), so every candidate is decoded and tested against the box before it is returned — which is what `morton_decode`, the interleave's inverse, is for, now in the shared file with the same kind of hand-pinned tests the interleave has. And the cursor counts the records *in the box*, not the records of the space, because those are the records the walk returns. The over-coverage — records examined versus records returned — is the number this operation is meant to publish, and it is not wired to the caller yet: the cost gate measures it by comparing against the userspace store, which counts the same thing. |
||
|
surface-camera-build
|
72a6bbe173 |
cubelinux: the key and the span move into the shared file, and the driver stops having its own
`morton_encode` existed twice — once in the driver, once (as the curve) in cube-core — and the two
had to agree byte for byte, because one side computes a key to look a record up and the other
computes it to lay an image out. Nothing structural held that agreement: only the gates, which
would notice afterwards. That is the same shape as the cube_format.rs finding recorded in
DESIGN-coordinate-surface.md §7.4 — a shared *file* whose shared half has no callers — so this
starts paying it off where it is cheap and exact.
The interleave now lives in cube_format.rs, which both builds compile, and the driver's copy is a
call to it. It is written as plain `while` loops rather than iterator chains because the kernel
build of this file has no `std` and no `alloc`, which is the constraint the whole arrangement
exists under.
Alongside it, two things the range work needs and neither side had:
key_cmp compare two keys as the 192-bit numbers they are. The key is big-endian, so this
is the comparison a sorted index performs, and it is stated once rather than
assumed at each site.
key_span the key span a box covers, (key(lo), key(hi)) inclusive. This is the bound a seek
needs, and — the point — it needs no decomposition at all: the interleave is
monotone in every axis, so every point in the box has a key between its corners'
keys. A sorted index can be binary-searched for the foot and scanned forward to the
head. The span is a BOUND and not the set: records outside the box can have keys
inside it, which is the classic Z-order amplification and is why a caller tests
membership per candidate. Confusing the bound for the set is close to the mistake
that put "an aligned box is one run" into two doc comments.
aligned_box_is_one_run
the condition, proved in crates/cube-format's tests and pinned there and in
cube-store: a power-of-two-aligned box is exactly one contiguous run of the key iff
max(k) - min(k) <= 1 AND the axes at the top level form a prefix of (x, y, z). A
size that is not a power of two is answered `false` rather than rounded, because a
caller passing one has a bug and "no" is the useful answer.
Verified on build #53: verify-enum (the kernel's walk against userspace's, which is the check that
would catch any change in the key), verify-boot-record, verify-syscall all pass.
|
||
|
surface-camera-build
|
087c6b0e60 |
cubelinux: the kernel records its own boot
The second half of "the OS stores itself". The store was already the kernel's; what was missing was the kernel *saying* something of its own rather than a client doing it. At its first write of a boot the driver now appends one record describing the boot it is having — its own version banner, the wall-clock time, and the store device it resolved — to a reserved space, through the same append path every other mutation uses. Three things had to be decided, and each had a wrong answer that looked right: WHERE IT HOOKS. "At store init" does not exist and must not be invented: there is no init-time open, deliberately, so there is no ordering to get wrong against the block driver that provides the device. An __initcall appending a record would reintroduce exactly that ordering problem. The moment is the FIRST WRITE, which needs no ordering at all and is the semantically right one — the kernel records itself when it becomes the writer. A boot in which the kernel only reads writes no record, which is honest rather than a gap. WHICH SPACE. 0xFD was the first choice, following the convention that a reserved space is one repeated byte. 0xFD is the OS KEYSTORE — the space the kill switch exists to destroy — and writing boot records into it would have been a serious bug. Only the userspace name table catches this (format_space in crates/cube-command) because the kernel keeps no table of space names, so the table is now written down where the constant is: 0x00 root, 0xFF edges, 0xFE portal, 0xFD keystore, 0xFC boot. The record lives at (0,0,0) in 0xFC: one record, the current boot. WHAT IT SAYS. boot=<epoch seconds> device=<resolved path> kernel=<the version banner>, banner last and unquoted so everything after the final = is the kernel's own words rather than a field this code parsed. Raw epoch seconds rather than a date: rendering a calendar date in the kernel is date arithmetic, and a caller with a clock can do it without a kernel bug being the reason a timestamp is wrong. The banner comes from linux_banner and the time from ktime_get_real_ts64. WHY IT IS OFF BY DEFAULT. Not caution, but an invariant. The gates' method is that the store the kernel produces is comparable, byte for byte, with the store userspace produces from the same mutations; a record the kernel injects that the caller never asked for would turn those comparisons into non-comparisons. So it is cube_boot_record=1 on the kernel command line, parsed in C beside cube_store= for the reason that parameter is in C (this kernel's Rust cannot express a string parameter), and kernel/verify-boot-record.sh is the gate that turns it on. A failure to record is logged and never propagated: the record is worth having and is not a precondition for the caller's write. It is attempted once per boot rather than retried per write, because a store that will not take it will not take it later, and one warning is information where a stream of them is noise. |
||
|
surface-camera-build
|
78540b5687 |
cubelinux: the fold reads the layout it writes
Found on the box, minutes after the live store was folded for the first time: the second fold answered -EINVAL. `build_merged` parsed the packed layout — `space | key | len | value`, repeated — so it could read a store that had never been folded and nothing else. The first fold reads packed and writes addressed; every fold after that reads addressed, which is what a store does for the rest of its life. As written, a store could be folded exactly once and then never again — the log would grow until it filled. No gate folded twice, which is why it got this far: verify-kernel-checkpoint.sh, verify-frontend.sh and verify-enum-cost.sh each folded a packed store once. The guest's bench folds twice now, and verify-enum-cost.sh insists on the second one — "a store can be folded once and then never again, which is not a layout" — so the case is covered rather than remembered. The v3 branch reads through the shared format's own arithmetic: the space table gives each space's index range, the index gives each key and the value's place, and the log is applied over the result exactly as before. The packed path is untouched. Verified: verify-enum-cost.sh (which now folds twice and still measures 1.0x for a 40x store) and verify-kernel-checkpoint.sh (the image a fold writes holds exactly what userspace holds). |
||
|
surface-camera-build
|
d569df710d |
cubelinux: the store's format is one file, and the kernel and userspace both include it
Two build systems cannot share a crate: the kernel's Rust build compiles what is in its own module
tree, and a cargo crate is not that. So they share a *file* — `drivers/cube/cube_format.rs`, which
the driver declares with `mod` and `crates/cube-format` includes by path. There is no copy to drift
from, which is the only arrangement that cannot go stale. What happened when v3 landed and userspace
did not is the argument: an hour of `unsupported-version`, one gate red, and two implementations of
one format each believing itself.
The file holds what both sides must agree about, and nothing else — pure functions over slices, no
allocation, no I/O, no logging:
* the constants every reader and writer derives its arithmetic from,
* the header, in all three versions, with refusal rather than guessing: a reader that invents an
extent can read somebody else's bytes,
* the v3 space table and index, and the three equalities a reader computes its addresses from,
* the packed record and the log entry, the two framings a fold and a walk have to parse alike,
* CRC-32, FNV-1a, and the digest line both sides print.
The driver's copies of the constants are now aliases of the shared ones, and the two functions that
*validate* a header — `parse_header` and the v3 geometry — delegate to it, because validation is
where a second description gets believed. The readers stay where they are: this driver streams from
a file with its own buffers while userspace already holds the whole image, and that difference is
real rather than duplicated.
Nothing changed in behaviour, which is the point: verify-enum.sh, verify-enum-cost.sh,
verify-frontend.sh, verify-kernel-checkpoint.sh and verify-kernel-append.sh all pass, and the shared
crate's own tests include a kernel-written image — ten records, three spaces, 110 value bytes — read
through its arithmetic.
|
||
|
surface-camera-build
|
1db3abce93 |
cubelinux: v3 — the image carries the addresses, so a coordinate is a place
A packed list (`space | key | len | value`, repeated) cannot answer "where is this coordinate":
record N's offset is the sum of every record before it. The key gives ORDER, and order is not an
address, so no coordinate could be turned into a place and not even a binary search was possible —
the middle record's offset is just as unknowable. Every lookup walked, every batch of a listing
walked again, and a read of one record cost as much as the store (measured: 110 ms per get, 119 ms
per `spaces`, 1,206 ms for a 6,127-record listing).
v3 puts the addresses in the image:
[header 46] magic, version, curve, extents, counts
[space table: space_count x 48] space | first index | records
[index: record_count x 40] key | value offset | value length
[values: packed, in index order]
A lookup is now a binary search over arithmetic addresses (`index_off + i * 40`); a listing starts
at its space's first index entry and streams, reading a span of values per batch; asking which
spaces exist is the space table; a spatial range is a contiguous run of index entries. An index
entry is 40 bytes against the packed frame's 64, so the image also gets smaller.
Measured by `verify-enum-cost.sh` between a 500-record store and a 20,000-record one:
listing the same 10-record space: 3.22 ms -> 3.27 ms (1.0x, store grew 40x)
asking which spaces exist: 3.25 ms -> 3.08 ms (0.9x)
reading a coordinate that is there: 1.07 ms -> 1.25 ms (1.2x)
reading one that is not: 1.19 ms -> 1.22 ms (1.0x)
Nothing scales with the store any more, which is the premise: knowing a coordinate is what lets you
reach it.
Two bugs found on the way, both of the kind only a real image shows. The index was written with a
4-byte value length (`Entry` carries it in a u32) while the header's arithmetic and both readers
assumed 8 — every entry four bytes short, the last of them overlapping the values. And the digest's
`bytes` figure added the layout's own header length, so two layouts of one store digested
differently; it is now the records' logical size, which is the same number either way.
Verified: verify-kernel-append.sh, verify-kernel-checkpoint.sh, verify-enum.sh, verify-enum-cost.sh
and verify-frontend.sh all pass, and the workspace tests pass — including a new fixture test in
cube-store-raw that reads a 700-byte image the kernel itself wrote, because a hand-built image
cannot catch a misunderstanding shared by the builder.
|
||
|
surface-camera-build
|
56665a303e |
cubelinux: read the store where it lies, instead of rebuilding it per call
Every read began by reading the whole device into kernel memory, parsing every
record of every space into a merged pool, and heapsorting the lot — then throwing
all of it away. So a `get` of one record cost as much as listing the store, every
batch of a listing paid it again, and the coordinate bought nothing mechanically:
knowing where a record is did not make reaching it any cheaper. Measured on the
workhorse: one `get` 110 ms, one `spaces` call 119 ms, one 6,127-record listing
1,206 ms.
The records are already in the order the coordinate computes — a checkpoint writes
them by (space, key) — so the kernel now walks them where they lie: zero-copy
values, no pool, no sort, and the log (small, and the only thing that can override
the image) consulted as an overlay. It reads the image and the log window rather
than the whole device, which is four images wide.
* `get` — the log's newest word on the coordinate, else the image walked to it.
* `enum` — one merge pass over two sorted sequences: the space's image records
and its log edits.
* `spaces` — from a table of where each space starts, cached by the control
block's generation. That table is one entry per space, not per record, and a
checkpoint is the only thing that invalidates it.
* `put`/`del`/`sync` — unchanged: a write is a log append, and the fold still
writes the whole sorted store.
Verified: verify-enum.sh (a 3,006-record listing identical to userspace's, record
for record) and verify-frontend.sh (socket -> front-end -> cube(2), including a
listing larger than one batch and the store the front-end leaves being
digest-identical to userspace's) both pass.
Not yet what it should be: listing a 10-record space in a 20,000-record store
still costs about six times what it costs in a 500-record one, so the space table
is not taking effect as intended and the lookup is not yet independent of store
size. That belongs in the image's layout — a coordinate cannot be turned into a
byte offset in a variable-length packed list, because a record's position is the
sum of every value before it — and the fix is a format one, not a caching one.
|
||
|
surface-camera-build
|
b9a7b8d8f3 |
cubelinux: cube(2) walks the store — CUBE_OP_ENUM and CUBE_OP_SPACES
Enumeration was the one operation the interface did not have, and the one a capability interface owes an explanation for: a listing is the opposite of "knowing a coordinate is the authorisation to use it". So the walk is bounded and explicit. A caller names the space, holds a cursor, and gets as many whole records as fit in the buffer it offered, packed `key(24) | value_len(u32) | value` in the store's own order. SPACES walks the distinct spaces that hold a record, one per call; range is that walk with the region as a filter, applied by the caller rather than by a second operation in the kernel. Its own argument block, versioned by its own size: SYSCALL_DEFINE2 peeks `size` and routes — 80 bytes is the coordinate block, 64 is this one. An interface that cannot grow has to be replaced, and this one grows by being given a new block. The end of a walk is the cursor alone. A batch holds as many whole records as fit, so it is FULL only when a record lands on the boundary and most batches come back short; a caller that reads "the buffer was not filled" as "the space is exhausted" truncates its listing to the first batch and cannot tell. ENUM answers a finished walk with no records and the cursor unmoved; SPACES answers it with -ENOENT, because the space after the last one is not there. That rule is in the uapi header because it is a contract detail, not an implementation one. The kernel caches no merged index: each call walks the records in order, asks the log — small, and the only thing that can override one — what the winner is, and skips what the cursor has already covered. O(records) a call, tens of milliseconds for the live store, which is worth more than a cache every write would have to invalidate. |
||
|
surface-camera-build
|
ed5ff764ba |
cubelinux: the release starts with a digit, because module tooling demands it
The store's sort no longer lives on the kernel stack (heapsort, see the parent commit's gate), and the release is now 6.19.3-cubelinux0.6 instead of CUBELinux.0.6 — depmod/mkinitramfs reject a version whose first character is not numeric, so the literal name cannot be uname -r. The box's own kernel uses the same shape (6.19.3-cube+); this is the same concession. |
||
|
CUBELinux build
|
e202f960d5 |
cubelinux: the digest's byte count is the store's, not the reader's
'bytes=' reported how much had been read, so the same records disagreed between a file (16,537,532) and a device slot holding them (67,108,864). The gates compare this line, so it has to mean the same thing on both sides: it is now the header plus the records — the length the store actually is. Found by moving the machine's real store into a device store and comparing. |
||
|
CUBELinux build
|
3ee59dafff |
CUBELinux.0.6: cube(2) — the coordinate interface
The write path was proven but unreachable: its operations lived behind a device node. This is the interface the decision chose (DESIGN-cube-interface.md) — one syscall number, an opcode, and a versioned argument block, with operations that are the verbs the command language already defines: put, get, del, sync. long cube(unsigned int op, struct cube_args __user *args) `size` comes first and is checked, because syscall numbers are permanent and an interface that cannot grow would have to be replaced. A coordinate is the space and its three axes; nothing here resolves a name and nothing enumerates. Split deliberately: the entry point, the user copies and the argument validation are in C (cube_syscall.c) because `SYSCALL_DEFINE*` is a C macro this kernel has no Rust equivalent for; everything that touches the store's bytes is in Rust, which passes the coordinate to the format code as its parts so that Morton encoding stays in the one module that must get it exactly right. A read that does not fit returns the size it needs rather than truncating — a short read would be worse than an error. Number 548: the x86_64 table says numbers 548 and above are available for non-x32 use. Gate (kernel/verify-syscall.sh): a static client in the initramfs does four writes (including an empty value and a second space), reads one back *through the same interface*, and folds with `sync`. Three different failures are separated — the calls failing (a broken ABI), a read not returning what a write stored (a wrong key encoding or index), and the folded image differing (a wrong format, order or merge). put 7,0,0 ok (21 bytes) put 8,0,0 ok (0 bytes) put 9,0,0 ok (28 bytes) put 1,2,3 ok (13 bytes) get 7,0,0 21 bytes: the kernel wrote this sync ok and the image left on the device is byte-identical to the one userspace writes from the same mutations, with the log empty. The interface's store is the same store. All nine gates pass on 0.6. |
||
|
CUBELinux build
|
2d4ed0fd9f |
CUBELinux.0.5: a torn log entry is overwritten, not buried
The write path's last unproven claim was the one everything else rests on: replay is prefix-trusting. Building the gate for it found that the claim was false as implemented — in *both* implementations, and in the same way. The log walkers advanced their offset past an entry's frame and only then checked the checksum. A corrupted entry therefore never moved the *replay* point (it was discarded, so the store looked right) but it did move the *append* point. The consequences, in order of how bad they are: - the kernel appended **after** the tear, burying the corruption inside a log that then looked well-formed — the exact opposite of the documented rule; - the userspace log truncated to a prefix that still contained the corrupt frame, so every later append started past it and the corruption stayed in the log forever; - and because each append then read back the same wrong prefix, every append wrote to the same offset and overwrote its predecessor — four mutations in, one on disk. The fix is one line of ordering in three places: compute the frame's end, check the length and the checksum, and only then move the offset. An entry that did not validate may not move the point that says where the log ends. Gate (kernel/verify-torn-tail.sh), on a store with an overwrite, an insertion and a deletion whose last entry has been torn by zeroing its tail in place: whole : 272 bytes, 3 records (the delete applied — the contrast) reference : 349 bytes, 4 records (userspace, torn bytes: the delete discarded) kernel : 349 bytes, 4 records appended : 667 bytes, 8 records (the kernel appended over the tear) expected : 667 bytes, 8 records (userspace: torn tail dropped, then the same four) Building the gate also corrected the gate itself: tearing a log by *truncating the device* is not a torn log, it is a smaller device — the capacity shrinks, writes past the end vanish into the page cache with no error anywhere, and the test measures an artifact. A torn write leaves the device the same size and corrupts bytes in place. All seven gates pass on 0.5. |
||
|
CUBELinux build
|
451754c7ac |
CUBELinux.0.4: the kernel folds its own log
Until now the kernel could append to a log but needed a *userspace* checkpoint to reclaim it — the dependency the write-authority decision was meant to remove. This is the fold. A store device is no longer an image at offset 0. It is: [control copy A][control copy B][slot 0][slot 1][log] with the control block saying which slot is live and how much log is in use, and two copies each carrying a generation and a checksum. Two slots buy atomicity: the folded image is written to the slot nothing is reading and flushed, and only then does the control change — one small write to the copy that is *not* current. At every instant there is either the old image with a log that still describes the mutations since it, or the new image with an empty log. A crash in the middle of the copy leaves the previous store intact, which is the entire reason for two slots. Folding into the only copy of an image is not atomic, so `sync` refuses on a bare image rather than pretending. `sync` is a real operation, not a test hook: a caller that wants the log reclaimed — a shutdown, a snapshot, a handover — is entitled to ask. Gate (kernel/verify-kernel-checkpoint.sh): the kernel writes four mutations and then folds its own log. The image it writes is compared with the one userspace writes from the same mutations byte for byte, not by digest — and it is identical, with the log empty afterwards. Three bugs came out of building this, all caught by the gates rather than by inspection: - the refactor that extracted the merge left the image walk *duplicated* inside the digest, re-adding image records after the log's entries — which gave them a newer sequence number and quietly resurrected records the log had deleted; - a v1 image has no extent, so it has no log either, and the new layout code was demanding both; - an unwritten log region is empty, not broken, and the first append was refusing it. The append gate's harness was also counting a *refused* write as accepted, which is how the second one hid. All six gates pass: three image reads (v1 curated, v1 snapshot at 35,318 records, v2 store), the log replay, the append with its SIGKILL durability test, and the checkpoint compared byte for byte. |
||
|
CUBELinux build
|
93e2c8c39b |
CUBELinux.0.3: the kernel writes, and is compared while writing
0.2 read the store. This appends to it: a mutation through /dev/cubelinux is written to the log and fsynced *before* the write is accepted, which is the kernel's version of the contract cube_duratest.py proves for the daemon. - the mutation comes in as the argument block the coordinate interface will pass: op(1) | space(32) | x(8) | y(8) | z(8) | len(4) | value[len]. The kernel Morton-encodes the point itself — a key written here has to be the key a userspace reader decodes, and that is a thing the gate would catch if it were merely similar; - the append lands after the log's valid prefix, so a torn tail is overwritten rather than appended to, exactly as the userspace log does it; - a log region that has never been written is zeros, not a log: the header is written first, the way the userspace log creates its file; - the entry's CRC covers space, key, length and value, computed the same way. The write is the byte plane — bytes at a coordinate, no header written beside them — because that is what a differential comparison against userspace's `cell put` can be exact about. The header tier sits above this. Gate (kernel/verify-kernel-append.sh), four mutations including an empty value and a record in a second space: reference : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 kernel : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 folded : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 survived : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 The third line is the one that matters: userspace *reads the log the kernel wrote*, folds it, and lands on the same store. Without it the first two would only show the kernel agreeing with itself. The fourth is a SIGKILL of the VM with no shutdown and therefore no flush for us. Not yet: the kernel folds nothing itself, so it still depends on a userspace checkpoint to reclaim its log. That is the next step, and it has its own gate. |
||
|
CUBELinux build
|
63b0a2f26e |
cubelinux: replay the write-ahead log, and be compared while doing it
A store on a device is [image][log]: the image is a checkpoint, the log is the
mutations since, in the framing cube-store/src/wal.rs writes (op | crc32 | space
| key | len | value, prefix-trusting recovery). The kernel now reads both and
reports the store they *describe* — the image with the log applied, in the order
a checkpoint would write it.
- the log's offset is fixed by geometry (the first 4 KiB boundary at or after the
image's declared extent), so no superblock is needed to find it;
- recovery is prefix-trusting, exactly as userspace does it: the first entry that
is short, mis-framed or fails its CRC ends the log;
- merging is by coordinate with the newest write winning, which is what makes an
overwrite an overwrite and a delete a delete;
- the digest is taken over the surviving records in sorted order, and `bytes`
reports the size a checkpoint of this store would produce — so the number can
be compared with the image userspace actually writes, not merely eyeballed.
Gate (kernel/verify-log-replay.sh), on a pair carrying an overwrite, an
insertion and a deletion:
reference : bytes=377 records=4 value_bytes=99 fnv1a64=a38c1db74b00966f
kernel : bytes=377 records=4 value_bytes=99 fnv1a64=a38c1db74b00966f
log entries=6 applied over the image
It failed the first time, which is the point: the coalescing kept the earliest
write of a coordinate rather than the latest, resurrecting records the log had
deleted — six records where the fold produced four. A differential gate is what
turns that class of error into a red line instead of a corrupted store.
The image-only gates still pass unchanged (v1 curated, v1 snapshot 35,318
records, v2 store): with no log to apply the digest is the image's own order, so
v1 readers and every existing tool keep seeing exactly what they saw before.
|
||
|
CUBELinux build
|
b54ba34dc0 |
cubelinux: read format v2 as well as v1
The store's write path is decided (append a log, fold it into the image at a checkpoint; DESIGN-cubelinux-write-path.md), and that decision forces the log to sit right after the image on the device. v1 had no extent and no count — it walked records until it met zero padding — so a v1 reader would have walked straight into the log's header and parsed it as a record. v2 states the image's byte extent and its record count, and this teaches the kernel reader both: - v1: walk from a 6-byte header until the trailing zeros, as before. - v2: walk from a 22-byte header, stop exactly at the declared count, and never read past the declared extent. A zero frame inside the count is a record, not padding — which is the ambiguity v1 could not resolve. - a v2 header whose extent does not cover the header itself is refused rather than guessed at, and a count that is not met counts as an error instead of quietly returning a shorter list. Gate, unchanged in method: the kernel's digest of /dev/vda must equal cube-image's digest of the same bytes. Passing on all three: v1 curated 11 records fnv1a64=161113085b1573b2 v1 snapshot 35,318 records fnv1a64=5e20f98455387b08 v2 store 4 records fnv1a64=20ecadb5cdc9c994 |
||
|
CUBELinux build
|
cee3e554d9 |
CUBELinux.0.2: the kernel reads the CUBE store off a block device
The first CUBE code in the kernel, and deliberately only a reader: the write
authority has not moved yet, and PLAN-kernel-cubelinux.md records both that
decision and the hazard that makes the order matter — a kernel writing while a
userspace daemon still holds the same image loses one of the two writers' work,
silently. A reader cannot do that.
- drivers/cube/: a Rust module exposing /dev/cubelinux. Reading it reads the
pinned image from the block device through the kernel's own file layer
(filp_open + kernel_read — the path this kernel version binds for Rust, and
the reason no C helper was needed), parses the records, and returns one line:
digest curve=0 bytes=8400896 records=35318 value_bytes=6139148 fnv1a64=5e20f98455387b08 errors=0
The work happens on read, not at init, so there is no initcall ordering to get
wrong against the block driver that provides the device.
- The format is restated in the kernel (32-byte space, 24-byte key, 8-byte LE
length, value), including the two rules the userspace parser documents: a value
that runs past the buffer is a truncated record, and an all-zero frame ends the
records only when every remaining byte is zero — the rule that keeps a real
record at the origin from being read as padding.
- The digest is the point. A record count alone lets two different images agree;
folding the bytes in means the kernel and userspace are *compared* rather than
assumed to agree. `cube-image digest` prints the same line in the same field
order, and the QEMU gate fails if they differ by a byte.
Gate, on both images:
curated 11 records, 4,096 bytes, fnv1a64=161113085b1573b2 — match
snapshot 35,318 records, 8,400,896 bytes, fnv1a64=5e20f98455387b08 — match
The tree carries CONFIG_CUBELINUX_STORE=y on top of defconfig + RUST; a tree
without it boots and simply has no /dev/cubelinux.
|