V3::encode wrote a hardcoded VERSION_V3. It had no callers, so the mistake
cost nothing — and then userspace's v4 writer became the first caller, and a
v4 geometry would have been published under a v3 header: every reader walking
a 42-byte index 40 bytes at a time, finding a store that is silently wrong.
One wrong byte, found before the first v4 image was written rather than after.
The first half of the flag substrate (DESIGN-flag-vocabularies.md): the shared
format file now defines VERSION_V4, whose index entry is `key | flags(u16) |
value_off | value_len`, and WAL version 2, whose entry carries the same mask
before its length. The mask is a raw u16 — its bits are a vocabulary's business,
never the format's.
Backward compatible, and pinned as such: a v3 index entry and a v1 log entry read
as a zero mask ("no class"), which a scan treats as matching nothing, so a store
folded before the flag existed degrades to "unclassified" rather than "matches
everything". `V3::decode` accepts both versions and `index_stride()` names the
one that differs; `wal_entry` keys its stride off the log's own version byte.
The readers and writers that actually move bytes (the driver's serialize and
append, and `cube-store-raw`) are separate and are the next commit; this is the
shared definition and the arithmetic a reader derives from it.
The seventh operation: `cube(2)` gains CUBE_OP_RANGE, a bounded walk of one
space's records that lie in a box. Its argument block is its own (`cube_range_args`,
versioned by `size` like the walk's), and the box travels as its six numbers for
the reason a coordinate does — the key it has to become is the driver's business.
The operation rests on the span that the shared file already owns. `key_span(lo, hi)`
bounds every key in the box because the interleave is monotone on each axis, so a
v3 image is **sought**: the fixed-stride index is binary-searched for the span's foot
(`Addressed::lower_bound`) and read forward to its head, merging the log's edits
exactly as a walk does. A packed v1/v2 image has no index to search, so its space is
walked with the same span used only to stop early — and the contract is the same
either way, so a caller is not told which path it got.
The trap that shaped the code, and the reason it is written the way it is: **the span
is a bound, not the set.** Keys of points outside the box fall inside it (Z-order
amplification), so every candidate is decoded and tested against the box before it
is returned — which is what `morton_decode`, the interleave's inverse, is for, now in
the shared file with the same kind of hand-pinned tests the interleave has. And the
cursor counts the records *in the box*, not the records of the space, because those
are the records the walk returns.
The over-coverage — records examined versus records returned — is the number this
operation is meant to publish, and it is not wired to the caller yet: the cost gate
measures it by comparing against the userspace store, which counts the same thing.
`morton_encode` existed twice — once in the driver, once (as the curve) in cube-core — and the two
had to agree byte for byte, because one side computes a key to look a record up and the other
computes it to lay an image out. Nothing structural held that agreement: only the gates, which
would notice afterwards. That is the same shape as the cube_format.rs finding recorded in
DESIGN-coordinate-surface.md §7.4 — a shared *file* whose shared half has no callers — so this
starts paying it off where it is cheap and exact.
The interleave now lives in cube_format.rs, which both builds compile, and the driver's copy is a
call to it. It is written as plain `while` loops rather than iterator chains because the kernel
build of this file has no `std` and no `alloc`, which is the constraint the whole arrangement
exists under.
Alongside it, two things the range work needs and neither side had:
key_cmp compare two keys as the 192-bit numbers they are. The key is big-endian, so this
is the comparison a sorted index performs, and it is stated once rather than
assumed at each site.
key_span the key span a box covers, (key(lo), key(hi)) inclusive. This is the bound a seek
needs, and — the point — it needs no decomposition at all: the interleave is
monotone in every axis, so every point in the box has a key between its corners'
keys. A sorted index can be binary-searched for the foot and scanned forward to the
head. The span is a BOUND and not the set: records outside the box can have keys
inside it, which is the classic Z-order amplification and is why a caller tests
membership per candidate. Confusing the bound for the set is close to the mistake
that put "an aligned box is one run" into two doc comments.
aligned_box_is_one_run
the condition, proved in crates/cube-format's tests and pinned there and in
cube-store: a power-of-two-aligned box is exactly one contiguous run of the key iff
max(k) - min(k) <= 1 AND the axes at the top level form a prefix of (x, y, z). A
size that is not a power of two is answered `false` rather than rounded, because a
caller passing one has a bug and "no" is the useful answer.
Verified on build #53: verify-enum (the kernel's walk against userspace's, which is the check that
would catch any change in the key), verify-boot-record, verify-syscall all pass.
Found on the box, minutes after the live store was folded for the first time: the second fold
answered -EINVAL. `build_merged` parsed the packed layout — `space | key | len | value`, repeated —
so it could read a store that had never been folded and nothing else. The first fold reads packed
and writes addressed; every fold after that reads addressed, which is what a store does for the rest
of its life. As written, a store could be folded exactly once and then never again — the log would
grow until it filled.
No gate folded twice, which is why it got this far: verify-kernel-checkpoint.sh, verify-frontend.sh
and verify-enum-cost.sh each folded a packed store once. The guest's bench folds twice now, and
verify-enum-cost.sh insists on the second one — "a store can be folded once and then never again,
which is not a layout" — so the case is covered rather than remembered.
The v3 branch reads through the shared format's own arithmetic: the space table gives each space's
index range, the index gives each key and the value's place, and the log is applied over the result
exactly as before. The packed path is untouched.
Verified: verify-enum-cost.sh (which now folds twice and still measures 1.0x for a 40x store) and
verify-kernel-checkpoint.sh (the image a fold writes holds exactly what userspace holds).
Two build systems cannot share a crate: the kernel's Rust build compiles what is in its own module
tree, and a cargo crate is not that. So they share a *file* — `drivers/cube/cube_format.rs`, which
the driver declares with `mod` and `crates/cube-format` includes by path. There is no copy to drift
from, which is the only arrangement that cannot go stale. What happened when v3 landed and userspace
did not is the argument: an hour of `unsupported-version`, one gate red, and two implementations of
one format each believing itself.
The file holds what both sides must agree about, and nothing else — pure functions over slices, no
allocation, no I/O, no logging:
* the constants every reader and writer derives its arithmetic from,
* the header, in all three versions, with refusal rather than guessing: a reader that invents an
extent can read somebody else's bytes,
* the v3 space table and index, and the three equalities a reader computes its addresses from,
* the packed record and the log entry, the two framings a fold and a walk have to parse alike,
* CRC-32, FNV-1a, and the digest line both sides print.
The driver's copies of the constants are now aliases of the shared ones, and the two functions that
*validate* a header — `parse_header` and the v3 geometry — delegate to it, because validation is
where a second description gets believed. The readers stay where they are: this driver streams from
a file with its own buffers while userspace already holds the whole image, and that difference is
real rather than duplicated.
Nothing changed in behaviour, which is the point: verify-enum.sh, verify-enum-cost.sh,
verify-frontend.sh, verify-kernel-checkpoint.sh and verify-kernel-append.sh all pass, and the shared
crate's own tests include a kernel-written image — ten records, three spaces, 110 value bytes — read
through its arithmetic.
surface-camera-build