A packed list (`space | key | len | value`, repeated) cannot answer "where is this coordinate":
record N's offset is the sum of every record before it. The key gives ORDER, and order is not an
address, so no coordinate could be turned into a place and not even a binary search was possible —
the middle record's offset is just as unknowable. Every lookup walked, every batch of a listing
walked again, and a read of one record cost as much as the store (measured: 110 ms per get, 119 ms
per `spaces`, 1,206 ms for a 6,127-record listing).
v3 puts the addresses in the image:
[header 46] magic, version, curve, extents, counts
[space table: space_count x 48] space | first index | records
[index: record_count x 40] key | value offset | value length
[values: packed, in index order]
A lookup is now a binary search over arithmetic addresses (`index_off + i * 40`); a listing starts
at its space's first index entry and streams, reading a span of values per batch; asking which
spaces exist is the space table; a spatial range is a contiguous run of index entries. An index
entry is 40 bytes against the packed frame's 64, so the image also gets smaller.
Measured by `verify-enum-cost.sh` between a 500-record store and a 20,000-record one:
listing the same 10-record space: 3.22 ms -> 3.27 ms (1.0x, store grew 40x)
asking which spaces exist: 3.25 ms -> 3.08 ms (0.9x)
reading a coordinate that is there: 1.07 ms -> 1.25 ms (1.2x)
reading one that is not: 1.19 ms -> 1.22 ms (1.0x)
Nothing scales with the store any more, which is the premise: knowing a coordinate is what lets you
reach it.
Two bugs found on the way, both of the kind only a real image shows. The index was written with a
4-byte value length (`Entry` carries it in a u32) while the header's arithmetic and both readers
assumed 8 — every entry four bytes short, the last of them overlapping the values. And the digest's
`bytes` figure added the layout's own header length, so two layouts of one store digested
differently; it is now the records' logical size, which is the same number either way.
Verified: verify-kernel-append.sh, verify-kernel-checkpoint.sh, verify-enum.sh, verify-enum-cost.sh
and verify-frontend.sh all pass, and the workspace tests pass — including a new fixture test in
cube-store-raw that reads a 700-byte image the kernel itself wrote, because a hand-built image
cannot catch a misunderstanding shared by the builder.
Every read began by reading the whole device into kernel memory, parsing every
record of every space into a merged pool, and heapsorting the lot — then throwing
all of it away. So a `get` of one record cost as much as listing the store, every
batch of a listing paid it again, and the coordinate bought nothing mechanically:
knowing where a record is did not make reaching it any cheaper. Measured on the
workhorse: one `get` 110 ms, one `spaces` call 119 ms, one 6,127-record listing
1,206 ms.
The records are already in the order the coordinate computes — a checkpoint writes
them by (space, key) — so the kernel now walks them where they lie: zero-copy
values, no pool, no sort, and the log (small, and the only thing that can override
the image) consulted as an overlay. It reads the image and the log window rather
than the whole device, which is four images wide.
* `get` — the log's newest word on the coordinate, else the image walked to it.
* `enum` — one merge pass over two sorted sequences: the space's image records
and its log edits.
* `spaces` — from a table of where each space starts, cached by the control
block's generation. That table is one entry per space, not per record, and a
checkpoint is the only thing that invalidates it.
* `put`/`del`/`sync` — unchanged: a write is a log append, and the fold still
writes the whole sorted store.
Verified: verify-enum.sh (a 3,006-record listing identical to userspace's, record
for record) and verify-frontend.sh (socket -> front-end -> cube(2), including a
listing larger than one batch and the store the front-end leaves being
digest-identical to userspace's) both pass.
Not yet what it should be: listing a 10-record space in a 20,000-record store
still costs about six times what it costs in a 500-record one, so the space table
is not taking effect as intended and the lookup is not yet independent of store
size. That belongs in the image's layout — a coordinate cannot be turned into a
byte offset in a variable-length packed list, because a record's position is the
sum of every value before it — and the fix is a format one, not a caching one.
Enumeration was the one operation the interface did not have, and the one a
capability interface owes an explanation for: a listing is the opposite of
"knowing a coordinate is the authorisation to use it". So the walk is bounded
and explicit. A caller names the space, holds a cursor, and gets as many whole
records as fit in the buffer it offered, packed `key(24) | value_len(u32) | value`
in the store's own order. SPACES walks the distinct spaces that hold a record,
one per call; range is that walk with the region as a filter, applied by the
caller rather than by a second operation in the kernel.
Its own argument block, versioned by its own size: SYSCALL_DEFINE2 peeks `size`
and routes — 80 bytes is the coordinate block, 64 is this one. An interface that
cannot grow has to be replaced, and this one grows by being given a new block.
The end of a walk is the cursor alone. A batch holds as many whole records as
fit, so it is FULL only when a record lands on the boundary and most batches
come back short; a caller that reads "the buffer was not filled" as "the space
is exhausted" truncates its listing to the first batch and cannot tell. ENUM
answers a finished walk with no records and the cursor unmoved; SPACES answers
it with -ENOENT, because the space after the last one is not there. That rule is
in the uapi header because it is a contract detail, not an implementation one.
The kernel caches no merged index: each call walks the records in order, asks
the log — small, and the only thing that can override one — what the winner is,
and skips what the cursor has already covered. O(records) a call, tens of
milliseconds for the live store, which is worth more than a cache every write
would have to invalidate.
The store's sort no longer lives on the kernel stack (heapsort, see the parent
commit's gate), and the release is now 6.19.3-cubelinux0.6 instead of
CUBELinux.0.6 — depmod/mkinitramfs reject a version whose first character is
not numeric, so the literal name cannot be uname -r. The box's own kernel uses
the same shape (6.19.3-cube+); this is the same concession.
surface-camera-build