Closing the last hole in the substrate: `CUBE_OP_ENUM` and `CUBE_OP_RANGE`
returned `key | len | value`, so a kernel- or daemon-backed store's
`entries_flagged`/`range_flagged` could only answer from the trait's default —
"no class" for every record in the store. Both walks now return the flag scan's
frame, class included, and both stores answer for real.
`Command::List` and `Command::Range` print it, because the record has one: a
listing that reported only lengths is what made a caller open the store to find
out what it was looking at. That is also what keeps the kernel's listing and
userspace's comparable in the same words, which `verify-enum` and
`verify-range` are built on.
Three parsers had to follow, and each was found by a test rather than by
reading:
* `CubeArgs`'s walk decoder in `cube-kernel` read the length at the wrong
offset — caught by asserting the class of one record and none for its
neighbours, so a decoder that read the wrong two bytes shows up as the
wrong *record* rather than as a length that happens to be right;
* the client's `listed` split a line on " [" and parsed the head, which is
now `<coord> flags=0xNNNN`, so every listing came back EMPTY rather than
failing — a parse that silently drops what it cannot read is the worst
version of this;
* the front-end gate looked for `root:799,0,0 [100B]`.
`verify-frontend` now asserts the class in a *listing* as well as in a point
read, which is the asymmetry the hole was made of: one store, two answers about
the same record.
`Reply` carries the record's class, so `CellGet` returns what `cell put flags=` wrote.
The JSON field is additive: a peer that does not send it reads as 0, which is what a
record written before the field existed means — the same answer rather than a wrong
one. `DaemonStore::get_flagged` takes it, so a remote store no longer answers "no
class" for a record that has one.
THE SEALING WRAPPER was the third instance of exactly that, and the front-end gate
found it: `SealedStore` forwarded `put_flagged` and then answered every read from the
trait's default. Worse, fixing `get_flagged` first was not enough — `entries_flagged`
and `range_flagged` were missing too, so a class survived a point read and vanished
from a listing. A wrapper has three read paths, not one, and the trait's default makes
each omission look like an answer.
So the trait now states the rule where it is implemented: a backend that stores masks
must override all three, the default's `flags: 0` is right only for a backend that
genuinely cannot carry one, and a *transport* that has no field for one produces the
same zero — so a backend of that second kind has to say so. `KernelStore` does: `get`
carries the class because `CUBE_OP_GET` fills it, and `entries`/`range` do not because
those frames have no field for it, which is why scanning a kernel store by class is
`CUBE_OP_FLAG_SCAN`'s job.
TWO GATE BUGS, both found while chasing that one. `verify-frontend` watched only
`cube-kernel/src` for changes, so an edit to `cube-store-seal` — exactly where the
class was being dropped — left the front-end binary stale and the gate testing the
previous behaviour: the gate whose own comment says "a stale front-end would test an
interface nobody has, which is how a green gate lies". It watches every crate now. And
it built its front-end-flavoured initramfs over the shared `initramfs.cpio.gz`, so a
gate was silently changing another gate's fixture; it builds to a path of its own.
`verify-flag-scan` now tells a boot that did not finish from a scan that answered
wrongly. That flake has now been seen three times, always inside a sweep and always
silent; two was a pattern, three is worth the distinction even before the cause.
`KernelStore::get_flagged` takes the mask out of the argument block, so the trait's
default no longer answers "no class" for a store that is holding one. That default
was the third instance of the same lie — MemStore had it, and a test caught it; this
one the transport could not express until the syscall learned to answer.
The guest's `get` prints what it got: `get x,y,z flags=0xNNNN N bytes: <value>`.
The flags sit before the count rather than after the value because the value is
variable-length and last, and the gates that parse this line read from `bytes: `
onward, so nothing that already read a value had to change.
`verify-flag-scan` now asserts it, on both paths: the class is read with the record
still in the log and again after the fold, and both must be the class it was written
with. A scan answers "which records are errors"; a read answers "what is *this*
record" — and they have to agree about the same bytes, which is what makes the class
a property of the record rather than of the query that found it.
The write half of the substrate, for the userspace toolchain. The mask is the
*writer's* to state, because the writer is the one that knows, and it goes in
the record's own frame rather than in a header `cell put` exists to avoid.
Named `flags=<hex>` rather than a bare number so a value that happens to start
with a digit is never mistaken for a class, and re-rendered only when it is
non-zero so a command without one round-trips to the same text.
Every layer forwards it, including the ones that could have dropped it
quietly: the remote client (a store that dropped it would make classifying a
record a local-only ability), the kernel-backed store (the mask rides in
`cube_args` to the same kernel path that classifies the boot record), and the
sealing wrapper. `cube_args` grows from 80 to 88 bytes, which the kernel's
size-first dispatch still tells apart from the other three blocks.
The kernel now folds into the addressed format, so userspace has to know it or a kernel-written
store is a store userspace cannot check — a fork of the format rather than a version of it. The raw
reader gained it: `parse` accepts v3, and `iter_records` walks the space table and the index,
handing out the same zero-copy records in the same (space, key) order as before.
The proof that this is one format and not two is a fixture: 700 bytes lifted out of a store the
kernel itself folded, committed at `crates/cube-store-raw/tests/fixtures/kernel-v3.img`, and an
integration test that reads it whole — ten records, 110 value bytes, three spaces in order, the
rehearsal's own values among them. A hand-built image cannot catch a bug that the builder and the
reader share, and that is exactly what the unit test in this crate did: it built its index with
8-byte lengths, so it never noticed that the kernel wrote 4-byte ones. The fixture did.
The digest's `bytes` figure was the layout's size, so the same store digested differently depending
on which format held it; it is now the records' logical size and the two agree. The gates that
compare a kernel image with a userspace one compare content and allow the layout to differ, saying
so where they do it.
Also here: the guest's `bench` verb and `verify-enum-cost.sh`, which is what turned all of this from
an argument into numbers — two stores forty times apart, four questions asked of both. It failed
when the walk was all there was, and it passes now.
Three things the rehearsal was getting wrong, all of them the kind that only
show up when the rehearsal is contained:
* the gate stopped the LIVE store. Its paths were overridden to temporary
files, but the unit it stopped was the real one, so a rehearsal took the
machine's front-end down for the length of a run. KILL_HOLDER_UNIT names the
unit to stop and the gate now asserts the real service did not move.
* the password was a hardcoded default, so the kill switch was "armed" with a
password that was in the transcript. Both rehearsal scripts mint a random
password per run, and no hash file exists unless someone arms one.
* the show: fireworks.rs draws straight onto the framebuffer in the guest that
is destroying its own disk — no browser, no JavaScript engine, native Rust
with mmap and a firework that keeps going as long as the destruction does.
With no framebuffer (serial-only, or -display none) it quietly does nothing,
because it is a witness to the destruction, not part of it.
install-box.sh is the two-pass box install this ran through: kernel first, then
the initramfs and the units, each pass verified before the next.
The interface gap this closes was recorded, not papered over: `cube(2)` had no
ENUM/SPACES, so `list`, `spaces` and `range` through the front-end answered with
an honest error. Now the client walks the kernel's store — entries(), spaces(),
range() — and the front-end serves all three from it.
Two gates, because the two halves fail differently:
* kernel/verify-enum.sh builds a 16 MiB device holding 3006 records over three
spaces — one overwritten, one deleted, one empty value, 3000 bulk — boots the
kernel, walks it through cube(2) and diffs that listing against the one
userspace produces from the same device. Identical, line for line.
* kernel/verify-frontend.sh asks through the socket: client -> front-end ->
cube(2). Its store now holds 800 records in `root`, more than one 64 KiB
batch, so `list` has to follow the cursor to reach root:799,0,0; `spaces`
and `range` are asked for too, and the store the front-end leaves is
digest-identical to the one those commands leave in userspace.
Building them found the bug worth keeping. A batch holds as many whole records
as fit, so it is FULL only when a record lands on the boundary — most batches
come back short. The client read "the buffer was not filled" as "the space is
exhausted" and so did the guest, and a 3006-record listing came back as 747
records looking exactly like a finished walk. The end of a walk is the cursor
alone. drain_batches() is that loop extracted so the rule is testable without a
kernel, with a test that fails on the old shape.
Three more seams the gates exposed and this fixes:
* glibc's syscall() reports the kernel's -ENOENT as -1 with errno set, so the
guest's end-of-walk test compared against -2 and never matched, and every
error print said "Operation not permitted" whatever the errno was.
* kernel/testbed/cube-client.c read the reply once, into 4 KiB. A listing is
tens of KiB, so the client would have reported a truncated answer as the
front-end's own; it reads to the newline now.
* an EINVAL from the walk block on a kernel that knows only the coordinate
block is now named for what it is, because "Invalid argument" from a call
whose argument this code built is not something a caller can act on.
kernel/testbed/cube.c gained the `walk` verb, which is how the gate reads the
kernel's own listing out of the guest.
glibc's syscall() returns -1 for *any* negative kernel return and stores the real
errno separately, so `-ret` on the error path is always 1 — EPERM. KernelStore::call
was computing `from_raw_os_error(-ret)`, so an absent record (the kernel's -ENOENT)
came back as "Operation not permitted".
That is why it hid this long: it only fires on the error path, and every early gate
read records that existed. It is also what the sealer's EPERM was hours ago — not
Morton codes, not a stray probe write; this. The fix reads std::io::Error::last_os_error()
when the syscall returns -1, which is what libc left in errno.
Proven by the encryption gate, which exercises exactly the missing-record path
(keys reads an absent keystore cell first): PASS, with a sealed envelope on the
device and no plaintext.
cube-frontend now wraps its kernel store in cube-store-seal's SealedStore. When
a key is available — `--key FILE`, or what systemd unsealed into
$CREDENTIALS_DIRECTORY — `cell put` seals and `cell get` opens, so the machine's
clients keep speaking the same language and the store holds ciphertext. With no
key, the wrapper passes envelopes through untouched, and `stats` reports
store=kernel+sealed=true/false so the state is visible rather than inferred.
The front-end's exact code path is now unit-tested without a kernel: the command
language over a sealed store round-trips — `cell put` leaves an envelope at rest,
`cell get` returns the plaintext — which is the claim the whole cutover depends
on, held by a test that runs in milliseconds instead of a boot.
SealedStore gained `inner()`, the raw ciphertext view, for that inspection and
for a process that deliberately wants what is at rest.
verify-sealed-read.sh — three runs over one sealed store device, none of which
writes:
run 1 the key is there → every record opens, to the bytes it held before
run 2 the key is gone → not one record opens; the store is ciphertext
run 3 a different key → not one record opens either, which is the check
that makes run 2 mean something: "unreadable" must
not be the same as "my reader is broken"
The store is a slice of the machine's own records, sealed offline with a
throwaway key (`cube-image seal`), so the shapes and sizes are real and the key
protecting them exists only for the run. The guest does the reads through
`cube(2)` with cube-seal's new `--open-check` mode, which takes no key for the
"after the kill" case — a read-only pass that still reaches the store and
reports that nothing opens.
Two things the first run of this gate caught, both fixed:
* the probe wrote. The sealer's put/get/delete probe was running in open-check
mode too, and one put plus one delete appended to the log and rewrote the
control block — so a run that was supposed to be read-only moved the store's
digest. The read path was never the writer; the probe was. It is now gated to
the sealing path, and the gate asserts the device is byte-identical across all
three runs to keep it that way.
* "no key" has to be an answer, not a refusal. With no key file the sealer used
to exit before reaching the store; it now proceeds keyless and counts every
sealed record as unopenable, which is the honest "after the kill" outcome.
Also: rehearse-erase.sh now measures reformatting in both its senses — mkfs with
and without its default discard, and `cube-image format` on a copy of this
machine's store (8192 of 8192 records survived a no-discard reformat; 200 of 200
of the store's own records survived its format call) — and testbed/store-survival.py
plus slice-store.py are the samplers and counters behind that.
The contained version of the thing asked for: the drive is a 64 MiB file this
gate creates, the machine is a VM, and the destruction happens inside the guest
against its own virtual disk. The host's disk is never a candidate — the guest
gets exactly one disk, and the pristine copy is kept so the aftermath can be
compared.
Two runs, because one of them proves nothing on its own:
run 1 kill password → accepted, blkdiscard ok, 67108864 bytes overwritten,
powered off. The image afterwards: first16 all zeros,
record text gone, 65536 KiB materialised.
run 2 wrong password → denied three times, disk byte-identical to pristine.
The password is not on the disk: the initramfs carries a salt and a hash, and
the guest compares a hash. (A real deployment should use the slow KDF the
store's recovery copy already uses; one SHA-256 is right for a rehearsal and
wrong for a secret a human typed.) It appears in the run-1 log only because it
was typed into that console.
Two bugs, both of which looked like "the guest ignored the password":
* `-nographic` muxes stdio with the QEMU *monitor*, so a line written to
stdin is eaten by the monitor and never reaches the guest's serial port.
`-display none -serial stdio -monitor none` gives the guest the port, and
input is sent after a delay and repeated, because bytes that arrive before
the guest brings the port up are dropped.
* the guest's initramfs had no /dev, so `/dev/vda` did not exist and the
destroy step reported "No such file or directory" for a disk that was right
there. It now mounts devtmpfs, sysfs and proc first.
Also in this commit: cube-seal takes coordinates in the language's own spelling
and parses them with the language's parser, because reading a record's key bytes
out of an image and calling them x, y, z is wrong — they are Morton codes, and
that mistake made the conversion rehearsal ask the kernel for coordinates that
never existed.
Two gates, both of which boot the thing rather than test its parts:
verify-frontend.sh serves the command language over a Unix socket from
/cube-frontend, which answers from the kernel's store through cube(2), and
compares what that leaves behind with what the same commands leave in
userspace. Building it found a real bug: the read took its length from the
syscall's *return* value, where the ABI reports it in the structure — so
every get returned an empty record and no error at all. Fixed in cube-kernel,
with the rule split into a function a test can hold (read_back).
verify-efi-boot.sh rehearses the boot loader half of a box boot, which
nothing else could: OVMF → GRUB in the ESP → ext4 /boot on the same NVMe disk
(GRUB reads disks through the firmware's block I/O, which is how a Surface
boots Linux at all) → kernel → initramfs → root. It boots the entry that
install-root.sh writes, so the installer is tested too, and it says why the
release comes from the linked kernel rather than from `make kernelrelease`
(a dirty tree appends '+', and that answer files modules where the running
kernel will never look — kernel-release.sh).
verify-box-preflight.sh is the read-only check that installing this on the
workhorse cannot cost it the way back: a different release, a module set that
covers what the running kernel loads (iwlwifi first — there is no wired NIC),
an entry in custom.cfg that the default does not mention, and a one-shot
grub-reboot to reach it once.
Also: the nvme rehearsal init now resolves root=UUID= as well as
root=PARTUUID= (the box uses UUID=), tested against a disk image before the
boot that depends on it; the installer refuses to overwrite a custom.cfg it
did not write; a test in cube-store that only passed when arithmetic wrapped
now computes the inclusive top instead; and every gate's QEMU gets
`timeout -k 5`, because thirteen orphans from earlier runs were still burning
cores hours later.
luulu