93e2c8c39b7bf73f9d089ade5bab2a2d629c1844
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 CUBELinux build
|
93e2c8c39b |
CUBELinux.0.3: the kernel writes, and is compared while writing
0.2 read the store. This appends to it: a mutation through /dev/cubelinux is written to the log and fsynced *before* the write is accepted, which is the kernel's version of the contract cube_duratest.py proves for the daemon. - the mutation comes in as the argument block the coordinate interface will pass: op(1) | space(32) | x(8) | y(8) | z(8) | len(4) | value[len]. The kernel Morton-encodes the point itself — a key written here has to be the key a userspace reader decodes, and that is a thing the gate would catch if it were merely similar; - the append lands after the log's valid prefix, so a torn tail is overwritten rather than appended to, exactly as the userspace log does it; - a log region that has never been written is zeros, not a log: the header is written first, the way the userspace log creates its file; - the entry's CRC covers space, key, length and value, computed the same way. The write is the byte plane — bytes at a coordinate, no header written beside them — because that is what a differential comparison against userspace's `cell put` can be exact about. The header tier sits above this. Gate (kernel/verify-kernel-append.sh), four mutations including an empty value and a record in a second space: reference : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 kernel : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 folded : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 survived : bytes=500 records=6 value_bytes=94 fnv1a64=6b679d39597a62b3 The third line is the one that matters: userspace *reads the log the kernel wrote*, folds it, and lands on the same store. Without it the first two would only show the kernel agreeing with itself. The fourth is a SIGKILL of the VM with no shutdown and therefore no flush for us. Not yet: the kernel folds nothing itself, so it still depends on a userspace checkpoint to reclaim its log. That is the next step, and it has its own gate. |
||
|
CUBELinux build
|
63b0a2f26e |
cubelinux: replay the write-ahead log, and be compared while doing it
A store on a device is [image][log]: the image is a checkpoint, the log is the
mutations since, in the framing cube-store/src/wal.rs writes (op | crc32 | space
| key | len | value, prefix-trusting recovery). The kernel now reads both and
reports the store they *describe* — the image with the log applied, in the order
a checkpoint would write it.
- the log's offset is fixed by geometry (the first 4 KiB boundary at or after the
image's declared extent), so no superblock is needed to find it;
- recovery is prefix-trusting, exactly as userspace does it: the first entry that
is short, mis-framed or fails its CRC ends the log;
- merging is by coordinate with the newest write winning, which is what makes an
overwrite an overwrite and a delete a delete;
- the digest is taken over the surviving records in sorted order, and `bytes`
reports the size a checkpoint of this store would produce — so the number can
be compared with the image userspace actually writes, not merely eyeballed.
Gate (kernel/verify-log-replay.sh), on a pair carrying an overwrite, an
insertion and a deletion:
reference : bytes=377 records=4 value_bytes=99 fnv1a64=a38c1db74b00966f
kernel : bytes=377 records=4 value_bytes=99 fnv1a64=a38c1db74b00966f
log entries=6 applied over the image
It failed the first time, which is the point: the coalescing kept the earliest
write of a coordinate rather than the latest, resurrecting records the log had
deleted — six records where the fold produced four. A differential gate is what
turns that class of error into a red line instead of a corrupted store.
The image-only gates still pass unchanged (v1 curated, v1 snapshot 35,318
records, v2 store): with no log to apply the digest is the image's own order, so
v1 readers and every existing tool keep seeing exactly what they saw before.
|
||
|
CUBELinux build
|
b54ba34dc0 |
cubelinux: read format v2 as well as v1
The store's write path is decided (append a log, fold it into the image at a checkpoint; DESIGN-cubelinux-write-path.md), and that decision forces the log to sit right after the image on the device. v1 had no extent and no count — it walked records until it met zero padding — so a v1 reader would have walked straight into the log's header and parsed it as a record. v2 states the image's byte extent and its record count, and this teaches the kernel reader both: - v1: walk from a 6-byte header until the trailing zeros, as before. - v2: walk from a 22-byte header, stop exactly at the declared count, and never read past the declared extent. A zero frame inside the count is a record, not padding — which is the ambiguity v1 could not resolve. - a v2 header whose extent does not cover the header itself is refused rather than guessed at, and a count that is not met counts as an error instead of quietly returning a shorter list. Gate, unchanged in method: the kernel's digest of /dev/vda must equal cube-image's digest of the same bytes. Passing on all three: v1 curated 11 records fnv1a64=161113085b1573b2 v1 snapshot 35,318 records fnv1a64=5e20f98455387b08 v2 store 4 records fnv1a64=20ecadb5cdc9c994 |
||
|
CUBELinux build
|
cee3e554d9 |
CUBELinux.0.2: the kernel reads the CUBE store off a block device
The first CUBE code in the kernel, and deliberately only a reader: the write
authority has not moved yet, and PLAN-kernel-cubelinux.md records both that
decision and the hazard that makes the order matter — a kernel writing while a
userspace daemon still holds the same image loses one of the two writers' work,
silently. A reader cannot do that.
- drivers/cube/: a Rust module exposing /dev/cubelinux. Reading it reads the
pinned image from the block device through the kernel's own file layer
(filp_open + kernel_read — the path this kernel version binds for Rust, and
the reason no C helper was needed), parses the records, and returns one line:
digest curve=0 bytes=8400896 records=35318 value_bytes=6139148 fnv1a64=5e20f98455387b08 errors=0
The work happens on read, not at init, so there is no initcall ordering to get
wrong against the block driver that provides the device.
- The format is restated in the kernel (32-byte space, 24-byte key, 8-byte LE
length, value), including the two rules the userspace parser documents: a value
that runs past the buffer is a truncated record, and an all-zero frame ends the
records only when every remaining byte is zero — the rule that keeps a real
record at the origin from being read as padding.
- The digest is the point. A record count alone lets two different images agree;
folding the bytes in means the kernel and userspace are *compared* rather than
assumed to agree. `cube-image digest` prints the same line in the same field
order, and the QEMU gate fails if they differ by a byte.
Gate, on both images:
curated 11 records, 4,096 bytes, fnv1a64=161113085b1573b2 — match
snapshot 35,318 records, 8,400,896 bytes, fnv1a64=5e20f98455387b08 — match
The tree carries CONFIG_CUBELINUX_STORE=y on top of defconfig + RUST; a tree
without it boots and simply has no /dev/cubelinux.
|
||
|
CUBELinux build
|
e4ae67ea7f |
CUBELinux.0.1: name the release, and build with this box's rustc
The tree is Linux 6.19.3 with the changes this machine's toolchain needs, and nothing else. No CUBE code yet — this is the base the coordinate interface will be built on, so it starts from a known-good bootable kernel. Naming: - VERSION/PATCHLEVEL/SUBLEVEL stay 6.19.3 (visible in `make kernelversion`), while the release string setlocalversion composes is CUBELinux.0.1, so `uname -r` and /lib/modules report this product rather than a Linux point release. The SCM suffix still applies: a dirty tree says so. rustc compatibility (rustc 1.100.0-nightly, clang 19.1.7): - scripts/generate_rust_target.rs emitted `"rustc-abi": "x86-softfloat"`, which this rustc rejects; it is `softfloat` now. - rust/Makefile's cmd_rustc_library did not pass -Zunstable-options, so the custom target spec would not load at all. - the generated bindings declare `strlen` with the kernel target's `c_char` (u8, from -funsigned-char) while rustc expects `*const i8`; the newer suspicious_runtime_symbol_definitions lint fires on that and -D warnings makes it fatal. Scoped to bindings.o and uapi.o, not to handwritten code. - three `'static` bounds the abstractions now need (irq handlers). - str.rs imported alloc::flags::* which prelude::* already provides, and `#![feature(used_with_arg)]` is stale now that the feature is stable; both are unused-feature/unused-import errors under -D warnings. Gates: bzImage builds (14,697,472 bytes) with CONFIG_RUST=y, virtio-blk and a serial console built in. Boot test in QEMU is next. |