cubelinux/cubelinux-kernel
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

cubelinux: the store's format is one file, and the kernel and userspace both include it

Browse Source 
Two build systems cannot share a crate: the kernel's Rust build compiles what is in its own module
tree, and a cargo crate is not that. So they share a *file* — `drivers/cube/cube_format.rs`, which
the driver declares with `mod` and `crates/cube-format` includes by path. There is no copy to drift
from, which is the only arrangement that cannot go stale. What happened when v3 landed and userspace
did not is the argument: an hour of `unsupported-version`, one gate red, and two implementations of
one format each believing itself.

The file holds what both sides must agree about, and nothing else — pure functions over slices, no
allocation, no I/O, no logging:

  * the constants every reader and writer derives its arithmetic from,
  * the header, in all three versions, with refusal rather than guessing: a reader that invents an
    extent can read somebody else's bytes,
  * the v3 space table and index, and the three equalities a reader computes its addresses from,
  * the packed record and the log entry, the two framings a fold and a walk have to parse alike,
  * CRC-32, FNV-1a, and the digest line both sides print.

The driver's copies of the constants are now aliases of the shared ones, and the two functions that
*validate* a header — `parse_header` and the v3 geometry — delegate to it, because validation is
where a second description gets believed. The readers stay where they are: this driver streams from
a file with its own buffers while userspace already holds the whole image, and that difference is
real rather than duplicated.

Nothing changed in behaviour, which is the point: verify-enum.sh, verify-enum-cost.sh,
verify-frontend.sh, verify-kernel-checkpoint.sh and verify-kernel-append.sh all pass, and the shared
crate's own tests include a kernel-written image — ten records, three spaces, 110 value bytes — read
through its arithmetic.
This commit is contained in:
surface-camera-build
2026-09-21 03:18:52 -04:00
parent 1db3abce93
commit d569df710d
2 changed files with 516 additions and 88 deletions
Download Patch File Download Diff File Expand all files Collapse all files
drivers/cube/cubelinux_store.rs
+54 -88
View File
@@ -42,6 +42,17 @@
use core::fmt::{self, Write};
// The store's format, in one file, shared with userspace.
//
// `crates/cube-format` includes this same path, so the layout, its constants and the arithmetic a
// reader derives from them have one description rather than one per build system. What is not here
// is I/O or allocation: this driver reads with its own buffers, and userspace already has the whole
// image. The constants below are aliases of the shared ones, and the two functions that *validate*
// a header delegate to it — validation is where a second description would be read as truth.
#[path = "cube_format.rs"]
mod cube_format;
use kernel::{
alloc::AllocError,
bindings, c_str,
@@ -64,11 +75,10 @@ module! {
/// The layout, restated here because the kernel cannot depend on the userspace crates.
/// `crates/cube-store-raw` is the source of truth; the digest comparison is what keeps
/// this copy honest.
const MAGIC: &[u8; 4] = b"CUBE";
const MAGIC: &[u8; 4] = cube_format::MAGIC;
/// The original format: magic, version, curve, then records until zero padding.
const VERSION_V1: u8 = 1;
const VERSION_V1: u8 = cube_format::VERSION_V1;
/// The packed format: the same, plus the image's byte extent and record count.
const VERSION: u8 = 2;
/// The addressed format: the same, plus a fixed-size index, a space table, and packed values.
///
/// v2 is a packed list — `space | key | len | value`, repeated — so record N's offset is the sum of
@@ -89,19 +99,19 @@ const VERSION: u8 = 2;
/// addresses (`index_off + i * 40`), a listing starts at its space's first index entry and streams,
/// asking which spaces exist is the space table, and a spatial range is a contiguous run of index
/// entries. An index entry is 40 bytes against v2's 64-byte frame, so the image also gets smaller.
const VERSION_V3: u8 = 3;
const HEADER_LEN_V1: usize = 6;
const HEADER_LEN_V2: usize = 6 + 8 + 8;
const VERSION_V3: u8 = cube_format::VERSION_V3;
const HEADER_LEN_V1: usize = cube_format::HEADER_LEN_V1;
const HEADER_LEN_V2: usize = cube_format::HEADER_LEN_V2;
/// magic(4) version(1) curve(1) image_bytes(8) record_count(8) space_count(8) index_off(8)
/// values_off(8).
const HEADER_LEN_V3: usize = 4 + 1 + 1 + 8 + 8 + 8 + 8 + 8;
const HEADER_LEN_V3: usize = cube_format::HEADER_LEN_V3;
/// `key(24) | value_off(8) | value_len(8)`.
const INDEX_ENTRY: usize = RAW_KEY_LEN + 8 + 8;
const INDEX_ENTRY: usize = cube_format::INDEX_ENTRY;
/// `space(32) | first index(8) | records(8)`.
const SPACE_ENTRY: usize = SPACE_ID_LEN + 8 + 8;
const SPACE_ID_LEN: usize = 32;
const RAW_KEY_LEN: usize = 24;
const RECORD_FIXED: usize = SPACE_ID_LEN + RAW_KEY_LEN + 8;
const SPACE_ENTRY: usize = cube_format::SPACE_ENTRY;
const SPACE_ID_LEN: usize = cube_format::SPACE_ID_LEN;
const RAW_KEY_LEN: usize = cube_format::RAW_KEY_LEN;
const RECORD_FIXED: usize = cube_format::RECORD_FIXED;
// The device the store lives on, as the `cube_store=` boot parameter resolved it.
//
@@ -144,12 +154,12 @@ const MAX_BYTES: usize = 128 * 1024 * 1024;
/// The log's own magic, distinct from the image's so a reader that opens the wrong one
/// cannot mistake it for the other.
const WAL_MAGIC: [u8; 4] = *b"CUBW";
const WAL_VERSION: u8 = 1;
const WAL_MAGIC: [u8; 4] = *cube_format::WAL_MAGIC;
const WAL_VERSION: u8 = cube_format::WAL_VERSION;
/// Log header: magic(4) + version(1) + curve tag(1).
const WAL_HEADER_LEN: usize = 6;
const WAL_HEADER_LEN: usize = cube_format::WAL_HEADER_LEN;
/// Log entry, before the value: op(1) + crc32(4) + space(32) + key(24) + len(4).
const ENTRY_FIXED: usize = 1 + 4 + 32 + 24 + 4;
const ENTRY_FIXED: usize = cube_format::ENTRY_FIXED;
/// The log begins at the first 4 KiB boundary at or after the image. Fixed by geometry so
/// no superblock is needed to find it, and stated by the v2 header's extent.
///
@@ -178,7 +188,7 @@ const OP_PUT: u8 = 1;
const OP_SYNC: u8 = 3;
/// FNV-1a offset basis.
const FNV_OFFSET: u64 = 0xcbf2_9ce4_8422_2325;
const FNV_OFFSET: u64 = cube_format::FNV_OFFSET;
/// A line of output, built in place. No allocation: the read path formats into a fixed
/// buffer, so it cannot fail for want of memory while holding a file open.
@@ -290,51 +300,21 @@ fn read_image(image: &mut KVVec<u8>) -> Result<()> {
/// value runs past the buffer is a truncated record and an error, and an all-zero frame is
/// *end of records* only when every remaining byte is also zero — otherwise a real record
/// at the origin, followed by padding, would be read as an empty one.
/// What a header says, in both formats.
struct Header {
version: u8,
curve: u8,
image_bytes: Option<u64>,
record_count: Option<u64>,
}
// What a header says, in every format, is the shared file's: `version`, `curve`, the image's
// extent, and how many records follow. Same fields, one description.
use cube_format::Header;
/// Read the header, or say what is wrong with it.
///
/// The shared file decides what a header means — that is the point of it being shared — and this
/// only translates a refusal into the words this driver logs.
fn parse_header(image: &[u8]) -> Result<Header, &'static str> {
if image.len() < 5 || &image[0..4] != MAGIC {
return Err("not-a-store");
}
let version = image[4];
let curve = image[5];
match version {
VERSION_V1 => Ok(Header {
version,
curve,
image_bytes: None,
record_count: None,
}),
VERSION => {
if image.len() < HEADER_LEN_V2 {
return Err("short-v2-header");
}
let mut word = [0u8; 8];
word.copy_from_slice(&image[6..14]);
let image_bytes = u64::from_le_bytes(word);
word.copy_from_slice(&image[14..22]);
let record_count = u64::from_le_bytes(word);
// A header that does not cover itself is corrupt, and guessing at the extent
// of an image means possibly reading somebody else's bytes.
if (image_bytes as usize) < HEADER_LEN_V2 {
return Err("bad-extent");
}
Ok(Header {
version,
curve,
image_bytes: Some(image_bytes),
record_count: Some(record_count),
})
}
_ => Err("unsupported-version"),
}
cube_format::parse_header(image).map_err(|bad| match bad {
cube_format::Bad::Magic => "not-a-store",
cube_format::Bad::Version(_) => "unsupported-version",
cube_format::Bad::Extent => "bad-extent",
cube_format::Bad::Tables => "bad-tables",
})
}
/// The v3 header: what a reader needs to address the image by arithmetic.
@@ -359,35 +339,21 @@ impl HeaderV3 {
}
fn decode(image: &[u8]) -> Result<Self, &'static str> {
if image.len() < HEADER_LEN_V3 || &image[0..4] != MAGIC || image[4] != VERSION_V3 {
return Err("not-a-v3-header");
}
let word = |at: usize| -> u64 {
let mut w = [0u8; 8];
w.copy_from_slice(&image[at..at + 8]);
u64::from_le_bytes(w)
};
let header = HeaderV3 {
image_bytes: word(6),
record_count: word(14),
space_count: word(22),
index_off: word(30),
values_off: word(38),
};
// Every extent inside the image and in order, or a reader would take somebody else's bytes
// for a record — the same rule the v2 header is held to.
if header.image_bytes < HEADER_LEN_V3 as u64 {
return Err("bad-extent");
}
let table_end = HEADER_LEN_V3 as u64 + header.space_count * SPACE_ENTRY as u64;
let index_end = header.index_off + header.record_count * INDEX_ENTRY as u64;
if header.index_off != table_end || header.values_off != index_end {
return Err("bad-tables");
}
if header.values_off > header.image_bytes {
return Err("tables-past-the-image");
}
Ok(header)
// The shared file owns this: the offsets a reader computes (`index_off + i * INDEX_ENTRY`)
// are only places if the index really starts there and really is that wide, and that is a
// statement about the format rather than about this driver.
let geometry = cube_format::V3::decode(image).map_err(|bad| match bad {
cube_format::Bad::Extent => "bad-extent",
_ => "bad-tables",
})?;
let header = parse_header(image)?;
Ok(HeaderV3 {
image_bytes: header.image_bytes.unwrap_or(0),
record_count: geometry.record_count,
space_count: geometry.space_count,
index_off: geometry.index_off,
values_off: geometry.values_off,
})
}
}