store: a get on the first slice of a chain returns the whole value
The arrangement vocabulary's second half — the producer and the userspace reader have existed since 0244-ish, and this is the kernel joining them for a caller who does not know it holds a chain. Bounded by the same 65,536 slices the userspace reader uses and by MAX_CHAIN_BYTES (CUBE_MAX_VALUE, 16 MiB: a longer chain could not be handed over in one call, so reading on would be work whose answer nobody can receive). A hole, a slice that claims to continue and does not, and a second beginning are each -EIO rather than a guessed end. A joined read reports START|END, because the flags describe the bytes handed over rather than the slice they came from — which is also why the userspace reader needed no change: it already stops at END_RECORD. It does NOT join a sealed chain, and that is a fact about the format: a sealed slice is an envelope with its own header and nonce, so N of them concatenated are not one openable value, and joining them belongs to whoever holds the key. When it cannot join (sealed, or a packed image with no index) it returns the slice with the record's own flags — CONTINUATION without END_RECORD says "this is a slice, not the whole value" — so the caller is told rather than misled. Reading those bits at all was safe because nothing uses them, and that was measured before the code was written: over the live store's 69,749 records, flag-scan finds 1 record in 0x0080, 137 in 0x0800, and zero in every one of 0x0001, 0x0002, 0x0003, 0x0004, 0x0008, 0x0010, 0x0020, 0x0040, 0x0100, 0x0200. Gated by kernel/verify-chain.sh: PASS, seven checks.
This commit is contained in:
surface-camera-build
@@ -3297,6 +3297,157 @@ unsafe fn v3_get(
|
||||
copy_out(value.as_slice(), buf, len)
|
||||
}
|
||||
|
||||
// ── the arrangement vocabulary: one value, several records ─────────────────────────────────────
|
||||
//
|
||||
// Three bits say where a record sits in a chain of records that together hold one value:
|
||||
// `START_RECORD` on the first slice, `CONTINUATION` on every slice that is followed by another,
|
||||
// `END_RECORD` on the last — so a single-slice value carries `START|END`, which is what a
|
||||
// self-contained record already is. `cube_core::WordFlags` allocates them,
|
||||
// `DESIGN-flag-vocabularies.md` §7 is the design, and `crates/cube-store/src/chain.rs` is the
|
||||
// producer and the reference reader.
|
||||
//
|
||||
// **The bits were allocated and unread, so "is anything already using them?" decided whether
|
||||
// reading them here is safe — and it was measured rather than assumed.** Over this machine's live
|
||||
// store, `cube-image flag-scan` finds 1 record in class 0x0080 and 137 in 0x0800, and nothing at all
|
||||
// in 0x0001, 0x0002, 0x0003, 0x0004, 0x0008, 0x0010, 0x0020, 0x0040, 0x0100 or 0x0200. So no read
|
||||
// that works today can change meaning because of the code below.
|
||||
const START_RECORD: u16 = 1 << 2;
|
||||
const END_RECORD: u16 = 1 << 3;
|
||||
const CONTINUATION: u16 = 1 << 5;
|
||||
|
||||
/// `cube-store-seal`'s flag: this record's value is an *envelope*, not plaintext.
|
||||
///
|
||||
/// It is what stops the kernel joining a chain, and the reason is in the format rather than in
|
||||
/// policy. A sealed slice carries its own header and its own nonce, so N of them concatenated are
|
||||
/// not one value anybody can open; joining those belongs to whoever holds the key, one envelope at
|
||||
/// a time, which is what `chain.rs` does.
|
||||
const SEALED_FLAG: u16 = 0x0800;
|
||||
|
||||
/// The most slices one chain may be joined from — the userspace reader's own ceiling, because two
|
||||
/// readers of one format that disagreed about the bound would disagree about what is valid.
|
||||
const MAX_CHAIN_SLICES: usize = 1 << 16;
|
||||
|
||||
/// The most bytes a joined chain may be: `CUBE_MAX_VALUE`, the largest value a caller can ask for in
|
||||
/// one call. A longer chain could not be handed over in one piece, so reading on to find its end
|
||||
/// would be work whose answer nobody can receive.
|
||||
const MAX_CHAIN_BYTES: usize = 16 * 1024 * 1024;
|
||||
|
||||
/// Whether these flags say the record *begins* a chain that continues past it.
|
||||
fn chain_starts(flags: u16) -> bool {
|
||||
flags & START_RECORD != 0 && flags & END_RECORD == 0
|
||||
}
|
||||
|
||||
/// Read the chain starting at `(x, y, z)` and hand the caller the joined value.
|
||||
///
|
||||
/// The first slice has already been read — this is only called once a read found a chain — so it is
|
||||
/// not read again to find out, and its length and flags come in as arguments.
|
||||
///
|
||||
/// The buffer rule is a single record's rule applied to the whole value: if the joined bytes do not
|
||||
/// fit, nothing is copied and the length is returned, so the caller sizes up and asks again. The log
|
||||
/// is consulted per slice, so a chain that is written but not yet folded joins exactly like one that
|
||||
/// has been folded.
|
||||
///
|
||||
/// # Safety
|
||||
/// `space` must point to 32 readable bytes; `buf` to `len` writable bytes; `out_flags` must be
|
||||
/// writable.
|
||||
unsafe fn v3_get_chain(
|
||||
space: *const u8,
|
||||
x: u64,
|
||||
y: u64,
|
||||
z: u64,
|
||||
first_len: usize,
|
||||
first_flags: u16,
|
||||
buf: *mut u8,
|
||||
len: usize,
|
||||
out_flags: *mut u16,
|
||||
) -> isize {
|
||||
// Pass one: how long is the value, and does the chain hold together at all? The sizes come from
|
||||
// the index — and the log — so this pass reads no value bytes, which is what keeps the
|
||||
// size-then-read pattern as cheap for a chain as it is for a record.
|
||||
let mut total = first_len;
|
||||
let mut flags = first_flags;
|
||||
let mut zz = z;
|
||||
let mut slices = 1usize;
|
||||
while flags & END_RECORD == 0 {
|
||||
if flags & CONTINUATION == 0 {
|
||||
// It neither continues nor ends. Where the value stops is not knowable, and inventing an
|
||||
// end would hand the caller bytes nobody wrote.
|
||||
return -5; // -EIO
|
||||
}
|
||||
if slices >= MAX_CHAIN_SLICES {
|
||||
// A chain with no end is a read with no end: the ceiling is the answer.
|
||||
return -5; // -EIO
|
||||
}
|
||||
zz += 1;
|
||||
slices += 1;
|
||||
let image = match Addressed::open() {
|
||||
Ok(Some(image)) => image,
|
||||
Ok(None) => return -95, // -EOPNOTSUPP
|
||||
Err(e) => return -(e.to_errno() as isize),
|
||||
};
|
||||
// The next slice is arithmetic, not a pointer: the same space at the same x and y, one `z`
|
||||
// further along. Nothing is stored to find it, so nothing can point wrongly.
|
||||
let (sp, key) = unsafe { coord_key(space, x, y, zz) };
|
||||
let mut slice_flags = 0u16;
|
||||
let n = unsafe { v3_get(image, sp, key, core::ptr::null_mut(), 0, &mut slice_flags) };
|
||||
if n == -2 {
|
||||
// A hole. A chain with a missing slice is not a shorter value, it is a corrupt one, and
|
||||
// returning the prefix would hand the caller something that looks smaller rather than
|
||||
// broken.
|
||||
return -5; // -EIO
|
||||
}
|
||||
if n < 0 {
|
||||
return n;
|
||||
}
|
||||
if slice_flags & (CONTINUATION | END_RECORD) == 0 || slice_flags & START_RECORD != 0 {
|
||||
// A record that belongs to no chain, or a second beginning: either way the chain this
|
||||
// read started from is not the chain it claimed to be.
|
||||
return -5; // -EIO
|
||||
}
|
||||
total = match total.checked_add(n as usize) {
|
||||
Some(t) if t <= MAX_CHAIN_BYTES => t,
|
||||
_ => return -7, // -E2BIG
|
||||
};
|
||||
flags = slice_flags;
|
||||
}
|
||||
// The flags describe the bytes the caller is handed, not the slice they came from: this is now a
|
||||
// whole self-contained value, so it answers `START|END`. That is also what lets the userspace
|
||||
// reader work unchanged against a joining store — it stops at `END_RECORD`, which is what it was
|
||||
// already written to do.
|
||||
unsafe { *out_flags = (first_flags & !CONTINUATION) | END_RECORD };
|
||||
if total > len {
|
||||
return total as isize;
|
||||
}
|
||||
// Pass two: copy, in z order. Both passes walk the same coordinates, so each slice is copied into
|
||||
// the place the first pass measured for it.
|
||||
let mut off = 0usize;
|
||||
let mut zz = z;
|
||||
for _ in 0..slices {
|
||||
let image = match Addressed::open() {
|
||||
Ok(Some(image)) => image,
|
||||
Ok(None) => return -95, // -EOPNOTSUPP
|
||||
Err(e) => return -(e.to_errno() as isize),
|
||||
};
|
||||
let (sp, key) = unsafe { coord_key(space, x, y, zz) };
|
||||
let mut slice_flags = 0u16;
|
||||
let n = unsafe {
|
||||
v3_get(image, sp, key, unsafe { buf.add(off) }, len - off, &mut slice_flags)
|
||||
};
|
||||
if n < 0 {
|
||||
return n;
|
||||
}
|
||||
off += n as usize;
|
||||
zz += 1;
|
||||
}
|
||||
// `off` and `total` come from the same coordinates read twice, so they must agree. A
|
||||
// disagreement means the store moved under the read, and a length nobody measured is the wrong
|
||||
// thing to return quietly.
|
||||
if off != total {
|
||||
return -5; // -EIO
|
||||
}
|
||||
total as isize
|
||||
}
|
||||
|
||||
/// `CUBE_OP_SPACES` against a v3 image: the space table, and whatever only the log writes.
|
||||
///
|
||||
/// # Safety
|
||||
@@ -4068,6 +4219,11 @@ unsafe fn v3_flag_scan_every(
|
||||
/// only if it says nothing about the coordinate does the image answer. Nothing is copied out of
|
||||
/// either: the value the caller gets is a slice of bytes already in memory.
|
||||
///
|
||||
/// A value written as a *chain* — several records at consecutive `z`, marked with the arrangement
|
||||
/// bits — comes back **joined**: a `get` on the first slice returns every slice's bytes in `z` order,
|
||||
/// because the kernel is the half that can walk a chain without the caller reading each slice itself.
|
||||
/// What joins and what deliberately does not is stated at the arrangement vocabulary below.
|
||||
///
|
||||
/// # Safety
|
||||
/// `space` must point to 32 readable bytes; `buf` to `len` writable bytes.
|
||||
#[unsafe(no_mangle)]
|
||||
@@ -4088,7 +4244,28 @@ pub unsafe extern "C" fn cubelinux_kernel_get(
|
||||
// A v3 image is addressed, so a read is a binary search and a few bytes. v1 and v2 images are
|
||||
// packed lists, which the walking reader below handles.
|
||||
match Addressed::open() {
|
||||
Ok(Some(image)) => return unsafe { v3_get(image, sp, key, buf, len, out_flags) },
|
||||
Ok(Some(image)) => {
|
||||
let mut flags = 0u16;
|
||||
let got = unsafe { v3_get(image, sp, key, buf, len, &mut flags) };
|
||||
// A record that begins a chain and does not end it holds one slice of a larger value, and
|
||||
// the kernel is the half that can join it: a caller would otherwise read every slice
|
||||
// itself just to learn how long the value is.
|
||||
//
|
||||
// Two cases do not join here, and both are said in the answer rather than approximated. A
|
||||
// *sealed* slice is an envelope with its own header and nonce, so joining those is not
|
||||
// joining a value — that belongs to whoever holds the key. A packed image has no index to
|
||||
// address slices by. In both, the caller gets the slice with the record's own flags, and
|
||||
// `CONTINUATION` without `END_RECORD` says plainly "this is a slice, not the whole
|
||||
// value": told, not misled.
|
||||
if got >= 0 && chain_starts(flags) && (flags & SEALED_FLAG) == 0 {
|
||||
return unsafe {
|
||||
v3_get_chain(space, x, y, z, got as usize, flags, buf, len, out_flags)
|
||||
};
|
||||
}
|
||||
// SAFETY: the caller guarantees a writable u16.
|
||||
unsafe { *out_flags = flags };
|
||||
return got;
|
||||
}
|
||||
Ok(None) => {}
|
||||
Err(e) => return -(e.to_errno() as isize),
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user