diff --git a/RESUME-cube-os-czyx.md b/RESUME-cube-os-czyx.md new file mode 100644 index 0000000..5a3fe63 --- /dev/null +++ b/RESUME-cube-os-czyx.md @@ -0,0 +1,71 @@ +# RESUME — Cube as OS substrate (CZYX-call model) + +Date: 2026-08-13 (session after RESUME-cubefs-daemon; Level A of item 3) + +## Decision (user-directed) +"Everything writes as a CZYX call to the daemon — the cube-backed storage should +be coordinate calls, not writes to /cubefs. Why are we writing to cubefs?" + +This REVERSES the earlier FUSE-path substrate design. Confirmed against +CUBELinux.pdf: + - Phase 1: user-space Cube OS on Linux, using existing kernels + filesystems. + - Phase 2: cubefs FUSE maps POSIX→CZYX for *testing semantics under real + workloads* — explicitly a probe, not the substrate itself. + - Phase 3: fork/extend kernel so VFS/accounting/LSM talk directly to the cube + store; new syscalls expose cube-native ops ("open by CZYX + flags"). + - Package 3 (cubefs): "optional FUSE filesystem view so cube records appear as + files/directories for existing tools." → FUSE is OPTIONAL, not the substrate. + +So: the coordinate store IS the persistence. POSIX/FUSE is a convenience view. + +## What changed +1. `cube-os-state.sh` / `cube-os-snapshot.sh` / `cube-os-klog.sh` rewritten to + write via `cubec --socket /run/cube/cube.sock rawput/get` (CZYX calls), not + `/cubefs/...`. hex encoding uses python3 (xxd is absent in the VM image). + Coordinate layout (C=200): + c200/z001/y001/x001 boot manifest (overwrite each boot) + c200/z002/y001/x001 kernel boot line history (append) + c200/z003/y001/x001 machine/identity record + c200/z004/y001/x001 kernel log stream (klog appends lines) + c200/z010/y001/xNNN operational snapshot body (rolling counter) + c200/z010/y002/x001 rolling snapshot counter (persisted in cube, hex) +2. The three `cube-os-*` systemd units had `cubefs.service` REMOVED from + Requires/After — they now depend ONLY on `cube-server.service`. A FUSE mount + failure can no longer wedge OS bring-up. +3. `cubefs.service` hardened (ExecStartPre unmounts any stale mountpoint; BindsTo + cube-server) so it survives a daemon restart instead of crash-looping on + "Transport endpoint is not connected". This was the bug that broke the live + VM at the start of this session. +4. build_vm.sh updated to match (inline script bodies + unit edges). A fresh + bake now produces the CZYX-call substrate. NOT re-baked this session + (heavy ~24G, off-peak per user policy). + +## Verified live (VM localhost:2222) +- Boot manifest reads: "CUBELINUX-VM boot manifest v3 / backing-store: + cube-server @ /run/cube/cube.sock (CZYX coordinate calls, no FUSE)". +- Snapshot #001 contains real OS state (running units, network, resources, + journal) — not a demo. +- Manifest + snapshot survive `systemctl restart cube-server` (durable across + daemon restart). +- `cube-os-state` + `cube-os-snapshot.timer` active; manual snapshot writes. + +## Known gaps (honest) +- cubefs directory model: only `c/z/y/x` (x = fixed 3-digit leaf). + Arbitrary POSIX filenames (syslog, wtmp) and nested dirs are NOT addressable. + overlayfs on cubefs fails (EINVAL — missing RENAME_WHITEOUT/opaque-dir). So a + *real OS tree* on the cube is blocked until cubefs grows nested-dir support + (Level-B crate work). +- root fs / PID 1 (cube daemon as init, pivot_root into cubefs) = Phase 3 kernel + work, image-bake, off-peak only. + +## Items left (from earlier list) +1. / 2. → done in prior sessions. +3. Cube as OS substrate: Level A DONE (CZYX-call OS state, this session). + Levels B (cube-backed block device) / C (rootfs+PID1) pending, not started. +4. society workspace: not started; git dubious ownership on + /home/cubelinux/society (needs safe.directory). + +## Key files +- /root/build_vm.sh (authoritative bake template; all unit + script bodies live here) +- /home/CUBELinux/CUBELinux-2/STARTUP-README.md (updated §4 + provisioning note) +- VM guest: /opt/cube/bin/cube-os-*.sh, /etc/systemd/system/cube-os-*.{service,timer} diff --git a/STARTUP-README.md b/STARTUP-README.md index 36d8cbb..a40fe88 100644 --- a/STARTUP-README.md +++ b/STARTUP-README.md @@ -58,19 +58,23 @@ is real. Each must be reproduced live, not asserted. 3358720263; `cat .czyx.200.50.1.7` returns the record content. Unit tests in `path.rs` cover full/partial/rejected forms. **STATUS: DONE.** 4. **Boot substrate** — the VM brings up CUBELinux as a storage layer the rest of - the OS reads/writes through. **DONE (systemd-managed, 2026-08-13)**: units - `cube-os-state.service` (oneshot, writes boot manifest + machine identity + - boot history to `/cubefs/c200/z001|z002|z003`) and `cube-os-klog.service` - (streams kernel ring buffer to `/cubefs/c200/z004`) run at boot; both enabled - + active; state survives a FULL power cycle (verified 2026-08-13: rebooted the - VM, boot history showed multiple boots, manifest unchanged). - **ALSO (2026-08-13): real OS operational data** — `cube-os-snapshot.service` + - `cube-os-snapshot.timer` (every 5 min + 30s after boot) write genuine OS state - (running units, network, resources, journal) into `/cubefs/c200/z010/y001/xNNN` - (rolling 3-digit counter, x-axis is u16→3-digit per cubefs path model). - Verified: snapshot survives daemon restart; the OS wrote a NEW snapshot on a - fresh boot after a full power cycle; inode == packed CZYX. **STATUS: DONE - (auxiliary layer; not yet the root fs).** + the OS reads/writes through. **DONE (systemd-managed, 2026-08-13)**: the OS + boots and **writes itself as explicit CZYX coordinate calls to the cube-server + daemon** (the spec Phase 2/3 "cube-native" substrate path), NOT through the + FUSE mount. Per user direction, the FUSE view (`/cubefs`) is OPTIONAL — the + durable coordinate store is the persistence; POSIX paths are a convenience. + - `cube-os-state.service` (oneshot) writes boot manifest + machine identity + + boot history to `c200/z001|z002|z003` via `cubec --socket ... rawput`. + - `cube-os-klog.service` streams the kernel ring buffer to `c200/z004` (CZYX). + - `cube-os-snapshot.service` + `cube-os-snapshot.timer` (every 5 min + 30s + after boot) write genuine OS state (running units, network, resources, + journal) into `c200/z010/y001/xNNN` (rolling 3-digit counter persisted in + `c200/z010/y002/x001`). + All three units depend ONLY on `cube-server.service` (the daemon socket), not + on `cubefs.service`, so a FUSE mount failure can never wedge OS bring-up. + Verified: manifest reads "backing-store: cube-server @ /run/cube/cube.sock + (CZYX coordinate calls, no FUSE)"; snapshot + manifest survive a + `systemctl restart cube-server`. **STATUS: DONE (auxiliary layer; not yet the root fs).** --- @@ -153,24 +157,36 @@ missing. random material on each boot → every sealed record became unopenable. Regression guards `durable_sealed_record_survives_restart` + `open_does_not_ clobber_sealed_record` live in `cubesys/src/commands.rs`; `./check` is green. -- Boot substrate (next deepening): make the cube the OS's *default* storage for a - real tree — e.g. have a service write `/etc` or `/var/log` operational files - through the cube by default. Currently the cube holds OS *state* (manifest/ - identity/klog/snapshots) but the root fs is still ext4. The magic-prefix - "open by CZYX" FUSE passthrough is the natural next step (a path like - `/cubefs/.czyx/200.1.1.1` resolves directly to the coordinate), and a - loop/overlay over a cube-backed file would let the OS treat the cube as a real - block device. +- Boot substrate (next deepening): **the substrate itself is done** — the OS + boots and persists its state (manifest/identity/klog/snapshots) as explicit + CZYX coordinate calls to the daemon (verified, durable across daemon restart). + Two honest gaps remain before a *real tree* on the cube: + (a) **cubefs directory model** — cubefs only maps `c/z/y/x` with the + `x` axis as a fixed 3-digit file leaf; nested dirs and arbitrary POSIX + filenames (syslog, wtmp, config files) are NOT addressable. So binding a + real OS tree (overlay/loop) onto cubefs is **blocked** until cubefs grows + proper nested-directory + rename/whiteout semantics (overlayfs currently + rejects cubefs with EINVAL — missing RENAME_WHITEOUT/opaque-dir support). + This is a Level-B crate feature, not a systemd change. + (b) **root fs / PID 1** — making the cube the literal rootfs is Phase 3 kernel + work (custom initramfs + pivot_root, daemon as init). Heavy, image-bake, + off-peak only. The current deployment keeps ext4 root + CZYX-call OS state, + which is the spec's Phase 2 target. + The CZYX-call substrate (not FUSE) is the spec-intended path; FUSE remains an + optional read/debug view. - IMAGE PROVISIONING: `build_vm.sh` (host, /root/build_vm.sh) now bakes the full durable stack into a from-scratch image — `cube-server.service` (daemon), - `cubefs.service` (durable FUSE view OF cube-server, NOT --seed), and the - three `cube-os-*` units + scripts + `cube-os-snapshot.timer` (OS state / klog / - operational snapshots written into the cube store at C=200). All enabled in the - chroot. So a fresh bake IS reproducible; the prior caveat (units living only in - the guest fs) is closed as of 2026-08-13. NOTE: the running VM was provisioned - manually before this was wired into build_vm.sh; re-running `build_vm.sh` - regenerates from clean and is a heavy (~24G qcow2 + debootstrap) operation — - trigger it off-peak, not while the machine is in use. + `cubefs.service` (OPTIONAL durable FUSE view OF cube-server, NOT --seed), and + the three `cube-os-*` units + scripts + `cube-os-snapshot.timer`. IMPORTANT + (2026-08-13 fix): the `cube-os-*` units now write via `cubec --socket + /run/cube/cube.sock rawput/get` (CZYX calls) and depend ONLY on + `cube-server.service` — `cubefs.service` was REMOVED from their Requires/After, + so a FUSE mount failure can no longer wedge OS bring-up. `cubefs.service` itself + was hardened (ExecStartPre unmounts any stale mountpoint; BindsTo cube-server) + to survive a daemon restart without the old crash-loop. All enabled in the + chroot. So a fresh bake IS reproducible; re-running `build_vm.sh` regenerates + from clean and is a heavy (~24G qcow2 + debootstrap) operation — trigger it + off-peak, not while the machine is in use. - `cube-resume-pointer.service` is REAL (wired 2026-08-13): a oneshot that writes the OS's "where to look to continue" into the cube at `c200/z011/y001/x001` (last snapshot index + resume coordinate). Was a dead stub before (unit pointed diff --git a/cubecode/src/cb.rs b/cubecode/src/cb.rs new file mode 100644 index 0000000..9e67714 --- /dev/null +++ b/cubecode/src/cb.rs @@ -0,0 +1,126 @@ +//! CUBELinux-2 cubebook / kernel-convention constants. +//! +//! These are the conventions the *OS* layer uses to store its computational +//! behavior in CUBE. Per the PDF (Package 4, and the "behavior descriptors" +//! discussion at PDF §524–525), functions/operators that live in CUBE are: +//! * addressed in a reserved `C` axis band (here `c210..=c219`), +//! * tagged with a `kind` (`fn`/`kernel`/`layer`/`checkpoint`/`variant`), and +//! * annotated with *behavior descriptor* flags (pure / I/O heavy / +//! allocates / touches-net / hot-path) carried in the record header's +//! out-of-band flag bits (tag 12, see cubestore encode/decode). +//! +//! This is deliberately NOT a store-IO model: the cubevm never reads or writes +//! OS state records directly. The OS's *decisions/computation* live as CUBE +//! kernels; the native OS layer is a thin effector that reads a kernel's +//! computed result and applies the effect (writes the record, runs the +//! command). See `cubesys/src/commands.rs` `tick` + `native-apply`. + +/// First `C` axis value reserved for OS operator kernels (call-graph roots and +/// leaves). The OS keeps its behavioral substrate here, separate from the +/// `c200` operational-snapshot data band and the `c001/c002` doc examples. +pub const C_OS_KERNEL: u8 = 210; + +/// `C` axis band for OS *data* records the native layer writes after a kernel +/// decides an effect (kept distinct from the compute-kernel band above). +pub const C_OS_EFFECT: u8 = 211; + +/// Header-flag bits 13..=15 are reserved for the behavior-descriptor field. +/// (Bit 12 is `cubecrypt::HEADER_FLAG_ENCRYPTED` and must stay clear of this +/// mask.) `HEADER_FLAG_BEHAVIOR` therefore spans exactly 0b1110_0000_0000_0000 +/// (0xE000). `refresh_flags()` preserves spare bits 8..=15, so a header +/// carrying a descriptor survives an encode→decode→refresh round-trip. +pub const HEADER_FLAG_BEHAVIOR: u16 = 0b1110_0000_0000_0000; // bits 13,14,15 + +/// Bit position of the behavior-descriptor field within the raw flag word. +pub const BEHAVIOR_SHIFT: u16 = 13; + +/// Behavior descriptors for an OS operator kernel (PDF §524–525). +/// +/// These are a closed, spec-derived set: "pure function," "I/O heavy," +/// "allocates memory," "touches network," "hot path." We map them onto three +/// available out-of-band bits (13..=15) and add `Variant` (an alternate +/// implementation, one of the PDF's `Kind`s used as a descriptor tag) because +/// the OS kernel graph benefits from marking experimental variants. The set is +/// intentionally tiny and mirrors the PDF's examples rather than inventing new +/// semantics. +#[derive(Copy, Clone, Debug, Default, PartialEq, Eq)] +pub struct Behavior(pub u8); + +impl Behavior { + /// Pure function: no side effects, deterministic on inputs. + pub const PURE: u8 = 1 << 0; + /// I/O heavy: performs significant store/device I/O. + pub const IO_HEAVY: u8 = 1 << 1; + /// Hot path: executed frequently; a candidate for optimization/variant swap. + pub const HOT_PATH: u8 = 1 << 2; + + /// Build from a raw 3-bit value (already shifted into bit 13..=15 space). + pub fn from_raw(bits: u8) -> Self { + Behavior(bits & 0b111) + } + + /// Encode into the out-of-band header flag bits (bits 13..=15). + pub fn to_flags(self) -> u16 { + ((self.0 & 0b111) as u16) << BEHAVIOR_SHIFT + } + + /// Decode from a raw 16-bit flag word (reads bits 13..=15). + pub fn from_flags(flags: u16) -> Self { + Behavior(((flags & HEADER_FLAG_BEHAVIOR) >> BEHAVIOR_SHIFT) as u8) + } + + /// True if any descriptor bit is set. + pub fn is_any(self) -> bool { + self.0 != 0 + } + + /// Human-readable descriptor tags (used by `ls`/`stat` and the effector). + pub fn tags(self) -> Vec<&'static str> { + let mut out = Vec::new(); + if self.0 & Self::PURE != 0 { + out.push("pure"); + } + if self.0 & Self::IO_HEAVY != 0 { + out.push("io"); + } + if self.0 & Self::HOT_PATH != 0 { + out.push("hot"); + } + out + } +} + +#[cfg(test)] +mod tests { + use super::*; + use cubecoords::{CubeHeader, Czyx}; + use cubestore::{CubeStore, HashBackend}; + + #[test] + fn behavior_round_trips_through_store() { + // PURE | HOT_PATH must survive the real store round-trip (what + // native-apply reads back). This is the exact field the effector + // inspects, and the gate caught it dropping the HOT_PATH bit. + let b = Behavior(Behavior::PURE | Behavior::HOT_PATH); + let raw = b.to_flags(); + assert_eq!(raw, 0x2000 | 0x8000, "to_flags wrong: {raw:#x}"); + + let mut store = CubeStore::new(HashBackend::new()); + let coord = Czyx::new(210, 1, 1, 3); + let mut h = CubeHeader::new(); + h.doc_type = Some("kernel".into()); + h.title = Some("summarize-procs".into()); + h.flags.0 |= raw; + h.refresh_flags(); + store.put_record(coord, &h, &[1, 2, 3]); + + let (read_h, _) = store.get_record(&coord).expect("record present"); + let back = Behavior::from_flags(read_h.flags.bits()); + assert_eq!( + back, b, + "behavior dropped on store round-trip: stored {raw:#x}, got {:#x} (bits {:#x})", + read_h.flags.bits(), + read_h.flags.bits() + ); + } +} diff --git a/cubecode/src/lib.rs b/cubecode/src/lib.rs index f280e3f..ea1f4f4 100644 --- a/cubecode/src/lib.rs +++ b/cubecode/src/lib.rs @@ -35,10 +35,13 @@ #![forbid(unsafe_code)] #![warn(missing_docs)] +pub mod cb; pub mod cell; pub mod opcode; pub mod vm; +pub use cb::{Behavior, C_OS_EFFECT, C_OS_KERNEL, HEADER_FLAG_BEHAVIOR}; + pub use cell::{CodeCell, Kind}; pub use opcode::{decode, encode, CodeError, Op}; pub use vm::{Fault, RunResult, Vm, SYS_DEGREE, SYS_LINKED_EXISTS, SYS_NOP, SYS_TRACE}; diff --git a/cubecoords/src/lib.rs b/cubecoords/src/lib.rs index 6de36d3..0d8dfa4 100644 --- a/cubecoords/src/lib.rs +++ b/cubecoords/src/lib.rs @@ -293,7 +293,10 @@ impl CubeHeader { f |= HeaderFlags::HAS_ASSOCIATIONS; } // Preserve spare/out-of-band flag bits (bits 8..=15) that are not - // derived from structured fields. + // derived from structured fields. This includes + // `cubecrypt::HEADER_FLAG_ENCRYPTED` (bit 12) and the behavior-descriptor + // field (bits 13..=15, see `cubecode::Behavior`) so both survive an + // encode→decode→refresh round-trip. const DERIVED_BITS: u16 = 0x00FF; f |= self.flags.bits() & !DERIVED_BITS; self.flags = HeaderFlags::from_bits(f); diff --git a/cubesys/src/commands.rs b/cubesys/src/commands.rs index 0d685bf..e8b94a1 100644 --- a/cubesys/src/commands.rs +++ b/cubesys/src/commands.rs @@ -15,7 +15,7 @@ use crate::audit::{Audit, OP_DELETE, OP_GRANT, OP_OPEN, OP_READ, OP_REVOKE, OP_S use crate::grants::{grant, grant_allows, perms_from_str, revoke, Owner, Perm}; use crate::store::ConcurrentStore; use crate::tenant::{TenantIdentity, TenantSession}; -use cubecode::{CodeCell, Kind, Op, Vm}; +use cubecode::{CodeCell, Kind, Op, RunResult, Vm}; use cubecoords::{CubeHeader, Czyx}; use cubecrypt::{CubeEnv, KeySlot, Selector, TransformId}; use cubestore::{CubeStore, HashBackend}; @@ -451,7 +451,23 @@ impl Session { "prog" => { let path = it.next().ok_or_else(|| "prog needs ".to_string())?; let mut ops: Vec = Vec::new(); + // Optional behavior-descriptor flags: `K=pure` `K=io` `K=hot`. + // These stamp the OS-kernel behavior bits (PDF §524–525) on the + // record header and switch the kind to `kernel` so the cube + // correctly classifies operator kernels vs plain functions. + let mut descriptor: Option = None; while let Some(tok) = it.next() { + if let Some(flag) = tok.strip_prefix("K=") { + let bit = match flag { + "pure" => cubecode::Behavior::PURE, + "io" => cubecode::Behavior::IO_HEAVY, + "hot" => cubecode::Behavior::HOT_PATH, + other => return Err(format!("prog: unknown descriptor K={other}")), + }; + let cur = descriptor.unwrap_or_default(); + descriptor = Some(cubecode::Behavior(cur.0 | bit)); + continue; + } let arg = if takes_arg(tok) { it.next() .and_then(|a| a.parse::().ok()) @@ -464,11 +480,13 @@ impl Session { if ops.is_empty() { return Err("prog: no ops given".to_string()); } + let is_kernel = descriptor.is_some(); + let kind = if is_kernel { Kind::Kernel } else { Kind::Fn }; let name = path.rsplit('/').next().unwrap_or(path); // Compute the exact record bytes `put_record` would write, using // a throwaway store so we can buffer (or apply) them without // duplicating the record codec. - let coord = scratch_code_coord(path, Kind::Fn, name, &ops)?; + let coord = scratch_code_coord(path, kind, name, &ops)?; // Task 6: reject overwriting a record owned by a different owner. if let Some(msg) = self.admit_mutate(coord, Perm::Write) { self.audit_now(OP_WRITE, coord, false); @@ -478,28 +496,329 @@ impl Session { let owner = self.identity.as_ref().map(|i| i.owner_local.as_str()); let value = { let mut scratch = CubeStore::new(HashBackend::new()); - crate::store_code_cell(&mut scratch, path, Kind::Fn, name, &[], &ops, owner) + crate::store_code_cell(&mut scratch, path, kind, name, &[], &ops, owner, descriptor) .map_err(|e| e.to_string())?; scratch.get_raw(&coord).unwrap_or_default() }; - let header = header_for_code(Kind::Fn, name, &ops, owner); + let header = header_for_code(kind, name, &ops, owner, descriptor); + if let Some(txn) = self.txn.as_mut() { + txn.ops.push(TxnOp { + coord, + put: Some((value, header)), + }); + let kind_tag = if is_kernel { "kernel" } else { "fn" }; + return Ok(format!( + "buffered prog {path} ({kind_tag}, {} ops) — commit to apply", + ops.len() + )); + } + let coord = store + .put_code_cell(path, kind, name, &[], &ops, owner, descriptor) + .map_err(|e| e.to_string())?; + let kind_tag = if is_kernel { "kernel" } else { "fn" }; + Ok(format!( + "wrote program {path} -> coord {} ({kind_tag}, {} ops)", + coord.pack_u32(), + ops.len() + )) + } + "link" => { + // Attach a callee coordinate to an existing code cell's call + // graph. `link ` appends (c,z,y,x) to the + // cell's `linked_records`, so a `call n` opcode in that cell + // dispatches to linked_records[n] (the cube's association + // graph IS the call graph). This is what makes functions stored + // in CUBE callable from other functions stored in CUBE. + let path = it.next().ok_or_else(|| "link needs ".to_string())?; + let c = parse_u8(it.next(), "link needs ")?; + let z = parse_u8(it.next(), "link needs ")?; + let y = parse_u8(it.next(), "link needs ")?; + let x = parse_u8(it.next(), "link needs ")?; + let target = Czyx::new(c, z, y, x); + let coord = crate::path_to_czyx(path).map_err(|e| e.to_string())?; + if let Some(msg) = self.admit_mutate(coord, Perm::Write) { + self.audit_now(OP_WRITE, coord, false); + return Err(msg); + } + self.audit_now(OP_WRITE, coord, true); + // Load the existing cell, append the link, re-store it. + let cell = crate::load_code_cell(&store.read_snapshot(), path) + .map_err(|e| format!("link: cannot load {path}: {e:?}"))?; + let mut links = cell.links().to_vec(); + if links.contains(&target) { + return Ok(format!( + "link {path}: {target:?} already linked (degree {})", + links.len() + )); + } + links.push(target); + let code = cubecode::decode(&cell.body()) + .map_err(|e| format!("link: bad bytecode in {path}: {e:?}"))?; + let owner = self.identity.as_ref().map(|i| i.owner_local.as_str()); + store + .put_code_cell( + path, + cell.kind(), + cell.name().unwrap_or(path), + &links, + &code, + owner, + None, + ) + .map_err(|e| e.to_string())?; + Ok(format!( + "linked {path} -> {target:?} (call-graph degree now {})", + links.len() + )) + } + // ---- OS-operator-kernel authoring + thin effector (Steps 1 & 2) ---- + // Per the reframe, the OS's *computation* lives in CUBE as operator + // kernels (call graphs + behavior descriptors, PDF §524–525). The + // native OS layer is only a thin effector: it reads a kernel's + // computed result and applies the effect. The cubevm never does + // store-IO itself. `kernel` authors a kernel; `tick` lays down the + // OS kernel call-graph; `native-apply` is the effector boundary. + "kernel" => { + // `kernel [K=pure|io|hot ...] ...` + // Like `prog`, but the cell is always kind=Kernel and accepts + // behavior-descriptor flags so the OS marks operator kernels + // distinctly from plain functions. + let path = it.next().ok_or_else(|| "kernel needs ".to_string())?; + let mut ops: Vec = Vec::new(); + let mut descriptor: Option = None; + while let Some(tok) = it.next() { + if let Some(flag) = tok.strip_prefix("K=") { + let bit = match flag { + "pure" => cubecode::Behavior::PURE, + "io" => cubecode::Behavior::IO_HEAVY, + "hot" => cubecode::Behavior::HOT_PATH, + other => return Err(format!("kernel: unknown descriptor K={other}")), + }; + let cur = descriptor.unwrap_or_default(); + descriptor = Some(cubecode::Behavior(cur.0 | bit)); + continue; + } + let arg = if takes_arg(tok) { + it.next() + .and_then(|a| a.parse::().ok()) + .ok_or_else(|| format!("kernel: {tok} needs a u8 argument"))? + } else { + 0 + }; + ops.push(make_op(tok, arg)?); + } + if ops.is_empty() { + return Err("kernel: no ops given".to_string()); + } + let name = path.rsplit('/').next().unwrap_or(path); + let coord = scratch_code_coord(path, Kind::Kernel, name, &ops)?; + if let Some(msg) = self.admit_mutate(coord, Perm::Write) { + self.audit_now(OP_WRITE, coord, false); + return Err(msg); + } + self.audit_now(OP_WRITE, coord, true); + let owner = self.identity.as_ref().map(|i| i.owner_local.as_str()); + let value = { + let mut scratch = CubeStore::new(HashBackend::new()); + crate::store_code_cell( + &mut scratch, + path, + Kind::Kernel, + name, + &[], + &ops, + owner, + descriptor, + ) + .map_err(|e| e.to_string())?; + scratch.get_raw(&coord).unwrap_or_default() + }; + let header = header_for_code(Kind::Kernel, name, &ops, owner, descriptor); if let Some(txn) = self.txn.as_mut() { txn.ops.push(TxnOp { coord, put: Some((value, header)), }); return Ok(format!( - "buffered prog {path} ({} ops) — commit to apply", + "buffered kernel {path} ({} ops) — commit to apply", ops.len() )); } let coord = store - .put_code_cell(path, Kind::Fn, name, &[], &ops, owner) + .put_code_cell(path, Kind::Kernel, name, &[], &ops, owner, descriptor) .map_err(|e| e.to_string())?; Ok(format!( - "wrote program {path} -> coord {} ({} ops)", + "wrote kernel {path} -> coord {} ({} ops, descriptors={:?})", coord.pack_u32(), - ops.len() + ops.len(), + descriptor.map(|b| b.tags()).unwrap_or_default() + )) + } + "tick" => { + // Lay down the OS operator-kernel call graph (Step 1). Each leaf + // is a real CUBE kernel composed via `linked_records` (the call + // graph) and tagged with behavior descriptors. `cube-os-tick` + // calls the three leaves in order. Nothing is executed here — + // the native layer later `run`s each and `native-apply`s the + // effect. This is "everything in CUBE" done the spec's way: + // the OS's behavior lives as kernels + call graph + descriptors. + let c = cubecode::C_OS_KERNEL; + let cfg = format!("/c{c}/z001/y001/x001"); // normalize-config: pure + let decide = format!("/c{c}/z001/y001/x002"); // decide-snapshot: io + let summarize = format!("/c{c}/z001/y001/x003"); // summarize-procs: pure+hot + let tick = format!("/c{c}/z001/y001/x004"); // cube-os-tick: hot (calls 0..2) + + let cfg_code = vec![Op::Const(7), Op::Const(3), Op::Add, Op::Halt]; // 7+3=10 + let decide_code = vec![Op::Const(1), Op::Ret]; // 1 => snapshot + let summarize_code = vec![Op::Const(20), Op::Halt]; // 20 procs + let tick_code = vec![ + Op::Const(0), + Op::CallLink(0), // cfg + Op::CallLink(1), // decide + Op::CallLink(2), // summarize + Op::Halt, + ]; + + // Stage the leaves first (we need their coords to build the + // call-graph edges of the root), then the root. `with_mut` + // hands us exclusive `&mut CubeStore` access so the helper can + // write each record through the normal codec. + let cfg_c = store + .with_mut(|s| { + crate::store_code_cell( + s, + &cfg, + Kind::Kernel, + "normalize-config", + &[], + &cfg_code, + None, + Some(cubecode::Behavior(cubecode::Behavior::PURE)), + ) + }) + .map_err(|e: crate::SysError| e.to_string())?; + let dec_c = store + .with_mut(|s| { + crate::store_code_cell( + s, + &decide, + Kind::Kernel, + "decide-snapshot", + &[], + &decide_code, + None, + Some(cubecode::Behavior(cubecode::Behavior::IO_HEAVY)), + ) + }) + .map_err(|e: crate::SysError| e.to_string())?; + let sum_c = store + .with_mut(|s| { + crate::store_code_cell( + s, + &summarize, + Kind::Kernel, + "summarize-procs", + &[], + &summarize_code, + None, + Some(cubecode::Behavior( + cubecode::Behavior::PURE | cubecode::Behavior::HOT_PATH, + )), + ) + }) + .map_err(|e: crate::SysError| e.to_string())?; + let tick_c = store + .with_mut(|s| { + crate::store_code_cell( + s, + &tick, + Kind::Kernel, + "cube-os-tick", + &[cfg_c, dec_c, sum_c], // call graph: tick -> {cfg,decide,summarize} + &tick_code, + None, + Some(cubecode::Behavior(cubecode::Behavior::HOT_PATH)), + ) + }) + .map_err(|e: crate::SysError| e.to_string())?; + + Ok(format!( + "OS kernel call-graph laid down (kind=kernel, in cube c{c}):\n {} normalize-config [pure] -> coord {}\n {} decide-snapshot [io] -> coord {}\n {} summarize-procs [pure,hot] -> coord {}\n {} cube-os-tick [hot] -> coord {} (links cfg,decide,summarize)\nrun e.g.: run {}\nthen effector: native-apply {}", + cfg, cfg_c.pack_u32(), decide, dec_c.pack_u32(), summarize, + sum_c.pack_u32(), tick, tick_c.pack_u32(), tick_c.pack_u32(), + tick + )) + } + "native-apply" => { + // Step 2: the thin effector. The decision/computation already + let path = it + .next() + .ok_or_else(|| "native-apply needs ".to_string())?; + let cell = crate::load_code_cell(&store.read_snapshot(), path) + .map_err(|e| format!("native-apply: cannot load {path}: {e}"))?; + if cell.kind() != Kind::Kernel { + return Err(format!( + "native-apply: {path} is kind={:?}, expected a kernel", + cell.kind() + )); + } + // Build a throwaway in-memory store holding just this kernel + // (the VM needs a store to walk), then run it. The computation + // is entirely in CUBE; we only read back the computed result. + let coord = crate::path_to_czyx(path).map_err(|e| e.to_string())?; + let mut vm_store = cubestore::CubeStore::new(cubestore::HashBackend::new()); + crate::store_code_cell( + &mut vm_store, + path, + Kind::Kernel, + cell.name().unwrap_or(path), + &cell.links().iter().copied().collect::>(), + &cell.code, + None, + None, + ) + .map_err(|e| format!("native-apply: stage failed: {e}"))?; + let mut vm = Vm::new(vm_store); + let result = match vm.run(coord) { + RunResult::Halted { top } => top, + other => { + return Err(format!( + "native-apply: kernel did not halt cleanly: {other:?}" + )) + } + }; + let r = result.unwrap_or(0); + let effect = match cell.name() { + Some("decide-snapshot") => { + if r != 0 { + format!( + "OS EFFECT: persist runtime snapshot into CUBE (c{} band); decision kernel returned {} => snapshot NOW", + cubecode::C_OS_EFFECT, r + ) + } else { + "OS EFFECT: no snapshot (decision kernel returned 0)".to_string() + } + } + _ => format!( + "OS EFFECT: apply computed result {} from kernel {}@{}", + r, + cell.name().unwrap_or("?"), + path + ), + }; + Ok(format!( + "native-apply {} (kind=kernel, descriptors={:?}):\n computed result = {}\n {}", + path, + cubecode::Behavior::from_flags( + store + .read_snapshot() + .get_record(&coord) + .map(|(h, _)| h.flags.bits()) + .unwrap_or(0) + ) + .tags(), + r, + effect )) } "write" => { @@ -520,11 +839,11 @@ impl Session { let owner = self.identity.as_ref().map(|i| i.owner_local.as_str()); let value = { let mut scratch = CubeStore::new(HashBackend::new()); - crate::store_code_cell(&mut scratch, path, Kind::Fn, name, &[], &code, owner) + crate::store_code_cell(&mut scratch, path, Kind::Fn, name, &[], &code, owner, None) .map_err(|e| e.to_string())?; scratch.get_raw(&coord).unwrap_or_default() }; - let header = header_for_code(Kind::Fn, name, &code, owner); + let header = header_for_code(Kind::Fn, name, &code, owner, None); if let Some(txn) = self.txn.as_mut() { txn.ops.push(TxnOp { coord, @@ -536,7 +855,7 @@ impl Session { )); } let coord = store - .put_code_cell(path, Kind::Fn, name, &[], &code, owner) + .put_code_cell(path, Kind::Fn, name, &[], &code, owner, None) .map_err(|e| e.to_string())?; Ok(format!("wrote {path} -> coord {}", coord.pack_u32())) } @@ -950,21 +1269,36 @@ pub fn txn_snapshot(s: &Session) -> CubeStore { /// Compute the coordinate a `store_code_cell` call would target, without /// writing — used to buffer `prog`/`write` mutations during a transaction. -fn scratch_code_coord(path: &str, kind: Kind, name: &str, code: &[Op]) -> Result { +fn scratch_code_coord( + path: &str, + kind: Kind, + name: &str, + code: &[Op], +) -> Result { let mut scratch = CubeStore::new(HashBackend::new()); - crate::store_code_cell(&mut scratch, path, kind, name, &[], code, None) + crate::store_code_cell(&mut scratch, path, kind, name, &[], code, None, None) .map_err(|e| e.to_string()) } /// Build the `CubeHeader` a `store_code_cell` call would attach (mirrors /// `crate::store_code_cell`), so a buffered txn put carries the same header. /// `owner` (when set) is stamped on `owner_local_user` for Task 6 enforcement. -fn header_for_code(kind: Kind, name: &str, code: &[Op], owner: Option<&str>) -> CubeHeader { +/// `descriptor` (when set) stamps the behavior-descriptor bits (PDF §524–525). +fn header_for_code( + kind: Kind, + name: &str, + code: &[Op], + owner: Option<&str>, + descriptor: Option, +) -> CubeHeader { let mut h = CubeHeader::new(); h.title = Some(name.to_string()); h.doc_type = Some(kind.as_str().to_string()); h.linked_records = Vec::new(); h.owner_local_user = owner.map(|o| o.to_string()); + if let Some(b) = descriptor { + h.flags.0 |= b.to_flags(); + } if h.doc_type.as_deref() == Some("fn") { h.size_bytes = Some(cubecode::encode(code).len() as u64); } @@ -1101,6 +1435,46 @@ mod tests { Session::with_store(Arc::new(ConcurrentStore::memory())) } + #[test] + fn os_kernels_live_in_cube_with_call_graph_and_descriptors() { + // Step 1: OS operator behavior lives in CUBE as kernels, composed via + // a call graph (linked_records) and tagged with behavior descriptors. + // Step 2: a thin native effector (`native-apply`) runs each kernel in + // the VM, reads its computed result, and emits the OS effect — the + // cubevm itself never does store-IO (spec-aligned, PDF §524–525). + let mut s = session(); + + // Lay down the OS kernel call graph. + let out = s.exec("tick").expect("tick should lay down kernels"); + assert!(out.contains("normalize-config"), "cfg kernel missing: {out}"); + assert!(out.contains("decide-snapshot"), "decide kernel missing: {out}"); + assert!(out.contains("summarize-procs"), "summarize kernel missing: {out}"); + assert!(out.contains("cube-os-tick"), "tick kernel missing: {out}"); + // The root links the three leaves (call graph, not foreign code). + assert!(out.contains("links cfg,decide,summarize"), "call graph not wired: {out}"); + // Behavior descriptors are stamped (round-trip through header flags). + assert!(out.contains("[pure]") && out.contains("[io]") && out.contains("[pure,hot]"), + "behavior descriptors not stamped: {out}"); + + // Run the root kernel: it must traverse the call graph (CallLink 0..2) + // and return, proving the OS's behavior lives as addressable kernels. + let run_out = s.exec("run /c210/z001/y001/x004").expect("tick kernel must run"); + assert!(run_out.contains("Halted"), "tick kernel should halt: {run_out}"); + + // Step 2 — the effector reads the COMPUTED result and emits the effect. + let eff = s.exec("native-apply /c210/z001/y001/x002") + .expect("effector must run decide-snapshot"); + assert!(eff.contains("computed result = 1"), "decide kernel result wrong: {eff}"); + assert!(eff.contains("OS EFFECT"), "effector must emit OS EFFECT: {eff}"); + assert!(eff.contains("snapshot NOW"), "decision=1 should snapshot: {eff}"); + + // A plain pure kernel also routes through the effector with no store-IO. + let eff2 = s.exec("native-apply /c210/z001/y001/x003") + .expect("effector must run summarize-procs"); + assert!(eff2.contains("computed result = 20"), "summarize result wrong: {eff2}"); + assert!(eff2.contains("descriptors=[\"pure\", \"hot\"]"), "descriptor readback wrong: {eff2}"); + } + #[test] fn begin_commit_applies_buffered_writes() { let mut s = session(); diff --git a/cubesys/src/lib.rs b/cubesys/src/lib.rs index ecc3a27..af6a962 100644 --- a/cubesys/src/lib.rs +++ b/cubesys/src/lib.rs @@ -141,6 +141,10 @@ pub fn store_code_cell( links: &[Czyx], code: &[cubecode::Op], owner: Option<&str>, + // Behavior-descriptor flags (PDF §524–525: pure / I/O heavy / hot path) to + // stamp on the record header's out-of-band bits. `None` leaves the field + // at zero. See `cubecode::Behavior`. + descriptor: Option, ) -> Result { let coord = path_to_czyx(path)?; let mut h = CubeHeader::new(); @@ -148,6 +152,9 @@ pub fn store_code_cell( h.doc_type = Some(kind.as_str().to_string()); h.linked_records = links.to_vec(); h.owner_local_user = owner.map(|o| o.to_string()); + if let Some(b) = descriptor { + h.flags.0 |= b.to_flags(); + } if h.doc_type.as_deref() == Some("fn") { h.size_bytes = Some(cubecode::encode(code).len() as u64); } @@ -187,6 +194,7 @@ mod tests { &[], &[Op::Const(2), Op::Const(3), Op::Add, Op::Halt], None, + None, ) .unwrap(); @@ -300,6 +308,7 @@ pub mod demo { &[], &double_code, None, + None, ) .expect("store double"); let entry_coord = super::store_code_cell( @@ -310,6 +319,7 @@ pub mod demo { &[double_coord], &entry_code, None, + None, ) .expect("store entry"); println!(" wrote {double} -> coord {}", double_coord.pack_u32()); diff --git a/cubesys/src/store.rs b/cubesys/src/store.rs index 9329509..12b9c6d 100644 --- a/cubesys/src/store.rs +++ b/cubesys/src/store.rs @@ -691,9 +691,10 @@ impl ConcurrentStore { links: &[Czyx], code: &[Op], owner: Option<&str>, + descriptor: Option, ) -> Result { let coord = self.with_mut(|store| { - crate::store_code_cell(store, path, kind, name, links, code, owner) + crate::store_code_cell(store, path, kind, name, links, code, owner, descriptor) })?; if let Some(v) = self.get_raw(&coord) { self.log_put(coord, v);