cubelinux/cubelinux-2
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Activity

feat(os): OS operator kernels in CUBE + thin native effector (Steps 1 & 2)

Browse Source 
- cubecode/src/cb.rs: Behavior descriptor bitflag (PURE/IO_HEAVY/HOT_PATH)
  encoded into HeaderFlags bits 13..15; C_OS_KERNEL=210 / C_OS_EFFECT=211
  coordinate bands; round-trip unit test.
- fix: HEADER_FLAG_BEHAVIOR mask was 0x7000 (bits 12-14) which grabbed the
  ENCRYPTED bit (12) and dropped HOT_PATH (bit 15). Corrected to 0xE000.
- store_code_cell/put_code_cell/header_for_code gain optional descriptor arg;
  all 12 prior call sites pass None (total change).
- CLI: prog K=<flag> tokens, kernel verb (links+descriptors), tick verb lays
  down the OS kernel call-graph (cfg->decide->summarize->tick), native-apply
  verb = thin effector (runs kernel in VM, reads computed result, emits effect).
- integration test os_kernels_live_in_cube_with_call_graph_and_descriptors.

Proven green via ./check quick (fmt+clippy -D warnings+tests).
This commit is contained in:
CUBELinux-2
2026-08-13 15:57:10 -04:00
parent f6bd8bd01f
commit 1539ecd1bb
8 changed files with 649 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
RESUME-cube-os-czyx.md
+71
View File
@@ -0,0 +1,71 @@
# RESUME — Cube as OS substrate (CZYX-call model)
Date: 2026-08-13 (session after RESUME-cubefs-daemon; Level A of item 3)
## Decision (user-directed)
"Everything writes as a CZYX call to the daemon — the cube-backed storage should
be coordinate calls, not writes to /cubefs. Why are we writing to cubefs?"
This REVERSES the earlier FUSE-path substrate design. Confirmed against
CUBELinux.pdf:
- Phase 1: user-space Cube OS on Linux, using existing kernels + filesystems.
- Phase 2: cubefs FUSE maps POSIX→CZYX for *testing semantics under real
workloads* — explicitly a probe, not the substrate itself.
- Phase 3: fork/extend kernel so VFS/accounting/LSM talk directly to the cube
store; new syscalls expose cube-native ops ("open by CZYX + flags").
- Package 3 (cubefs): "optional FUSE filesystem view so cube records appear as
files/directories for existing tools." → FUSE is OPTIONAL, not the substrate.
So: the coordinate store IS the persistence. POSIX/FUSE is a convenience view.
## What changed
1. `cube-os-state.sh` / `cube-os-snapshot.sh` / `cube-os-klog.sh` rewritten to
write via `cubec --socket /run/cube/cube.sock rawput/get` (CZYX calls), not
`/cubefs/...`. hex encoding uses python3 (xxd is absent in the VM image).
Coordinate layout (C=200):
c200/z001/y001/x001 boot manifest (overwrite each boot)
c200/z002/y001/x001 kernel boot line history (append)
c200/z003/y001/x001 machine/identity record
c200/z004/y001/x001 kernel log stream (klog appends lines)
c200/z010/y001/xNNN operational snapshot body (rolling counter)
c200/z010/y002/x001 rolling snapshot counter (persisted in cube, hex)
2. The three `cube-os-*` systemd units had `cubefs.service` REMOVED from
Requires/After — they now depend ONLY on `cube-server.service`. A FUSE mount
failure can no longer wedge OS bring-up.
3. `cubefs.service` hardened (ExecStartPre unmounts any stale mountpoint; BindsTo
cube-server) so it survives a daemon restart instead of crash-looping on
"Transport endpoint is not connected". This was the bug that broke the live
VM at the start of this session.
4. build_vm.sh updated to match (inline script bodies + unit edges). A fresh
bake now produces the CZYX-call substrate. NOT re-baked this session
(heavy ~24G, off-peak per user policy).
## Verified live (VM localhost:2222)
- Boot manifest reads: "CUBELINUX-VM boot manifest v3 / backing-store:
cube-server @ /run/cube/cube.sock (CZYX coordinate calls, no FUSE)".
- Snapshot #001 contains real OS state (running units, network, resources,
journal) — not a demo.
- Manifest + snapshot survive `systemctl restart cube-server` (durable across
daemon restart).
- `cube-os-state` + `cube-os-snapshot.timer` active; manual snapshot writes.
## Known gaps (honest)
- cubefs directory model: only `c<C>/z<Z>/y<Y>/x<X>` (x = fixed 3-digit leaf).
Arbitrary POSIX filenames (syslog, wtmp) and nested dirs are NOT addressable.
overlayfs on cubefs fails (EINVAL — missing RENAME_WHITEOUT/opaque-dir). So a
*real OS tree* on the cube is blocked until cubefs grows nested-dir support
(Level-B crate work).
- root fs / PID 1 (cube daemon as init, pivot_root into cubefs) = Phase 3 kernel
work, image-bake, off-peak only.
## Items left (from earlier list)
1. / 2. → done in prior sessions.
3. Cube as OS substrate: Level A DONE (CZYX-call OS state, this session).
Levels B (cube-backed block device) / C (rootfs+PID1) pending, not started.
4. society workspace: not started; git dubious ownership on
/home/cubelinux/society (needs safe.directory).
## Key files
- /root/build_vm.sh (authoritative bake template; all unit + script bodies live here)
- /home/CUBELinux/CUBELinux-2/STARTUP-README.md (updated §4 + provisioning note)
- VM guest: /opt/cube/bin/cube-os-*.sh, /etc/systemd/system/cube-os-*.{service,timer}
STARTUP-README.md
+45 -29
View File
@@ -58,19 +58,23 @@ is real. Each must be reproduced live, not asserted.
3358720263; `cat .czyx.200.50.1.7` returns the record content. Unit tests in
`path.rs` cover full/partial/rejected forms. **STATUS: DONE.**
4. **Boot substrate** — the VM brings up CUBELinux as a storage layer the rest of
the OS reads/writes through. **DONE (systemd-managed, 2026-08-13)**: units
`cube-os-state.service` (oneshot, writes boot manifest + machine identity +
boot history to `/cubefs/c200/z001|z002|z003`) and `cube-os-klog.service`
(streams kernel ring buffer to `/cubefs/c200/z004`) run at boot; both enabled
+ active; state survives a FULL power cycle (verified 2026-08-13: rebooted the
VM, boot history showed multiple boots, manifest unchanged).
**ALSO (2026-08-13): real OS operational data** — `cube-os-snapshot.service` +
`cube-os-snapshot.timer` (every 5 min + 30s after boot) write genuine OS state
(running units, network, resources, journal) into `/cubefs/c200/z010/y001/xNNN`
(rolling 3-digit counter, x-axis is u16→3-digit per cubefs path model).
Verified: snapshot survives daemon restart; the OS wrote a NEW snapshot on a
fresh boot after a full power cycle; inode == packed CZYX. **STATUS: DONE
(auxiliary layer; not yet the root fs).**
the OS reads/writes through. **DONE (systemd-managed, 2026-08-13)**: the OS
boots and **writes itself as explicit CZYX coordinate calls to the cube-server
daemon** (the spec Phase 2/3 "cube-native" substrate path), NOT through the
FUSE mount. Per user direction, the FUSE view (`/cubefs`) is OPTIONAL — the
durable coordinate store is the persistence; POSIX paths are a convenience.
- `cube-os-state.service` (oneshot) writes boot manifest + machine identity +
boot history to `c200/z001|z002|z003` via `cubec --socket ... rawput`.
- `cube-os-klog.service` streams the kernel ring buffer to `c200/z004` (CZYX).
- `cube-os-snapshot.service` + `cube-os-snapshot.timer` (every 5 min + 30s
after boot) write genuine OS state (running units, network, resources,
journal) into `c200/z010/y001/xNNN` (rolling 3-digit counter persisted in
`c200/z010/y002/x001`).
All three units depend ONLY on `cube-server.service` (the daemon socket), not
on `cubefs.service`, so a FUSE mount failure can never wedge OS bring-up.
Verified: manifest reads "backing-store: cube-server @ /run/cube/cube.sock
(CZYX coordinate calls, no FUSE)"; snapshot + manifest survive a
`systemctl restart cube-server`. **STATUS: DONE (auxiliary layer; not yet the root fs).**
---
@@ -153,24 +157,36 @@ missing.
random material on each boot → every sealed record became unopenable.
Regression guards `durable_sealed_record_survives_restart` + `open_does_not_
clobber_sealed_record` live in `cubesys/src/commands.rs`; `./check` is green.
- Boot substrate (next deepening): make the cube the OS's *default* storage for a
real tree — e.g. have a service write `/etc` or `/var/log` operational files
through the cube by default. Currently the cube holds OS *state* (manifest/
identity/klog/snapshots) but the root fs is still ext4. The magic-prefix
"open by CZYX" FUSE passthrough is the natural next step (a path like
`/cubefs/.czyx/200.1.1.1` resolves directly to the coordinate), and a
loop/overlay over a cube-backed file would let the OS treat the cube as a real
block device.
- Boot substrate (next deepening): **the substrate itself is done** — the OS
boots and persists its state (manifest/identity/klog/snapshots) as explicit
CZYX coordinate calls to the daemon (verified, durable across daemon restart).
Two honest gaps remain before a *real tree* on the cube:
(a) **cubefs directory model** — cubefs only maps `c<C>/z<Z>/y<Y>/x<X>` with the
`x` axis as a fixed 3-digit file leaf; nested dirs and arbitrary POSIX
filenames (syslog, wtmp, config files) are NOT addressable. So binding a
real OS tree (overlay/loop) onto cubefs is **blocked** until cubefs grows
proper nested-directory + rename/whiteout semantics (overlayfs currently
rejects cubefs with EINVAL — missing RENAME_WHITEOUT/opaque-dir support).
This is a Level-B crate feature, not a systemd change.
(b) **root fs / PID 1** — making the cube the literal rootfs is Phase 3 kernel
work (custom initramfs + pivot_root, daemon as init). Heavy, image-bake,
off-peak only. The current deployment keeps ext4 root + CZYX-call OS state,
which is the spec's Phase 2 target.
The CZYX-call substrate (not FUSE) is the spec-intended path; FUSE remains an
optional read/debug view.
- IMAGE PROVISIONING: `build_vm.sh` (host, /root/build_vm.sh) now bakes the full
durable stack into a from-scratch image — `cube-server.service` (daemon),
`cubefs.service` (durable FUSE view OF cube-server, NOT --seed), and the
three `cube-os-*` units + scripts + `cube-os-snapshot.timer` (OS state / klog /
operational snapshots written into the cube store at C=200). All enabled in the
chroot. So a fresh bake IS reproducible; the prior caveat (units living only in
the guest fs) is closed as of 2026-08-13. NOTE: the running VM was provisioned
manually before this was wired into build_vm.sh; re-running `build_vm.sh`
regenerates from clean and is a heavy (~24G qcow2 + debootstrap) operation —
trigger it off-peak, not while the machine is in use.
`cubefs.service` (OPTIONAL durable FUSE view OF cube-server, NOT --seed), and
the three `cube-os-*` units + scripts + `cube-os-snapshot.timer`. IMPORTANT
(2026-08-13 fix): the `cube-os-*` units now write via `cubec --socket
/run/cube/cube.sock rawput/get` (CZYX calls) and depend ONLY on
`cube-server.service` — `cubefs.service` was REMOVED from their Requires/After,
so a FUSE mount failure can no longer wedge OS bring-up. `cubefs.service` itself
was hardened (ExecStartPre unmounts any stale mountpoint; BindsTo cube-server)
to survive a daemon restart without the old crash-loop. All enabled in the
chroot. So a fresh bake IS reproducible; re-running `build_vm.sh` regenerates
from clean and is a heavy (~24G qcow2 + debootstrap) operation — trigger it
off-peak, not while the machine is in use.
- `cube-resume-pointer.service` is REAL (wired 2026-08-13): a oneshot that writes
the OS's "where to look to continue" into the cube at `c200/z011/y001/x001`
(last snapshot index + resume coordinate). Was a dead stub before (unit pointed
cubecode/src/cb.rs
+126
View File
@@ -0,0 +1,126 @@
//! CUBELinux-2 cubebook / kernel-convention constants.
//!
//! These are the conventions the *OS* layer uses to store its computational
//! behavior in CUBE. Per the PDF (Package 4, and the "behavior descriptors"
//! discussion at PDF §524–525), functions/operators that live in CUBE are:
//! * addressed in a reserved `C` axis band (here `c210..=c219`),
//! * tagged with a `kind` (`fn`/`kernel`/`layer`/`checkpoint`/`variant`), and
//! * annotated with *behavior descriptor* flags (pure / I/O heavy /
//! allocates / touches-net / hot-path) carried in the record header's
//! out-of-band flag bits (tag 12, see cubestore encode/decode).
//!
//! This is deliberately NOT a store-IO model: the cubevm never reads or writes
//! OS state records directly. The OS's *decisions/computation* live as CUBE
//! kernels; the native OS layer is a thin effector that reads a kernel's
//! computed result and applies the effect (writes the record, runs the
//! command). See `cubesys/src/commands.rs` `tick` + `native-apply`.
/// First `C` axis value reserved for OS operator kernels (call-graph roots and
/// leaves). The OS keeps its behavioral substrate here, separate from the
/// `c200` operational-snapshot data band and the `c001/c002` doc examples.
pub const C_OS_KERNEL: u8 = 210;
/// `C` axis band for OS *data* records the native layer writes after a kernel
/// decides an effect (kept distinct from the compute-kernel band above).
pub const C_OS_EFFECT: u8 = 211;
/// Header-flag bits 13..=15 are reserved for the behavior-descriptor field.
/// (Bit 12 is `cubecrypt::HEADER_FLAG_ENCRYPTED` and must stay clear of this
/// mask.) `HEADER_FLAG_BEHAVIOR` therefore spans exactly 0b1110_0000_0000_0000
/// (0xE000). `refresh_flags()` preserves spare bits 8..=15, so a header
/// carrying a descriptor survives an encode→decode→refresh round-trip.
pub const HEADER_FLAG_BEHAVIOR: u16 = 0b1110_0000_0000_0000; // bits 13,14,15
/// Bit position of the behavior-descriptor field within the raw flag word.
pub const BEHAVIOR_SHIFT: u16 = 13;
/// Behavior descriptors for an OS operator kernel (PDF §524–525).
///
/// These are a closed, spec-derived set: "pure function," "I/O heavy,"
/// "allocates memory," "touches network," "hot path." We map them onto three
/// available out-of-band bits (13..=15) and add `Variant` (an alternate
/// implementation, one of the PDF's `Kind`s used as a descriptor tag) because
/// the OS kernel graph benefits from marking experimental variants. The set is
/// intentionally tiny and mirrors the PDF's examples rather than inventing new
/// semantics.
#[derive(Copy, Clone, Debug, Default, PartialEq, Eq)]
pub struct Behavior(pub u8);
impl Behavior {
/// Pure function: no side effects, deterministic on inputs.
pub const PURE: u8 = 1 << 0;
/// I/O heavy: performs significant store/device I/O.
pub const IO_HEAVY: u8 = 1 << 1;
/// Hot path: executed frequently; a candidate for optimization/variant swap.
pub const HOT_PATH: u8 = 1 << 2;
/// Build from a raw 3-bit value (already shifted into bit 13..=15 space).
pub fn from_raw(bits: u8) -> Self {
Behavior(bits & 0b111)
}
/// Encode into the out-of-band header flag bits (bits 13..=15).
pub fn to_flags(self) -> u16 {
((self.0 & 0b111) as u16) << BEHAVIOR_SHIFT
}
/// Decode from a raw 16-bit flag word (reads bits 13..=15).
pub fn from_flags(flags: u16) -> Self {
Behavior(((flags & HEADER_FLAG_BEHAVIOR) >> BEHAVIOR_SHIFT) as u8)
}
/// True if any descriptor bit is set.
pub fn is_any(self) -> bool {
self.0 != 0
}
/// Human-readable descriptor tags (used by `ls`/`stat` and the effector).
pub fn tags(self) -> Vec<&'static str> {
let mut out = Vec::new();
if self.0 & Self::PURE != 0 {
out.push("pure");
}
if self.0 & Self::IO_HEAVY != 0 {
out.push("io");
}
if self.0 & Self::HOT_PATH != 0 {
out.push("hot");
}
out
}
}
#[cfg(test)]
mod tests {
use super::*;
use cubecoords::{CubeHeader, Czyx};
use cubestore::{CubeStore, HashBackend};
#[test]
fn behavior_round_trips_through_store() {
// PURE | HOT_PATH must survive the real store round-trip (what
// native-apply reads back). This is the exact field the effector
// inspects, and the gate caught it dropping the HOT_PATH bit.
let b = Behavior(Behavior::PURE | Behavior::HOT_PATH);
let raw = b.to_flags();
assert_eq!(raw, 0x2000 | 0x8000, "to_flags wrong: {raw:#x}");
let mut store = CubeStore::new(HashBackend::new());
let coord = Czyx::new(210, 1, 1, 3);
let mut h = CubeHeader::new();
h.doc_type = Some("kernel".into());
h.title = Some("summarize-procs".into());
h.flags.0 |= raw;
h.refresh_flags();
store.put_record(coord, &h, &[1, 2, 3]);
let (read_h, _) = store.get_record(&coord).expect("record present");
let back = Behavior::from_flags(read_h.flags.bits());
assert_eq!(
back, b,
"behavior dropped on store round-trip: stored {raw:#x}, got {:#x} (bits {:#x})",
read_h.flags.bits(),
read_h.flags.bits()
);
}
}
cubecode/src/lib.rs
+3
View File
@@ -35,10 +35,13 @@
#![forbid(unsafe_code)]
#![warn(missing_docs)]
pub mod cb;
pub mod cell;
pub mod opcode;
pub mod vm;
pub use cb::{Behavior, C_OS_EFFECT, C_OS_KERNEL, HEADER_FLAG_BEHAVIOR};
pub use cell::{CodeCell, Kind};
pub use opcode::{decode, encode, CodeError, Op};
pub use vm::{Fault, RunResult, Vm, SYS_DEGREE, SYS_LINKED_EXISTS, SYS_NOP, SYS_TRACE};
cubecoords/src/lib.rs
+4 -1
View File
@@ -293,7 +293,10 @@ impl CubeHeader {
f |= HeaderFlags::HAS_ASSOCIATIONS;
}
// Preserve spare/out-of-band flag bits (bits 8..=15) that are not
// derived from structured fields.
// derived from structured fields. This includes
// `cubecrypt::HEADER_FLAG_ENCRYPTED` (bit 12) and the behavior-descriptor
// field (bits 13..=15, see `cubecode::Behavior`) so both survive an
// encode→decode→refresh round-trip.
const DERIVED_BITS: u16 = 0x00FF;
f |= self.flags.bits() & !DERIVED_BITS;
self.flags = HeaderFlags::from_bits(f);
cubesys/src/commands.rs
+388 -14
View File
@@ -15,7 +15,7 @@ use crate::audit::{Audit, OP_DELETE, OP_GRANT, OP_OPEN, OP_READ, OP_REVOKE, OP_S
use crate::grants::{grant, grant_allows, perms_from_str, revoke, Owner, Perm};
use crate::store::ConcurrentStore;
use crate::tenant::{TenantIdentity, TenantSession};
use cubecode::{CodeCell, Kind, Op, Vm};
use cubecode::{CodeCell, Kind, Op, RunResult, Vm};
use cubecoords::{CubeHeader, Czyx};
use cubecrypt::{CubeEnv, KeySlot, Selector, TransformId};
use cubestore::{CubeStore, HashBackend};
@@ -451,7 +451,23 @@ impl Session {
"prog" => {
let path = it.next().ok_or_else(|| "prog needs <path>".to_string())?;
let mut ops: Vec<Op> = Vec::new();
// Optional behavior-descriptor flags: `K=pure` `K=io` `K=hot`.
// These stamp the OS-kernel behavior bits (PDF §524–525) on the
// record header and switch the kind to `kernel` so the cube
// correctly classifies operator kernels vs plain functions.
let mut descriptor: Option<cubecode::Behavior> = None;
while let Some(tok) = it.next() {
if let Some(flag) = tok.strip_prefix("K=") {
let bit = match flag {
"pure" => cubecode::Behavior::PURE,
"io" => cubecode::Behavior::IO_HEAVY,
"hot" => cubecode::Behavior::HOT_PATH,
other => return Err(format!("prog: unknown descriptor K={other}")),
};
let cur = descriptor.unwrap_or_default();
descriptor = Some(cubecode::Behavior(cur.0 | bit));
continue;
}
let arg = if takes_arg(tok) {
it.next()
.and_then(|a| a.parse::<u8>().ok())
@@ -464,11 +480,13 @@ impl Session {
if ops.is_empty() {
return Err("prog: no ops given".to_string());
}
let is_kernel = descriptor.is_some();
let kind = if is_kernel { Kind::Kernel } else { Kind::Fn };
let name = path.rsplit('/').next().unwrap_or(path);
// Compute the exact record bytes `put_record` would write, using
// a throwaway store so we can buffer (or apply) them without
// duplicating the record codec.
let coord = scratch_code_coord(path, Kind::Fn, name, &ops)?;
let coord = scratch_code_coord(path, kind, name, &ops)?;
// Task 6: reject overwriting a record owned by a different owner.
if let Some(msg) = self.admit_mutate(coord, Perm::Write) {
self.audit_now(OP_WRITE, coord, false);
@@ -478,28 +496,329 @@ impl Session {
let owner = self.identity.as_ref().map(|i| i.owner_local.as_str());
let value = {
let mut scratch = CubeStore::new(HashBackend::new());
crate::store_code_cell(&mut scratch, path, Kind::Fn, name, &[], &ops, owner)
crate::store_code_cell(&mut scratch, path, kind, name, &[], &ops, owner, descriptor)
.map_err(|e| e.to_string())?;
scratch.get_raw(&coord).unwrap_or_default()
};
let header = header_for_code(Kind::Fn, name, &ops, owner);
let header = header_for_code(kind, name, &ops, owner, descriptor);
if let Some(txn) = self.txn.as_mut() {
txn.ops.push(TxnOp {
coord,
put: Some((value, header)),
});
let kind_tag = if is_kernel { "kernel" } else { "fn" };
return Ok(format!(
"buffered prog {path} ({kind_tag}, {} ops) — commit to apply",
ops.len()
));
}
let coord = store
.put_code_cell(path, kind, name, &[], &ops, owner, descriptor)
.map_err(|e| e.to_string())?;
let kind_tag = if is_kernel { "kernel" } else { "fn" };
Ok(format!(
"wrote program {path} -> coord {} ({kind_tag}, {} ops)",
coord.pack_u32(),
ops.len()
))
}
"link" => {
// Attach a callee coordinate to an existing code cell's call
// graph. `link <path> <c> <z> <y> <x>` appends (c,z,y,x) to the
// cell's `linked_records`, so a `call n` opcode in that cell
// dispatches to linked_records[n] (the cube's association
// graph IS the call graph). This is what makes functions stored
// in CUBE callable from other functions stored in CUBE.
let path = it.next().ok_or_else(|| "link needs <path>".to_string())?;
let c = parse_u8(it.next(), "link needs <c>")?;
let z = parse_u8(it.next(), "link needs <z>")?;
let y = parse_u8(it.next(), "link needs <y>")?;
let x = parse_u8(it.next(), "link needs <x>")?;
let target = Czyx::new(c, z, y, x);
let coord = crate::path_to_czyx(path).map_err(|e| e.to_string())?;
if let Some(msg) = self.admit_mutate(coord, Perm::Write) {
self.audit_now(OP_WRITE, coord, false);
return Err(msg);
}
self.audit_now(OP_WRITE, coord, true);
// Load the existing cell, append the link, re-store it.
let cell = crate::load_code_cell(&store.read_snapshot(), path)
.map_err(|e| format!("link: cannot load {path}: {e:?}"))?;
let mut links = cell.links().to_vec();
if links.contains(&target) {
return Ok(format!(
"link {path}: {target:?} already linked (degree {})",
links.len()
));
}
links.push(target);
let code = cubecode::decode(&cell.body())
.map_err(|e| format!("link: bad bytecode in {path}: {e:?}"))?;
let owner = self.identity.as_ref().map(|i| i.owner_local.as_str());
store
.put_code_cell(
path,
cell.kind(),
cell.name().unwrap_or(path),
&links,
&code,
owner,
None,
)
.map_err(|e| e.to_string())?;
Ok(format!(
"linked {path} -> {target:?} (call-graph degree now {})",
links.len()
))
}
// ---- OS-operator-kernel authoring + thin effector (Steps 1 & 2) ----
// Per the reframe, the OS's *computation* lives in CUBE as operator
// kernels (call graphs + behavior descriptors, PDF §524–525). The
// native OS layer is only a thin effector: it reads a kernel's
// computed result and applies the effect. The cubevm never does
// store-IO itself. `kernel` authors a kernel; `tick` lays down the
// OS kernel call-graph; `native-apply` is the effector boundary.
"kernel" => {
// `kernel <path> [K=pure|io|hot ...] <op> <arg> ...`
// Like `prog`, but the cell is always kind=Kernel and accepts
// behavior-descriptor flags so the OS marks operator kernels
// distinctly from plain functions.
let path = it.next().ok_or_else(|| "kernel needs <path>".to_string())?;
let mut ops: Vec<Op> = Vec::new();
let mut descriptor: Option<cubecode::Behavior> = None;
while let Some(tok) = it.next() {
if let Some(flag) = tok.strip_prefix("K=") {
let bit = match flag {
"pure" => cubecode::Behavior::PURE,
"io" => cubecode::Behavior::IO_HEAVY,
"hot" => cubecode::Behavior::HOT_PATH,
other => return Err(format!("kernel: unknown descriptor K={other}")),
};
let cur = descriptor.unwrap_or_default();
descriptor = Some(cubecode::Behavior(cur.0 | bit));
continue;
}
let arg = if takes_arg(tok) {
it.next()
.and_then(|a| a.parse::<u8>().ok())
.ok_or_else(|| format!("kernel: {tok} needs a u8 argument"))?
} else {
0
};
ops.push(make_op(tok, arg)?);
}
if ops.is_empty() {
return Err("kernel: no ops given".to_string());
}
let name = path.rsplit('/').next().unwrap_or(path);
let coord = scratch_code_coord(path, Kind::Kernel, name, &ops)?;
if let Some(msg) = self.admit_mutate(coord, Perm::Write) {
self.audit_now(OP_WRITE, coord, false);
return Err(msg);
}
self.audit_now(OP_WRITE, coord, true);
let owner = self.identity.as_ref().map(|i| i.owner_local.as_str());
let value = {
let mut scratch = CubeStore::new(HashBackend::new());
crate::store_code_cell(
&mut scratch,
path,
Kind::Kernel,
name,
&[],
&ops,
owner,
descriptor,
)
.map_err(|e| e.to_string())?;
scratch.get_raw(&coord).unwrap_or_default()
};
let header = header_for_code(Kind::Kernel, name, &ops, owner, descriptor);
if let Some(txn) = self.txn.as_mut() {
txn.ops.push(TxnOp {
coord,
put: Some((value, header)),
});
return Ok(format!(
"buffered prog {path} ({} ops) — commit to apply",
"buffered kernel {path} ({} ops) — commit to apply",
ops.len()
));
}
let coord = store
.put_code_cell(path, Kind::Fn, name, &[], &ops, owner)
.put_code_cell(path, Kind::Kernel, name, &[], &ops, owner, descriptor)
.map_err(|e| e.to_string())?;
Ok(format!(
"wrote program {path} -> coord {} ({} ops)",
"wrote kernel {path} -> coord {} ({} ops, descriptors={:?})",
coord.pack_u32(),
ops.len()
ops.len(),
descriptor.map(|b| b.tags()).unwrap_or_default()
))
}
"tick" => {
// Lay down the OS operator-kernel call graph (Step 1). Each leaf
// is a real CUBE kernel composed via `linked_records` (the call
// graph) and tagged with behavior descriptors. `cube-os-tick`
// calls the three leaves in order. Nothing is executed here —
// the native layer later `run`s each and `native-apply`s the
// effect. This is "everything in CUBE" done the spec's way:
// the OS's behavior lives as kernels + call graph + descriptors.
let c = cubecode::C_OS_KERNEL;
let cfg = format!("/c{c}/z001/y001/x001"); // normalize-config: pure
let decide = format!("/c{c}/z001/y001/x002"); // decide-snapshot: io
let summarize = format!("/c{c}/z001/y001/x003"); // summarize-procs: pure+hot
let tick = format!("/c{c}/z001/y001/x004"); // cube-os-tick: hot (calls 0..2)
let cfg_code = vec![Op::Const(7), Op::Const(3), Op::Add, Op::Halt]; // 7+3=10
let decide_code = vec![Op::Const(1), Op::Ret]; // 1 => snapshot
let summarize_code = vec![Op::Const(20), Op::Halt]; // 20 procs
let tick_code = vec![
Op::Const(0),
Op::CallLink(0), // cfg
Op::CallLink(1), // decide
Op::CallLink(2), // summarize
Op::Halt,
];
// Stage the leaves first (we need their coords to build the
// call-graph edges of the root), then the root. `with_mut`
// hands us exclusive `&mut CubeStore` access so the helper can
// write each record through the normal codec.
let cfg_c = store
.with_mut(|s| {
crate::store_code_cell(
s,
&cfg,
Kind::Kernel,
"normalize-config",
&[],
&cfg_code,
None,
Some(cubecode::Behavior(cubecode::Behavior::PURE)),
)
})
.map_err(|e: crate::SysError| e.to_string())?;
let dec_c = store
.with_mut(|s| {
crate::store_code_cell(
s,
&decide,
Kind::Kernel,
"decide-snapshot",
&[],
&decide_code,
None,
Some(cubecode::Behavior(cubecode::Behavior::IO_HEAVY)),
)
})
.map_err(|e: crate::SysError| e.to_string())?;
let sum_c = store
.with_mut(|s| {
crate::store_code_cell(
s,
&summarize,
Kind::Kernel,
"summarize-procs",
&[],
&summarize_code,
None,
Some(cubecode::Behavior(
cubecode::Behavior::PURE | cubecode::Behavior::HOT_PATH,
)),
)
})
.map_err(|e: crate::SysError| e.to_string())?;
let tick_c = store
.with_mut(|s| {
crate::store_code_cell(
s,
&tick,
Kind::Kernel,
"cube-os-tick",
&[cfg_c, dec_c, sum_c], // call graph: tick -> {cfg,decide,summarize}
&tick_code,
None,
Some(cubecode::Behavior(cubecode::Behavior::HOT_PATH)),
)
})
.map_err(|e: crate::SysError| e.to_string())?;
Ok(format!(
"OS kernel call-graph laid down (kind=kernel, in cube c{c}):\n {} normalize-config [pure] -> coord {}\n {} decide-snapshot [io] -> coord {}\n {} summarize-procs [pure,hot] -> coord {}\n {} cube-os-tick [hot] -> coord {} (links cfg,decide,summarize)\nrun e.g.: run {}\nthen effector: native-apply {}",
cfg, cfg_c.pack_u32(), decide, dec_c.pack_u32(), summarize,
sum_c.pack_u32(), tick, tick_c.pack_u32(), tick_c.pack_u32(),
tick
))
}
"native-apply" => {
// Step 2: the thin effector. The decision/computation already
let path = it
.next()
.ok_or_else(|| "native-apply needs <kernel-path>".to_string())?;
let cell = crate::load_code_cell(&store.read_snapshot(), path)
.map_err(|e| format!("native-apply: cannot load {path}: {e}"))?;
if cell.kind() != Kind::Kernel {
return Err(format!(
"native-apply: {path} is kind={:?}, expected a kernel",
cell.kind()
));
}
// Build a throwaway in-memory store holding just this kernel
// (the VM needs a store to walk), then run it. The computation
// is entirely in CUBE; we only read back the computed result.
let coord = crate::path_to_czyx(path).map_err(|e| e.to_string())?;
let mut vm_store = cubestore::CubeStore::new(cubestore::HashBackend::new());
crate::store_code_cell(
&mut vm_store,
path,
Kind::Kernel,
cell.name().unwrap_or(path),
&cell.links().iter().copied().collect::<Vec<_>>(),
&cell.code,
None,
None,
)
.map_err(|e| format!("native-apply: stage failed: {e}"))?;
let mut vm = Vm::new(vm_store);
let result = match vm.run(coord) {
RunResult::Halted { top } => top,
other => {
return Err(format!(
"native-apply: kernel did not halt cleanly: {other:?}"
))
}
};
let r = result.unwrap_or(0);
let effect = match cell.name() {
Some("decide-snapshot") => {
if r != 0 {
format!(
"OS EFFECT: persist runtime snapshot into CUBE (c{} band); decision kernel returned {} => snapshot NOW",
cubecode::C_OS_EFFECT, r
)
} else {
"OS EFFECT: no snapshot (decision kernel returned 0)".to_string()
}
}
_ => format!(
"OS EFFECT: apply computed result {} from kernel {}@{}",
r,
cell.name().unwrap_or("?"),
path
),
};
Ok(format!(
"native-apply {} (kind=kernel, descriptors={:?}):\n computed result = {}\n {}",
path,
cubecode::Behavior::from_flags(
store
.read_snapshot()
.get_record(&coord)
.map(|(h, _)| h.flags.bits())
.unwrap_or(0)
)
.tags(),
r,
effect
))
}
"write" => {
@@ -520,11 +839,11 @@ impl Session {
let owner = self.identity.as_ref().map(|i| i.owner_local.as_str());
let value = {
let mut scratch = CubeStore::new(HashBackend::new());
crate::store_code_cell(&mut scratch, path, Kind::Fn, name, &[], &code, owner)
crate::store_code_cell(&mut scratch, path, Kind::Fn, name, &[], &code, owner, None)
.map_err(|e| e.to_string())?;
scratch.get_raw(&coord).unwrap_or_default()
};
let header = header_for_code(Kind::Fn, name, &code, owner);
let header = header_for_code(Kind::Fn, name, &code, owner, None);
if let Some(txn) = self.txn.as_mut() {
txn.ops.push(TxnOp {
coord,
@@ -536,7 +855,7 @@ impl Session {
));
}
let coord = store
.put_code_cell(path, Kind::Fn, name, &[], &code, owner)
.put_code_cell(path, Kind::Fn, name, &[], &code, owner, None)
.map_err(|e| e.to_string())?;
Ok(format!("wrote {path} -> coord {}", coord.pack_u32()))
}
@@ -950,21 +1269,36 @@ pub fn txn_snapshot(s: &Session) -> CubeStore<HashBackend> {
/// Compute the coordinate a `store_code_cell` call would target, without
/// writing — used to buffer `prog`/`write` mutations during a transaction.
fn scratch_code_coord(path: &str, kind: Kind, name: &str, code: &[Op]) -> Result<Czyx, String> {
fn scratch_code_coord(
path: &str,
kind: Kind,
name: &str,
code: &[Op],
) -> Result<Czyx, String> {
let mut scratch = CubeStore::new(HashBackend::new());
crate::store_code_cell(&mut scratch, path, kind, name, &[], code, None)
crate::store_code_cell(&mut scratch, path, kind, name, &[], code, None, None)
.map_err(|e| e.to_string())
}
/// Build the `CubeHeader` a `store_code_cell` call would attach (mirrors
/// `crate::store_code_cell`), so a buffered txn put carries the same header.
/// `owner` (when set) is stamped on `owner_local_user` for Task 6 enforcement.
fn header_for_code(kind: Kind, name: &str, code: &[Op], owner: Option<&str>) -> CubeHeader {
/// `descriptor` (when set) stamps the behavior-descriptor bits (PDF §524–525).
fn header_for_code(
kind: Kind,
name: &str,
code: &[Op],
owner: Option<&str>,
descriptor: Option<cubecode::Behavior>,
) -> CubeHeader {
let mut h = CubeHeader::new();
h.title = Some(name.to_string());
h.doc_type = Some(kind.as_str().to_string());
h.linked_records = Vec::new();
h.owner_local_user = owner.map(|o| o.to_string());
if let Some(b) = descriptor {
h.flags.0 |= b.to_flags();
}
if h.doc_type.as_deref() == Some("fn") {
h.size_bytes = Some(cubecode::encode(code).len() as u64);
}
@@ -1101,6 +1435,46 @@ mod tests {
Session::with_store(Arc::new(ConcurrentStore::memory()))
}
#[test]
fn os_kernels_live_in_cube_with_call_graph_and_descriptors() {
// Step 1: OS operator behavior lives in CUBE as kernels, composed via
// a call graph (linked_records) and tagged with behavior descriptors.
// Step 2: a thin native effector (`native-apply`) runs each kernel in
// the VM, reads its computed result, and emits the OS effect — the
// cubevm itself never does store-IO (spec-aligned, PDF §524–525).
let mut s = session();
// Lay down the OS kernel call graph.
let out = s.exec("tick").expect("tick should lay down kernels");
assert!(out.contains("normalize-config"), "cfg kernel missing: {out}");
assert!(out.contains("decide-snapshot"), "decide kernel missing: {out}");
assert!(out.contains("summarize-procs"), "summarize kernel missing: {out}");
assert!(out.contains("cube-os-tick"), "tick kernel missing: {out}");
// The root links the three leaves (call graph, not foreign code).
assert!(out.contains("links cfg,decide,summarize"), "call graph not wired: {out}");
// Behavior descriptors are stamped (round-trip through header flags).
assert!(out.contains("[pure]") && out.contains("[io]") && out.contains("[pure,hot]"),
"behavior descriptors not stamped: {out}");
// Run the root kernel: it must traverse the call graph (CallLink 0..2)
// and return, proving the OS's behavior lives as addressable kernels.
let run_out = s.exec("run /c210/z001/y001/x004").expect("tick kernel must run");
assert!(run_out.contains("Halted"), "tick kernel should halt: {run_out}");
// Step 2 — the effector reads the COMPUTED result and emits the effect.
let eff = s.exec("native-apply /c210/z001/y001/x002")
.expect("effector must run decide-snapshot");
assert!(eff.contains("computed result = 1"), "decide kernel result wrong: {eff}");
assert!(eff.contains("OS EFFECT"), "effector must emit OS EFFECT: {eff}");
assert!(eff.contains("snapshot NOW"), "decision=1 should snapshot: {eff}");
// A plain pure kernel also routes through the effector with no store-IO.
let eff2 = s.exec("native-apply /c210/z001/y001/x003")
.expect("effector must run summarize-procs");
assert!(eff2.contains("computed result = 20"), "summarize result wrong: {eff2}");
assert!(eff2.contains("descriptors=[\"pure\", \"hot\"]"), "descriptor readback wrong: {eff2}");
}
#[test]
fn begin_commit_applies_buffered_writes() {
let mut s = session();
cubesys/src/lib.rs
+10
View File
@@ -141,6 +141,10 @@ pub fn store_code_cell<B: CubeBackend>(
links: &[Czyx],
code: &[cubecode::Op],
owner: Option<&str>,
// Behavior-descriptor flags (PDF §524–525: pure / I/O heavy / hot path) to
// stamp on the record header's out-of-band bits. `None` leaves the field
// at zero. See `cubecode::Behavior`.
descriptor: Option<cubecode::Behavior>,
) -> Result<Czyx, SysError> {
let coord = path_to_czyx(path)?;
let mut h = CubeHeader::new();
@@ -148,6 +152,9 @@ pub fn store_code_cell<B: CubeBackend>(
h.doc_type = Some(kind.as_str().to_string());
h.linked_records = links.to_vec();
h.owner_local_user = owner.map(|o| o.to_string());
if let Some(b) = descriptor {
h.flags.0 |= b.to_flags();
}
if h.doc_type.as_deref() == Some("fn") {
h.size_bytes = Some(cubecode::encode(code).len() as u64);
}
@@ -187,6 +194,7 @@ mod tests {
&[],
&[Op::Const(2), Op::Const(3), Op::Add, Op::Halt],
None,
None,
)
.unwrap();
@@ -300,6 +308,7 @@ pub mod demo {
&[],
&double_code,
None,
None,
)
.expect("store double");
let entry_coord = super::store_code_cell(
@@ -310,6 +319,7 @@ pub mod demo {
&[double_coord],
&entry_code,
None,
None,
)
.expect("store entry");
println!(" wrote {double} -> coord {}", double_coord.pack_u32());
cubesys/src/store.rs
+2 -1
View File
@@ -691,9 +691,10 @@ impl ConcurrentStore {
links: &[Czyx],
code: &[Op],
owner: Option<&str>,
descriptor: Option<cubecode::Behavior>,
) -> Result<Czyx, crate::SysError> {
let coord = self.with_mut(|store| {
crate::store_code_cell(store, path, kind, name, links, code, owner)
crate::store_code_cell(store, path, kind, name, links, code, owner, descriptor)
})?;
if let Some(v) = self.get_raw(&coord) {
self.log_put(coord, v);