CUBELinux-2
1539ecd1bb
- cubecode/src/cb.rs: Behavior descriptor bitflag (PURE/IO_HEAVY/HOT_PATH) encoded into HeaderFlags bits 13..15; C_OS_KERNEL=210 / C_OS_EFFECT=211 coordinate bands; round-trip unit test. - fix: HEADER_FLAG_BEHAVIOR mask was 0x7000 (bits 12-14) which grabbed the ENCRYPTED bit (12) and dropped HOT_PATH (bit 15). Corrected to 0xE000. - store_code_cell/put_code_cell/header_for_code gain optional descriptor arg; all 12 prior call sites pass None (total change). - CLI: prog K=<flag> tokens, kernel verb (links+descriptors), tick verb lays down the OS kernel call-graph (cfg->decide->summarize->tick), native-apply verb = thin effector (runs kernel in VM, reads computed result, emits effect). - integration test os_kernels_live_in_cube_with_call_graph_and_descriptors. Proven green via ./check quick (fmt+clippy -D warnings+tests).
197 lines
12 KiB
Markdown
197 lines
12 KiB
Markdown
# CUBELinux-2 — STARTUP README (read this on every boot/resume)
|
||
|
||
> Canonical spec: `/home/CUBELinux/html/files/CUBELinux.pdf`
|
||
> Companion design docs (in `/home/CUBELinux/`): `DESIGN-coordinate-layer.md`,
|
||
> `DESIGN-os-entities.md`, `PLAN-v1.md`, `REVIEW-cubelinux-pdf.md`,
|
||
> `INTEGRATION-cube-agent.md`.
|
||
> Session-history mirror: SQLite `/root/.hermes/state.db` (search messages for
|
||
> "CUBELinux.pdf" / crate names). CUBE `hermes` namespace holds HIST-QNA + MEMORY-NOTE.
|
||
|
||
---
|
||
|
||
## 0. What this project is (one paragraph)
|
||
|
||
CUBELinux is a coordinate-addressed store where **a position in space resolves to
|
||
bytes**. Data is keyed by `Coord { space: SpaceId, point: Point }` (C is a *space
|
||
selector*, not a spatial axis — resolved in PLAN-v1 / DESIGN-os-entities). The
|
||
PDF's vision (lines 1395-1426) is a 3-phase climb:
|
||
|
||
- **Phase 1 (DONE):** user-space Rust daemons/libs on Linux — `cubed` (daemon),
|
||
`cubefs` (FUSE view), `cubec`/`cube` (CLI).
|
||
- **Phase 2 (BUILD TARGET):** CubeFS FUSE filesystem mapping POSIX calls → CZYX,
|
||
so the cube is a *real* filesystem substrate testable under real workloads.
|
||
- **Phase 3 (END GOAL):** OS services (VFS, process accounting, paging, logging,
|
||
tracing) talk directly to the cube store; syscalls like "open by CZYX + flags".
|
||
|
||
The current standing directive (2026-08-13) is to **build CUBELinux *within the
|
||
VM* so it becomes the file store / underlying storage for the kernel and OS** —
|
||
i.e. push from Phase 1 toward Phase 2/3, making the cube the boot/root storage
|
||
substrate, not just an auxiliary `/cubefs` mount.
|
||
|
||
---
|
||
|
||
## 1. Verification points (from the spec, not invented)
|
||
|
||
These are the reference checks to satisfy before claiming the OS-storage binding
|
||
is real. Each must be reproduced live, not asserted.
|
||
|
||
1. **Phase 2 FS correctness** — FUSE mount maps POSIX `open/read/write/rename/
|
||
unlink` to CZYX `put/get/delete` with correct Null-cube metadata (ACLs,
|
||
timestamps, associations). `./check mount` = **57 passed / 0 failed**
|
||
(incl. durable-across-restart). In the VM, `/cubefs` is mounted live and
|
||
read/write/metadata/xattr work via coordinate paths. **STATUS: VERIFIED.**
|
||
2. **Durability under daemon restart** — a FUSE write survives a `cubed` restart
|
||
(WAL + checkpoint). Proven (commit `f40b448`, snapshot
|
||
`fixed-durable-20260813`). Re-verified 2026-08-13: OS state written to
|
||
`/cubefs/c200/...` survived a `systemctl restart cube-server cubefs`. **STATUS: VERIFIED.**
|
||
3. **Phase 3 syscall surface** — "open by CZYX + flags": a record addressable
|
||
directly by coordinate, not path. **SHIPPED 2026-08-13**:
|
||
(a) `cube open <path> <K.Z.Y.X> <tf>` and `cube seal ...` are first-class CLI
|
||
commands in `cubesys/src/bin/cube.rs` (reachable in-process + over the durable
|
||
daemon). Verified: dispatch reaches the crypto/run layer.
|
||
(b) **`.czyx.C.Z.Y.X` FUSE magic-prefix (2026-08-13)** — `cubefs/src/path.rs`
|
||
`parse_dot_czyx` + `cubefs/src/fuse.rs::lookup` intercept: a path component
|
||
`.czyx.200.50.1.7` resolves to the SAME inode as the canonical
|
||
`/cubefs/c200/z050/y001/x007` and reads/writes the real record. This is the
|
||
Phase-3 "open by CZYX" at the filesystem level — the kernel addresses the
|
||
record by coordinate. LIVE VERIFIED: stat both spellings -> identical inode
|
||
3358720263; `cat .czyx.200.50.1.7` returns the record content. Unit tests in
|
||
`path.rs` cover full/partial/rejected forms. **STATUS: DONE.**
|
||
4. **Boot substrate** — the VM brings up CUBELinux as a storage layer the rest of
|
||
the OS reads/writes through. **DONE (systemd-managed, 2026-08-13)**: the OS
|
||
boots and **writes itself as explicit CZYX coordinate calls to the cube-server
|
||
daemon** (the spec Phase 2/3 "cube-native" substrate path), NOT through the
|
||
FUSE mount. Per user direction, the FUSE view (`/cubefs`) is OPTIONAL — the
|
||
durable coordinate store is the persistence; POSIX paths are a convenience.
|
||
- `cube-os-state.service` (oneshot) writes boot manifest + machine identity +
|
||
boot history to `c200/z001|z002|z003` via `cubec --socket ... rawput`.
|
||
- `cube-os-klog.service` streams the kernel ring buffer to `c200/z004` (CZYX).
|
||
- `cube-os-snapshot.service` + `cube-os-snapshot.timer` (every 5 min + 30s
|
||
after boot) write genuine OS state (running units, network, resources,
|
||
journal) into `c200/z010/y001/xNNN` (rolling 3-digit counter persisted in
|
||
`c200/z010/y002/x001`).
|
||
All three units depend ONLY on `cube-server.service` (the daemon socket), not
|
||
on `cubefs.service`, so a FUSE mount failure can never wedge OS bring-up.
|
||
Verified: manifest reads "backing-store: cube-server @ /run/cube/cube.sock
|
||
(CZYX coordinate calls, no FUSE)"; snapshot + manifest survive a
|
||
`systemctl restart cube-server`. **STATUS: DONE (auxiliary layer; not yet the root fs).**
|
||
|
||
---
|
||
|
||
## 2. Current verified state (2026-08-13, refreshed on resume)
|
||
|
||
Host daemons (root/luulu): `cubed` (luulu --user, active+enabled),
|
||
`llama-server` (active), `cube-upload` (active+enabled). Cube socket at
|
||
`/run/user/1000/cubelinux/cubed.sock`.
|
||
|
||
CUBELinux-2 repo (`/home/CUBELinux/CUBELinux-2`): arc complete P1–P5, working
|
||
tree clean, HEAD `f40b448`. Canonical gate: `./check` (uses
|
||
`--features cubefs/mount`).
|
||
|
||
VM (`/var/lib/cubevm/cubelinux-vm.qcow2`, Debian-trixie + Xfce): carries
|
||
`cube-server` (daemon, `/run/cube/cube.sock`), `cubefs-mount`, `cubec`; systemd
|
||
units `cubefs.service`, `cube-server.service`, `cube-resume-pointer.service` all
|
||
enabled. **VM is NOT auto-started on host boot** — bring it up with
|
||
`bash /root/run_vm_window.sh` (visible GTK window, luulu-owned) or
|
||
`/root/run_vm_direct.sh` (serial to file). Snapshot `fixed-durable-20260813`
|
||
captures the working durable stack.
|
||
|
||
`build_vm.sh`: rebuilds cube binaries with `--features cubefs/mount` into a
|
||
clean staging dir before copying (fixes NEXT STEP #1), aborts if `cubefs-mount`
|
||
missing.
|
||
|
||
---
|
||
|
||
## 3. Resume procedure (every startup)
|
||
|
||
1. Read this file + `RESUME-cubefs-daemon.md` + `REPORT-INDEX.md`.
|
||
2. `python3` against `/root/.hermes/state.db` for thread review (sessions table).
|
||
3. CUBE `hermes` = MEMORY-NOTE; HIST-QNA is the per-exchange log (enforced).
|
||
4. Check live state: `systemctl --user -M luulu@ is-active cubed`,
|
||
`systemctl is-active cube-upload`, `pgrep -af qemu`, and SSH guest port 2222.
|
||
5. Decide the next Phase-2/3 step from §1 verification points; build, then
|
||
verify with `./check mount` and a real VM workload.
|
||
|
||
---
|
||
|
||
## 4. Hard rules (from PLAN-v1 / INTEGRATION-cube-agent)
|
||
|
||
- `cube-core` stays `#![no_std]` — the kernel-portability test. Never pull `std`
|
||
into it.
|
||
- Curve = Morton-first behind a `Curve` trait; Hilbert stays in-tree. Curve
|
||
choice decided by `cube-bench` on real NVMe, never by argument.
|
||
- C is a space selector, not an axis. `Coord = { space, point }`; cross-space is
|
||
a `Portal` edge, never arithmetic.
|
||
- CUBE (Rust crates) is canonical. The local LLM agent adapts to CUBE via
|
||
`cube_bridge.py` → `cubecli`/`cubed`, never the reverse.
|
||
- Axis width is `u64`/`u256` (NOT the PDF's `4 × u8` — see REVIEW §2). Tri-channel
|
||
and obfuscation-as-security are discarded (REVIEW §3/§5).
|
||
|
||
---
|
||
|
||
## 5. Open items
|
||
|
||
- Phase 3 key-management flow: **REAL (wired 2026-08-13).** New `cubecrypt::keyinit`
|
||
module (`ensure_os_keystore`, idempotent) mints the OS Null-space key cells
|
||
(default `gcm` @ `c000/z020/y000/x001`, and `xts` @ `c000/z020/y001/x001`) ONCE,
|
||
then reuses them. Exposed as `cube keyinit` / `cubec keyinit` and run at boot by
|
||
the new `cube-os-keyinit.service` (Before= the OS state/snapshot/resume units,
|
||
After=cube-server). The `seal`/`open` CLI now accept `auto` as the key-cell
|
||
argument, resolving to the OS default key — i.e. an OS service can
|
||
`cube seal <C.Z.Y.X> auto <transform>` / `cube open <C.Z.Y.X> auto <transform>`
|
||
with NO human-supplied key, satisfying "open by CZYX + flags unattended".
|
||
Verified live against the daemon (durable store): 1st `keyinit` issued 2 cells,
|
||
2nd issued 0 (idempotent); seal+open round-trip by CZYX+auto succeeds.
|
||
DURABILITY (FIXED 2026-08-13): the daemon store is NOT ephemeral — three fixes
|
||
make key cells + sealed records survive a cold reboot, all verified live via
|
||
`systemctl restart cube-server` (the VM reboot path):
|
||
- Fix A (store.rs): checkpoint boundary now persists `wal.committed_seq`
|
||
(highest fsync'd) not `wal.seq()` (next-to-assign); the old value skipped
|
||
every WAL entry since the last checkpoint → silent data loss on reboot.
|
||
- Fix B (commands.rs `open`): executes the decrypted program in an isolated
|
||
`read_snapshot()` clone instead of `put_raw`ing plaintext over the sealed
|
||
envelope, so re-open after reboot no longer returns `EnvelopeTooShort`.
|
||
- Fix C (commands.rs `keyinit`): the OS key cells are `log_put`-flushed to the
|
||
WAL, so they fold into the base snapshot and the SAME key material decrypts
|
||
after reboot (keyinit #2 issues 0, not 2). Without this, keyinit re-minted
|
||
random material on each boot → every sealed record became unopenable.
|
||
Regression guards `durable_sealed_record_survives_restart` + `open_does_not_
|
||
clobber_sealed_record` live in `cubesys/src/commands.rs`; `./check` is green.
|
||
- Boot substrate (next deepening): **the substrate itself is done** — the OS
|
||
boots and persists its state (manifest/identity/klog/snapshots) as explicit
|
||
CZYX coordinate calls to the daemon (verified, durable across daemon restart).
|
||
Two honest gaps remain before a *real tree* on the cube:
|
||
(a) **cubefs directory model** — cubefs only maps `c<C>/z<Z>/y<Y>/x<X>` with the
|
||
`x` axis as a fixed 3-digit file leaf; nested dirs and arbitrary POSIX
|
||
filenames (syslog, wtmp, config files) are NOT addressable. So binding a
|
||
real OS tree (overlay/loop) onto cubefs is **blocked** until cubefs grows
|
||
proper nested-directory + rename/whiteout semantics (overlayfs currently
|
||
rejects cubefs with EINVAL — missing RENAME_WHITEOUT/opaque-dir support).
|
||
This is a Level-B crate feature, not a systemd change.
|
||
(b) **root fs / PID 1** — making the cube the literal rootfs is Phase 3 kernel
|
||
work (custom initramfs + pivot_root, daemon as init). Heavy, image-bake,
|
||
off-peak only. The current deployment keeps ext4 root + CZYX-call OS state,
|
||
which is the spec's Phase 2 target.
|
||
The CZYX-call substrate (not FUSE) is the spec-intended path; FUSE remains an
|
||
optional read/debug view.
|
||
- IMAGE PROVISIONING: `build_vm.sh` (host, /root/build_vm.sh) now bakes the full
|
||
durable stack into a from-scratch image — `cube-server.service` (daemon),
|
||
`cubefs.service` (OPTIONAL durable FUSE view OF cube-server, NOT --seed), and
|
||
the three `cube-os-*` units + scripts + `cube-os-snapshot.timer`. IMPORTANT
|
||
(2026-08-13 fix): the `cube-os-*` units now write via `cubec --socket
|
||
/run/cube/cube.sock rawput/get` (CZYX calls) and depend ONLY on
|
||
`cube-server.service` — `cubefs.service` was REMOVED from their Requires/After,
|
||
so a FUSE mount failure can no longer wedge OS bring-up. `cubefs.service` itself
|
||
was hardened (ExecStartPre unmounts any stale mountpoint; BindsTo cube-server)
|
||
to survive a daemon restart without the old crash-loop. All enabled in the
|
||
chroot. So a fresh bake IS reproducible; re-running `build_vm.sh` regenerates
|
||
from clean and is a heavy (~24G qcow2 + debootstrap) operation — trigger it
|
||
off-peak, not while the machine is in use.
|
||
- `cube-resume-pointer.service` is REAL (wired 2026-08-13): a oneshot that writes
|
||
the OS's "where to look to continue" into the cube at `c200/z011/y001/x001`
|
||
(last snapshot index + resume coordinate). Was a dead stub before (unit pointed
|
||
at a nonexistent bin). Now deployed live in the VM and baked by build_vm.sh.
|
||
- OPTIONAL (cosmetic): launch the "CUBE Shell" desktop launcher in the VM to
|
||
confirm end-to-end at the file level (wiring already verified:
|
||
cube-term → cubec → /run/cube/cube.sock).
|