CUBELinux-2
f6bd8bd01f
- Fix A (store.rs): checkpoint boundary persists wal.committed_seq (highest fsync'd) instead of wal.seq() (next-to-assign), which skipped all WAL entries since last checkpoint -> silent data loss on reboot. - Fix B (commands.rs open): run decrypted program in isolated read_snapshot() clone instead of put_raw plaintext over sealed envelope (stopped reboot-time EnvelopeTooShort / clobber). - Fix C (commands.rs keyinit): flush OS key cells to WAL via log_put so they fold into base snapshot and survive reboot (keyinit #2 issues 0, not 2); previously re-minted random material each boot -> sealed records unopenable. - Regression guards durable_sealed_record_survives_restart + open_does_not_clobber_sealed_record in cubesys/src/commands.rs. - STARTUP-README: replace stale 'EPHEMERAL across restarts' caveat with the fixed/verified durability note. Verified live: systemctl restart cube-server (VM reboot path) -> sealed record decrypts+executes after reboot; key cell byte-identical; keyinit idempotent.
181 lines
10 KiB
Markdown
181 lines
10 KiB
Markdown
# CUBELinux-2 — STARTUP README (read this on every boot/resume)
|
||
|
||
> Canonical spec: `/home/CUBELinux/html/files/CUBELinux.pdf`
|
||
> Companion design docs (in `/home/CUBELinux/`): `DESIGN-coordinate-layer.md`,
|
||
> `DESIGN-os-entities.md`, `PLAN-v1.md`, `REVIEW-cubelinux-pdf.md`,
|
||
> `INTEGRATION-cube-agent.md`.
|
||
> Session-history mirror: SQLite `/root/.hermes/state.db` (search messages for
|
||
> "CUBELinux.pdf" / crate names). CUBE `hermes` namespace holds HIST-QNA + MEMORY-NOTE.
|
||
|
||
---
|
||
|
||
## 0. What this project is (one paragraph)
|
||
|
||
CUBELinux is a coordinate-addressed store where **a position in space resolves to
|
||
bytes**. Data is keyed by `Coord { space: SpaceId, point: Point }` (C is a *space
|
||
selector*, not a spatial axis — resolved in PLAN-v1 / DESIGN-os-entities). The
|
||
PDF's vision (lines 1395-1426) is a 3-phase climb:
|
||
|
||
- **Phase 1 (DONE):** user-space Rust daemons/libs on Linux — `cubed` (daemon),
|
||
`cubefs` (FUSE view), `cubec`/`cube` (CLI).
|
||
- **Phase 2 (BUILD TARGET):** CubeFS FUSE filesystem mapping POSIX calls → CZYX,
|
||
so the cube is a *real* filesystem substrate testable under real workloads.
|
||
- **Phase 3 (END GOAL):** OS services (VFS, process accounting, paging, logging,
|
||
tracing) talk directly to the cube store; syscalls like "open by CZYX + flags".
|
||
|
||
The current standing directive (2026-08-13) is to **build CUBELinux *within the
|
||
VM* so it becomes the file store / underlying storage for the kernel and OS** —
|
||
i.e. push from Phase 1 toward Phase 2/3, making the cube the boot/root storage
|
||
substrate, not just an auxiliary `/cubefs` mount.
|
||
|
||
---
|
||
|
||
## 1. Verification points (from the spec, not invented)
|
||
|
||
These are the reference checks to satisfy before claiming the OS-storage binding
|
||
is real. Each must be reproduced live, not asserted.
|
||
|
||
1. **Phase 2 FS correctness** — FUSE mount maps POSIX `open/read/write/rename/
|
||
unlink` to CZYX `put/get/delete` with correct Null-cube metadata (ACLs,
|
||
timestamps, associations). `./check mount` = **57 passed / 0 failed**
|
||
(incl. durable-across-restart). In the VM, `/cubefs` is mounted live and
|
||
read/write/metadata/xattr work via coordinate paths. **STATUS: VERIFIED.**
|
||
2. **Durability under daemon restart** — a FUSE write survives a `cubed` restart
|
||
(WAL + checkpoint). Proven (commit `f40b448`, snapshot
|
||
`fixed-durable-20260813`). Re-verified 2026-08-13: OS state written to
|
||
`/cubefs/c200/...` survived a `systemctl restart cube-server cubefs`. **STATUS: VERIFIED.**
|
||
3. **Phase 3 syscall surface** — "open by CZYX + flags": a record addressable
|
||
directly by coordinate, not path. **SHIPPED 2026-08-13**:
|
||
(a) `cube open <path> <K.Z.Y.X> <tf>` and `cube seal ...` are first-class CLI
|
||
commands in `cubesys/src/bin/cube.rs` (reachable in-process + over the durable
|
||
daemon). Verified: dispatch reaches the crypto/run layer.
|
||
(b) **`.czyx.C.Z.Y.X` FUSE magic-prefix (2026-08-13)** — `cubefs/src/path.rs`
|
||
`parse_dot_czyx` + `cubefs/src/fuse.rs::lookup` intercept: a path component
|
||
`.czyx.200.50.1.7` resolves to the SAME inode as the canonical
|
||
`/cubefs/c200/z050/y001/x007` and reads/writes the real record. This is the
|
||
Phase-3 "open by CZYX" at the filesystem level — the kernel addresses the
|
||
record by coordinate. LIVE VERIFIED: stat both spellings -> identical inode
|
||
3358720263; `cat .czyx.200.50.1.7` returns the record content. Unit tests in
|
||
`path.rs` cover full/partial/rejected forms. **STATUS: DONE.**
|
||
4. **Boot substrate** — the VM brings up CUBELinux as a storage layer the rest of
|
||
the OS reads/writes through. **DONE (systemd-managed, 2026-08-13)**: units
|
||
`cube-os-state.service` (oneshot, writes boot manifest + machine identity +
|
||
boot history to `/cubefs/c200/z001|z002|z003`) and `cube-os-klog.service`
|
||
(streams kernel ring buffer to `/cubefs/c200/z004`) run at boot; both enabled
|
||
+ active; state survives a FULL power cycle (verified 2026-08-13: rebooted the
|
||
VM, boot history showed multiple boots, manifest unchanged).
|
||
**ALSO (2026-08-13): real OS operational data** — `cube-os-snapshot.service` +
|
||
`cube-os-snapshot.timer` (every 5 min + 30s after boot) write genuine OS state
|
||
(running units, network, resources, journal) into `/cubefs/c200/z010/y001/xNNN`
|
||
(rolling 3-digit counter, x-axis is u16→3-digit per cubefs path model).
|
||
Verified: snapshot survives daemon restart; the OS wrote a NEW snapshot on a
|
||
fresh boot after a full power cycle; inode == packed CZYX. **STATUS: DONE
|
||
(auxiliary layer; not yet the root fs).**
|
||
|
||
---
|
||
|
||
## 2. Current verified state (2026-08-13, refreshed on resume)
|
||
|
||
Host daemons (root/luulu): `cubed` (luulu --user, active+enabled),
|
||
`llama-server` (active), `cube-upload` (active+enabled). Cube socket at
|
||
`/run/user/1000/cubelinux/cubed.sock`.
|
||
|
||
CUBELinux-2 repo (`/home/CUBELinux/CUBELinux-2`): arc complete P1–P5, working
|
||
tree clean, HEAD `f40b448`. Canonical gate: `./check` (uses
|
||
`--features cubefs/mount`).
|
||
|
||
VM (`/var/lib/cubevm/cubelinux-vm.qcow2`, Debian-trixie + Xfce): carries
|
||
`cube-server` (daemon, `/run/cube/cube.sock`), `cubefs-mount`, `cubec`; systemd
|
||
units `cubefs.service`, `cube-server.service`, `cube-resume-pointer.service` all
|
||
enabled. **VM is NOT auto-started on host boot** — bring it up with
|
||
`bash /root/run_vm_window.sh` (visible GTK window, luulu-owned) or
|
||
`/root/run_vm_direct.sh` (serial to file). Snapshot `fixed-durable-20260813`
|
||
captures the working durable stack.
|
||
|
||
`build_vm.sh`: rebuilds cube binaries with `--features cubefs/mount` into a
|
||
clean staging dir before copying (fixes NEXT STEP #1), aborts if `cubefs-mount`
|
||
missing.
|
||
|
||
---
|
||
|
||
## 3. Resume procedure (every startup)
|
||
|
||
1. Read this file + `RESUME-cubefs-daemon.md` + `REPORT-INDEX.md`.
|
||
2. `python3` against `/root/.hermes/state.db` for thread review (sessions table).
|
||
3. CUBE `hermes` = MEMORY-NOTE; HIST-QNA is the per-exchange log (enforced).
|
||
4. Check live state: `systemctl --user -M luulu@ is-active cubed`,
|
||
`systemctl is-active cube-upload`, `pgrep -af qemu`, and SSH guest port 2222.
|
||
5. Decide the next Phase-2/3 step from §1 verification points; build, then
|
||
verify with `./check mount` and a real VM workload.
|
||
|
||
---
|
||
|
||
## 4. Hard rules (from PLAN-v1 / INTEGRATION-cube-agent)
|
||
|
||
- `cube-core` stays `#![no_std]` — the kernel-portability test. Never pull `std`
|
||
into it.
|
||
- Curve = Morton-first behind a `Curve` trait; Hilbert stays in-tree. Curve
|
||
choice decided by `cube-bench` on real NVMe, never by argument.
|
||
- C is a space selector, not an axis. `Coord = { space, point }`; cross-space is
|
||
a `Portal` edge, never arithmetic.
|
||
- CUBE (Rust crates) is canonical. The local LLM agent adapts to CUBE via
|
||
`cube_bridge.py` → `cubecli`/`cubed`, never the reverse.
|
||
- Axis width is `u64`/`u256` (NOT the PDF's `4 × u8` — see REVIEW §2). Tri-channel
|
||
and obfuscation-as-security are discarded (REVIEW §3/§5).
|
||
|
||
---
|
||
|
||
## 5. Open items
|
||
|
||
- Phase 3 key-management flow: **REAL (wired 2026-08-13).** New `cubecrypt::keyinit`
|
||
module (`ensure_os_keystore`, idempotent) mints the OS Null-space key cells
|
||
(default `gcm` @ `c000/z020/y000/x001`, and `xts` @ `c000/z020/y001/x001`) ONCE,
|
||
then reuses them. Exposed as `cube keyinit` / `cubec keyinit` and run at boot by
|
||
the new `cube-os-keyinit.service` (Before= the OS state/snapshot/resume units,
|
||
After=cube-server). The `seal`/`open` CLI now accept `auto` as the key-cell
|
||
argument, resolving to the OS default key — i.e. an OS service can
|
||
`cube seal <C.Z.Y.X> auto <transform>` / `cube open <C.Z.Y.X> auto <transform>`
|
||
with NO human-supplied key, satisfying "open by CZYX + flags unattended".
|
||
Verified live against the daemon (durable store): 1st `keyinit` issued 2 cells,
|
||
2nd issued 0 (idempotent); seal+open round-trip by CZYX+auto succeeds.
|
||
DURABILITY (FIXED 2026-08-13): the daemon store is NOT ephemeral — three fixes
|
||
make key cells + sealed records survive a cold reboot, all verified live via
|
||
`systemctl restart cube-server` (the VM reboot path):
|
||
- Fix A (store.rs): checkpoint boundary now persists `wal.committed_seq`
|
||
(highest fsync'd) not `wal.seq()` (next-to-assign); the old value skipped
|
||
every WAL entry since the last checkpoint → silent data loss on reboot.
|
||
- Fix B (commands.rs `open`): executes the decrypted program in an isolated
|
||
`read_snapshot()` clone instead of `put_raw`ing plaintext over the sealed
|
||
envelope, so re-open after reboot no longer returns `EnvelopeTooShort`.
|
||
- Fix C (commands.rs `keyinit`): the OS key cells are `log_put`-flushed to the
|
||
WAL, so they fold into the base snapshot and the SAME key material decrypts
|
||
after reboot (keyinit #2 issues 0, not 2). Without this, keyinit re-minted
|
||
random material on each boot → every sealed record became unopenable.
|
||
Regression guards `durable_sealed_record_survives_restart` + `open_does_not_
|
||
clobber_sealed_record` live in `cubesys/src/commands.rs`; `./check` is green.
|
||
- Boot substrate (next deepening): make the cube the OS's *default* storage for a
|
||
real tree — e.g. have a service write `/etc` or `/var/log` operational files
|
||
through the cube by default. Currently the cube holds OS *state* (manifest/
|
||
identity/klog/snapshots) but the root fs is still ext4. The magic-prefix
|
||
"open by CZYX" FUSE passthrough is the natural next step (a path like
|
||
`/cubefs/.czyx/200.1.1.1` resolves directly to the coordinate), and a
|
||
loop/overlay over a cube-backed file would let the OS treat the cube as a real
|
||
block device.
|
||
- IMAGE PROVISIONING: `build_vm.sh` (host, /root/build_vm.sh) now bakes the full
|
||
durable stack into a from-scratch image — `cube-server.service` (daemon),
|
||
`cubefs.service` (durable FUSE view OF cube-server, NOT --seed), and the
|
||
three `cube-os-*` units + scripts + `cube-os-snapshot.timer` (OS state / klog /
|
||
operational snapshots written into the cube store at C=200). All enabled in the
|
||
chroot. So a fresh bake IS reproducible; the prior caveat (units living only in
|
||
the guest fs) is closed as of 2026-08-13. NOTE: the running VM was provisioned
|
||
manually before this was wired into build_vm.sh; re-running `build_vm.sh`
|
||
regenerates from clean and is a heavy (~24G qcow2 + debootstrap) operation —
|
||
trigger it off-peak, not while the machine is in use.
|
||
- `cube-resume-pointer.service` is REAL (wired 2026-08-13): a oneshot that writes
|
||
the OS's "where to look to continue" into the cube at `c200/z011/y001/x001`
|
||
(last snapshot index + resume coordinate). Was a dead stub before (unit pointed
|
||
at a nonexistent bin). Now deployed live in the VM and baked by build_vm.sh.
|
||
- OPTIONAL (cosmetic): launch the "CUBE Shell" desktop launcher in the VM to
|
||
confirm end-to-end at the file level (wiring already verified:
|
||
cube-term → cubec → /run/cube/cube.sock).
|