RESUME POINT — cubefs FUSE <-> cube-server daemon durability wiring

Saved: 2026-08-13 (user paused session; pick up from here).

GOAL (from task list)

Make cubefs a real FUSE view of the running cube-server daemon's durable store, and prove a FUSE write lands in the daemon WAL and survives a daemon restart. (Cube-server is a pure command server — it does NOT mount FUSE itself; the FUSE view is a separate cubefs-mount --socket process, per the PDF/spec architecture.)

WHAT IS DONE (compiles + tests green)

Raw-coordinate command family added to cubesys/src/commands.rs dispatch: rawget <c> <z> <y> <x>, rawput <c> <z> <y> <x> <hex>, rawdel <c> <z> <y> <x>, rawkeys, rawscan <c> [z] [y]. Helpers added: parse_u8, hex_encode, hex_decode. (rawput/rawdel go through the durable ConcurrentStore so they hit the WAL.)
cubefs/src/backend.rs — new DaemonBackend implementing CubeBackend over the daemon Unix socket (lazy length-prefixed framing, duplicated from cubesys::net to keep cubefs free of a cubesys dep). Registered in cubefs/src/lib.rs (pub mod backend; pub use backend::DaemonBackend;). DaemonBackend is Clone (clones share socket path, reconnect lazily).
cubefs/src/bin/cubefs_mount.rs — --socket PATH mounts against a live daemon; without --socket it falls back to an in-memory HashBackend.
cargo build --workspace GREEN. cargo test --workspace GREEN. (Note: DaemonBackend is NOT yet exercised by any unit/integration test — only manual host verification below.)

WHAT IS PROVEN (host-level, same kernel/code path as the VM)

A. Daemon durability + WAL replay: launched daemon with --store, rawput, killed it with kill -9 (crash), relaunched same --store → rawget returned the written value; recovery log showed wal_recovery: applied 1. So the DAEMON half of the durability story works. B. Direct rawput from a Python client checkpoints to store.json (4-byte record appeared after the checkpoint interval). Confirmed daemon is alive, reachable, and durable on the host.

WHAT IS BROKEN (the open bug) — RESOLVED 2026-08-13 (commit `0300def`)

Two separate bugs hid behind the "FUSE write didn't reach the daemon" symptom:

--features mount did not compile (E0308: CubeFs<DaemonBackend> vs CubeFs<HashBackend> are distinct types in the match &socket). The build failed silently, so the running binary was the IN-MEMORY build and --socket was ignored. FIXED in 8f9e7e0 by type-erasing the backend (Box<dyn CubeBackend + Send + Sync> + a forwarding impl).
Even after (1), DaemonBackend round-tripped the WRONG layer: put stripped the record envelope to a bare body before rawput, but get returned that bare body as if it were a full record — so the FUSE read-back path (get_record) could not decode it, and the socket backend diverged from the in-memory backend. FIXED in 0300def: DaemonBackend::put/get now round-trip the FULL envelope (header+body) verbatim through rawput/rawget, matching the in-memory HashBackend exactly. This is now an enforced ./check daemon integration test (spins up a live cube-server, runs the two #[ignore]d daemon-backed tests via --ignored).

GATE STATUS (2026-08-13, final)

./check (fmt+test+clippy -D warnings) GREEN.
./check daemon GREEN — socket-backed record round-trip against a real daemon.
./check mount GREEN — NOW runs the regression suite against BOTH the in-memory --seed mount AND a daemon-backed --socket mount (with its own fresh daemon), plus a durable across restart assertion. 57/57 pass. This is the regression guard exercising the real kernel VFS path for BOTH backends.

COMPLETED THIS SESSION (chronological)

Fixed DaemonBackend envelope round-trip (commit 0300def).
Fixed --features mount compile break (commit 8f9e7e0).
Enforced both via ./check daemon (the two #[ignore]d integration tests).
Host: ./check + ./check daemon + ./check mount all GREEN (mount now covers the daemon-backed path too).
VM (cubelinux-vm.qcow2, was pre-fix): rebuilt cube-server + cubefs-mount from the fixed source; rewired cubefs.service to --socket /run/cube/cube.sock with Requires=cube-server.service; fixed TWO real deployment bugs found only because the new gate exercised the socket path: (a) systemd ordering cycle (cube-server Wanted cubefs, cubefs Required cube-server) — broke the cycle by dropping the cubefs dependency from cube-server.service. (b) ExecStartPost chmod 666 raced the daemon's socket creation under the restart loop and failed the unit — made it tolerant+delayed. Proved a FUSE write survives a daemon restart (WAL persistence) on the VM. Injected SSH key for cube+root to drive the VM. Snapshot fixed-durable-20260813 captures the working durable stack.
./check mount stage hardened (commit da3aa3a): daemon-backed leg + durability assertion; run_fs_tests is now backend-agnostic (self-seeds its fixture).

NEXT STEPS (remaining, low priority)

Add a --features mount build to the VM IMAGE BUILD STEP (the run_vm / build script) so a from-scratch image bake includes the FUSE adapter — the host ./check already covers it; only the image-provisioning script is behind.
Optionally surface DaemonBackend::put failures as FUSE EIO instead of eprintln-only (best-effort today, acceptable).
Visually launch the "CUBE Shell" desktop launcher to confirm end-to-end (file level already verified: cube-term -> cubec -> /run/cube/cube.sock).

KEY FILES

/home/CUBELinux/CUBELinux-2/cubesys/src/commands.rs (raw* commands + helpers)
/home/CUBELinux/CUBELinux-2/cubefs/src/backend.rs (DaemonBackend — FIX HERE)
/home/CUBELinux/CUBELinux-2/cubefs/src/lib.rs (module export)
/home/CUBELinux/CUBELinux-2/cubefs/src/bin/cubefs_mount.rs (--socket wiring)
/home/CUBELinux/CUBELinux-2/cubestore/src/lib.rs (CubeStore::put_raw L358)
/home/CUBELinux/CUBELinux-2/cubefs/src/vfs.rs (CubeFs::write L392)

TEST HARNESS NOTES

Launch daemon/mount as terminal(background=true) so Hermes tracks them.
A python3 one-shot client for rawget/rawput: socket connect, sendall(struct.pack('<I',len(payload))+payload), read 4-byte len then body.
Daemon flags used: --socket S --store STORE.json --recovery-log REC.ndjson --allow-anonymous --checkpoint-ms 400 --wal-fsync-ms 30

6.4 KiB Raw Blame History