fix(cubefs): make FUSE mount a durable view of cube-server daemon store

The cubefs-mount --socket path was never actually built: CubeFs<B> is
generic, so the --socket (DaemonBackend) and default (HashBackend) arms
of the match were incompatible types (E0308), and --features mount failed
to compile. The running binary was therefore the in-memory build, so
--socket was silently ignored and every FUSE write went to RAM and never
reached the daemon (rawget/rawkeys returned none / 0 keys, WAL stayed 0).

Fix: type-erase the backend. Add  (forwards to inner) in cubestore, and build
 in cubefs-mount via
Box::new(DaemonBackend::new(p)) / Box::new(HashBackend::new()). Keeps
cubefs free of a cubesys dep (acyclic graph).

Verified end-to-end on host (shared code path as the VM): a FUSE write
via  lands in the daemon store (rawget returns the
record, 5 keys present, WAL grows), and survives a  of the
daemon + relaunch with the same --store (byte-identical read-back).

Also includes (from RESUME-cubefs-daemon.md): cubesys raw* command family
(rawget/rawput/rawdel/rawkeys/rawscan + parse/hex helpers) and the
DaemonBackend client + smoke tests. Report/verification docs added.

Note: VM cubefs.service still mounts in-memory (no --socket); update the
unit to  as a
follow-up so the deployed VM FUSE is durable too.
This commit is contained in:
CUBELinux-2
2026-08-13 05:40:59 -04:00
parent ad73f42e46
commit 8f9e7e0025
10 changed files with 574 additions and 13 deletions
+77
View File
@@ -0,0 +1,77 @@
# RESUME POINT — cubefs FUSE <-> cube-server daemon durability wiring
Saved: 2026-08-13 (user paused session; pick up from here).
## GOAL (from task list)
Make cubefs a real FUSE view of the running `cube-server` daemon's durable
store, and prove a FUSE write lands in the daemon WAL and survives a daemon
restart. (Cube-server is a pure command server — it does NOT mount FUSE
itself; the FUSE view is a separate `cubefs-mount --socket` process, per the
PDF/spec architecture.)
## WHAT IS DONE (compiles + tests green)
1. Raw-coordinate command family added to `cubesys/src/commands.rs` dispatch:
`rawget <c> <z> <y> <x>`, `rawput <c> <z> <y> <x> <hex>`,
`rawdel <c> <z> <y> <x>`, `rawkeys`, `rawscan <c> [z] [y]`.
Helpers added: `parse_u8`, `hex_encode`, `hex_decode`.
(rawput/rawdel go through the durable `ConcurrentStore` so they hit the WAL.)
2. `cubefs/src/backend.rs` — new `DaemonBackend` implementing `CubeBackend`
over the daemon Unix socket (lazy length-prefixed framing, duplicated from
`cubesys::net` to keep cubefs free of a cubesys dep). Registered in
`cubefs/src/lib.rs` (`pub mod backend; pub use backend::DaemonBackend;`).
`DaemonBackend` is Clone (clones share socket path, reconnect lazily).
3. `cubefs/src/bin/cubefs_mount.rs``--socket PATH` mounts against a live
daemon; without `--socket` it falls back to an in-memory `HashBackend`.
4. `cargo build --workspace` GREEN. `cargo test --workspace` GREEN.
(Note: `DaemonBackend` is NOT yet exercised by any unit/integration test —
only manual host verification below.)
## WHAT IS PROVEN (host-level, same kernel/code path as the VM)
A. Daemon durability + WAL replay: launched daemon with `--store`, `rawput`,
killed it with `kill -9` (crash), relaunched same `--store``rawget`
returned the written value; recovery log showed
`wal_recovery: applied 1`. So the DAEMON half of the durability story works.
B. Direct `rawput` from a Python client checkpoints to `store.json` (4-byte
record appeared after the checkpoint interval). Confirmed daemon is alive,
reachable, and durable on the host.
## WHAT IS BROKEN (the open bug) — RESOLVED 2026-08-13
FUSE write did NOT reach the daemon. Root cause was NOT a connection-caching bug
(the earlier `RefCell` cache was already removed in favour of fresh-per-call
sockets). The real blocker: `cubefs-mount` requires `--features mount` to build,
and at that point it did NOT compile (E0308: `CubeFs<DaemonBackend>` vs
`CubeFs<HashBackend>` are different types in the `match &socket`). The build was
silently failing, so the running binary was the IN-MEMORY build and `--socket`
was ignored. Fixed by type-erasing the backend (`Box<dyn CubeBackend + Send +
Sync>` + a forwarding impl in cubestore). Now compiles and the FUSE mount reaches
the daemon; durability across a daemon `kill -9` is proven (see
VERIFICATION-cubefs-daemon.md, REVISED).
## NEXT STEPS (current)
1. Repeat the single-daemon + single-mount e2e INSIDE the VM to confirm the
deployed artifact behaves identically (host proof is on the shared code path).
NOTE: the VM's `cubefs.service` currently launches `cubefs-mount /cubefs
--seed` WITHOUT `--socket`, so it is in-memory there too — update the unit to
`cubefs-mount /cubefs --socket /run/cube/cube.sock --seed` (and ensure the
daemon is ordered Before= cubefs.service) so the VM FUSE is durable.
2. Add a `--features mount` build to the VM image's build step and to `./check`
so the durable mount can't silently regress to in-memory again.
3. Optionally surface `DaemonBackend::put` failures as FUSE EIO instead of
eprintln-only (best-effort today, acceptable).
## KEY FILES
- /home/CUBELinux/CUBELinux-2/cubesys/src/commands.rs (raw* commands + helpers)
- /home/CUBELinux/CUBELinux-2/cubefs/src/backend.rs (DaemonBackend — FIX HERE)
- /home/CUBELinux/CUBELinux-2/cubefs/src/lib.rs (module export)
- /home/CUBELinux/CUBELinux-2/cubefs/src/bin/cubefs_mount.rs (--socket wiring)
- /home/CUBELinux/CUBELinux-2/cubestore/src/lib.rs (CubeStore::put_raw L358)
- /home/CUBELinux/CUBELinux-2/cubefs/src/vfs.rs (CubeFs::write L392)
## TEST HARNESS NOTES
- Launch daemon/mount as `terminal(background=true)` so Hermes tracks them.
- A `python3` one-shot client for rawget/rawput:
socket connect, sendall(struct.pack('<I',len(payload))+payload),
read 4-byte len then body.
- Daemon flags used:
`--socket S --store STORE.json --recovery-log REC.ndjson --allow-anonymous
--checkpoint-ms 400 --wal-fsync-ms 30`