Files

T

CUBELinux-2 8f9e7e0025 fix(cubefs): make FUSE mount a durable view of cube-server daemon store

The cubefs-mount --socket path was never actually built: CubeFs<B> is
generic, so the --socket (DaemonBackend) and default (HashBackend) arms
of the match were incompatible types (E0308), and --features mount failed
to compile. The running binary was therefore the in-memory build, so
--socket was silently ignored and every FUSE write went to RAM and never
reached the daemon (rawget/rawkeys returned none / 0 keys, WAL stayed 0).

Fix: type-erase the backend. Add  (forwards to inner) in cubestore, and build
 in cubefs-mount via
Box::new(DaemonBackend::new(p)) / Box::new(HashBackend::new()). Keeps
cubefs free of a cubesys dep (acyclic graph).

Verified end-to-end on host (shared code path as the VM): a FUSE write
via  lands in the daemon store (rawget returns the
record, 5 keys present, WAL grows), and survives a  of the
daemon + relaunch with the same --store (byte-identical read-back).

Also includes (from RESUME-cubefs-daemon.md): cubesys raw* command family
(rawget/rawput/rawdel/rawkeys/rawscan + parse/hex helpers) and the
DaemonBackend client + smoke tests. Report/verification docs added.

Note: VM cubefs.service still mounts in-memory (no --socket); update the
unit to  as a
follow-up so the deployed VM FUSE is durable too.

2026-08-13 05:40:59 -04:00

4.5 KiB

Raw Blame History

RESUME POINT — cubefs FUSE <-> cube-server daemon durability wiring

Saved: 2026-08-13 (user paused session; pick up from here).

GOAL (from task list)

Make cubefs a real FUSE view of the running cube-server daemon's durable store, and prove a FUSE write lands in the daemon WAL and survives a daemon restart. (Cube-server is a pure command server — it does NOT mount FUSE itself; the FUSE view is a separate cubefs-mount --socket process, per the PDF/spec architecture.)

WHAT IS DONE (compiles + tests green)

Raw-coordinate command family added to cubesys/src/commands.rs dispatch: rawget <c> <z> <y> <x>, rawput <c> <z> <y> <x> <hex>, rawdel <c> <z> <y> <x>, rawkeys, rawscan <c> [z] [y]. Helpers added: parse_u8, hex_encode, hex_decode. (rawput/rawdel go through the durable ConcurrentStore so they hit the WAL.)
cubefs/src/backend.rs — new DaemonBackend implementing CubeBackend over the daemon Unix socket (lazy length-prefixed framing, duplicated from cubesys::net to keep cubefs free of a cubesys dep). Registered in cubefs/src/lib.rs (pub mod backend; pub use backend::DaemonBackend;). DaemonBackend is Clone (clones share socket path, reconnect lazily).
cubefs/src/bin/cubefs_mount.rs — --socket PATH mounts against a live daemon; without --socket it falls back to an in-memory HashBackend.
cargo build --workspace GREEN. cargo test --workspace GREEN. (Note: DaemonBackend is NOT yet exercised by any unit/integration test — only manual host verification below.)

WHAT IS PROVEN (host-level, same kernel/code path as the VM)

A. Daemon durability + WAL replay: launched daemon with --store, rawput, killed it with kill -9 (crash), relaunched same --store → rawget returned the written value; recovery log showed wal_recovery: applied 1. So the DAEMON half of the durability story works. B. Direct rawput from a Python client checkpoints to store.json (4-byte record appeared after the checkpoint interval). Confirmed daemon is alive, reachable, and durable on the host.

WHAT IS BROKEN (the open bug) — RESOLVED 2026-08-13

FUSE write did NOT reach the daemon. Root cause was NOT a connection-caching bug (the earlier RefCell cache was already removed in favour of fresh-per-call sockets). The real blocker: cubefs-mount requires --features mount to build, and at that point it did NOT compile (E0308: CubeFs<DaemonBackend> vs CubeFs<HashBackend> are different types in the match &socket). The build was silently failing, so the running binary was the IN-MEMORY build and --socket was ignored. Fixed by type-erasing the backend (Box<dyn CubeBackend + Send + Sync> + a forwarding impl in cubestore). Now compiles and the FUSE mount reaches the daemon; durability across a daemon kill -9 is proven (see VERIFICATION-cubefs-daemon.md, REVISED).

NEXT STEPS (current)

Repeat the single-daemon + single-mount e2e INSIDE the VM to confirm the deployed artifact behaves identically (host proof is on the shared code path). NOTE: the VM's cubefs.service currently launches cubefs-mount /cubefs --seed WITHOUT --socket, so it is in-memory there too — update the unit to cubefs-mount /cubefs --socket /run/cube/cube.sock --seed (and ensure the daemon is ordered Before= cubefs.service) so the VM FUSE is durable.
Add a --features mount build to the VM image's build step and to ./check so the durable mount can't silently regress to in-memory again.
Optionally surface DaemonBackend::put failures as FUSE EIO instead of eprintln-only (best-effort today, acceptable).

KEY FILES

/home/CUBELinux/CUBELinux-2/cubesys/src/commands.rs (raw* commands + helpers)
/home/CUBELinux/CUBELinux-2/cubefs/src/backend.rs (DaemonBackend — FIX HERE)
/home/CUBELinux/CUBELinux-2/cubefs/src/lib.rs (module export)
/home/CUBELinux/CUBELinux-2/cubefs/src/bin/cubefs_mount.rs (--socket wiring)
/home/CUBELinux/CUBELinux-2/cubestore/src/lib.rs (CubeStore::put_raw L358)
/home/CUBELinux/CUBELinux-2/cubefs/src/vfs.rs (CubeFs::write L392)

TEST HARNESS NOTES

Launch daemon/mount as terminal(background=true) so Hermes tracks them.
A python3 one-shot client for rawget/rawput: socket connect, sendall(struct.pack('<I',len(payload))+payload), read 4-byte len then body.
Daemon flags used: --socket S --store STORE.json --recovery-log REC.ndjson --allow-anonymous --checkpoint-ms 400 --wal-fsync-ms 30

4.5 KiB Raw Blame History