fix(cubefs): make FUSE mount a durable view of cube-server daemon store
The cubefs-mount --socket path was never actually built: CubeFs<B> is generic, so the --socket (DaemonBackend) and default (HashBackend) arms of the match were incompatible types (E0308), and --features mount failed to compile. The running binary was therefore the in-memory build, so --socket was silently ignored and every FUSE write went to RAM and never reached the daemon (rawget/rawkeys returned none / 0 keys, WAL stayed 0). Fix: type-erase the backend. Add (forwards to inner) in cubestore, and build in cubefs-mount via Box::new(DaemonBackend::new(p)) / Box::new(HashBackend::new()). Keeps cubefs free of a cubesys dep (acyclic graph). Verified end-to-end on host (shared code path as the VM): a FUSE write via lands in the daemon store (rawget returns the record, 5 keys present, WAL grows), and survives a of the daemon + relaunch with the same --store (byte-identical read-back). Also includes (from RESUME-cubefs-daemon.md): cubesys raw* command family (rawget/rawput/rawdel/rawkeys/rawscan + parse/hex helpers) and the DaemonBackend client + smoke tests. Report/verification docs added. Note: VM cubefs.service still mounts in-memory (no --socket); update the unit to as a follow-up so the deployed VM FUSE is durable too.
This commit is contained in:
CUBELinux-2
@@ -0,0 +1,21 @@
|
|||||||
|
# REPORT INDEX — locations of verification/session reports
|
||||||
|
|
||||||
|
Hardfile index (per user 2026-08-13: "keep a hardfile log of each location of
|
||||||
|
these reports and summaries within the session store database"). Each row logs
|
||||||
|
where a report/summary lives on disk AND its coordinate in the session store
|
||||||
|
(CUBE `hermes` namespace), so they can be found later without grepping history.
|
||||||
|
|
||||||
|
Format: `DATE | TOPIC | DISK PATH | CUBE COORD (hermes ns) | NOTES`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
2026-08-13 | cubefs<->cube-server FUSE durability: bug fix + byte-proof write reached daemon | /home/CUBELinux/CUBELinux-2/VERIFICATION-cubefs-daemon.md | hermes: (mirror pending) | record-codec byte dump proving CubeFs write persisted to daemon; DaemonBackend per-call-conn fix
|
||||||
|
2026-08-13 | RESUME POINT cubefs daemon durability task | /home/CUBELinux/CUBELinux-2/RESUME-cubefs-daemon.md | hermes: (mirror pending) | full task state, next steps, key files
|
||||||
|
2026-08-13 | HIST-QNA: user paused session, asked to save resume point | (CUBE hermes only) | hermes:c43ec674e18e24dc3866323d77db316961e42bb74f1581d6cfca7587120558d6:17772457101911163865,8235721494341014670,4908589020773913290 | logged before pause
|
||||||
|
|
||||||
|
## To mirror a disk report into CUBE hermes (so it has a coord):
|
||||||
|
# as luulu, with cubed daemon up:
|
||||||
|
# cd /home/luulu/.cubelinux-agent && export XDG_RUNTIME_DIR=/run/user/1000
|
||||||
|
# python3 -c "import cube_bridge as cb; cb.set_socket('/run/user/1000/cubelinux/cubed.sock'); \
|
||||||
|
# cb.cube_write('hermes','REPORT:<topic>:<date>', open('<disk path>').read())"
|
||||||
|
# then record the returned coord in the table above.
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# RESUME POINT — cubefs FUSE <-> cube-server daemon durability wiring
|
||||||
|
|
||||||
|
Saved: 2026-08-13 (user paused session; pick up from here).
|
||||||
|
|
||||||
|
## GOAL (from task list)
|
||||||
|
Make cubefs a real FUSE view of the running `cube-server` daemon's durable
|
||||||
|
store, and prove a FUSE write lands in the daemon WAL and survives a daemon
|
||||||
|
restart. (Cube-server is a pure command server — it does NOT mount FUSE
|
||||||
|
itself; the FUSE view is a separate `cubefs-mount --socket` process, per the
|
||||||
|
PDF/spec architecture.)
|
||||||
|
|
||||||
|
## WHAT IS DONE (compiles + tests green)
|
||||||
|
1. Raw-coordinate command family added to `cubesys/src/commands.rs` dispatch:
|
||||||
|
`rawget <c> <z> <y> <x>`, `rawput <c> <z> <y> <x> <hex>`,
|
||||||
|
`rawdel <c> <z> <y> <x>`, `rawkeys`, `rawscan <c> [z] [y]`.
|
||||||
|
Helpers added: `parse_u8`, `hex_encode`, `hex_decode`.
|
||||||
|
(rawput/rawdel go through the durable `ConcurrentStore` so they hit the WAL.)
|
||||||
|
2. `cubefs/src/backend.rs` — new `DaemonBackend` implementing `CubeBackend`
|
||||||
|
over the daemon Unix socket (lazy length-prefixed framing, duplicated from
|
||||||
|
`cubesys::net` to keep cubefs free of a cubesys dep). Registered in
|
||||||
|
`cubefs/src/lib.rs` (`pub mod backend; pub use backend::DaemonBackend;`).
|
||||||
|
`DaemonBackend` is Clone (clones share socket path, reconnect lazily).
|
||||||
|
3. `cubefs/src/bin/cubefs_mount.rs` — `--socket PATH` mounts against a live
|
||||||
|
daemon; without `--socket` it falls back to an in-memory `HashBackend`.
|
||||||
|
4. `cargo build --workspace` GREEN. `cargo test --workspace` GREEN.
|
||||||
|
(Note: `DaemonBackend` is NOT yet exercised by any unit/integration test —
|
||||||
|
only manual host verification below.)
|
||||||
|
|
||||||
|
## WHAT IS PROVEN (host-level, same kernel/code path as the VM)
|
||||||
|
A. Daemon durability + WAL replay: launched daemon with `--store`, `rawput`,
|
||||||
|
killed it with `kill -9` (crash), relaunched same `--store` → `rawget`
|
||||||
|
returned the written value; recovery log showed
|
||||||
|
`wal_recovery: applied 1`. So the DAEMON half of the durability story works.
|
||||||
|
B. Direct `rawput` from a Python client checkpoints to `store.json` (4-byte
|
||||||
|
record appeared after the checkpoint interval). Confirmed daemon is alive,
|
||||||
|
reachable, and durable on the host.
|
||||||
|
|
||||||
|
## WHAT IS BROKEN (the open bug) — RESOLVED 2026-08-13
|
||||||
|
FUSE write did NOT reach the daemon. Root cause was NOT a connection-caching bug
|
||||||
|
(the earlier `RefCell` cache was already removed in favour of fresh-per-call
|
||||||
|
sockets). The real blocker: `cubefs-mount` requires `--features mount` to build,
|
||||||
|
and at that point it did NOT compile (E0308: `CubeFs<DaemonBackend>` vs
|
||||||
|
`CubeFs<HashBackend>` are different types in the `match &socket`). The build was
|
||||||
|
silently failing, so the running binary was the IN-MEMORY build and `--socket`
|
||||||
|
was ignored. Fixed by type-erasing the backend (`Box<dyn CubeBackend + Send +
|
||||||
|
Sync>` + a forwarding impl in cubestore). Now compiles and the FUSE mount reaches
|
||||||
|
the daemon; durability across a daemon `kill -9` is proven (see
|
||||||
|
VERIFICATION-cubefs-daemon.md, REVISED).
|
||||||
|
|
||||||
|
## NEXT STEPS (current)
|
||||||
|
1. Repeat the single-daemon + single-mount e2e INSIDE the VM to confirm the
|
||||||
|
deployed artifact behaves identically (host proof is on the shared code path).
|
||||||
|
NOTE: the VM's `cubefs.service` currently launches `cubefs-mount /cubefs
|
||||||
|
--seed` WITHOUT `--socket`, so it is in-memory there too — update the unit to
|
||||||
|
`cubefs-mount /cubefs --socket /run/cube/cube.sock --seed` (and ensure the
|
||||||
|
daemon is ordered Before= cubefs.service) so the VM FUSE is durable.
|
||||||
|
2. Add a `--features mount` build to the VM image's build step and to `./check`
|
||||||
|
so the durable mount can't silently regress to in-memory again.
|
||||||
|
3. Optionally surface `DaemonBackend::put` failures as FUSE EIO instead of
|
||||||
|
eprintln-only (best-effort today, acceptable).
|
||||||
|
|
||||||
|
## KEY FILES
|
||||||
|
- /home/CUBELinux/CUBELinux-2/cubesys/src/commands.rs (raw* commands + helpers)
|
||||||
|
- /home/CUBELinux/CUBELinux-2/cubefs/src/backend.rs (DaemonBackend — FIX HERE)
|
||||||
|
- /home/CUBELinux/CUBELinux-2/cubefs/src/lib.rs (module export)
|
||||||
|
- /home/CUBELinux/CUBELinux-2/cubefs/src/bin/cubefs_mount.rs (--socket wiring)
|
||||||
|
- /home/CUBELinux/CUBELinux-2/cubestore/src/lib.rs (CubeStore::put_raw L358)
|
||||||
|
- /home/CUBELinux/CUBELinux-2/cubefs/src/vfs.rs (CubeFs::write L392)
|
||||||
|
|
||||||
|
## TEST HARNESS NOTES
|
||||||
|
- Launch daemon/mount as `terminal(background=true)` so Hermes tracks them.
|
||||||
|
- A `python3` one-shot client for rawget/rawput:
|
||||||
|
socket connect, sendall(struct.pack('<I',len(payload))+payload),
|
||||||
|
read 4-byte len then body.
|
||||||
|
- Daemon flags used:
|
||||||
|
`--socket S --store STORE.json --recovery-log REC.ndjson --allow-anonymous
|
||||||
|
--checkpoint-ms 400 --wal-fsync-ms 30`
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# VERIFICATION: cubefs FUSE view <-> cube-server daemon durability (REVISED 2026-08-13)
|
||||||
|
|
||||||
|
Hard record for later review. Date: 2026-08-13 (session after restart).
|
||||||
|
|
||||||
|
## Critical correction to the prior version of this file
|
||||||
|
The earlier "proof" in this file was measured against `CubeFs` DIRECTLY
|
||||||
|
(`cubefs_daemon_smoke` test calling `fs.create`+`fs.write` on a `CubeFs<DaemonBackend>`),
|
||||||
|
NOT against the actual `cubefs-mount --socket` FUSE binary. The FUSE binary
|
||||||
|
(`cubefs-mount`) requires `--features mount` to build, and at that time it did
|
||||||
|
NOT compile: `CubeFs<B>` is generic, so the `--socket` (DaemonBackend) and
|
||||||
|
default (HashBackend) arms were incompatible types (E0308). The build was
|
||||||
|
silently failing, so the running `cubefs-mount` was the IN-MEMORY build — which
|
||||||
|
is why `--socket` was ignored and every FUSE write went to RAM and never reached
|
||||||
|
the daemon (rawget/rawkeys returned none / 0 keys, WAL stayed 0 bytes). The
|
||||||
|
prior "proof" therefore did NOT verify the FUSE mount.
|
||||||
|
|
||||||
|
## The actual bug that blocked durable FUSE (now FIXED)
|
||||||
|
- `cubefs/src/bin/cubefs_mount.rs`: the `match &socket` produced
|
||||||
|
`CubeFs<DaemonBackend>` vs `CubeFs<HashBackend>` — incompatible; `--features mount`
|
||||||
|
failed to compile (E0308), so the durable mount was never shippable.
|
||||||
|
- Fix: type-erase the backend. Added `impl CubeBackend for Box<dyn CubeBackend +
|
||||||
|
Send + Sync>` (forwards to inner) in `cubestore/src/lib.rs`, and changed the
|
||||||
|
mount to build `CubeFs<Box<dyn CubeBackend + Send + Sync>>` via
|
||||||
|
`Box::new(DaemonBackend::new(p))` / `Box::new(HashBackend::new())`. Both arms
|
||||||
|
are now the same concrete type; `--features mount` compiles.
|
||||||
|
- This keeps `cubefs` free of a `cubesys` dependency (DaemonBackend duplicates
|
||||||
|
the framing locally), so the dependency graph stays acyclic.
|
||||||
|
|
||||||
|
## Proof the FUSE mount reaches the daemon AND survives a crash (REAL, this run)
|
||||||
|
Harness: `bash /tmp/verify_cubefs_daemon_host.sh` (daemon + one mount, both on
|
||||||
|
the same socket, then `kill -9` daemon and relaunch).
|
||||||
|
|
||||||
|
1. `mkdir -p /tmp/verify/mnt/c012/z003/y004`
|
||||||
|
2. `echo cubefs-e2e-proof-<ts> > /tmp/verify/mnt/c012/z003/y004/x007`
|
||||||
|
3. FUSE read-back returns `cubefs-e2e-proof-<ts>` (write visible through FUSE).
|
||||||
|
4. Daemon `rawget 12 3 4 7` -> `ok: <record hex>`; `rawkeys` -> `ok: 5 keys`
|
||||||
|
(volume meta + the new record landed in the daemon store).
|
||||||
|
5. `cube-store.json.wal` grew to **956 bytes** (durable WAL written).
|
||||||
|
6. `kill -9` the daemon, relaunch with the same `--store`.
|
||||||
|
7. Post-crash `rawget 12 3 4 7` returns the SAME record bytes; post-crash FUSE
|
||||||
|
read-back returns `cubefs-e2e-proof-<ts>`. **Survived the daemon crash.**
|
||||||
|
|
||||||
|
Decoded body of the rawget record: `cubefs-e2e-proof-1786613763` — byte-exact
|
||||||
|
match to what was written through the FUSE mount.
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
cubefs is now a genuine FUSE view of the running `cube-server` daemon's durable
|
||||||
|
store, and a FUSE write lands in the daemon WAL and survives a daemon restart.
|
||||||
|
The "cubefs as boot-time data root" integration is functionally proven on the
|
||||||
|
host (shared code path as the VM). Next: repeat the same single-daemon +
|
||||||
|
single-mount e2e INSIDE the VM to confirm the deployed artifact, then wire
|
||||||
|
`--socket` into the VM's `cubefs.service` (currently it mounts without
|
||||||
|
`--socket`, i.e. in-memory — that is why the VM FUSE looked like it worked but
|
||||||
|
never persisted to the daemon).
|
||||||
|
|
||||||
|
## Test files
|
||||||
|
- cubefs/tests/cubefs_daemon_smoke.rs (CubeFs+DaemonBackend, needs live daemon via CUBE_SOCK)
|
||||||
|
- cubefs/tests/daemon_backend_smoke.rs (DaemonBackend put/get/keys, needs live daemon)
|
||||||
|
- /tmp/verify_cubefs_daemon_host.sh (full FUSE mount e2e on host)
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
//! `DaemonBackend` — a [`CubeBackend`] that talks to a running `cube-server`
|
||||||
|
//! over its Unix-domain socket.
|
||||||
|
//!
|
||||||
|
//! This is what makes cubefs a *real* FUSE view of the daemon's durable store
|
||||||
|
//! (PDF Package 3: "cubefs: optional FUSE filesystem view so cube records
|
||||||
|
//! appear as files/directories"), instead of an isolated in-memory map that
|
||||||
|
//! loses every write on unmount. The daemon serves the raw coordinate
|
||||||
|
//! commands (`rawget`/`rawput`/`rawdel`/`rawkeys`/`rawscan`) over the same
|
||||||
|
//! text protocol it uses for `cubec`; `DaemonBackend` is the matching client.
|
||||||
|
//!
|
||||||
|
//! Wire format: length-prefixed UTF-8 frames (see `cubesys::net`). We reuse
|
||||||
|
//! the framing by hand so this crate stays free of any cubesys dependency —
|
||||||
|
//! that keeps the dependency graph acyclic (cubefs is already depended on by
|
||||||
|
//! cubesys, so adding a cubesys dep here would create the very cycle we are
|
||||||
|
//! avoiding).
|
||||||
|
|
||||||
|
use cubecoords::Czyx;
|
||||||
|
use cubestore::CubeBackend;
|
||||||
|
use std::io::{Read, Write};
|
||||||
|
use std::os::unix::net::UnixStream;
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
/// A [`CubeBackend`] backed by a `cube-server` Unix socket.
|
||||||
|
///
|
||||||
|
/// Cheaply cloneable: each clone opens its own connection on first use, so
|
||||||
|
/// the backend can be shared across the FUSE worker threads without a global
|
||||||
|
/// mutex on the socket (the daemon already serializes per-command internally
|
||||||
|
/// via its store lock).
|
||||||
|
pub struct DaemonBackend {
|
||||||
|
socket: PathBuf,
|
||||||
|
connect_timeout: Duration,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Clone for DaemonBackend {
|
||||||
|
/// Clones share the socket path but NOT any open connection — each clone
|
||||||
|
/// opens its own connection per call, so FUSE worker threads never contend
|
||||||
|
/// on one shared `UnixStream` (which is not `Clone` anyway).
|
||||||
|
fn clone(&self) -> Self {
|
||||||
|
DaemonBackend {
|
||||||
|
socket: self.socket.clone(),
|
||||||
|
connect_timeout: self.connect_timeout,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DaemonBackend {
|
||||||
|
/// Connect to the daemon at `socket`. The connection is established lazily
|
||||||
|
/// on the first `get`/`put`/`delete` (so construction never blocks).
|
||||||
|
pub fn new(socket: impl Into<PathBuf>) -> Self {
|
||||||
|
DaemonBackend {
|
||||||
|
socket: socket.into(),
|
||||||
|
connect_timeout: Duration::from_secs(5),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The default daemon socket path (`$XDG_RUNTIME_DIR/cube/cube.sock`).
|
||||||
|
pub fn default_socket() -> PathBuf {
|
||||||
|
if let Ok(r) = std::env::var("XDG_RUNTIME_DIR") {
|
||||||
|
PathBuf::from(r).join("cube/cube.sock")
|
||||||
|
} else {
|
||||||
|
PathBuf::from("/run/cube/cube.sock")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn connect(&self) -> std::io::Result<UnixStream> {
|
||||||
|
let s = UnixStream::connect(&self.socket)?;
|
||||||
|
s.set_read_timeout(Some(Duration::from_secs(10)))?;
|
||||||
|
s.set_write_timeout(Some(self.connect_timeout))?;
|
||||||
|
Ok(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn rpc(&self, cmd: &str) -> Result<String, String> {
|
||||||
|
// Open a FRESH connection per call. A cached stream goes stale the
|
||||||
|
// instant the daemon restarts (kill -9 / crash / redeploy): the socket
|
||||||
|
// is still `Some` but dead, so every later write fails silently and
|
||||||
|
// FUSE reports success while nothing reaches the store. That was the
|
||||||
|
// bug. The daemon already serializes per command, so per-call
|
||||||
|
// connections are correct and contention-free.
|
||||||
|
let mut s = self
|
||||||
|
.connect()
|
||||||
|
.map_err(|e| format!("daemon socket {cmd}: {e}"))?;
|
||||||
|
write_frame(&mut s, cmd).map_err(|e| format!("daemon socket {cmd}: {e}"))?;
|
||||||
|
read_frame(&mut s).map_err(|e| format!("daemon socket {cmd}: {e}"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CubeBackend for DaemonBackend {
|
||||||
|
fn put(&mut self, key: Czyx, value: Vec<u8>) {
|
||||||
|
let hex = to_hex(&value);
|
||||||
|
let cmd = format!(
|
||||||
|
"rawput {} {} {} {} {}",
|
||||||
|
key.c, key.z, key.y, key.x, hex
|
||||||
|
);
|
||||||
|
// A failed durability write is reported via stderr; the FUSE layer
|
||||||
|
// surfaces the prior successful state to the kernel. We do not panic
|
||||||
|
// here because an unreachable daemon should not crash the mount — it
|
||||||
|
// should surface as a write error to the caller (best-effort here).
|
||||||
|
if let Err(e) = self.rpc(&cmd) {
|
||||||
|
eprintln!("cubefs: rawput {key:?} failed: {e}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn get(&self, key: &Czyx) -> Option<Vec<u8>> {
|
||||||
|
let cmd = format!("rawget {} {} {} {}", key.c, key.z, key.y, key.x);
|
||||||
|
match self.rpc(&cmd) {
|
||||||
|
Ok(resp) => {
|
||||||
|
let resp = resp.trim();
|
||||||
|
if let Some(hex) = resp.strip_prefix("ok: ") {
|
||||||
|
from_hex(hex)
|
||||||
|
} else {
|
||||||
|
// "none" or an error reply => absent.
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("cubefs: rawget {key:?} failed: {e}");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn delete(&mut self, key: &Czyx) {
|
||||||
|
let cmd = format!("rawdel {} {} {} {}", key.c, key.z, key.y, key.x);
|
||||||
|
if let Err(e) = self.rpc(&cmd) {
|
||||||
|
eprintln!("cubefs: rawdel {key:?} failed: {e}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn keys(&self) -> Vec<Czyx> {
|
||||||
|
match self.rpc("rawkeys") {
|
||||||
|
Ok(resp) => parse_key_list(&resp),
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("cubefs: rawkeys failed: {e}");
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn scan_prefix(&self, c: u8, z: Option<u8>, y: Option<u8>) -> Vec<Czyx> {
|
||||||
|
let cmd = match (z, y) {
|
||||||
|
(Some(z), Some(y)) => format!("rawscan {c} {z} {y}"),
|
||||||
|
(Some(z), None) => format!("rawscan {c} {z}"),
|
||||||
|
_ => format!("rawscan {c}"),
|
||||||
|
};
|
||||||
|
match self.rpc(&cmd) {
|
||||||
|
Ok(resp) => parse_key_list(&resp),
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("cubefs: rawscan {c} {z:?} {y:?} failed: {e}");
|
||||||
|
Vec::new()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Parse the `ok: N keys\n<space-separated pack_u32 list>` reply into
|
||||||
|
/// [`Czyx`] coordinates. Tolerant of the `ok: N keys` prefix being absent.
|
||||||
|
fn parse_key_list(resp: &str) -> Vec<Czyx> {
|
||||||
|
let body = resp.trim();
|
||||||
|
// Drop the "ok: N keys" summary line if present; the payload is the rest.
|
||||||
|
let payload = match body.split_once('\n') {
|
||||||
|
Some((head, rest)) if head.starts_with("ok:") => rest,
|
||||||
|
_ => body,
|
||||||
|
};
|
||||||
|
payload
|
||||||
|
.split_whitespace()
|
||||||
|
.filter_map(|tok| tok.parse::<u32>().ok())
|
||||||
|
.map(Czyx::unpack_u32)
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- local copies of the length-prefixed framing used by cubesys::net ---
|
||||||
|
// Duplicated (not imported) to keep cubefs free of a cubesys dependency.
|
||||||
|
|
||||||
|
fn write_frame<W: Write>(w: &mut W, payload: &str) -> std::io::Result<()> {
|
||||||
|
let bytes = payload.as_bytes();
|
||||||
|
w.write_all(&(bytes.len() as u32).to_le_bytes())?;
|
||||||
|
w.write_all(bytes)?;
|
||||||
|
w.flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_frame<R: Read>(r: &mut R) -> std::io::Result<String> {
|
||||||
|
let mut len_buf = [0u8; 4];
|
||||||
|
r.read_exact(&mut len_buf)?;
|
||||||
|
let len = u32::from_le_bytes(len_buf) as usize;
|
||||||
|
let mut buf = vec![0u8; len];
|
||||||
|
r.read_exact(&mut buf)?;
|
||||||
|
String::from_utf8(buf).map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn to_hex(b: &[u8]) -> String {
|
||||||
|
let mut s = String::with_capacity(b.len() * 2);
|
||||||
|
for byte in b {
|
||||||
|
s.push_str(&format!("{byte:02x}"));
|
||||||
|
}
|
||||||
|
s
|
||||||
|
}
|
||||||
|
|
||||||
|
fn from_hex(s: &str) -> Option<Vec<u8>> {
|
||||||
|
if s.len() % 2 != 0 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let bytes = s.as_bytes();
|
||||||
|
let mut out = Vec::with_capacity(s.len() / 2);
|
||||||
|
let mut i = 0;
|
||||||
|
while i < bytes.len() {
|
||||||
|
let hi = (bytes[i] as char).to_digit(16)?;
|
||||||
|
let lo = (bytes[i + 1] as char).to_digit(16)?;
|
||||||
|
out.push(((hi << 4) | lo) as u8);
|
||||||
|
i += 2;
|
||||||
|
}
|
||||||
|
Some(out)
|
||||||
|
}
|
||||||
@@ -1,25 +1,31 @@
|
|||||||
//! `cubefs-mount` — mount a cube as a POSIX filesystem.
|
//! `cubefs-mount` — mount a cube as a POSIX filesystem.
|
||||||
//!
|
//!
|
||||||
//! Usage: `cubefs-mount <mountpoint> [--label NAME] [--seed] [--allow-other]`
|
//! Usage:
|
||||||
|
//! cubefs-mount <mountpoint> [--label NAME] [--seed] [--allow-other]
|
||||||
|
//! [--socket PATH]
|
||||||
|
//!
|
||||||
|
//! `--socket PATH` mounts the **daemon's** durable store over a Unix socket
|
||||||
|
//! (see `cubefs::DaemonBackend`). A write through the FUSE mount then lands in
|
||||||
|
//! the daemon's WAL and survives an unmount — or a daemon restart — exactly as
|
||||||
|
//! the PDF's "cubefs is a FUSE view of the cube" intends. Without `--socket`
|
||||||
|
//! the mount uses the in-memory `HashBackend` (ephemeral; every write is lost
|
||||||
|
//! on unmount), which is still useful for namespace-mapping tests and for
|
||||||
|
//! machines with no `cube-server` running.
|
||||||
//!
|
//!
|
||||||
//! `--allow-other` lets users other than the mounting user reach the
|
//! `--allow-other` lets users other than the mounting user reach the
|
||||||
//! filesystem. Without it the kernel rejects them at the mountpoint before any
|
//! filesystem. Without it the kernel rejects them at the mountpoint before any
|
||||||
//! request reaches us, so multi-user ACL behaviour cannot be observed.
|
//! request reaches us, so multi-user ACL behaviour cannot be observed.
|
||||||
//!
|
|
||||||
//! The backing store is the in-memory [`HashBackend`] for now: Package 3's job
|
|
||||||
//! is the *namespace mapping*, and a durable on-disk backend is a cubestore
|
|
||||||
//! concern that gets swapped in by changing one type parameter here. `--seed`
|
|
||||||
//! populates a few records so the mount has something to `ls`.
|
|
||||||
|
|
||||||
use cubefs::fuse::CubeFuse;
|
use cubefs::{CubeFs, DaemonBackend};
|
||||||
use cubefs::CubeFs;
|
use cubestore::{CubeBackend, CubeStore, HashBackend};
|
||||||
use cubestore::{CubeStore, HashBackend};
|
|
||||||
use std::process::ExitCode;
|
use std::process::ExitCode;
|
||||||
|
|
||||||
fn main() -> ExitCode {
|
fn main() -> ExitCode {
|
||||||
let args: Vec<String> = std::env::args().collect();
|
let args: Vec<String> = std::env::args().collect();
|
||||||
let Some(mountpoint) = args.get(1).filter(|a| !a.starts_with("--")) else {
|
let Some(mountpoint) = args.get(1).filter(|a| !a.starts_with("--")) else {
|
||||||
eprintln!("usage: cubefs-mount <mountpoint> [--label NAME] [--seed]");
|
eprintln!(
|
||||||
|
"usage: cubefs-mount <mountpoint> [--label NAME] [--seed] [--socket PATH] [--allow-other]"
|
||||||
|
);
|
||||||
return ExitCode::from(2);
|
return ExitCode::from(2);
|
||||||
};
|
};
|
||||||
let label = args
|
let label = args
|
||||||
@@ -30,8 +36,29 @@ fn main() -> ExitCode {
|
|||||||
.unwrap_or_else(|| "cube0".to_string());
|
.unwrap_or_else(|| "cube0".to_string());
|
||||||
let seed = args.iter().any(|a| a == "--seed");
|
let seed = args.iter().any(|a| a == "--seed");
|
||||||
let allow_other = args.iter().any(|a| a == "--allow-other");
|
let allow_other = args.iter().any(|a| a == "--allow-other");
|
||||||
|
let socket = args
|
||||||
|
.iter()
|
||||||
|
.position(|a| a == "--socket")
|
||||||
|
.and_then(|i| args.get(i + 1).cloned());
|
||||||
|
|
||||||
let mut fs = CubeFs::new(CubeStore::new(HashBackend::new()));
|
// Backing store: the daemon's durable store (over a socket) when `--socket`
|
||||||
|
// is given, otherwise an ephemeral in-memory map. Both are type-erased to
|
||||||
|
// `Box<dyn CubeBackend + Send + Sync>` so `CubeFs` has a single concrete
|
||||||
|
// type regardless of which backend was chosen at runtime.
|
||||||
|
let mut fs: CubeFs<Box<dyn CubeBackend + Send + Sync>> = match &socket {
|
||||||
|
Some(path) => {
|
||||||
|
eprintln!("cubefs: mounting daemon store at socket {path}");
|
||||||
|
CubeFs::new(CubeStore::new(
|
||||||
|
Box::new(DaemonBackend::new(path.clone())) as Box<dyn CubeBackend + Send + Sync>,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
None => {
|
||||||
|
eprintln!("cubefs: mounting IN-MEMORY store (writes are NOT durable; use --socket PATH to mount the daemon's store)");
|
||||||
|
CubeFs::new(CubeStore::new(
|
||||||
|
Box::new(HashBackend::new()) as Box<dyn CubeBackend + Send + Sync>,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
};
|
||||||
fs.format(&label);
|
fs.format(&label);
|
||||||
|
|
||||||
if seed {
|
if seed {
|
||||||
@@ -73,9 +100,10 @@ fn main() -> ExitCode {
|
|||||||
opts.push(fuser::MountOption::AllowOther);
|
opts.push(fuser::MountOption::AllowOther);
|
||||||
}
|
}
|
||||||
eprintln!(
|
eprintln!(
|
||||||
"mounting cubefs at {mountpoint} (label={label}, seed={seed}, allow_other={allow_other}) — ctrl-c to unmount"
|
"mounting cubefs at {mountpoint} (label={label}, socket={:?}, seed={seed}, allow_other={allow_other}) — ctrl-c to unmount",
|
||||||
|
socket
|
||||||
);
|
);
|
||||||
match fuser::mount2(CubeFuse::new(fs), mountpoint, &opts) {
|
match fuser::mount2(cubefs::fuse::CubeFuse::new(fs), mountpoint, &opts) {
|
||||||
Ok(()) => ExitCode::SUCCESS,
|
Ok(()) => ExitCode::SUCCESS,
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
eprintln!("mount failed: {e}");
|
eprintln!("mount failed: {e}");
|
||||||
|
|||||||
@@ -39,6 +39,7 @@
|
|||||||
#![forbid(unsafe_code)]
|
#![forbid(unsafe_code)]
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
|
|
||||||
|
pub mod backend;
|
||||||
pub mod nullspace;
|
pub mod nullspace;
|
||||||
pub mod path;
|
pub mod path;
|
||||||
pub mod vfs;
|
pub mod vfs;
|
||||||
@@ -46,6 +47,7 @@ pub mod vfs;
|
|||||||
#[cfg(feature = "mount")]
|
#[cfg(feature = "mount")]
|
||||||
pub mod fuse;
|
pub mod fuse;
|
||||||
|
|
||||||
|
pub use backend::DaemonBackend;
|
||||||
pub use nullspace::{Acl, JournalEntry, JournalOp, NullSpace, VolumeMeta};
|
pub use nullspace::{Acl, JournalEntry, JournalOp, NullSpace, VolumeMeta};
|
||||||
pub use path::{czyx_to_ino, ino_to_czyx, parse_path, render_path, PathError, ROOT_INO};
|
pub use path::{czyx_to_ino, ino_to_czyx, parse_path, render_path, PathError, ROOT_INO};
|
||||||
pub use vfs::{Attr, CubeFs, FsError, Kind};
|
pub use vfs::{Attr, CubeFs, FsError, Kind};
|
||||||
|
|||||||
@@ -0,0 +1,25 @@
|
|||||||
|
// Ad-hoc smoke test: CubeFs + DaemonBackend (NO FUSE) against a live daemon.
|
||||||
|
// Bisects whether the FUSE layer or the cubefs/cubestore layer is the culprit.
|
||||||
|
// Run: CUBE_SOCK=<sock> cargo test --test cubefs_daemon_smoke -- --nocapture
|
||||||
|
use cubecoords::Czyx;
|
||||||
|
use cubefs::{CubeFs, DaemonBackend};
|
||||||
|
use cubestore::{CubeBackend, CubeStore};
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn cubefs_create_write_reaches_daemon() {
|
||||||
|
let sock = std::env::var("CUBE_SOCK").expect("set CUBE_SOCK");
|
||||||
|
let mut fs = CubeFs::new(CubeStore::new(DaemonBackend::new(sock)));
|
||||||
|
fs.format("cube0");
|
||||||
|
let path = "/c011/z007/y003/x009";
|
||||||
|
fs.create(path, 0, 0, 0o644).expect("create");
|
||||||
|
fs.write(path, 0, b"fuse-proof-XYZ", 0, 0).expect("write");
|
||||||
|
// Now ask the daemon directly via a second backend.
|
||||||
|
let mut probe = DaemonBackend::new(std::env::var("CUBE_SOCK").unwrap());
|
||||||
|
let k = Czyx::new(11, 7, 3, 9);
|
||||||
|
let got = probe.get(&k);
|
||||||
|
println!("daemon-side get(11,7,3,9) => {:?}", got);
|
||||||
|
assert_eq!(got.as_deref(), Some(&b"fuse-proof-XYZ"[..]), "CubeFs write must reach daemon store");
|
||||||
|
let keys = probe.keys();
|
||||||
|
println!("daemon keys => {:?}", keys);
|
||||||
|
assert!(keys.contains(&k));
|
||||||
|
}
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
// Ad-hoc smoke test for DaemonBackend end-to-end against a live cube-server.
|
||||||
|
// Run with: cargo test --test daemon_backend_smoke -- --nocapture
|
||||||
|
// Requires CUBE_SOCK env var pointing at a running cube-server socket.
|
||||||
|
use cubecoords::Czyx;
|
||||||
|
use cubefs::DaemonBackend;
|
||||||
|
use cubestore::CubeBackend;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn daemon_backend_put_get_roundtrip() {
|
||||||
|
let sock = std::env::var("CUBE_SOCK").expect("set CUBE_SOCK to a live cube-server socket");
|
||||||
|
let mut b = DaemonBackend::new(sock);
|
||||||
|
let k = Czyx::new(9, 2, 2, 1);
|
||||||
|
b.put(k, b"fuse-persist-proof".to_vec());
|
||||||
|
let got = b.get(&k);
|
||||||
|
println!("DaemonBackend::get(9,2,2,1) => {:?}", got);
|
||||||
|
assert_eq!(got.as_deref(), Some(&b"fuse-persist-proof"[..]), "daemon backend put/get roundtrip");
|
||||||
|
// also exercise keys()
|
||||||
|
let keys = b.keys();
|
||||||
|
println!("DaemonBackend::keys() => {:?}", keys);
|
||||||
|
assert!(keys.contains(&k), "keys() should include coord after put");
|
||||||
|
}
|
||||||
@@ -99,6 +99,28 @@ impl CubeBackend for HashBackend {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Type-erased backend: lets `CubeFs`/`CubeStore` hold either a `HashBackend`
|
||||||
|
/// or a `DaemonBackend` behind one concrete type, so callers (e.g.
|
||||||
|
/// `cubefs-mount`, which chooses the backend at runtime from `--socket`) don't
|
||||||
|
/// have to be generic over `B`. Forwards every call to the inner backend.
|
||||||
|
impl CubeBackend for Box<dyn CubeBackend + Send + Sync> {
|
||||||
|
fn put(&mut self, key: Czyx, value: Vec<u8>) {
|
||||||
|
(**self).put(key, value)
|
||||||
|
}
|
||||||
|
fn get(&self, key: &Czyx) -> Option<Vec<u8>> {
|
||||||
|
(**self).get(key)
|
||||||
|
}
|
||||||
|
fn delete(&mut self, key: &Czyx) {
|
||||||
|
(**self).delete(key)
|
||||||
|
}
|
||||||
|
fn keys(&self) -> Vec<Czyx> {
|
||||||
|
(**self).keys()
|
||||||
|
}
|
||||||
|
fn scan_prefix(&self, c: u8, z: Option<u8>, y: Option<u8>) -> Vec<Czyx> {
|
||||||
|
(**self).scan_prefix(c, z, y)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// A record store: a header + body addressed by a [`Czyx`] label.
|
/// A record store: a header + body addressed by a [`Czyx`] label.
|
||||||
///
|
///
|
||||||
/// Decision: we serialize the header and body as a single byte buffer with a
|
/// Decision: we serialize the header and body as a single byte buffer with a
|
||||||
|
|||||||
@@ -557,6 +557,64 @@ impl Session {
|
|||||||
store.delete_raw(&coord);
|
store.delete_raw(&coord);
|
||||||
Ok(format!("deleted {path}"))
|
Ok(format!("deleted {path}"))
|
||||||
}
|
}
|
||||||
|
// --- Raw coordinate API (used by cubefs' socket-backed backend,
|
||||||
|
// and any client that wants to address the store by CZYX
|
||||||
|
// directly instead of by path). These mutate the SAME durable
|
||||||
|
// `ConcurrentStore` the daemon serves, so a write through here
|
||||||
|
// is immediately visible to `ls`/`stat`/FUSE and is folded
|
||||||
|
// into the WAL + checkpoint like any other write. ---
|
||||||
|
"rawget" => {
|
||||||
|
let c = parse_u8(it.next(), "rawget needs <c>")?;
|
||||||
|
let z = parse_u8(it.next(), "rawget needs <z>")?;
|
||||||
|
let y = parse_u8(it.next(), "rawget needs <y>")?;
|
||||||
|
let x = parse_u8(it.next(), "rawget needs <x>")?;
|
||||||
|
let coord = Czyx::new(c, z, y, x);
|
||||||
|
match store.get_raw(&coord) {
|
||||||
|
Some(v) => Ok(format!("ok: {}", hex_encode(&v))),
|
||||||
|
None => Ok("none".to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
"rawput" => {
|
||||||
|
let c = parse_u8(it.next(), "rawput needs <c>")?;
|
||||||
|
let z = parse_u8(it.next(), "rawput needs <z>")?;
|
||||||
|
let y = parse_u8(it.next(), "rawput needs <y>")?;
|
||||||
|
let x = parse_u8(it.next(), "rawput needs <x>")?;
|
||||||
|
let hex = it
|
||||||
|
.next()
|
||||||
|
.ok_or_else(|| "rawput needs <hex-bytes>".to_string())?;
|
||||||
|
let val = hex_decode(hex)
|
||||||
|
.ok_or_else(|| "rawput: value must be hex".to_string())?;
|
||||||
|
let coord = Czyx::new(c, z, y, x);
|
||||||
|
store.put_raw(coord, val);
|
||||||
|
Ok(format!("ok: wrote {}", coord.pack_u32()))
|
||||||
|
}
|
||||||
|
"rawdel" => {
|
||||||
|
let c = parse_u8(it.next(), "rawdel needs <c>")?;
|
||||||
|
let z = parse_u8(it.next(), "rawdel needs <z>")?;
|
||||||
|
let y = parse_u8(it.next(), "rawdel needs <y>")?;
|
||||||
|
let x = parse_u8(it.next(), "rawdel needs <x>")?;
|
||||||
|
let coord = Czyx::new(c, z, y, x);
|
||||||
|
store.delete_raw(&coord);
|
||||||
|
Ok(format!("ok: deleted {}", coord.pack_u32()))
|
||||||
|
}
|
||||||
|
"rawkeys" => {
|
||||||
|
let ks: Vec<String> =
|
||||||
|
store.keys().iter().map(|k| k.pack_u32().to_string()).collect();
|
||||||
|
Ok(format!("ok: {} keys", ks.len()))
|
||||||
|
.map(|s| if ks.is_empty() { s } else { format!("{s}\n{}", ks.join(" ")) })
|
||||||
|
}
|
||||||
|
"rawscan" => {
|
||||||
|
let c = parse_u8(it.next(), "rawscan needs <c>")?;
|
||||||
|
let z = it.next().and_then(|t| t.parse::<u8>().ok());
|
||||||
|
let y = it.next().and_then(|t| t.parse::<u8>().ok());
|
||||||
|
let ks: Vec<String> = store
|
||||||
|
.scan_prefix(c, z, y)
|
||||||
|
.iter()
|
||||||
|
.map(|k| k.pack_u32().to_string())
|
||||||
|
.collect();
|
||||||
|
Ok(format!("ok: {} keys", ks.len()))
|
||||||
|
.map(|s| if ks.is_empty() { s } else { format!("{s}\n{}", ks.join(" ")) })
|
||||||
|
}
|
||||||
"grant" => {
|
"grant" => {
|
||||||
// Issue a permission grant (Task 6b / PDF flags 5-19). Only an
|
// Issue a permission grant (Task 6b / PDF flags 5-19). Only an
|
||||||
// identified owner may grant (under --require-identity); in the
|
// identified owner may grant (under --require-identity); in the
|
||||||
@@ -855,6 +913,41 @@ pub fn parse_coord(s: &str) -> Option<cubecoords::Czyx> {
|
|||||||
Some(cubecoords::Czyx::new(nums[0], nums[1], nums[2], nums[3]))
|
Some(cubecoords::Czyx::new(nums[0], nums[1], nums[2], nums[3]))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Parse a single `u8` axis token: decimal (`7`) or hex (`0x07`).
|
||||||
|
fn parse_u8(t: Option<&str>, what: &str) -> Result<u8, String> {
|
||||||
|
let t = t.ok_or_else(|| what.to_string())?;
|
||||||
|
t.parse::<u8>()
|
||||||
|
.or_else(|_| u8::from_str_radix(t.trim_start_matches("0x"), 16))
|
||||||
|
.map_err(|_| format!("{what} (got '{t}')"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Encode bytes as a lowercase hex string (used by the raw coordinate API so
|
||||||
|
/// payloads survive the text socket framing).
|
||||||
|
fn hex_encode(b: &[u8]) -> String {
|
||||||
|
let mut s = String::with_capacity(b.len() * 2);
|
||||||
|
for byte in b {
|
||||||
|
s.push_str(&format!("{byte:02x}"));
|
||||||
|
}
|
||||||
|
s
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decode a hex string into bytes. Rejects odd length / non-hex.
|
||||||
|
fn hex_decode(s: &str) -> Option<Vec<u8>> {
|
||||||
|
if s.len() % 2 != 0 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
let bytes = s.as_bytes();
|
||||||
|
let mut out = Vec::with_capacity(s.len() / 2);
|
||||||
|
let mut i = 0;
|
||||||
|
while i < bytes.len() {
|
||||||
|
let hi = (bytes[i] as char).to_digit(16)?;
|
||||||
|
let lo = (bytes[i + 1] as char).to_digit(16)?;
|
||||||
|
out.push(((hi << 4) | lo) as u8);
|
||||||
|
i += 2;
|
||||||
|
}
|
||||||
|
Some(out)
|
||||||
|
}
|
||||||
|
|
||||||
/// Parse a cubevm op name (case-insensitive) into an [`Op`]. `arg` is the
|
/// Parse a cubevm op name (case-insensitive) into an [`Op`]. `arg` is the
|
||||||
/// operand byte for ops that take one (const/load/store/jmp/jz/jnz/call/ret/
|
/// operand byte for ops that take one (const/load/store/jmp/jz/jnz/call/ret/
|
||||||
/// syscall); it is ignored for argument-less ops.
|
/// syscall); it is ignored for argument-less ops.
|
||||||
|
|||||||
Reference in New Issue
Block a user