Commit Graph
43 Commits
Author SHA1 Message Date
CUBELinux-2 314df8e42d docs: finalize RESUME — mount gate hardened (daemon-backed leg), VM deployment bugs fixed, snapshot taken 2026-08-13 07:08:14 -04:00
CUBELinux-2 da3aa3a07a test: ./check mount now also exercises the daemon-backed --socket FUSE path
The mount stage previously only tested the in-memory --seed mount, so the
daemon-backed write path (DaemonBackend) was never regression-guarded -- which
is exactly how a VM shipping an old cube-server (lacking the raw* commands)
shipped undetected. The mount stage now runs the full regression suite against
BOTH a --seed mount and a --socket mount (with its own fresh daemon), and adds
a durability-across-restart assertion proving the FUSE view is a real durable
store. run_fs_tests is now backend-agnostic (self-seeds its fixture).
2026-08-13 07:07:43 -04:00
CUBELinux-2 28e5bcc091 docs: finalize session status — gates green (check/daemon/mount), VM durable e2e + launcher verified 2026-08-13 06:35:01 -04:00
CUBELinux-2 0e3f18d106 docs: mark VM durable-socket e2e done (NEXT STEP 1) in RESUME doc
VM qcow2 was pre-fix; rebuilt cubefs-mount from fixed source inside the VM,
rewired cubefs.service to --socket /run/cube/cube.sock with
Requires=cube-server.service, and proved a FUSE write survives a daemon
restart via the durable store+WAL. ./check also green inside the VM.
2026-08-13 06:25:39 -04:00
CUBELinux-2 dd269e2b44 docs: record DaemonBackend envelope round-trip fix + gate status in RESUME doc 2026-08-13 05:58:40 -04:00
CUBELinux-2 0300def300 fix(cubefs): round-trip full record envelope through DaemonBackend + gate hardening
- DaemonBackend::put/get now round-trip the FULL record envelope (header+body)
  verbatim via rawput/rawget instead of stripping to a bare body. Before this,
  a socket-backed mount wrote the bare body but get returned it as if it were a
  record, breaking the FUSE read-back path (get_record could not decode it).
  This diverged from the in-memory backend and silently corrupted reads.
- Cargo fmt --check now passes (was failing; the committed tree was never a
  clean ./check, which is why the durable mount could regress undetected).
- Fix two new clippy lints (manual_is_multiple_of) in cubefs/backend.rs and
  cubesys/commands.rs so the -D warnings gate is green.
- Daemon-backed integration tests (cubefs_daemon_smoke, daemon_backend_smoke)
  now use the real CubeStore<DaemonBackend> record path and are #[ignore]d so
  the default gate (no daemon) stays green, while ./check daemon spins up a live
  cube-server and runs them via --ignored. This makes the durable-socket
  contract an actual enforced test, not a manual ad-hoc script.
2026-08-13 05:58:12 -04:00
CUBELinux-2 8f9e7e0025 fix(cubefs): make FUSE mount a durable view of cube-server daemon store
The cubefs-mount --socket path was never actually built: CubeFs<B> is
generic, so the --socket (DaemonBackend) and default (HashBackend) arms
of the match were incompatible types (E0308), and --features mount failed
to compile. The running binary was therefore the in-memory build, so
--socket was silently ignored and every FUSE write went to RAM and never
reached the daemon (rawget/rawkeys returned none / 0 keys, WAL stayed 0).

Fix: type-erase the backend. Add  (forwards to inner) in cubestore, and build
 in cubefs-mount via
Box::new(DaemonBackend::new(p)) / Box::new(HashBackend::new()). Keeps
cubefs free of a cubesys dep (acyclic graph).

Verified end-to-end on host (shared code path as the VM): a FUSE write
via  lands in the daemon store (rawget returns the
record, 5 keys present, WAL grows), and survives a  of the
daemon + relaunch with the same --store (byte-identical read-back).

Also includes (from RESUME-cubefs-daemon.md): cubesys raw* command family
(rawget/rawput/rawdel/rawkeys/rawscan + parse/hex helpers) and the
DaemonBackend client + smoke tests. Report/verification docs added.

Note: VM cubefs.service still mounts in-memory (no --socket); update the
unit to  as a
follow-up so the deployed VM FUSE is durable too.
2026-08-13 05:40:59 -04:00
CUBELinux-2 ad73f42e46 fix(audit): eliminate O(n)/unbounded audit-path bottlenecks (100% ok under load)
Root cause of the ~4% error rate in audit-enabled runs (run-qc6newt3:
96.30% ok, op6 mean 367ms max 3003ms) was the audit path's three
compounding costs, isolated iteratively under the real 8-user x 150s
model-B-with-audit stress harness:

  1. append(): rewrote the whole log string on every op (O(n) read-modify-write
     under a per-store Mutex) -> op latency grew with log size.
  2. dump(): walked 1..=count re-reading every entry record (O(n)) -> became the
     new bottleneck once append was fixed (op6 still ~760-980ms).
  3. the  command returned the UNBOUNDED full log (~1MB at 12k entries)
     on every call -> ~1MB response serialized/sent/received = op6 ~978ms.

Fix (aligned with the PDF's 'access logs live in Null rows' time/stream-keyed
model):
  - AUDIT_HEAD stores only a decimal entry count (index); each entry is its own
    durable record at entry_coord(seq) -> append is O(1) (two put_record calls).
  - ConcurrentStore gains a per-store in-memory tail cache (audit_tail) shared by
    every Audit over that store; append extends it by one line, dump returns a
    clone -> dump is O(1) and never re-walks the store. Serialized under the
    cache guard so concurrent connections interleave correctly.
  - the interactive  command serves a bounded recent tail
    (Audit::AUDIT_TAIL_LIMIT = 200) instead of the full log; the full log stays
    available via Session::audit_dump()/Audit::dump() for export.

Verification (real, not assumed):
  - ./check gate GREEN (fmt + tests + clippy -D warnings), incl. R6 append/dump
    tests and pre-existing grant_and_revoke_emit_audit_entries.
  - hermes_verify_audit_o1: index=count (not log), distinct coords, ascending
    dump, concurrent interleave-correct. PASS.
  - model-B-with-audit re-run (8 users x 150s): 110,647 ops, 100.00% ok, 0
    failures; op6 mean 11.9ms (p99 46.9ms, max 124.9ms) vs 367ms pre-fix. Final
    run evidence: /root/cube-stress/run-kkaogy3b.
  - Auth confirmed a non-factor (zero rejections) across all runs.

docs/stress-comparison-20260811.md: corrected the bogus '~0.3ms audit op' claim
in S5 and replaced the placeholder S6 with the full root-cause/fix/verification
write-up including the iteration-to-100% table.
2026-08-11 21:31:42 -04:00
CUBELinux-2 15ce36d488 docs: auth-model A/B comparison — error rate is op-mix, not auth model
Two harness drivers (model A per-command auth, model B persistent auth-once)
run across the real release cube-server to settle the "auth-each-time had ~0%
errors" memory. Conclusion: error rate is driven by the slow `audit` op /
3s socket cap, NOT the auth model — with audit removed, model A hits 94.81%
and model B 100%. Adds docs/stress-comparison-20260811.md §5 and the two
reusable harness scripts under tools/.
2026-08-11 19:09:44 -04:00
CUBELinux-2 4ab938fde8 tools/cubec: fix CLI so ./check stress works; separate client flags from command
cubec forwarded its OWN leading options (--socket/--tenant/...) verbatim
into the command payload sent to the daemon, so 'cubec --socket SOCK
"prog ..."' made the server reject '--socket' as an unknown command.
stress.sh therefore produced 'error: unknown command: --socket' on every
sample and measured nothing.

Split cubec arg parsing into client-option vs command-payload so flags are
consumed locally and only the command reaches the daemon. The canonical
'./check stress' stage now drives a real daemon and samples stats.
2026-08-11 18:33:28 -04:00
CUBELinux-2 ec84fbc725 cubesys: coalesce WAL fsync on commit; stop over-reporting durable seq
commit_txn issued an unconditional fsync per COMMIT, so N concurrent
writers serialized behind N disk syncs (p99 hit the 3s socket timeout
under 8 writers). Add Wal::sync_upto: committers queue on an fsync_gate,
the first one flushes the whole accumulated buffer, and waiters that find
committed_seq past their target return with zero I/O. N commits now cost
~1 fsync with the same durability guarantee.

Also fix a durability over-report: flush_pending stamped committed_seq
from the LIVE seq counter, so sequences taken by appenders that had not
yet buffered their bytes were reported durable. Track max_seq alongside
the pending buffer and advance committed_seq only to what was written.

Wire --wal-fsync-ms / --checkpoint-ms in cube-server (previously
hardcoded to defaults, so the documented knob did nothing).

Both regressions are mutation-verified: each test fails when its bug is
reintroduced.
2026-08-11 18:09:29 -04:00
CUBELinux-2 c87fef0514 fix(cubesys): unhang R4 handshake tests + drop unused import
- run_handshake now spawns auth_handshake on its own thread; the old
  code read the CHALLENGE frame before the daemon ever wrote one,
  deadlocking all 5 r4_handshake_tests (>60s hang). Join for the
  daemon verdict.
- remove unused TenantId/TenantIdentity import in
  grant_and_revoke_emit_audit_entries (clippy -D warnings failure).
- verify_hello call already passes all 6 args (psk, &nonce, tenant,
  &owner_local, owner_remote.as_deref(), sig); confirmed against
  cubecrypt::verify_hello signature.

./check: ALL CHECKS PASSED (fmt+tests+clippy -D warnings).
2026-08-11 16:38:38 -04:00
CUBELinux-2 94bddda3dd feat(cubesys/cubecrypt): R4 challenge-response auth + R5/R6 wiring
- cubecrypt/src/auth.rs: HMAC-SHA256 signed HELLO (sign_hello/verify_hello),
  random_nonce_hex entropy source (plan R4)
- cube-server: --auth-key enables CHALLENGE/HELLO handshake; auth_handshake is
  a module-level free fn (run(self) consumes self, so the thread closure can
  only reach the captured psk). Resolves the earlier E0425 compile failure
- cubec.rs client: --auth-key builds a signed HELLO frame
- commands.rs: audit op constants + read-gating hooks wired into admit_read
- audit.rs: per-tenant append-only audit log in a Null-cube range (plan R6)
- clippy -D warnings clean; full ./check gate ALL CHECKS PASSED (61 tests)
2026-08-11 15:59:10 -04:00
hermes 78cd5c0046 chore(cubesys): clear clippy -D warnings so full ./check gate is green
cargo clippy --fix applied style nits (redundant return, ?-operator,
map_or simplify) plus doc-comment list indentation. The full ./check gate
(fmt+test+clippy -D warnings) now passes; R5 read-gate + grant tests
remain green (39 cubesys lib tests).
2026-08-11 15:25:14 -04:00
hermes 8e31c15f69 docs: mark R5 done, reframe R1 in plan (owner/grant-native read gate) 2026-08-11 15:12:30 -04:00
hermes 0e579a02dd feat(cubesys): R5 — read-gating in daemon path (owner/grant-native)
Closes the read-privacy gap flagged in plan R1/R5. Rather than fork the
FUSE uid/gid Acl model (whose identity is POSIX uid, incompatible with the
daemon's name-based HELLO identity), read-gating is done owner/grant-native:

- commands.rs: new admit_read(coord) mirroring admit_mutate (owner ->
  read-grant -> deny; unowned records world-readable). Wired into
  (read+execute) and  (metadata read), gated by enforce_owner exactly
  like writes.
- Tests: read_gate_blocks_non_owner_and_allows_read_grant (bob denied,
  allowed after alice grants read), read_gate_requires_identity_under_enforce
  (anonymous stat rejected under --require-identity).
- Full ./check quick green: 39 cubesys lib tests, clippy -D warnings clean.

Note in plan: Acl lift (R1) is NOT a literal fork; the daemon reuses the
owner/grant enforcement concept, not the POSIX Acl struct.
2026-08-11 15:11:51 -04:00
hermes e77c650e9c feat(cubesys): Task 6b — permission grants (PDF flags 5-19)
Add a delegated-grant auth layer:
- cubesys/src/grants.rs: Grant/Owner/Perm model, GRANT_BUCKET at
  Czyx::new(0,1,0,1), grant/revoke/grant_allows, std-only JSON codec.
- Stored via put_record with a doc_type='grant-table' header so it
  survives checkpoint/restore (raw put_raw was dropped on dump_store).
- commands.rs: GRANT/REVOKE opcodes + admit_mutate() enforcement hook
  (owner -> grant -> deny). GRANT/REVOKE require HELLO identity under
  --require-identity.
- Enforce owner-match contract preserved (legacy/tests stay green).
- 11 new grant tests; full ./check quick = 115 pass, clippy -D clean.
2026-08-11 15:01:35 -04:00
CUBELinux-2 16cbf6c3cb feat(cubesys): Task 6 (B) — require HELLO identity, gate seal/open
Adopted recommendation (B): make owner authority a hard guarantee instead
of the non-breaking opt-in. A mutating op now requires a stamped HELLO
identity; anonymous writes are rejected. seal/open (destructive writes)
are gated the same way, and seal stamps the owner onto the encrypted record.

Design / non-breaking bridge:
- Session gains enforce_owner: bool (default false) so library/REPL/unit
  tests stay permissive — the 26 prior tests + 3 Task-6 tests are unchanged.
- owner_violation() gains require_identity: the (false) path keeps legacy
  behaviour; the (true) path rejects no-identity mutating ops.
- The daemon flips enforce_owner=true on every connection (both HELLO and
  no-HELLO branches), implementing the default --require-identity policy.
- Added --allow-anonymous escape hatch so legacy cubec/stress.sh (which
  send no HELLO) keep working; stress.sh now passes --allow-anonymous.
- Session::set_enforce_owner() accessor so the daemon (separate bin) can
  set the private field.

Verification:
- ./check quick: EXIT=0, fmt+clippy clean, 28 cubesys lib tests (added
  enforce_owner_requires_identity, seal_open_respect_owner).
- Ad-hoc daemon verifier (LE framing) against the rebuilt cube-server:
  anonymous prog/del rejected, HELLO'd owner first-claim + self-overwrite
  allowed, cross-owner overwrite rejected. ALL PASS.

Note: seal's demo key-cell crypto path (KeyCellMissing on the synthetic key
material) is a pre-existing quirk unrelated to this change; the gate fires
before crypto, so the test verifies the gate, not the crypto.
2026-08-11 14:12:42 -04:00
CUBELinux-2 abc1b56d24 feat(cubesys): Task 6 — owner enforcement on mutating commands
Stamp owner_local_user on records written via prog/write and gate the
mutating paths (prog, write, del — both live and buffered txn) so a
session may only create or overwrite a record whose owner_local_user
matches its HELLO-declared identity.

Design (logical + expedient for the whole project):
- Owner is the durable record-level CubeHeader.owner_local_user field,
  so enforcement is replay-safe and works across daemon restart.
- Enforcement is opt-in/non-breaking: gated only when the session has a
  stamped identity AND the record has an owner. First write by an owner
  claims an unowned coord; a session with no identity (tests, legacy)
  writes freely.
- COMMIT re-checks owner on each buffered op before applying, so a
  concurrent cross-owner commit between BEGIN and COMMIT is rejected
  (txn is restored for retry, not silently dropped).
- seal/open (encrypted raw put/del) left ungated for now: their headers
  are not owner-stamped yet — tracked as follow-up.

Verification:
- ./check quick: EXIT=0, fmt+clippy clean, 26 cubesys tests (added
  owner_enforcement_blocks_cross_owner_overwrite,
  owner_enforcement_allows_first_claim_and_same_owner,
  for_tenant_carries_identity).
- Ad-hoc daemon verifier over real cube-server socket (LE framing):
  cross-owner overwrite + delete rejected, same-owner + first-claim
  allowed, no-HELLO legacy writes allowed. ALL PASS.
2026-08-11 13:41:24 -04:00
CUBELinux-2 c36f64c78d fix(cubesys): make transaction commits durable + replayable
Two correctness bugs found via ad-hoc daemon verification (T5 was
compile-verified only before):

1. decode_wal dropped WalOp::Txn entries on replay: the txn encoder emits
   {"seq","op":"txn","batch"} with NO c/z/y/x fields, but decode_wal
   read c/z/y/x unconditionally -> field_u8("c") returned Err -> the
   whole entry was skipped. Committed transactions silently vanished on
   restart. Fix: branch on op=='txn' before the c/z/y/x extraction.

2. commit was not synchronously durable: append_txn only buffered to the
   WAL pending buffer; fsync happened on the 25ms group thread. A
   clean stop within that window lost the commit. Fix: commit_txn now
   calls wal.flush_pending() (fsync) before returning, so COMMIT is
   durable on return -- a real transaction boundary.

Adds unit test commit_replays_from_wal_without_checkpoint (would have
failed before fix 1). Ad-hoc verifier exercises all 3 changed paths on
the live cube-server socket.
2026-08-11 13:09:24 -04:00
CUBELinux-2 308e20852c feat(cubesys): Task 5 transactions (BEGIN/COMMIT/ROLLBACK) + HELLO identity wiring
- Session gains txn: Option<Txn> (BEGIN snapshot + buffered ops) and
  identity: Option<TenantIdentity> (owner enforcement hook for Tasks 6+).
- prog/write/del buffer into the open txn; commit_txn applies the whole
  batch under one write lock + a SINGLE WalOp::Txn entry (atomic + durable
  replay). ROLLBACK discards. Reads consult the BEGIN snapshot (isolation).
- cube-server: HELLO-resolved tenant yields Some(ts); default-tenant path
  consistent; Session built once per connection so txns span frames.
- WalEntry gains batch field; decode_wal + encode_wal handle Txn replay.
- 4 new T5 tests: buffer/commit, rollback, snapshot isolation, durable
  reopen via WAL replay. ./check quick green.
2026-08-11 12:50:54 -04:00
CUBELinux-2 6209955b46 feat(cubesys): Task 4 reader/writer sharding (Mutex -> RwLock)
ConcurrentStore.inner is now Arc<RwLock<CubeStore>>: all read paths take the
read side, all mutations + checkpoint take the write side. Readers no longer
exclude each other and overlap an active writer (verified by
concurrent_reads_dont_block_on_writer + cube-bench Task 4 section). WAL,
checkpoint, and coordinate encoding are untouched, so durability/replay is
unchanged (.check green).

Honest finding recorded in docs/task4-reader-writer-sharding.md: on this 8-core
host std RwLock removes reader-vs-reader exclusion (correct) but shows no
wall-clock speedup for short reads (cache-line bounce on one shared lock). Real
read-throughput scaling would need sharded/lock-free storage, left as a
follow-up decision rather than invented.
2026-08-11 12:20:08 -04:00
CUBELinux-2 7f18218aca fix(cubesys): HELLO ACK + ad-hoc E2E proof of multi-tenant routing (Task 3)
The wire protocol required an explicit HELLO acknowledgement frame; a
HELLO-only client would otherwise block on the next recv. Add the ACK.

Ad-hoc E2E (real built cube-server, real 4-byte LE framed Unix socket):
  - SHARED mode: a HELLO tenant and a no-HELLO client share ONE store
    (write visible across both) — backward-compatible with cubec/stress.sh.
  - PER-TENANT mode (--tenant-dir): two HELLO tenants are HARD-isolated
    (T2 and default tenant cannot see T1's record).

Verifier: /tmp/hermes-verify-driver.sh + /tmp/hermes-verify-e2e.py
2026-08-11 11:50:05 -04:00
CUBELinux-2 50c1b55f71 feat(cubesys): wire cube-server to multi-tenant registry + HELLO (Task 3)
- TenantRegistry gains a true single-store 'shared' mode so the legacy
  --store PATH invocation (and stress.sh / old cubec) keeps serving one
  global store, while still parsing HELLO frames.
- cube-server now holds an Arc<TenantRegistry>, resolves each connection to
  its tenant's TenantSession, and stamps the client identity from
  HELLO <tenant> <owner_local> [<owner_remote>].
- Add --tenant-dir DIR for real per-tenant disk isolation (opt-in).
- Add TenantIdentity + TenantRegistry::parse_hello and a shared-session
  integration test.
2026-08-11 11:26:36 -04:00
CUBELinux-2 cf10cfcf51 feat(cubesys): per-tenant durable stores on disk (Task 2)
TenantConfig::Disk opens each tenant's ConcurrentStore under its own
sanitized subdir of store_dir (path separators -> '_', no traversal escape),
reusing the durable WAL+checkpoint machinery (incl. 49698af delta-path fix).
Registry takes a config; get_or_provision returns io::Result so a disk
open failure surfaces instead of silently falling back to memory.
Tests: tenant_store_is_isolated_on_disk (separate dirs + survives reopen,
B still isolated) + tenant_id_cannot_traverse_store_dir. ./check green.
2026-08-11 10:59:47 -04:00
CUBELinux-2 f5a8a2120c feat(cubesys): tenant id + registry skeleton (Task 1)
Add TenantId (from-str, normalized, non-empty) and TenantRegistry that
provisions one isolated TenantSession per tenant behind an RwLock<HashMap>.
In-memory store only for now; disk-backed per-tenant stores + daemon wiring
land in Tasks 2-3. 4 unit tests pass, ./check green.
2026-08-11 10:53:38 -04:00
CUBELinux-2 75b11f939e docs: stress/compare report + concurrent multi-tenant plan (owner-grant model) 2026-08-11 04:49:58 -04:00
CUBELinux-2 49698af9fc fix(cubesys): delta path mismatch in ConcurrentStore checkpoint/reopen
open() derived the delta path as "${db_path}.delta" (e.g. "db3.json.delta")
while checkpoint_store() writes via db_path.with_extension("delta")
(e.g. "db3.delta"). On reopen, load_base_plus_delta therefore read a
never-written path and silently skipped the delta, so post-checkpoint
changes were lost. Use db_p.with_extension("delta") in both places.

Also drop a no-op cp_seq.max(0) (u64 >= 0 always) to clear the clippy
-W clippy::unnecessary_min_or_max lint.

Verified: ./check all green; incremental_checkpoint_delta_model,
durable_checkpoint_and_replay, wal_recovery_after_crash pass in isolation.
2026-08-11 04:04:05 -04:00
CUBELinux-2 7ccb29aa6a cubesys: durable WAL + ConcurrentStore (NDJSON, group-commit fsync, recovery log)
- Add persist.rs: std-only NDJSON snapshot of the HashBackend store
  (no serde) for the durable checkpoint + load_into_store replay.
- Add store.rs: ConcurrentStore = Mutex<HashBackend> live store + WAL
  (newline-delimited JSON, group-commit fsync, idempotent seq-numbered
  replay) + durable JSON checkpoint + bg flusher + startup replay.
- Recovery events (checkpoint failure, WAL fsync failure, WAL replay)
  are written to a recovery.ndjson you asked to keep as the written backup
  log, so any fall-back to JSON is recorded 'in writing'.
- Refactor cube-server to thread-per-connection over ConcurrentStore.
- query_doc_type / scan_prefix / linked_to / delete_raw added.

Verified: ./check (fmt, 7 unit tests, clippy -D warnings) all green;
./check stress drove 22,080 prog+run pairs (~368/s) over 60s, daemon
survived, latency prog~9us/run~13us mean.
2026-08-11 03:18:59 -04:00
hermes bbaa36a32c check: add ./check stress stage; fix unreachable bench/mount dispatch
- ./check stress spins a fresh cube-server+cubec built from this tree on a
  throwaway socket/store and drives ~150s of real prog/run traffic while
  sampling the per-command latency + per-C telemetry.
- tools/stress.sh: never touches the production daemon; STRESS_SECONDS override.
- Fixed latent control-flow bug: the old '|| exit 0' guards made 'bench' and
  'mount' subcommands unreachable after the gate (they exited early). Now uses
  explicit run_* flags and only exits at the true end.
2026-08-11 01:46:49 -04:00
cube-agent 4f9cb2e75e Add cube-bench (correctness-gated microbenchmarks) + daemon stats telemetry
- cube-bench crate: real-code-path throughput/latency over cubestore,
  cubecrypt (aes/gcm/chacha/xts), cubecode VM, and cubesys Session.
  Every section asserts correctness before timing. Wired into ./check
  as an opt-in 'bench' stage.
- cubesys Session: per-command latency histogram + per-C-namespace record
  counts, exposed via a new 'stats' command over the live socket.
- Deployed rebuilt cube-server to /home/luulu/.cubelinux/bin and
  restarted the system cube.service; verified stats live.
2026-08-11 01:15:32 -04:00
CUBELinux-2 06ea3252eb cubesys: add cube systemd daemon (cube-server) + socket client (cubec)
Implements the requested cube service: a long-lived daemon that holds ONE
CubeStore for its whole lifetime and serves the cube command language over a
Unix-domain socket, plus cubec to talk to it.

- cubesys::commands: factored the single command interpreter (Session::exec)
  so cube REPL, cubec client, and the daemon run identical logic
- cubesys::net: dependency-free length-framed AF_UNIX transport
- cubesys::persist: dependency-free JSON snapshot (atomic tmp+rename) so the
  store -- including sealed/encrypted records -- survives daemon restarts
- cube-server: listens on $XDG_RUNTIME_DIR/cube/cube.sock, snapshots to
  $XDG_STATE_HOME/cube/cube-store.json, replays on startup
- cubec: one-shot + REPL client over the socket
- cube.rs trimmed to a thin REPL/script/demo driver (help text updated)
- /etc/systemd/system/cube.service: runs as luulu, ProtectSystem=strict,
  RestrictAddressFamilies=AF_UNIX, Restart=on-failure; enabled + active
- integration.md documents the daemon + caveat (open rewrites plaintext)

Verified: ./check (fmt+tests+clippy -D warnings) green; ./check mount (27
FUSE e2e) green; socket CLI round-trips; sealed record survived a full
service restart and reopened+r with original value.
2026-08-11 00:10:45 -04:00
CUBELinux-2 35e5183193 feat(system): bind cubefs+cubecode+cubecrypt into one running system (cubesys)
Integrates Packages 3-5 over a single shared CubeStore, the literal
CUBELinux premise (data addressed by coordinate, not path). Adds the
cubesys crate (lib + cube CLI + cube-demo) proving two end-to-end
properties: a cubefs path IS a runnable code cell at the same coordinate,
and a sealed record reopens and runs on the same store.

Two latent cross-crate bugs surfaced and fixed while integrating:
- cubecoords: refresh_flags() now preserves out-of-band flag bits
  (8..=15), so cubecrypt's HEADER_FLAG_ENCRYPTED survives refresh.
- cubestore: record codec now serializes raw flag bits (TLV tag 12) so
  the encrypted bit survives the store round-trip.

All gates green (./check, incl. cubefs --features mount).
2026-08-10 23:42:38 -04:00
CUBELinux-2 b8823d25be test(cubecrypt): add persistent XTS roundtrip + non-AEAD test (was only ad-hoc verified)
XTS was previously covered only by a deleted throwaway example. Fold the
verification into the committed suite so it runs under ./check: 16-multiple
roundtrip, short-plaintext pad/strip roundtrip, multi-block sector, and the
correct non-AEAD wrong-key property (wrong key opens to different garbage).
2026-08-10 21:01:42 -04:00
CUBELinux-2 37e8255cb1 feat(cubelinux-2): Package 5 — cubecrypt: Null-cube env selects key material + transform (AES-GCM, ChaCha20-Poly1305, XTS) over CZYX 2026-08-10 20:48:08 -04:00
CUBELinux-2 01f61f13f4 fix(cubecode): share one data stack across CALL_LINK frames
Ad-hoc verification (factorial via recursive CALL_LINK) surfaced a real defect:
each exec_cell had its own private stack, but the design (and the doc
contract) is that callees run on the SHARED data stack so a caller passes
args by leaving them on the stack and reads the callee result there after
RET. With per-frame stacks, a callee that popped its argument faulted with
PcOutOfRange.

- exec_cell now takes &mut Vec<u8> (the one shared stack) instead of owning
  a fresh one; run() owns it and threads it through recursion.
- Added regression test shared_stack_passes_args_across_cube_edges (caller
  leaves 21, callee doubles it via Store/Load, caller sees 42) so the
  convention is locked by ./check, not just ad-hoc.

The prior commit's unit tests didn't exercise cross-frame stack args, which
is why the bug slipped through; suite is now 50 tests green (incl. 14
cubecode).
2026-08-10 19:59:17 -04:00
CUBELinux-2 4484eba83d feat(cubelinux-2): Package 4 — cubecode (cubevm): code/data mapping + safe cube-addressed bytecode VM over CZYX
Implements the PDF's Package 4 (cubevm/cubecode): the cube as a substrate for
storing and introspecting code + AI artifacts.

- opcode.rs: decode/encode codec for a deterministic, safe bytecode (28 ops:
  stack arithmetic/logic/shift, comparisons, jumps, CALL_LINK, RET, SYSCALL,
  DUP/DROP). Body is always decoded through this codec before execution, so
  the cube never runs code that didn't survive decode.
- cell.rs: CodeCell + Kind (Fn/Kernel/Layer/Checkpoint/Variant/Other). Kind
  rides in the record doc_type; the call graph is the cube's linked_records,
  so CALL_LINK n executes linked_records[n].
- vm.rs: stack-based interpreter over CubeStore. CALL_LINK follows cube edges;
  call depth is bounded (no infinite cube loops); deterministic + no unsafe +
  no float, so runs are reproducible (prereq for 'navigate and reconstruct
  experiments'). Host syscalls introspect the cube (degree/link-exists/trace).

Design decision (the PDF's interesting tension): it says body may be
'bytecode, machine code, or serialized model weights' but the load-bearing
clause is a DSL/runtime that 'walks cube links to load and dispatch functions'
= addressable code-as-data. On this hardware a foreign-machine-code JIT is
neither safe nor needed, so we built the safe bytecode VM. cubetrace/
cubedbt/cubeai (execution capture + ML over traces) are the next layer and are
excluded here.

Verified: ./check (fmt+tests+clippy -D warnings) green; cubecode: 13 tests
(arith, div-by-zero fault, CALL_LINK edge-follow, bad-link fault, recursion
depth bound, syscall trace/degree). Gate: cde1e62 -> +Package4.
2026-08-10 19:55:08 -04:00
CUBELinux-2 cde1e62b10 build: ./check — canonical verification gate for CUBELinux-2
The workspace had no single verification entrypoint, so every claim of
'green' rested on an ad-hoc command chosen after the fact. ./check makes it
one command, mirroring the convention already used by the original build.

  ./check          fmt + tests + clippy -D warnings
  ./check quick    tests only
  ./check mount    the above + live FUSE end-to-end (root, 27 assertions)

The FUSE adapter compiles in every stage (--features cubefs/mount) so the
kernel-facing code can never silently rot behind a feature flag. The live
mount stage is opt-in because it needs root, /dev/fuse and attr — but it is
the only stage that exercises the real kernel VFS path, and it encodes the
two Package 3 regressions (mkdir -p to depth 4, cross-user ACLs) that every
unit test missed.

Adopting clippy -D warnings immediately paid for itself: it failed on two
pre-existing Package 1 defects that per-crate greps had let through —
a needless_range_loop in TriEnc::unpack_6 and a doc comment orphaned by a
blank line in HeaderFlags. Both fixed.

Verified: ./check green; ./check mount green (27/27, no leftover mounts).
2026-08-10 19:28:41 -04:00
CUBELinux-2 a308f8422d feat(cubelinux-2): Package 3 — cubefs (POSIX/FUSE namespace over CZYX)
Implements the PDF's Package 3 with new code:

- path: bijective POSIX path <-> Czyx mapping (/c001/z002/y003/x004).
  Axis-letter + 3-digit zero-padded canonical names so lexical order equals
  numeric order and each coordinate has exactly one spelling. Inode IS the
  packed u32 coordinate — no inode side table.
- nullspace: the PDF's 'use Null cubes for ACLs, xattrs, journaling, volume
  metadata', with the Z-plane allocation fixed and documented (Z=1 volume,
  Z=2 ACL, Z=3 xattr, Z=4 journal ring). ACL/xattr tables are FNV
  hash-bucketed with exact-match resolution inside the bucket, because 4
  axes of subject cannot injectively mirror into 2 axes of Null space.
  Journal is a bounded ring; wraps are detectable via a monotonic counter.
- vfs: the whole filesystem, kernel-free and unit-testable — lookup,
  readdir, create/read/write/truncate/unlink, mkdir/rmdir, ACL enforcement,
  xattrs, journaling, POSIX errno mapping.
- fuse (feature 'mount'): thin kernel adapter, zero TTL (the store is
  writable out-of-band, so cached metadata would go stale).
- cubestore: added the PDF's 'optional scanning primitives' (keys,
  scan_prefix) and the Package 2 association API (associate, linked_to)
  that cubefs needs for directory listings.

Two defects were found by LIVE MOUNT testing and fixed, not by unit tests:
 1. mkdir succeeded then the kernel's revalidating lookup returned ENOENT,
    so 'mkdir -p' could never reach depth 4. Directories were purely
    inferred from records, making an empty directory unrepresentable. Fixed
    with an explicit Null-space directory marker; rmdir removes it; readdir
    merges markers in. 5 regression tests added.
 2. Multi-user ACL behaviour was untestable because the mount lacked
    AllowOther — the kernel returned EACCES at the mountpoint before any
    request reached us. Added --allow-other.

Verified: 58 unit tests pass; clippy clean; live mount exercised with cat,
echo, dd, truncate, cp, chmod, chown, getfattr/setfattr, mkdir -p, rmdir,
find, a 200-record write loop, and cross-user reads/writes as luulu.
2026-08-10 19:23:59 -04:00
CUBELinux-2 c23a45def2 tooling: remove HIST-QNA capture entirely — user directive: CUBE historical logging is dead, notes only 2026-08-10 19:02:39 -04:00
CUBELinux-2 dd87ce8822 tooling: remove prune path — history is never deleted
User clarified: the earlier prune was a ONE-TIME cleanup of an over-backfill
bug, NOT a recurring retention cap. Saved history must persist. Removed
cmd_prune / --prune-older-than-hours entirely; the script now only captures
forward (default) or seeds a bounded window once. No code path deletes cube
entries. Verified: help has no prune arg, forward/seed dry-runs change no
state, live timer runs forward-only.
2026-08-10 18:27:28 -04:00
CUBELinux-2 aaaccb0780 tooling: histcapture.py (auto HIST-QNA capture) + systemd timer
Captures user/assistant messages from /root/.hermes/state.db into the
'hermes' cube namespace as TYPE: HIST-QNA entries. Forward-only by default
(no backfill); --seed-hours N for bounded one-time population; idempotent;
--prune-older-than-hours to undo over-backfill. systemd timer runs every
5 min. Tested: 24h seed = 1171 msgs; prune cleared a 3164-entry backfill
to 44, then re-seeded correctly.
2026-08-10 18:22:49 -04:00
CUBELinux-2 6e13b13eea feat(cubelinux-2): Package 1 — cubecoords + cubestore from PDF spec
New code (no recycling from prior build). Implements:
- cubecoords: Czyx 4-axis coordinate (pack/unpack u32), Null-class
  classification (Total / Cube 1-4 / User), TriWord tri-channel 64-bit
  codec (6 ASCII + 4 control bits), HeaderFlags + CubeHeader (derived
  flags). All coding decisions documented inline.
- cubestore: CubeBackend trait, HashBackend (HashMap<u32>), CubeStore with
  dependency-free length-prefixed record codec (header TLV + body).
- 8 unit tests, all passing; cargo test clean (0 warn/err).

Per directive: separate git repo; current /home/CUBELinux kept as working
tool; scoped to AI-OS-excluded PDF vision.
2026-08-10 18:03:04 -04:00