whitepaper: publish post 3 (CUBE Integration into ESP32 Modules); sync posts 1-2 to live
- add whitepaper/cubelinux-esp32-integration.html — third blog post: an architectural proposal for carrying the coordinate substrate onto ESP32-class microcontrollers, as a CUBE client or a standalone node. Same format/layout as the existing posts; explicitly marked a preliminary draft, not implemented. - update whitepaper/cubelinux-whitepapers.html — listing now shows three drafts. - sync whitepaper/cubelinux-final-whitepaper.html and cubelinux-neural-networks-addendum.html to the live cubelinux.com versions. The local copies were stale (post 1 was the Aug-25 build) and both lacked the live head meta block (description + Open Graph / Twitter / canonical).
This commit is contained in:
luulu
@@ -0,0 +1,297 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
|
||||
<title>CUBELinux — CUBE Integration into ESP32 Modules</title>
|
||||
<meta name="description" content="Proposal: carrying the CUBELinux coordinate substrate onto ESP32-class microcontrollers, as a CUBE client or a standalone node.">
|
||||
<style>
|
||||
:root{
|
||||
--gold:#d8af68; --gold-hi:#fff9b1; --cyan:#39d7ff; --bg:#0c0f14;
|
||||
--panel:#141a22; --panel2:#1b232e; --ink:#e8eaeb; --muted:#8a97a6;
|
||||
--ok:#5ee0a0; --line:#26303c; --warn:#e8a040;
|
||||
}
|
||||
*{box-sizing:border-box}
|
||||
html,body{margin:0;height:100%}
|
||||
body{
|
||||
background:radial-gradient(1200px 600px at 80% -10%, #15212e 0%, var(--bg) 55%);
|
||||
color:var(--ink); font:15px/1.55 -apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;
|
||||
-webkit-tap-highlight-color:transparent;
|
||||
}
|
||||
.wrap{max-width:780px;margin:0 auto;padding:18px 16px 40px}
|
||||
header{margin-bottom:18px;border-bottom:1px solid var(--line);padding-bottom:14px}
|
||||
.logo{width:36px;height:36px;border-radius:10px;background:linear-gradient(135deg,var(--gold),var(--cyan));box-shadow:0 0 20px rgba(57,215,255,.3);flex-shrink:0}
|
||||
h1{font-size:20px;margin:8px 0 2px;font-weight:700;letter-spacing:.2px}
|
||||
h1 span{color:var(--cyan)}
|
||||
.meta{color:var(--muted);font-size:12px;margin-top:4px}
|
||||
.meta b{color:var(--ink);font-weight:600}
|
||||
h2{
|
||||
font-size:14px;margin:24px 0 6px;padding-bottom:4px;border-bottom:1px solid var(--line);
|
||||
color:var(--gold);letter-spacing:.4px;text-transform:uppercase;font-weight:700;
|
||||
}
|
||||
h3{font-size:13.5px;margin:14px 0 4px;color:var(--cyan)}
|
||||
p{margin:6px 0}
|
||||
ul,ol{margin:6px 0 6px 18px;padding:0}
|
||||
li{margin:2px 0}
|
||||
blockquote{
|
||||
border-left:3px solid var(--gold);margin:10px 0;padding:4px 12px;
|
||||
background:var(--panel);border-radius:0 8px 8px 0;color:var(--ink);
|
||||
}
|
||||
code{
|
||||
background:var(--panel2);border:1px solid var(--line);border-radius:5px;
|
||||
padding:1px 5px;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:13px;
|
||||
color:var(--cyan);
|
||||
}
|
||||
pre{
|
||||
background:var(--panel);border:1px solid var(--line);border-radius:10px;
|
||||
padding:10px 12px;overflow-x:auto;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;
|
||||
font-size:12.5px;line-height:1.45;margin:8px 0;
|
||||
}
|
||||
table{border-collapse:collapse;width:100%;margin:8px 0;font-size:13px}
|
||||
th,td{border:1px solid var(--line);padding:5px 8px;text-align:left}
|
||||
th{background:var(--panel2);color:var(--gold);font-weight:600}
|
||||
tr:nth-child(even){background:var(--panel)}
|
||||
.note{background:var(--panel);border:1px solid var(--line);border-left:3px solid var(--warn);border-radius:8px;padding:8px 12px;margin:10px 0;color:var(--ink)}
|
||||
.note b{color:var(--warn)}
|
||||
.pill{display:inline-block;background:var(--panel2);border:1px solid var(--line);border-radius:10px;padding:1px 8px;font-size:11px;color:var(--muted);margin-right:4px}
|
||||
.pill.core{background:rgba(94,224,160,.12);border-color:var(--ok);color:var(--ok)}
|
||||
.pill.opt{background:var(--panel2);border-color:var(--line);color:var(--muted)}
|
||||
footer{margin-top:32px;padding-top:12px;border-top:1px solid var(--line);color:var(--muted);font-size:12px}
|
||||
.diagram{background:var(--panel);border:1px solid var(--line);border-radius:10px;padding:10px 12px;margin:8px 0;font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-size:12.5px;line-height:1.4}
|
||||
.diagram .layer{padding:2px 0;border-bottom:1px dashed var(--line)}
|
||||
.diagram .layer:last-child{border-bottom:none}
|
||||
.diagram .arrow{color:var(--cyan);text-align:center;padding:2px 0}
|
||||
</style>
|
||||
<!-- Open Graph / Twitter / canonical (site prep) -->
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:title" content="CUBELinux — CUBE Integration into ESP32 Modules">
|
||||
<meta property="og:description" content="Proposal: carrying the CUBELinux coordinate substrate onto ESP32-class microcontrollers, as a CUBE client or a standalone node.">
|
||||
<meta property="og:url" content="https://cubelinux.com/cubelinux-esp32-integration.html">
|
||||
<meta property="og:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<meta name="twitter:card" content="summary_large_image">
|
||||
<meta name="twitter:title" content="CUBELinux — CUBE Integration into ESP32 Modules">
|
||||
<meta name="twitter:description" content="Proposal: carrying the CUBELinux coordinate substrate onto ESP32-class microcontrollers, as a CUBE client or a standalone node.">
|
||||
<meta name="twitter:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<link rel="canonical" href="https://cubelinux.com/cubelinux-esp32-integration.html">
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
|
||||
<header>
|
||||
<div class="logo"></div>
|
||||
<h1>CUBELINUX <span>·</span> CUBE Integration into ESP32 Modules</h1>
|
||||
<div class="meta"><b>Preliminary Draft — CUBELinux Project</b> — 2026-09-09. Architectural proposal, not implementation.</div>
|
||||
</header>
|
||||
|
||||
<div class="note"><b>Status.</b> This is a proposal and a direction, not a built system. It describes how the CUBELinux coordinate substrate could be carried onto ESP32-class microcontrollers — as a client of a host CUBE daemon, or as a compact standalone CUBE node — and what a mesh of such devices would make possible. Nothing described here is implemented. The central thesis: because the coordinate primitive is already <code>no_std</code> and allocation-free, the device becomes a first-class <em>space</em> in the same coordinate graph as everything else, and the association-edge and Portal machinery turns a device fleet into one addressable, capability-gated graph.</div>
|
||||
|
||||
<h2 id="intro">1. Introduction</h2>
|
||||
|
||||
<p>CUBELinux addresses every byte by a coordinate — a 256-bit space selector (<code>C</code>) plus three spatial axes (<code>Z</code>, <code>Y</code>, <code>X</code>) — rather than by path, hostname, or inode. The coordinate primitive lives in the <code>cube-core</code> crate, which is <code>no_std</code>, allocation-free, and <code>forbid(unsafe_code)</code>, and which already compiles for a bare-metal target (<code>thumbv7em-none-eabihf</code>, ARM Cortex-M without an operating system). That property was not incidental: the crate's own documentation treats the <code>no_std</code> boundary as the standing test of whether the layer could live where there is no OS at all.</p>
|
||||
|
||||
<p>This paper takes that boundary seriously and asks what happens when the thing on the other side is an ESP32. The ESP32 family is a widely deployed class of low-cost microcontroller with integrated Wi-Fi, a few hundred kilobytes of SRAM, and a few megabytes of flash — the class of part found inside smart plugs, thermostats, sensors, and industrial gateways. Espressif maintains first-class Rust support for the family, so the toolchain question is largely answered. The open question is architectural: what does it mean for a device to be addressed by coordinate, to hold its own configuration and telemetry at coordinates, and to be reachable from a host only through an explicit Portal?</p>
|
||||
|
||||
<p>The scope of this paper is architectural. We describe the device as a coordinate space, the two deployment tracks (client and node), the precise boundary between what ports to a microcontroller and what does not, the graph and Portal model applied to a fleet, and the agent monitoring and automation layer that sits on top. We close with an honest account of what is missing. The central thesis: CUBE does not merely add a faster way to store bytes on a device — it makes <strong>monitoring, memory, and control the same traversable graph</strong>.</p>
|
||||
|
||||
<h2 id="device-space">2. The ESP32 as a Coordinate Space</h2>
|
||||
|
||||
<h3>2.1 Device identity as a space selector</h3>
|
||||
|
||||
<p>In CUBELinux, movement along the spatial axes is arithmetic, but movement between spaces is not: a different space requires an explicit <code>Portal</code>. This distinction is exactly what a device fleet needs. Each ESP32 is assigned its own <code>SpaceId</code> — a 256-bit value — and is therefore not an address on a bus or an entry in a registry, but a coordinate system in its own right. The device's whole state space is <code>c<SpaceId>/z…/y…/x…</code>.</p>
|
||||
|
||||
<p>Two consequences follow immediately. First, there is no collision between devices: two devices are two spaces, and their coordinates are not interleaved. Second, because <code>SpaceId</code> is 256 bits, a device space that was never disclosed cannot be guessed — the whitepaper's capability claim is structural, not a policy bolted on top.</p>
|
||||
|
||||
<h3>2.2 Device state as coordinates</h3>
|
||||
|
||||
<p>Everything a device knows about itself is placed at a coordinate. Because CUBELinux stores metadata in reserved Null-space coordinates rather than in a side table, the record's header and the record's body are addressed by the same coordinate — there is no separate metadata store to drift out of sync. A device therefore does not need a parallel configuration format, a schema registry, or a documentation table. Its configuration, telemetry, and health are all coordinates.</p>
|
||||
|
||||
<table>
|
||||
<tr><th>Device aspect</th><th>Representation</th><th>Kind</th></tr>
|
||||
<tr><td>Capabilities, flags, firmware version</td><td>header at the device root coordinate</td><td>header</td></tr>
|
||||
<tr><td>Configuration (Wi-Fi, sample rate, GPIO maps)</td><td>records in the device's config subtree</td><td>data</td></tr>
|
||||
<tr><td>Telemetry and health (uptime, sensor values, faults)</td><td>records in the device's health subtree</td><td>data</td></tr>
|
||||
<tr><td>"This device feeds that one"</td><td>an association edge in the reserved edge space</td><td>EDG</td></tr>
|
||||
<tr><td>Host ↔ device reachability</td><td>a portal descriptor in the reserved portal space</td><td>portal</td></tr>
|
||||
</table>
|
||||
|
||||
<p>The consequence is that "what is this device?" and "is this device healthy?" are not application-specific queries. They are reads at known coordinates, and the same reads work whether they are issued on the device itself or against the host's mirror of the device space.</p>
|
||||
|
||||
<h3>2.3 The device space and the host space</h3>
|
||||
|
||||
<p>A deployed system therefore has at least two spaces: the device's own space, and the host's coordinating space. These are separate cubes. Movement between them is a Portal, not an arithmetic step — which is the whitepaper's model applied literally. A host that can read a device's health subtree was given a portal to it; a host that was not, cannot construct the address.</p>
|
||||
|
||||
<h2 id="tracks">3. Two Deployment Tracks</h2>
|
||||
|
||||
<p>There are two coherent ways to place CUBE on the device, and they differ in where the authoritative store lives.</p>
|
||||
|
||||
<h3>3.1 Track A — ESP32 as a CUBE client</h3>
|
||||
|
||||
<p>The device keeps its configuration and state in Espressif's non-volatile storage (NVS) and speaks a coordinate-shaped request protocol to a host CUBE daemon over MQTT or TCP. A write is expressed as a coordinate write — <code>write c<space>/z…/y…/x…</code> — and a health check is a read at a coordinate. The device holds a local mirror so it keeps operating when the network drops; the host holds the coordinating store.</p>
|
||||
|
||||
<p><strong>New machinery required.</strong> A transport (MQTT or a small TCP framing) and an NVS-backed record mirror on the device. The coordinate types, the record encoding, and the command vocabulary already exist on the host side.</p>
|
||||
|
||||
<p><strong>Where the authority sits.</strong> On the host. The device is a well-behaved participant in the host's graph.</p>
|
||||
|
||||
<h3>3.2 Track B — ESP32 as a standalone CUBE node</h3>
|
||||
|
||||
<p>The device bakes the coordinate layer and the record codec into its firmware, adds an NVS or SD-backed <code>Store</code> implementation, and runs a listener that speaks the CUBE command protocol over TCP or BLE. A host, or a peer device, then writes and reads coordinates against the device directly, and the device is authoritative for its own space.</p>
|
||||
|
||||
<p><strong>New machinery required.</strong> An embedded storage backend, and a listener replacing the host's Unix-domain socket transport. The coordinate types and the canonical record format are already written to be usable without an operating system.</p>
|
||||
|
||||
<p><strong>Where the authority sits.</strong> On the device. This is the version in which a device is a genuine node rather than a peripheral.</p>
|
||||
|
||||
<div class="diagram">
|
||||
<div class="layer">┌──────────────────────────── HOST SPACE ────────────────────────────┐</div>
|
||||
<div class="layer">│ coordinating store · adjacency index · portal catalogue · agent │</div>
|
||||
<div class="layer">└───────────────┬───────────────────────────────────┬─────────────────┘</div>
|
||||
<div class="layer arrow">│ Portal (Track A: MQTT / TCP) │ Portal (Track B: TCP / BLE)</div>
|
||||
<div class="layer">┌───────────────▼───────────────┐ ┌───────────────▼─────────────────┐</div>
|
||||
<div class="layer">│ DEVICE SPACE — client │ │ DEVICE SPACE — standalone node │</div>
|
||||
<div class="layer">│ NVS mirror · config · health │ │ NVS/SD store · config · health │</div>
|
||||
<div class="layer">│ authoritative store: host │ │ authoritative store: device │</div>
|
||||
<div class="layer">└────────────────────────────────┘ └─────────────────────────────────┘</div>
|
||||
</div>
|
||||
|
||||
<table>
|
||||
<tr><th></th><th>Track A — client</th><th>Track B — standalone node</th></tr>
|
||||
<tr><td>Device storage</td><td>NVS mirror</td><td>NVS / SD, canonical</td></tr>
|
||||
<tr><td>Transport</td><td>MQTT or TCP to host daemon</td><td>TCP or BLE listener</td></tr>
|
||||
<tr><td>New work</td><td>transport + NVS mirror</td><td>embedded <code>Store</code> + transport</td></tr>
|
||||
<tr><td>Authority</td><td>host</td><td>device</td></tr>
|
||||
<tr><td>Relative effort</td><td>low</td><td>substantial (days to weeks)</td></tr>
|
||||
<tr><td>Offline operation</td><td>local mirror only</td><td>fully autonomous</td></tr>
|
||||
</table>
|
||||
|
||||
<h2 id="portability">4. What Ports and What Does Not</h2>
|
||||
|
||||
<p>The portability boundary is not a matter of opinion; it is visible in the crates. Two of them were written for targets without an operating system. The rest assume a host.</p>
|
||||
|
||||
<h3>4.1 The crates that port</h3>
|
||||
|
||||
<ul>
|
||||
<li><strong><code>cube-core</code></strong> — <code>#![no_std]</code>, <code>#![forbid(unsafe_code)]</code>, allocation-free. Defines <code>SpaceId</code>, <code>Point</code>, <code>Coord</code>, the <code>Curve</code> trait (Morton / RowMajor / Hilbert), and <code>Portal</code>. It already compiles for bare-metal ARM, and the corresponding <code>riscv32</code> bare-metal targets are a configuration change rather than a rewrite.</li>
|
||||
<li><strong><code>cube-store-raw</code></strong> — <code>#![no_std]</code> and allocation-free by default. Implements the canonical on-disk record format (magic, version, curve tag; then <code>[SpaceId][Key][len][value]</code> records) as zero-copy header parsing and record iteration. Its own documentation names the consumer: a kernel-mode driver that reads blocks without a heap. A flash-backed device store is the same shape of consumer.</li>
|
||||
</ul>
|
||||
|
||||
<h3>4.2 The layers that do not</h3>
|
||||
|
||||
<ul>
|
||||
<li><strong><code>cube-store</code></strong> — standard-library based. It uses the host filesystem, a write-ahead log with group-commit fsync, a JSON snapshot, and atomic rename. A microcontroller has no host filesystem and no fsync, and its flash has a finite erase-cycle budget that a fsync-per-burst log would spend quickly.</li>
|
||||
<li><strong><code>cube-index</code></strong> — the derived adjacency and portal indices. They are rebuilt in memory and kept current as edges mutate. That is correct on a host; it is not a fleet-sized structure for a few hundred kilobytes of SRAM.</li>
|
||||
<li><strong><code>cube-daemon</code> / <code>cubesys</code></strong> — a Unix-domain socket server managed by systemd. Unix sockets and service managers are host concepts. The command protocol itself is transport-agnostic and portable; the socket type is not.</li>
|
||||
<li><strong><code>cubefs</code></strong> — a FUSE front-end. Linux-only.</li>
|
||||
</ul>
|
||||
|
||||
<h3>4.3 Embedded replacements</h3>
|
||||
|
||||
<table>
|
||||
<tr><th>Crate / layer</th><th>std?</th><th>Ports to ESP32</th><th>Replacement or note</th></tr>
|
||||
<tr><td><code>cube-core</code></td><td>no_std</td><td>yes</td><td>as-is; retarget the build</td></tr>
|
||||
<tr><td><code>cube-store-raw</code></td><td>no_std</td><td>yes</td><td>as-is; parse/iterate over a flash block</td></tr>
|
||||
<tr><td><code>cube-store</code></td><td>std</td><td>no</td><td>needs an NVS / SD-backed <code>Store</code></td></tr>
|
||||
<tr><td><code>cube-index</code></td><td>std</td><td>host only</td><td>device keeps only its own slice, or scans</td></tr>
|
||||
<tr><td><code>cube-daemon</code></td><td>std + Unix socket</td><td>no</td><td>TCP or BLE listener speaking the protocol</td></tr>
|
||||
<tr><td><code>cubefs</code></td><td>std + FUSE</td><td>no</td><td>Linux only; not applicable</td></tr>
|
||||
<tr><td><code>cube-agent</code></td><td>std</td><td>host only</td><td>runs where the model and the index run</td></tr>
|
||||
</table>
|
||||
|
||||
<div class="note"><b>Read this as the boundary.</b> The coordinate model and the record format cross to the device. The durable store, the daemon, and the filesystem view do not. Any claim that CUBE "installs on anything" must be read as "<em>the coordinate layer ports; the host stack is replaced</em>."</div>
|
||||
|
||||
<h2 id="graph">5. The Graph and Portal Model at the Edge</h2>
|
||||
|
||||
<h3>5.1 Association edges</h3>
|
||||
|
||||
<p>CUBELinux does not keep relationships in a side table. An association is a first-class record — <code>Edge { from, to, kind, weight }</code> — stored at its own coordinate inside a reserved edge space. A device therefore does not merely hold values; it holds the <em>edges</em> that describe what it feeds, what feeds it, and what it is part of. Because edges are records, they can carry their own header metadata, and they cannot silently diverge from the data they describe.</p>
|
||||
|
||||
<p>Read-optimised traversal is provided by an adjacency index that answers "outgoing edges of this node" and "incoming edges of this node" in time proportional to that node's degree, with a follow operation that walks from a node to a target. The index is derived — rebuilt from the store and updated on mutation — so it can never become a second source of truth.</p>
|
||||
|
||||
<h3>5.2 Portals and capability</h3>
|
||||
|
||||
<p>A <code>Portal</code> is a directed edge from one coordinate to another, possibly crossing a space boundary. Its semantics are deliberately strict:</p>
|
||||
|
||||
<blockquote>A portal is traversed only by standing exactly on its mouth. Traversal is not something that can be done from a distance. The reverse of a portal is a distinct edge and must be created explicitly to be usable.</blockquote>
|
||||
|
||||
<p>Applied to a fleet, this is the authority model. A device is reachable from another space only where a portal exists. A monitoring agent that has been handed a portal to a device's health subtree can read that subtree and nothing else — not because a policy table said so, but because it does not possess the 256-bit selector for the rest. Portals are themselves data, stored in a reserved portal space and keyed by their source space, so every portal leaving a given space can be enumerated with a single prefix scan. That is a discovery primitive: one can ask what a device is wired to without reading the device.</p>
|
||||
|
||||
<div class="note"><b>The gap that matters.</b> Capability here is <em>by construction</em>, not yet <em>by enforcement</em>. The traversal operation checks only that the caller is standing on the mouth; it does not check that the caller holds the portal. "This agent may touch the pump and not the controller" is therefore a property that must be <strong>built</strong> — a gate that refuses a cross-space read or write unless the caller presents the portal to the target space. Until that gate exists, the authority model is a design property rather than a guarantee.</div>
|
||||
|
||||
<h3>5.3 Health as a graph walk</h3>
|
||||
|
||||
<p>Because a device's relationships are edges and its state is coordinate-addressable, a health check is not a list of endpoint calls. It is a walk. Starting from a coordinate — a production line, a room, a subsystem — one follows outgoing edges to every device it comprises, reads each device's health coordinate, and follows incoming edges to establish what is upstream when something has faulted. The same walk answers "what does this affect?" and "what feeds this?", in either direction, in time proportional to the local degree. In a conventional installation that question is answered by joining a device registry, a topology table, and a historian, and the join is only as correct as the last synchronisation.</p>
|
||||
|
||||
<h2 id="agents">6. Agent Monitoring and Automation</h2>
|
||||
|
||||
<h3>6.1 Why recall is a graph walk</h3>
|
||||
|
||||
<p>The whitepaper's retrieval model for memory is association-based: each memory is a record, and recall means traversing edges from one memory to a linked memory, at constant cost per hop and independently of the curve chosen. This is described in the whitepaper as the single highest-leverage idea in the design, because it turns memory from a scan problem into a graph-walk problem.</p>
|
||||
|
||||
<p>The consequence for an edge deployment is the reason this paper exists. The agent's memory is a graph walk. The installation's topology is a graph walk. They are the same operation over the same structure. An agent that has been asked "why did line 3 stop?" walks edges the same way it recalls a prior incident — and the walk that answers the operational question is the walk that retrieves the relevant history.</p>
|
||||
|
||||
<h3>6.2 The monitoring and automation layer</h3>
|
||||
|
||||
<p>A monitoring agent on this substrate does four things, all through one interface:</p>
|
||||
|
||||
<ol>
|
||||
<li><strong>Observe.</strong> Read a coordinate, or walk the graph from a coordinate, to establish state.</li>
|
||||
<li><strong>Recall.</strong> Follow association edges to prior records — incidents, previous readings, prior decisions — using the same traversal.</li>
|
||||
<li><strong>Decide.</strong> A reasoning loop consumes observations and recalled associations. The agent implementation that exists is a ReAct-style loop over a fixed tool set, driving a local model, with its conversation and observations persisted into the cube as durable cross-session memory.</li>
|
||||
<li><strong>Act.</strong> Write a coordinate. A control action is a coordinate write at a known location, subject to the same portal gate as any other write.</li>
|
||||
</ol>
|
||||
|
||||
<p>Observation, memory, and control are therefore not three subsystems. They are reads, walks, and writes against one coordinate graph.</p>
|
||||
|
||||
<h3>6.3 In factory automation</h3>
|
||||
|
||||
<p>A line is modelled as a space, its machines as coordinates within it, and its material flow as edges. A fault triggers a walk: incoming edges identify upstream contributors, outgoing edges identify downstream exposure. A maintenance agent is handed portals to the assets it is responsible for and to nothing else, so its reach is exactly its remit. Because state and relationships share one structure, the agent's recollection of how a similar fault resolved is retrieved by the same traversal that located the fault — the diagnostic path and the memory path coincide.</p>
|
||||
|
||||
<h3>6.4 In home automation</h3>
|
||||
|
||||
<p>A home is a space; rooms, devices, and sensors are coordinates; "this camera watches that door" and "this plug powers that lamp" are edges. An automation such as "when the door opens and the house is empty, raise the lights" is a walk from the door to what observes it and what should respond, rather than a rule whose trigger conditions are maintained separately from the topology. The distinguishing property is the same as in the factory: the assistant's memory of the house and the house's own state are addressed identically, so they cannot disagree about what is connected to what.</p>
|
||||
|
||||
<h2 id="challenges">7. Challenges and Limitations</h2>
|
||||
|
||||
<h3>7.1 Enforcement is not built</h3>
|
||||
|
||||
<p>The capability model described in §5.2 is by construction only. No access-control logic exists in the coordinate, header, index, or store layers; the authorisation code that exists in the project lives in the host daemon and agent, and governs command and file access on the host. A portal gate — refusing a cross-space operation unless the caller holds the portal — is the missing keystone, and it must be built before any claim about a unified authority model is honest.</p>
|
||||
|
||||
<h3>7.2 The device integration is not built</h3>
|
||||
|
||||
<p>Neither track exists. There is no NVS-backed store, no MQTT or BLE transport, and no device-space provisioning. The coordinate layer and the record codec are written to permit these; they are not implemented.</p>
|
||||
|
||||
<h3>7.3 The index does not fit a microcontroller</h3>
|
||||
|
||||
<p>The adjacency and portal indices are derived, in-memory structures. On a host this is unproblematic. On a device with a few hundred kilobytes of SRAM, only the device's own slice is affordable; a whole-fleet index on an MCU is not. The practical consequence is that the fleet-wide graph belongs on the host, with devices as its leaf spaces — which is precisely the split that Track A assumes.</p>
|
||||
|
||||
<h3>7.4 Durability on flash</h3>
|
||||
|
||||
<p>The host store achieves durability through a write-ahead log, group-commit fsync, and checkpointing to a snapshot. None of these primitives exist on a microcontroller, and flash erase cycles are finite. A device-backed store must instead map records onto a wear-levelled medium — non-volatile storage on the device, or a block device where one is present — and the crash-recovery guarantee must be re-established for that medium rather than assumed from the host design.</p>
|
||||
|
||||
<h3>7.5 The inference tier is deferred</h3>
|
||||
|
||||
<p>The project's AI/ML crates are explicitly deferred; the reasoning loop that exists is the agent implementing a ReAct-style tool loop against a local model on a host. On-device inference is a separate, unresolved question, and this paper does not assume it. The device is a space in the graph; the reasoning happens where the model and the index can run.</p>
|
||||
|
||||
<h3>7.6 The microcontroller is not a small server</h3>
|
||||
|
||||
<p>The most common misreading of this proposal is that CUBE "installs" on an ESP32 as it would on a host. It does not. A microcontroller has no operating system unless one is supplied, no host filesystem, no process model, no Unix sockets, and no FUSE. What crosses to the device is the coordinate model and the record encoding. Everything above that boundary is replaced by device-appropriate machinery. The proposal is a port of an addressing model, not a deployment of a host stack.</p>
|
||||
|
||||
<h2 id="conclusion">8. Conclusion</h2>
|
||||
|
||||
<p>The coordinate layer of CUBELinux was written without an operating system in mind, and that decision is what makes an edge deployment coherent. A device can be a space; its configuration, telemetry, and health can be coordinates; its relationships can be first-class edges; and reachability to it can be an explicit portal rather than an address that anything may construct.</p>
|
||||
|
||||
<p>Two tracks follow from this. As a client, a device is a well-behaved participant in a host's graph, and the new work is a transport and a non-volatile mirror. As a standalone node, a device is authoritative for its own space, and the new work is an embedded store and a listener in place of the host's socket. In both cases the coordinate types and the canonical record format are reused rather than rewritten.</p>
|
||||
|
||||
<p>The strongest argument for the approach is not storage speed. It is that state, relationships, reachability, and agent memory become one addressable structure, so that monitoring and recall are the same traversal and control is a write at a coordinate. That unification is the substance of the proposal; performance is a secondary consideration and, in the measurements available, was never the limiting factor.</p>
|
||||
|
||||
<p>Two pieces stand between this proposal and a system: the device-side integration, and the portal gate that converts capability-by-construction into capability-by-enforcement. The second is the more important, because it is what makes a scoped agent — one that can read a pump's health and cannot touch a controller — a guarantee rather than an intention.</p>
|
||||
|
||||
<p>Future work: the portal gate; an NVS and SD-backed <code>Store</code>; a retarget of <code>cube-core</code> and <code>cube-store-raw</code> to a microcontroller target; a device-space provisioning path; and a fleet demonstration in which the whole installation is one adjacency index on a host and each device is one space within it.</p>
|
||||
|
||||
<footer>
|
||||
<p>CUBELinux — CUBE Integration into ESP32 Modules · Preliminary Draft · 2026-09-09</p>
|
||||
<p style="margin-top:6px"><a href="/spec.php" style="color:var(--cyan)">Sign up for the CUBELinux Newsletter →</a> — get update summaries as they happen.</p>
|
||||
<p style="margin-top:6px"><span class="pill opt">ARCHITECTURAL PROPOSAL</span> Not implemented · <span class="pill core">ORIGINAL WORK</span> CUBELinux project</p>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -4,6 +4,7 @@
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
|
||||
<title>CUBELinux — A Coordinate-Addressed OS · Final Whitepaper</title>
|
||||
<meta name="description" content="The full CUBELinux whitepaper: coordinate addressing, the kernel-resident data layer, and WordFlags.">
|
||||
<style>
|
||||
:root{
|
||||
--gold:#d8af68; --gold-hi:#fff9b1; --cyan:#39d7ff; --bg:#0c0f14;
|
||||
@@ -64,6 +65,17 @@
|
||||
.pill.def{background:var(--panel2);border-color:var(--line);color:var(--muted)}
|
||||
.pill.drop{background:rgba(232,160,64,.1);border-color:var(--warn);color:var(--warn)}
|
||||
</style>
|
||||
<!-- Open Graph / Twitter / canonical (site prep) -->
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:title" content="CUBELinux — A Coordinate-Addressed OS · Final Whitepaper">
|
||||
<meta property="og:description" content="The full CUBELinux whitepaper: coordinate addressing, the kernel-resident data layer, and WordFlags.">
|
||||
<meta property="og:url" content="https://cubelinux.com/cubelinux-final-whitepaper.html">
|
||||
<meta property="og:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<meta name="twitter:card" content="summary_large_image">
|
||||
<meta name="twitter:title" content="CUBELinux — A Coordinate-Addressed OS · Final Whitepaper">
|
||||
<meta name="twitter:description" content="The full CUBELinux whitepaper: coordinate addressing, the kernel-resident data layer, and WordFlags.">
|
||||
<meta name="twitter:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<link rel="canonical" href="https://cubelinux.com/cubelinux-final-whitepaper.html">
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
@@ -84,7 +96,9 @@
|
||||
<li><a href="#schema">4. The OS Entity Schema</a></li>
|
||||
<li><a href="#crates">5. The Crate Structure</a></li>
|
||||
<li><a href="#kernel">6. The Kernel Build</a></li>
|
||||
<li><a href="#status">7. Project Status (2026-08-25)</a></li>
|
||||
<li><a href="#status">7. The Journey (2026-08-21 → 2026-09-06)</a></li>
|
||||
<li><a href="#state">8. Where We Are Now</a></li>
|
||||
<li><a href="#next">9. Future Testing & Implementations to Explore</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
@@ -176,6 +190,7 @@
|
||||
<p>Encode cost on 10 million points: row-major 1.4 ns/op, Morton 141.6 ns/op, Hilbert 456.2 ns/op. Hilbert is 326x slower to encode than row-major and 3.2x slower than Morton. The current Hilbert implementation is a naive bit-at-a-time loop and could be made several times faster, but it will never approach row-major's single memcpy.</p>
|
||||
|
||||
<p>The Curve trait paid for itself: three implementations, swapped by type parameter, and the winner was decided by measurement in one sitting.</p>
|
||||
<blockquote><strong>Re-verified 2026-09-06.</strong> We re-ran this exact bake-off in release and reproduced it: <strong>Morton 140.6 ns/pt</strong> (draft 141.6), <strong>Hilbert 454.9</strong> (456.2), <strong>row-major 1.7</strong> (1.4); row-major's unaligned blowup <strong>~77.5 MB / 196 seeks</strong> (draft 79 MB). The conclusion survives; the numbers hold within ~1–2%.</blockquote>
|
||||
|
||||
<h3>2.4 The key layout (192-bit = 24-byte fixed)</h3>
|
||||
|
||||
@@ -258,7 +273,7 @@
|
||||
<li>Versus Redis: comparable raw speed, but Redis is network + multi-client; CUBELinux is a local Unix-socket single-writer coordinate store with EDG graph-walk and hard per-namespace partitioning that Redis does not model.</li>
|
||||
</ul>
|
||||
|
||||
<p>Bottom line: CUBELinux-2 is now in the "durable, coordinate-addressed, sub-15us mean command latency" zone — faster than SQLite's durable path, lighter than RocksDB/LMDB for its single-writer local niche, but not yet a concurrent/multi-tenant DB.</p>
|
||||
<p>Bottom line: CUBELinux-2 is now in the "durable, coordinate-addressed, sub-15us mean command latency" zone — faster than SQLite's durable path, lighter than RocksDB/LMDB for its single-writer local niche, but <strong>not yet a concurrent/multi-tenant DB</strong>. The concrete gap and the path to it are in §9 (concurrency/sharding + multi-writer persistence); the first sharded-read step is measured there.</p>
|
||||
|
||||
<h3>3.5 The audit-path bug and its O(1) fix</h3>
|
||||
|
||||
@@ -278,7 +293,7 @@
|
||||
|
||||
<p><strong>Association-graph model (recommended):</strong> each memory is a leaf keyed by memory_id (hash → 120-bit leaf). Recall = traverse EDG edges (memory → linked memory). That is O(1)-per-hop point/edge lookup — curve-independent. Morton is fine; the edge index does the work.</p>
|
||||
|
||||
<p><strong>Feature-ball model (alternative):</strong> memory leaf = quantized feature vector (fx, fy, fz); recall = nearest neighbours in a ball. Caveat: the bench's aligned_boxes decomposition is axis-aligned only — it cannot represent a ball. Hilbert clusters balls materially better than Morton, so if you commit to feature-ball recall, Hilbert may win the recall workload despite its 411 ns/pt encode tax. Unproven — the bench did not test balls.</p>
|
||||
<p><strong>Feature-ball model (alternative):</strong> memory leaf = quantized feature vector (fx, fy, fz); recall = nearest neighbours in a ball. Caveat: the bench's aligned_boxes decomposition is axis-aligned only — it cannot represent a ball. The bench initially only tested axis-aligned boxes, so this was marked unproven — it has since been tested (2026-09-06): <strong>Morton 2716, row-major 2724, Hilbert 2730</strong> runs for random ball data (2798 ball pts), and a 3-way tie (5165/5166/5191) for clustered features. <strong>Hilbert does not win feature-ball recall.</strong> A sphere does not align with any curve's monotone blocks, so recall is served by the curve-independent <em>1-seek bounding-cube path</em> + in-memory filter; the aligned_boxes decomposition is axis-aligned and cannot represent a ball, but the bounding-cube path makes the curve choice largely irrelevant for recall. (The open question there is the distributed/replicated read path, not the curve.) <strong>Resolved balling algorithm (2026-09-06):</strong> the correct structure for feature-ball / nearest-neighbour-in-ball recall is a <em>metric VP-tree</em> (or a multi-layer HNSW graph), not a curve. A VP-tree over the feature vectors gives <strong>exact 100% recall with ~277 node visits per query</strong> (bounded, log-like) vs any curve reading ~1 run per ball point. It is CUBE-addressable: each tree node = a feature-vector record, its children = <code>linked_records</code> edges. So the feature-ball <em>idea</em> is sound — the curve was the wrong tool for it. <strong>Now built as a real CUBE index</strong> (<code>cubecode::ballindex</code>): the VP-tree is stored as records (<code>doc_type="vp-node"</code>, body = pivot + radius, children = <code>linked_records</code>), and <code>ball_stats</code> walks it. Measured at N=2000: <strong>exact 100% recall, ~125 node-visits/query</strong> (each visit = one record read) — bounded, log-like, vs any curve's ~1 run per ball point.</p>
|
||||
|
||||
<h3>4.2 Sessions / threads (SES)</h3>
|
||||
|
||||
@@ -309,12 +324,12 @@
|
||||
|
||||
<p><strong>NOT PROVEN</strong> (needed before declaring "Morton serves the OS"):</p>
|
||||
<ol>
|
||||
<li>Feature-ball recall — bench tested axis-aligned boxes only.</li>
|
||||
<li>Feature-ball recall — bench tested axis-aligned boxes only. <em>(Since measured 2026-09-06:</em> the three curves give essentially the same run count for a ball — <strong>Morton 2716, row-major 2724, Hilbert 2730</strong> (2798 ball pts) for random, and a 3-way tie (5165/5166/5191) for clustered features. <strong>Hilbert does not win feature-ball.</strong> Ball recall is the curve-independent 1-seek bounding-cube path + in-memory filter; the curve choice is <em>not</em> the lever there.)</li>
|
||||
<li>Edge-walk throughput — the EDG association model is unbenched (the AdjacencyIndex numbers are a first data point, not a full traversal micro-bench).</li>
|
||||
<li>Session replay / project prefix scan at robotics scale — should bench append+replay under a synthetic control-loop tick.</li>
|
||||
</ol>
|
||||
|
||||
<p><strong>Recommendation:</strong> ship Morton first, behind the Curve trait, with space as a routing prefix and class/leaf as the Morton-ordered tail. Build the EDG association index as the primary memory-retrieval path. Keep Hilbert in the trait for a future feature-ball MEM mode. Do NOT claim Morton "serves AI memory/sessions/projects" until #1–#3 are benched.</p>
|
||||
<p><strong>Recommendation:</strong> ship Morton first, behind the Curve trait, with space as a routing prefix and class/leaf as the Morton-ordered tail. Build the EDG association index as the primary memory-retrieval path. Keep Hilbert in the trait — but it is <em>not</em> better for feature-ball recall (see above), so do not reach for it to fix that workload. Since 2026-09-06 the original #1–#3 are benched: feature-ball measured (curves tie), edge-walk ~8.9M edges/ms in-memory / ~5.0M on a real store, session replay a 1-seek prefix scan for every curve.</p>
|
||||
|
||||
<h3>4.7 The power of metatags — why CUBELinux beats hierarchy for retrieval</h3>
|
||||
|
||||
@@ -334,13 +349,13 @@
|
||||
|
||||
<table>
|
||||
<tr><th>PDF crate</th><th>Status</th><th>Notes</th></tr>
|
||||
<tr><td>cubecoords</td><td><span class="pill on">BUILT</span></td><td>→ cube-core (no_std): SpaceId, Point, Coord, Curve trait with Morton/RowMajor/Hilbert, Portal. Compiles for thumbv7em-none-eabihf (bare-metal ARM).</td></tr>
|
||||
<tr><td>tricoding</td><td><span class="pill drop">DROPPED</span></td><td>25% density regression as spec'd (tri-channel packs 6 ASCII + 4 ctrl into 64 bits = 52/64 used, 12 wasted = 25% fewer characters per word than plain 8 ASCII). Deferred off critical path; general compressors will beat it on real corpora.</td></tr>
|
||||
<tr><td>cubecoords</td><td><span class="pill on">BUILT</span></td><td>→ cube-core (no_std): SpaceId, Point, Coord, Curve trait with Morton/RowMajor/Hilbert, Portal; <code>TriWord</code>/<code>WordFlags</code> (16-bit per-word flag field) + <code>HeaderFlags</code>. Compiles for thumbv7em-none-eabihf (bare-metal ARM).</td></tr>
|
||||
<tr><td>tricoding</td><td><span class="pill on">REVISED</span></td><td>The <em>body</em> packing was dropped (density regression + compressors win) — but the 12 "wasted" bits + 4 control bits were realized as a <strong>16-bit per-word flag field</strong> (<code>cubecoords::WordFlags</code>, <code>pack_flags</code>/<code>unpack_flags</code>): real inline metadata, free on raw size. Measurements: raw stays +33% (6 chars/word); zstd +59% with flags reserved, +73% carrying real metadata. See §8.1.</td></tr>
|
||||
<tr><td>cubeheader</td><td><span class="pill on">BUILT</span></td><td>Null-space metadata model. Header flags, associations as first-class edges. The best original idea in the PDF.</td></tr>
|
||||
<tr><td>cubestore</td><td><span class="pill on">BUILT</span></td><td>Store trait + MemStore + FileBackedStore + RawBlockStore + WAL/checkpoint. 24-byte on-disk key, format pinned.</td></tr>
|
||||
<tr><td>cubecrypt</td><td><span class="pill on">BUILT</span></td><td>Record-level encryption (AES-GCM, XTS, ChaCha) with derived ≥256-bit identifiers. Standard primitives. Obfuscation-as-security discarded.</td></tr>
|
||||
<tr><td>cubefs</td><td><span class="pill on">BUILT</span></td><td>FUSE front-end. Maps c<C>/z<Z>/y<Y>/x<X> to coordinates. --socket mode is a genuine FUSE view of the daemon's durable store.</td></tr>
|
||||
<tr><td>cubetrace</td><td><span class="pill def">DEFERRED</span></td><td>DBI via FFI. Not built.</td></tr>
|
||||
<tr><td>cubetrace</td><td><span class="pill on">BUILT</span></td><td>EXECUTION CAPTURE + REPLAY realized (cubecode::vm <code>run_captured</code>/<code>replay</code>); captured traces stored as <code>Kind::Layer</code> records, plus golden-regression and lineage graph-walk (<code>store_trace</code>/<code>replay_trace</code>/<code>capture_golden</code>/<code>verify_golden</code>/<code>lineage</code>) + CLI. See §8.5.</td></tr>
|
||||
<tr><td>cubedbt</td><td><span class="pill def">DEFERRED</span></td><td>Replay engine. Not built.</td></tr>
|
||||
<tr><td>cubeai-core</td><td><span class="pill def">DEFERRED</span></td><td>AI/ML tier. Excluded on this hardware.</td></tr>
|
||||
<tr><td>cubeai-agent</td><td><span class="pill def">DEFERRED</span></td><td>AI/ML tier. Excluded on this hardware.</td></tr>
|
||||
@@ -358,36 +373,47 @@
|
||||
|
||||
<p>The kernel's relevance to CUBELinux is the block-device seam: the cube-store RawBlock backend targets a real NVMe partition through the same BlockDevice trait that FileBlockDevice (regular file stand-in) satisfies. The kernel driver would be a third impl of that trait, and the on-disk format is already pinned (see §3.2), so the kernel driver is a backend impl, not a format change.</p>
|
||||
|
||||
<h2 id="status">7. Project Status — 2026-08-25</h2>
|
||||
<h2 id="status">7. The Journey: 2026-08-21 → 2026-09-06</h2>
|
||||
|
||||
<p>The FUSE proxy work (cube-fuse-proxy, a standalone crate in the workspace) reached the M2(a) gate on 2026-08-25:</p>
|
||||
<p>This is the blow-by-blow of what was built and tried — the story behind the facts in §1–§6, told as we lived it. Each step led to the next, and several open questions were settled by measurement rather than argument.</p>
|
||||
|
||||
<h3>7.1 Where we started (the 2026-08-25 milestone)</h3>
|
||||
<p>The FUSE proxy (cube-fuse-proxy, a standalone crate) reached the M2(a) gate on 2026-08-25: it compiles cleanly (0 errors, 5 trivial warnings), passes <code>./check</code> (fmt + tests + clippy) green, mounts a FUSE filesystem via <code>fuser::mount2</code>, and its proof binary (main.rs, 383 lines) demonstrates seed read/write/remount through the mount. Two tests were added (root-dir readdir, root-dir "."/".." lookup) and two real bugs fixed (the <code>|| true</code> short-circuit removed; root lookup handles "." and ".."). M0 and M1 are committed; M2(a) is demonstrated. That is where the draft left off on 2026-08-25.</p>
|
||||
|
||||
<h3>7.2 Then we went back to the "dropped" tricoding crate</h3>
|
||||
<p>§5 records tricoding as DROPPED because the <em>body</em> packing was a 25% density regression. But that evaluation missed what the 12 "wasted" bits were <em>for</em>. We first measured the body read path to settle whether a tri-channel body was even worth it: <strong>raw body 34,890 MB/s</strong> (memcpy) vs tri-channel <code>unpack_flags</code> <strong>9,062 MB/s</strong> — raw is ~3.9× faster and 33% smaller. So the body stays raw.</p>
|
||||
<p>That freed the 12 wasted bits plus the 4 control bits to be a <strong>16-bit per-word flag/metadata field</strong>, realized as <code>cubecoords::WordFlags(pub u16)</code> co-packed with 6 data bytes in a <code>TriWord</code> (bits 48–63), via <code>TriEnc::pack_flags(WordFlags, [u8;6])</code> / <code>unpack_flags</code> — backward-compatible with the old <code>pack_6</code> (its 4 control bits are the high nibble). Measured, the flag field costs <strong>zero raw size</strong> (still 6 chars/word); compression is <strong>+59%</strong> with flags reserved and <strong>+73%</strong> carrying real metadata. The 16 bits map to CUBE Null-space semantics: arrangement, start/end-of-record, header-vs-payload, continuation, data-type, permissions, encrypted, compressed, stego, null/lookup, checksum.</p>
|
||||
<p>We then wired it through the stack: each record's header carries the record's <code>WordFlags</code> as a fixed <code>u16</code> (header tag 14, alongside the per-record HeaderFlags tag 12), written by <code>encode_header</code>/<code>decode_header</code> and preserved by <code>refresh_flags</code> — so reading the flags needs no body decode. The OS exposes it end-to-end: <code>CubeStore::scan_by_word_flag</code> / <code>ConcurrentStore::scan_by_word_flag</code>, <code>cube show-flags <C.Z.Y.X></code>, <code>cube scan-word-flags <flag|0xbits></code>, and OS-created records are stamped via <code>cubesys::default_word_flags(kind, descriptor)</code> (START|END|IS_HEADER + type=code for Fn/Kernel + ENCRYPTED for security-sensitive).</p>
|
||||
|
||||
<h3>7.3 Then we built cubetrace: capture → replay → artifact</h3>
|
||||
<p>The "cubetrace" layer §5 had marked DEFERRED. We built it. <code>Vm::run_captured(entry)</code> records the <em>exact</em> ordered instruction trace a program actually executed — the "machine code as it runs"; branches resolved, call frames marked. <code>replay(&[Op])</code> interprets that trace on a fresh stack + frame-based locals, with <strong>no access to the original program</strong>, reproducing the observable behaviour (data-stack result + SYS_TRACE output). The trace is stored as a first-class <code>Kind::Layer</code> record linked back to its source (<code>store_trace</code>/<code>replay_trace</code>), so runs persist as linked CZYX records. We added golden-regression (<code>capture_golden</code>/<code>verify_golden</code>: re-run fresh <em>and</em> replay the stored trace, asserting both match the recorded output) and a lineage graph-walk (<code>lineage</code>). The CLI exposes it: <code>trace-capture</code>, <code>trace-replay</code>, <code>golden-capture</code>, <code>trace-verify</code>, <code>trace-list</code>, <code>trace-links</code>.</p>
|
||||
<p>A small solitaire state machine is the worked example: we captured its executed-op trace, stored it as a Layer record, and replayed it to reproduce the game's actions purely from the captured trace — <em>not from its source</em>.</p>
|
||||
|
||||
<h3>7.4 Then we re-verified everything</h3>
|
||||
<p>We ran the full workspace test suite (<strong>~184 tests, 0 failures</strong> (full workspace suite): cubeai 5, cubecode 21, cubecoords 8, cubecrypt 12, cubedbt 4, cubefs 53, cubestore 17, cubesys 54 + cube-server 5, cubetrace 3, <strong>cube-mvw 2</strong>) and re-ran both benches in release. The whitepaper's own numbers reproduced within ~1–2% (see the notes woven into §2.3 and §4.6). The store performance reproduced too: put_raw <strong>237.6 ns/op</strong> (draft 236) and ~<strong>4208k put/s</strong> (draft ~4237k). The one revision: feature-ball recall does <em>not</em> favor Hilbert — the curves tie (Morton 2716 / row-major 2724 / Hilbert 2730 for random; a 3-way tie for clustered), and ball recall is the curve-independent 1-seek bounding-cube path. So the ship-Morton-first recommendation survives; Hilbert stays in the trait but is not for balls.</p>
|
||||
|
||||
<h2 id="state">8. Where We Are Now</h2>
|
||||
<p>CUBELinux is a <strong>coordinate-addressed OS substrate</strong> that addresses bytes by <code>Coord { SpaceId, Point }</code> behind a Curve trait. It has a durable WAL+delta-checkpoint store with a pinned 24-byte on-disk key; a record header that carries both per-record <code>HeaderFlags</code> and a per-word <strong>16-bit <code>WordFlags</code> flag field</strong> (header tag 14); first-class associations (EDG) with an AdjacencyIndex; a safe, deterministic, cube-addressed bytecode VM (<code>cubecode</code>); running <code>cubefs</code> (FUSE) and <code>cubecrypt</code>; and a custom kernel that has been built, installed, and booted. New since 08-25: <strong>cubetrace</strong> (execution capture → replay → store-as-<code>Kind::Layer</code> → golden-regression → lineage), per-word flag queries and inspection, a CUBE VP-tree balling index (<code>cubecode::ballindex</code>), and a full re-verification pass. <strong>And now a real concurrent DB OS backend:</strong> <code>cube-mvw</code> implements <code>CubeBackend</code>, so <code>CubeStore<MvwBackend></code> is a durable, sharded, <strong>multi-writer MVCC</strong> store with <strong>per-shard WALs + group-commit + LSM-lite compaction</strong> and crash recovery — the DB OS is becoming a reality. <strong>~184 tests green</strong> (full workspace, incl. cube-mvw); both benches reproduced.</p>
|
||||
<p><em>Faces it can wear:</em> AI/ML experiment lineage & reproducibility; behavior cloning/imitation (the Solitaire idea generalized); security/forensics/malware behavioral traces; replayable audit/compliance event sourcing; golden-trace regression; agent/LLM session capture; robotics control-loop analysis. (The full-use-case list is in the companion <a href="cubelinux-neural-networks-addendum.html" style="color:var(--cyan)">neural-network addendum</a>.)</p>
|
||||
|
||||
<h2 id="next">9. Future Testing & Implementations to Explore</h2>
|
||||
<p>What we plan to try next — each a measurable step, not a promise:</p>
|
||||
<ul>
|
||||
<li>Compiles cleanly — 0 errors, 5 trivial warnings (unused bindings in stub callbacks: ENOSUS, umask, fh, lock_owner, reply).</li>
|
||||
<li>Passes <code>./check</code> (fmt + tests + clippy), exit 0 — green gate.</li>
|
||||
<li>Mounts a FUSE filesystem via <code>fuser::mount2</code>.</li>
|
||||
<li>Proof binary (main.rs, 383 lines) demonstrates seed read/write/remount through the mount.</li>
|
||||
<li><strong>2 tests added 2026-08-25, both passing:</strong> root-dir readdir (space dirs visible), root-dir "."/".." lookup.</li>
|
||||
<li><strong>2 real bugs fixed 2026-08-25:</strong> root readdir <code>|| true</code> short-circuit removed; root lookup now handles "." and ".." explicitly.</li>
|
||||
<li><strong>Wire WordFlags deeper (partly done 2026-09-06).</strong> The per-word flag stripe + tag-14 are now surfaced in the file-attribute view: <code>cubefs::vfs::Attr</code> carries <code>word_flags</code>/<code>header_flags</code> (from the record header, tag 14/12) and <code>cube stat <path></code> prints them named (verified: <code>word_flags=0x009c (tri6|start|end|header|type=code)</code>). Still open: expose as read-only FUSE xattrs (<code>user.cube.word_flags</code>) and add a per-word flag index for faster <code>scan_by_word_flag</code>.</li>
|
||||
<li><strong>Real-machine trace capture.</strong> CUBEVM is a safe <code>u8</code> bytecode VM; feed a real tracer (Intel PT / QEMU / a JIT listener) into the same trace-record + replay machinery — the mechanism generalizes beyond the VM.</li>
|
||||
<li><strong>Concurrency / sharding (first step measured 2026-09-06).</strong> A single <code>RwLock</code> over one map <em>serializes even readers</em> via atomic count contention: it drops to ~10 M reads/s at 16 threads. Sharding key→shard with a per-shard <code>RwLock</code> scales: <strong>shard-64 hits ~75–80 M reads/s at 16 threads (~7–9× the single lock)</strong>. That is the read side of a multi-tenant DB. Writes need the same sharding plus MVCC/timestamps and a durable concurrent WAL; then per-tenant isolation (each <code>SpaceId</code> is already a namespace boundary), capacity/quota, and transaction isolation complete the picture. <strong>Writes measured (2026-09-06):</strong> a single write lock <em>serializes</em> writers (0.7–2.7 M writes/s at 8–16 threads), while shard-64 reaches <strong>~12.2 M writes/s at 8 threads (~4.5×)</strong>; it saturates at 16 threads (per-shard HashMap rehash/cache contention — a full multi-tenant DB needs per-shard LSM/B-tree + MVCC, not a sharded HashMap). <strong>Step 2 BUILT as a real DB backend (2026-09-06):</strong> <code>cube-mvw</code> is now a <strong>CUBELinux workspace crate</strong> whose <code>MvwBackend</code> implements <code>CubeBackend</code> — so <code>CubeStore<MvwBackend></code> is a durable, sharded, <strong>multi-writer MVCC</strong> store with <strong>per-shard WALs</strong> (parallel durable commits), <strong>LSM-lite compaction</strong> (memtable + WAL delta + compacted checkpoint base), and crash recovery. Tests (2): MVCC latest-version-wins + WAL recovery (drop+reopen restores), and true concurrent multi-writer (8 threads × 2000 via interior-mutability <code>put_shared</code>, all readable); <code>CubeStore<MvwBackend></code> records round-trip. Bench (16 shards): ~394–488k durable puts/s (per-shard WAL), <strong>~8.1M gets/s</strong>, ~5 ms compact. <strong>Group-commit</strong> (batch one fsync per shard for many ops) gives ~<strong>1.1×</strong> more durable writes at this page-cache-warm scale (the write path here is bounded by memtable ops, not fsync; it matters more on colder/larger/remote storage). <em>No tables layer is used:</em> the metatag/coordinate/association model is the schema; per-shard WAL + group-commit + compaction are pure storage-engine on CUBE's own record/block format.</li>
|
||||
<li><strong>Multi-writer / terabyte-scale persistence.</strong> The store is single-writer local and page-cache-warm; scaling to concurrent multi-writer and terabyte/multi-tenant needs the sharded write path + MVCC + a concurrent WAL (LMDB-style copy-on-write base). This follows from the sharding step above.</li>
|
||||
<li><strong>Real kernel driver.</strong> The RawBlock backend targets an NVMe partition through the same BlockDevice trait; a third impl (a <code>.ko</code>) is a backend, not a format change (the format is already pinned).</li>
|
||||
<li><strong>cubedbt + cubeai tiers.</strong> The replay-engine (cubedbt) and AI/ML tiers (cubeai-core/agent) remain deferred; the trace toolchain now feeds their inputs.</li>
|
||||
<li><strong>Feature-ball: the balling index is now built (<code>cubecode::ballindex</code>).</strong> The correct <em>balling algorithm</em> is a metric <strong>VP-tree</strong> (or multi-layer HNSW graph) over feature vectors, stored as CUBE records + <code>linked_records</code> edges — exact 100% recall at ~125 node-visits/query (N=2000). Still open: re-bench at larger scale and add an HNSW variant for approximate top-k at very large N.</li>
|
||||
<li><strong>The neural-network addendum.</strong> State-spanning NNs as optional optimizations over the deterministic core (prefetching, coordinate-conditioned compression, anomaly detection, learned indexing, adaptive caching).</li>
|
||||
<li><strong>CI golden harness (done 2026-09-06).</strong> A <code>ci-golden.sh</code> runs the capture→store→replay→verify golden chain (execution-capture + replay, golden-regression drift, trace-as-Layer, lineage), the CLI <code>trace-verify</code> end-to-end (must PASS), and the sharded-read concurrency step — failing the build on any regression.</li>
|
||||
</ul>
|
||||
|
||||
<p>Remaining gaps (incomplete-but-acceptable-for-a-proof):</p>
|
||||
<ul>
|
||||
<li>Tests cover only root readdir/lookup; remaining crate functionality untested.</li>
|
||||
<li>Work uncommitted (cube-fuse-proxy/ and cube-spawn/ untracked on <code>feat/kernelize-cube-rawblock</code>).</li>
|
||||
<li>create doesn't persist to store (design note, acceptable for proof).</li>
|
||||
<li>flush/release both persist — redundant but harmless.</li>
|
||||
<li>cube-spawn (Phase 3 harness) untracked and untested — forensic report said "compiles, 11 tests pass" but not re-verified.</li>
|
||||
</ul>
|
||||
|
||||
<p>M0 and M1 are committed. M2(a) is demonstrated but uncommitted. PLAN-kernelize-cube.md §3 still lists M2(a) as "pending user go-ahead." User go-ahead received 2026-08-25.</p>
|
||||
|
||||
<p>FileBackedStore durability (WAL + delta checkpoint, 250ms/200-op burst cap, crash loss bounded) is proven and correct — this is the backing store the FUSE proxy uses, so the proxy inherits durability for flushed writes.</p>
|
||||
|
||||
<footer>
|
||||
<p>CUBELinux — A Coordinate-Addressed OS · Final Draft Whitepaper · Prepared 2026-08-21 · Updated 2026-08-25</p>
|
||||
<p>CUBELinux — A Coordinate-Addressed OS · Final Draft Whitepaper · Prepared 2026-08-21 · Updated 2026-09-06</p>
|
||||
<p><a href="/spec.php" style="color:var(--cyan)">Sign up for the CUBELinux Newsletter →</a> — get update summaries as they happen.</p>
|
||||
<p><span class="pill on">BUILT</span> 7 crates · <span class="pill def">DEFERRED</span> 3 · <span class="pill drop">DROPPED</span> 1 · <span class="pill on">KERNEL BOOTED</span></p>
|
||||
<p><span class="pill on">BUILT</span> 8 crates · <span class="pill def">DEFERRED</span> 3 · <span class="pill on">WordFlags REVISED</span> (tricoding → 16-bit flag stripe) · <span class="pill on">KERNEL BOOTED</span></p>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
|
||||
<title>CUBELinux — State-Spanning Neural Networks for the CZYX Coordinate Filesystem</title>
|
||||
<meta name="description" content="Addendum: state-spanning neural networks for the CZYX coordinate filesystem.">
|
||||
<style>
|
||||
:root{
|
||||
--gold:#d8af68; --gold-hi:#fff9b1; --cyan:#39d7ff; --bg:#0c0f14;
|
||||
@@ -61,6 +62,17 @@
|
||||
.diagram .layer:last-child{border-bottom:none}
|
||||
.diagram .arrow{color:var(--cyan);text-align:center;padding:2px 0}
|
||||
</style>
|
||||
<!-- Open Graph / Twitter / canonical (site prep) -->
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:title" content="CUBELinux — State-Spanning Neural Networks (CZYX)">
|
||||
<meta property="og:description" content="Addendum: state-spanning neural networks for the CZYX coordinate filesystem.">
|
||||
<meta property="og:url" content="https://cubelinux.com/cubelinux-neural-networks-addendum.html">
|
||||
<meta property="og:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<meta name="twitter:card" content="summary_large_image">
|
||||
<meta name="twitter:title" content="CUBELinux — State-Spanning Neural Networks (CZYX)">
|
||||
<meta name="twitter:description" content="Addendum: state-spanning neural networks for the CZYX coordinate filesystem.">
|
||||
<meta name="twitter:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<link rel="canonical" href="https://cubelinux.com/cubelinux-neural-networks-addendum.html">
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
|
||||
@@ -4,6 +4,7 @@
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1, viewport-fit=cover" />
|
||||
<title>CUBELinux — Whitepapers</title>
|
||||
<meta name="description" content="The CUBELinux whitepapers: a coordinate-addressed OS where the database lives in the kernel.">
|
||||
<style>
|
||||
:root{
|
||||
--gold:#d8af68; --gold-hi:#fff9b1; --cyan:#39d7ff; --bg:#0c0f14;
|
||||
@@ -55,6 +56,17 @@
|
||||
a.read.optional:hover{background:rgba(57,215,255,.14)}
|
||||
footer{margin-top:32px;padding-top:12px;border-top:1px solid var(--line);color:var(--muted);font-size:12px}
|
||||
</style>
|
||||
<!-- Open Graph / Twitter / canonical (site prep) -->
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:title" content="CUBELinux — Whitepapers">
|
||||
<meta property="og:description" content="The CUBELinux whitepapers: a coordinate-addressed OS where the database lives in the kernel.">
|
||||
<meta property="og:url" content="https://cubelinux.com/cubelinux-whitepapers.html">
|
||||
<meta property="og:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<meta name="twitter:card" content="summary_large_image">
|
||||
<meta name="twitter:title" content="CUBELinux — Whitepapers">
|
||||
<meta name="twitter:description" content="The CUBELinux whitepapers: a coordinate-addressed OS where the database lives in the kernel.">
|
||||
<meta name="twitter:image" content="https://cubelinux.com/images/logo-header@2x.png">
|
||||
<link rel="canonical" href="https://cubelinux.com/cubelinux-whitepapers.html">
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
@@ -62,23 +74,26 @@
|
||||
<header>
|
||||
<div class="logo"></div>
|
||||
<h1>CUBELINUX <span>·</span> Whitepapers</h1>
|
||||
<div class="meta"><b>Two drafts, posted 2026-08-25.</b> Pick one to read in full — each is a standalone HTML page.</div>
|
||||
<div class="meta"><b>Three drafts — two posted 2026-08-25, one posted 2026-09-09.</b> Pick one to read in full — each is a standalone HTML page.</div>
|
||||
</header>
|
||||
|
||||
<div class="note"><b>About these drafts.</b> The whitepaper documents the design, verified on-disk state, and completed kernel build of the CUBELinux coordinate-addressed OS. The addendum is a separate, clearly-labeled <em>preliminary exploration</em> of how state-spanning neural networks might one day layer on top of the CZYX filesystem — nothing in it is built or committed.</div>
|
||||
<div class="note"><b>About these drafts.</b> The whitepaper documents the design, verified on-disk state, and completed kernel build of the CUBELinux coordinate-addressed OS. The addendum is a separate, clearly-labeled <em>preliminary exploration</em> of how state-spanning neural networks might one day layer on top of the CZYX filesystem — nothing in it is built or committed. The ESP32 integration proposal is likewise a <em>preliminary exploration</em> of carrying the coordinate substrate onto microcontrollers, and is likewise not built.</div>
|
||||
|
||||
<div class="note" style="border-left-color:var(--cyan)"><b>CUBELinux Newsletter.</b> Get update summaries as they happen — new builds, verified milestones, and project progress. <a href="/spec.php" style="color:var(--cyan)">Sign up for the newsletter →</a></div>
|
||||
<div class="note" style="border-left-color:var(--cyan)"><b>CUBELinux Newsletter.</b> Get update summaries as they happen — new builds, verified milestones, and project progress. <br><a class="read" style="display:inline-block;margin-top:8px;color:var(--cyan)" href="/spec.php">Sign up for the newsletter →</a></div>
|
||||
|
||||
<h2>The drafts</h2>
|
||||
|
||||
<article>
|
||||
<h3>CUBELinux — A Coordinate-Addressed OS</h3>
|
||||
<div class="date">Final draft · 2026-08-21 · updated 2026-08-25 with FUSE proxy milestone</div>
|
||||
<p>The primary whitepaper. Documents the core claim (a position in space resolves to bytes), the coordinate primitive (256-bit space selector + Morton-ordered leaf behind a Curve trait), the storage layer (WAL + delta checkpoint, 250ms/200-op burst cap), the OS entity schema (MEM/SES/PRJ/EDG, with metatags as first-class citizens), the crate structure (7 built / 3 deferred / 1 dropped), and the custom kernel that has been built, installed, and booted. §7 closes with the FUSE proxy reaching M2(a): 2 tests added, 2 bugs fixed, `./check` green.</p>
|
||||
<div class="date">Final draft · 2026-08-21 · updated 2026-09-06 with WordFlags + cubetrace + verification pass</div>
|
||||
<p>The primary whitepaper. Documents the core claim (a position in space resolves to bytes), the coordinate primitive (256-bit space selector + Morton-ordered leaf behind a Curve trait), the storage layer (WAL + delta checkpoint, 250ms/200-op burst cap), the OS entity schema (MEM/SES/PRJ/EDG, with metatags as first-class citizens), the crate structure (7 built / 3 deferred / 1 dropped), and the custom kernel that has been built, installed, and booted. §7 closes with the FUSE proxy reaching M2(a): 2 tests added, 2 bugs fixed, `./check` green.<br/><br/><strong>Updated 2026-09-06</strong> — the tri-channel word's 12 "wasted" bits were realized as a <strong>16-bit per-word flag field</strong> (<code>cubecoords::WordFlags</code>, header tag 14) with <code>scan_by_word_flag</code>/<code>show-flags</code>; the <strong>cubetrace</strong> layer is now built (execution capture → replay → store as <code>Kind::Layer</code> records → golden-regression → lineage walk, with the Solitaire example); and the whitepaper's own benchmarks were <strong>re-tested in release</strong> — ~180 tests green, encode costs/box bake-off/session-replay/edge-walk reproduced within ~1–2%, with one revision: feature-ball recall does <em>not</em> favor Hilbert. See §8–9.</p>
|
||||
<footer>
|
||||
<div class="tags">
|
||||
<span class="pill primary">primary deliverable</span>
|
||||
<span class="pill primary">WordFlags 16-bit flag field</span>
|
||||
<span class="pill primary">cubetrace capture + replay</span>
|
||||
<span class="pill booted">kernel built + booted</span>
|
||||
<span class="pill booted">~180 tests green</span>
|
||||
<span class="pill standard">durable WAL store</span>
|
||||
<span class="pill standard">FUSE proxy M2(a)</span>
|
||||
</div>
|
||||
@@ -101,8 +116,23 @@
|
||||
</footer>
|
||||
</article>
|
||||
|
||||
<article>
|
||||
<h3>CUBE Integration into ESP32 Modules</h3>
|
||||
<div class="date">Preliminary draft · 2026-09-09</div>
|
||||
<p>A proposal for carrying the coordinate substrate onto ESP32-class microcontrollers. Describes the device as a first-class coordinate space — its own 256-bit <code>SpaceId</code>, with configuration, telemetry, and health held at coordinates and its relationships held as first-class association edges. Presents two deployment tracks: the device as a CUBE <em>client</em> speaking a coordinate protocol to a host daemon over MQTT/TCP, and the device as a standalone CUBE <em>node</em> with a non-volatile store and its own listener. Draws the precise portability boundary — <code>cube-core</code> and <code>cube-store-raw</code> are <code>no_std</code> and cross to the device, while the durable store, daemon, and FUSE view do not — and closes on the graph-and-Portal model for a device fleet plus the agent monitoring layer, in which recall and topology are the same graph walk. Central thesis: CUBE does not merely store bytes on a device faster; it makes monitoring, memory, and control one traversable, capability-gated coordinate graph.</p>
|
||||
<footer>
|
||||
<div class="tags">
|
||||
<span class="pill optional">architectural proposal</span>
|
||||
<span class="pill explored">edge / embedded</span>
|
||||
<span class="pill standard">not implemented</span>
|
||||
<span class="pill original">original work</span>
|
||||
</div>
|
||||
<a class="read optional" href="cubelinux-esp32-integration.html">Read the proposal →</a>
|
||||
</footer>
|
||||
</article>
|
||||
|
||||
<footer>
|
||||
<p>CUBELinux — Whitepapers · Posted 2026-08-25 · <a href="/" style="color:var(--cyan)">Back to CUBELinux.com</a></p>
|
||||
<p>CUBELinux — Whitepapers · Posted 2026-08-25 · updated 2026-09-09 · <a href="/" style="color:var(--cyan)">Back to CUBELinux.com</a></p>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user