From f78fe31c923f5f0863ee3a2656754ef1163d7fd4 Mon Sep 17 00:00:00 2001 From: Igor Lins e Silva <4753812+igorls@users.noreply.github.com> Date: Wed, 5 Aug 2026 23:59:58 -0300 Subject: [PATCH] docs: scope the mesh docs to what 3.7.0 actually ships The website pages arrived from the full RFC 004 branch and documented the memory op-log, `mempalace oplog`, and automatic bidirectional memory convergence. None of that is in this slice: the CLI has logstream, artifact, and replica, and memory moves by one-way `replica pull`. - cli.md: drop the `mempalace oplog` section (the command does not exist); note that full convergence is staged. - replicated-palace.md: split the sync table into shipping (coordination, multi-master) vs next (memory ops, bidirectional); say plainly that two machines capturing independently do not merge yet. - shared-brain.md: mark step 3 a pull you re-run, not a converging loop. - RFC 004: record which steps landed. --- docs/rfcs/004-replicated-palace.md | 2 ++ website/concepts/replicated-palace.md | 44 ++++++++++++++++----------- website/guide/shared-brain.md | 19 +++++++++--- website/reference/cli.md | 34 +++++++-------------- 4 files changed, 54 insertions(+), 45 deletions(-) diff --git a/docs/rfcs/004-replicated-palace.md b/docs/rfcs/004-replicated-palace.md index 54c5b5a..02ea6df 100644 --- a/docs/rfcs/004-replicated-palace.md +++ b/docs/rfcs/004-replicated-palace.md @@ -1,6 +1,8 @@ # RFC 004: The Replicated Palace Status: Draft complete — all sections drafted (storage: mac-claude; transport/lifecycle/appendix: windows-claude); awaiting Igor's review + +Shipped so far: step 0 (logstream multi-master replication) and step 1 (memory read replicas — snapshot pull, local fold, distributed embedding), plus the transport seam and the estate endpoint. Step 2a (the memory op-log, anti-entropy, fold/promote) and the v4 content-pure id migration + write-flip are designed here but not yet landed on develop — they carry a palace migration and are staged for a later release. Owners: mac-claude (storage layers, §6–§9), windows-claude (transport & lifecycle, §5 and Appendix A), decided by Igor Created: 2026-07-02 Branch: `feat/shared-brain-dogfood` diff --git a/website/concepts/replicated-palace.md b/website/concepts/replicated-palace.md index 2dba83c..e07020d 100644 --- a/website/concepts/replicated-palace.md +++ b/website/concepts/replicated-palace.md @@ -28,7 +28,7 @@ an edge case. │ agents → 127.0.0.1 │ ops │ agents → 127.0.0.1 │ ops │ agents → local │ │ ┌─────────────────┐ │ ◀─────▶ │ ┌─────────────────┐ │◀────▶│ ┌──────────────┐ │ │ │ mempalace hub │ │ │ │ mempalace hub │ │ │ │ mempalace hub│ │ - │ │ op-log │ │ │ │ op-log │ │ │ │ op-log │ │ + │ │ event log │ │ │ │ event log │ │ │ │ event log │ │ │ │ derived index │ │ │ │ derived index │ │ │ │ derived index│ │ │ └─────────────────┘ │ │ └─────────────────┘ │ │ └──────────────┘ │ └─────────────────────┘ └─────────────────────┘ └──────────────────┘ @@ -39,9 +39,11 @@ an edge case. per-hub bearer tokens. The transport lives behind a seam (`MEMPALACE_TRANSPORT`), so a decentralized mesh-identity transport can replace tokens without touching anything above it. -2. **Sync** — append-only logs of *ops*, merged by union. Every mutation - (a filed drawer, a knowledge-graph fact, a coordination event) becomes an - immutable, provenance-stamped op that travels between replicas. +2. **Sync** — append-only logs of *ops*, merged by union. Each op is + immutable and provenance-stamped, and travels between replicas. Today + coordination events and artifacts move this way; memory content moves by + one-way replica pull until the memory op-log lands (see + [What syncs today](#what-syncs-today)). 3. **Derived state** — vector indexes and caches are rebuilt or folded locally, never copied. **Sync the facts, derive the senses**: ops are kilobytes; vector indexes are gigabytes. Every machine remembers @@ -71,19 +73,26 @@ converge through a common peer. Gossip, in the practical sense. | Layer | Mechanism | Status | |---|---|---| | Coordination events + artifacts (the [agent logstream](/concepts/agent-logstream)) | op sync, multi-master | shipping | -| Memory content (drawers) | snapshot bootstrap + memory ops | shipping (ops in shadow) | -| Knowledge graph | snapshot bootstrap + memory ops | shipping (ops in shadow) | +| Memory content (drawers) | snapshot pull + local fold (**one-way**) | shipping | +| Knowledge graph | snapshot pull + local fold (**one-way**) | shipping | | Vectors | never synced — derived locally, or folded from a peer's [vector cache](#distributed-embedding) | shipping | +| Memory content (drawers), bidirectional | memory op-log + anti-entropy | next | | Organization (wings/rooms/tunnels as ops) | op vocabulary reserved | next | -"Shadow" is deliberate engineering honesty: the memory op-log currently runs -in a **dual-write shadow** — every drawer and knowledge-graph write also -emits an op, ops travel and fold into peer replicas, but each machine's -vector store remains its system of record until the shadow proves itself. -`mempalace oplog verify` replays the op-log against the live store and must -stay clean over a real usage period before ops become authoritative. A fold -never mutates a locally-authored drawer from a remote op during the shadow; -conflicts are counted and preserved for review, not applied silently. +Read the split carefully, because it is the difference between what works +today and what the rest of this page describes. **Coordination is already +multi-master**: any agent on any machine appends events, and the logstream +converges in both directions. **Memory is not yet.** Drawers and graph facts +move via `mempalace replica pull` — a one-way, insert-only fold from an +origin you name. Two machines that each capture their own conversations do +not merge; each pulls what it wants from the other. + +The **memory op-log** — provenance-stamped ops for every drawer and graph +write, anti-entropy sync, and a fold that resolves cross-replica edits by +last-writer-wins — is the mechanism that closes that gap. It is designed +(RFC 004 step 2a) and staged for a later release, along with the +content-pure id recipe it depends on. Until it lands, treat each replica's +own captures as authoritative locally. ## Bootstrapping a new machine @@ -112,8 +121,9 @@ drawers carry a `replica_origin` stamp naming where they came from — ask "who is X?" on any machine and the answer arrives with its provenance. Once the hub starts, the background sync loop (every -`MEMPALACE_SYNC_INTERVAL` seconds, default 15) keeps everything converging: -logstream events, memory ops, and the fold — no cron jobs, no manual steps. +`MEMPALACE_SYNC_INTERVAL` seconds, default 15) keeps the logstream +converging on its own — no cron jobs, no manual steps. Memory pulls are +still an explicit `mempalace replica pull` until the memory op-log lands. ## Distributed embedding @@ -179,4 +189,4 @@ Two things replication is **not**: agents included - [Agent Logstream](/concepts/agent-logstream) — the coordination layer that pioneered the op-sync machinery -- [CLI reference](/reference/cli) — `mempalace replica` and `mempalace oplog` +- [CLI reference](/reference/cli) — `mempalace replica` diff --git a/website/guide/shared-brain.md b/website/guide/shared-brain.md index e2dedc3..9e01531 100644 --- a/website/guide/shared-brain.md +++ b/website/guide/shared-brain.md @@ -372,16 +372,25 @@ Joining the mesh is three steps per machine: ``` The hub's background loop picks up `peers.json` changes within one sync - cycle — coordination events, memory ops, and the fold all converge every + cycle — coordination events and artifacts converge every `MEMPALACE_SYNC_INTERVAL` seconds (default 15) with no further action. -3. **Bootstrap the memory** once, with the local hub stopped: +3. **Pull the memory**, with the local hub stopped: `mempalace replica pull --with-vectors` folds every peer's authored content — and their precomputed vectors — into the local palace. See the [CLI reference](/reference/cli#mempalace-replica). After that, delegation works exactly as described in this guide — but an -agent's inbox, and the palace behind it, survive any single machine -sleeping. `GET /sync/peers` on any hub shows the estate: which peers were +agent's inbox survives any single machine sleeping. + +::: warning Coordination converges on its own; memory does not yet +Step 2 is continuous and bidirectional. Step 3 is a **one-way pull you +re-run**: it folds what the peers have authored into this machine, and it is +insert-only and resumable, so re-running it heals any gap. What it does not +do is merge — a drawer edited on two machines will not reconcile itself. +Bidirectional memory convergence is the memory op-log (RFC 004 step 2a), +staged for a later release. Until then, put `mempalace replica pull` on a +schedule if you want each machine to stay current. +::: `GET /sync/peers` on any hub shows the estate: which peers were reachable last round, their version vectors, and any replicas known only through gossip. @@ -402,4 +411,4 @@ the mesh is a first-class citizen of it. - [Agent Logstream](/concepts/agent-logstream) — the event/artifact model in depth - [Remote / Team Server](/guide/remote-server) — full hub deployment: tokens, TLS, backends, Docker/systemd - [MCP Integration](/guide/mcp-integration) — the memory tools every connected agent gets -- [CLI Reference](/reference/cli#mempalace-logstream) — `mempalace logstream`, `mempalace artifact`, `mempalace replica`, `mempalace oplog` +- [CLI Reference](/reference/cli#mempalace-logstream) — `mempalace logstream`, `mempalace artifact`, `mempalace replica` diff --git a/website/reference/cli.md b/website/reference/cli.md index 0c1f117..e801d27 100644 --- a/website/reference/cli.md +++ b/website/reference/cli.md @@ -243,27 +243,15 @@ origins to be quiescent (no active mines). Pulls are insert-only and resumable — re-running heals any gap. Raise `MEMPALACE_SYNC_HTTP_TIMEOUT` (seconds, default 30) for large bootstraps. -## `mempalace oplog` +A running hub syncs the logstream automatically every +`MEMPALACE_SYNC_INTERVAL` seconds (default 15). The CLI verbs exist for +bootstraps, offline machines, and inspection. -The canonical memory op-log (RFC 004 step 2a — currently in dual-write -shadow). Every drawer and knowledge-graph write also lands as a -provenance-stamped op in `oplog.sqlite3`; ops travel between replicas and -fold into their stores. - -```bash -mempalace oplog status --json # counts, kind histogram, version vector -mempalace oplog sync # pull missing ops from peers -mempalace oplog fold # apply pulled ops to the local store (hub stopped) -mempalace oplog verify # replay ops vs the live store — the cutover gate -``` - -| Subcommand | Description | -|------------|-------------| -| `status` | Op counts, per-kind histogram, and this replica's version vector | -| `sync` | Anti-entropy pull of missing memory ops (`--peer URL --token T`, or all peers) | -| `fold` | Apply pulled remote ops to the local store — stop the hub first; a running hub folds on its own sync cadence | -| `verify` | Replay the op-log against the live store; exits `1` on divergence | - -A running hub does all of this automatically every `MEMPALACE_SYNC_INTERVAL` -seconds (default 15): logstream sync, memory-op sync, then the fold. The CLI -verbs exist for bootstraps, offline machines, and inspection. +::: tip Full convergence is staged +`mempalace replica pull` gives you **read replicas**: a one-way fold of +drawers and graph facts from an origin. Bidirectional convergence — where +every replica is an equal writer and edits merge automatically — is RFC 004 +step 2a (the memory op-log), staged for a later release. Until then, treat +each replica's own captures as authoritative locally and pull from the +origins that hold what you want mirrored. +:::