diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 3355115..680296d 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -64,6 +64,7 @@ export default withMermaid( { text: 'Cursor IDE Hooks', link: '/guide/cursor-hooks' }, { text: 'Configuration', link: '/guide/configuration' }, { text: 'Remote / Team Server', link: '/guide/remote-server' }, + { text: 'Shared Brain', link: '/guide/shared-brain' }, ], }, ], diff --git a/website/guide/shared-brain.md b/website/guide/shared-brain.md new file mode 100644 index 0000000..e3c11a2 --- /dev/null +++ b/website/guide/shared-brain.md @@ -0,0 +1,320 @@ +# Shared Brain: One Palace for Your Whole Agent Fleet + +Run one MemPalace hub and let every agent you work with — Claude Code on your +Mac, Codex on a Windows box, OpenCode on a laptop, a Hermes bot on a home +server — read, write, and coordinate through the **same palace**. One memory, +many minds. This guide takes you from zero to a working fleet. + +## What you're building + +``` + mac-claude ──────┐ stdio auto-proxy ┌─ mempalace serve + mac-codex ───────┤ or HTTP (loopback) │ (one host owns the palace) + ├───────────────────────────▶│ + windows-codex ───┤ HTTPS + bearer token │ drawers + KG + diary ← memory + laptop-opencode ─┘ (tailnet / proxy) │ logstream + artifacts ← coordination + └────────────────────────── +``` + +One hub process owns the palace. Every agent — local or remote — talks to it +over MCP. The hub gives your fleet **two distinct layers**: + +| | Memory (drawers, KG, diary) | Logstream (events, artifacts) | +|---|---|---| +| Holds | Durable knowledge worth recalling | Active work moving between agents | +| Access | Semantic search | Structured filters + long-poll | +| Examples | Decisions, facts, people, outcomes | Delegations, replies, patches, acks | + +Rule of thumb: if another agent should **act** on it, it's an event. If a +future session should **know** it, it's a drawer. A concluded delegation +usually produces both — the events carried the work, a drawer records the +outcome. The event model is covered in depth in +[Agent Logstream](/concepts/agent-logstream). + +## 1. Start the hub + +Pick the machine that will own the palace (the one with your data, or the one +with a GPU for embedding) and start the hub on loopback: + +```bash +mempalace serve --host 127.0.0.1 --port 8765 +``` + +That's it for a single-machine fleet. One `serve` process holds the palace's +writer lease and safely serializes concurrent writes from every client — +never point two server processes at the same palace. + +## 2. Connect the local agents + +Agents on the hub machine need **zero reconfiguration**. If you've already +set them up with the normal stdio server +([MCP Integration](/guide/mcp-integration)): + +```bash +claude mcp add mempalace -- python -m mempalace.mcp_server +codex mcp add mempalace -- python -m mempalace.mcp_server +``` + +…each stdio process checks for a live hub serving its palace and +**auto-proxies every request to it** instead of opening its own database +handles. The check runs per request (a tiny local read of the hub's +registration file), so plugins and desktop apps join the shared brain — and +follow a restarted hub — without touching their config. Set +`MEMPALACE_HUB_FORWARD=0` to opt out. + +Local agents that speak HTTP natively can also connect directly to +`http://127.0.0.1:8765/mcp`. + +## 3. Bring in remote machines + +For agents on other machines, keep the loopback bind and front it with a +tailnet or HTTPS reverse proxy at a name like `memory.example.com`. The full +hub-side recipe — bearer tokens, `MEMPALACE_MCP_EXTRA_ALLOWED_HOSTS` for the +fronted hostname, TLS options, networked storage backends, Docker/systemd — +lives in [Remote / Team Server](/guide/remote-server); follow that guide +once, then connect each remote agent: + +```bash +claude mcp add --transport http mempalace https://memory.example.com/mcp \ + --header "Authorization: Bearer $MEMPALACE_MCP_HTTP_TOKEN" +``` + +One trap worth calling out: the hub **auto-generates a bearer token only for +non-loopback binds**. A loopback bind fronted by a proxy is tokenless unless +you set one explicitly — mint one and pass it at startup: + +```bash +mempalace serve --host 127.0.0.1 --port 8765 --token "$(openssl rand -hex 32)" +``` + +(or export `MEMPALACE_MCP_HTTP_TOKEN` before starting the hub). + +::: warning Never expose the logstream unauthenticated +Events and artifacts carry work metadata and patch contents. The hub's +bearer-token policy covers them — don't weaken it with `--allow-insecure` +outside a trusted proxy setup, and verify the token is actually set when +fronting a loopback bind. +::: + +## 4. Give every agent a name + +Every agent needs **one stable identity** in `-` format: +`mac-claude`, `mac-codex`, `windows-codex`, `laptop-opencode`. This is the +`from_agent` on every event it writes and the `to_agent` others use to reach +it. Never rotate names and never impersonate another agent — the append-only +event trail is only auditable if identities are stable. + +## 5. Wire the protocol into each agent + +Agents don't discover the etiquette on their own; you teach it once, in +their instruction files. The canonical copy lives in +[`integrations/shared/coordination-protocol.md`](https://github.com/MemPalace/mempalace/blob/develop/integrations/shared/coordination-protocol.md) +— that file is the single source of truth and the version below tracks it. +Copy the snippet verbatim (so the rules never drift per-agent), replacing +`` with the agent's identity: + +```text +## MemPalace shared brain + +You share a MemPalace hub with other agents. Your agent identity is + — use it as from_agent/created_by in every MemPalace call. + +Memory (recall + writing): +- Before answering about past work, decisions, people, or projects, + search the palace (mempalace_search; mempalace_kg_query for + relational/temporal facts). Quote results verbatim — never paraphrase + stored content. If the palace has nothing, say so; don't guess. +- File durable outcomes (decisions, conclusions, learned facts) with + mempalace_add_drawer. When a fact changes: mempalace_kg_invalidate the + old fact, then mempalace_kg_add the new one. Don't file secrets or + tokens. + +Coordination (logstream): +- Check your inbox when starting work and before long tasks: + mempalace_event_list with to_agent= (new since your last + seen event id). +- To delegate: mempalace_event_append (type=task.request, stream= + project/, room=delegation, correlation_id=task_..., status=open, + body = goal + branch + base commit + definition of done), then + mempalace_event_wait on that correlation_id for the reply. +- When you accept a task: ack it with status=claimed. Deliver code as a + patch via mempalace_patch_submit (never just push a branch and go + silent). If blocked, reply with status=blocked and verbatim notes. +- When you receive a patch: mempalace_artifact_get, verify sha256, + apply only with explicit user-visible intent, run the stated tests, + then mempalace_event_ack with status=applied or failed. +- Events are append-only and verbatim. Close every loop — no task you + touched stays open without an applied/failed/blocked ack. +``` + +Where it goes depends on the harness: + +| Harness | Instruction file | +|---|---| +| Claude Code | `~/.claude/CLAUDE.md` | +| Codex CLI | `~/.codex/AGENTS.md` | +| OpenCode | `~/.config/opencode/AGENTS.md` | +| Hermes | the agent's `SOUL.md` | + +The memory half composes with the +[recall protocol](https://github.com/MemPalace/mempalace/blob/develop/integrations/shared/recall-protocol.md); +link the canonical files rather than restating them. + +## 6. Run your first delegation + +The canonical loop: request → claimed → patch.ready → verify → apply → ack. +The steps below use the [CLI one-liners](/reference/cli#mempalace-logstream) +because they're the easiest way to follow (and debug) a loop; agents drive +the same operations through the matching MCP tools. Every `--json` result +includes the event or artifact `id` — the `evt_...` / `art_...` values in +later steps come from the previous command's output (e.g. `| jq -r .id`). + +**1. Request** — `mac-claude` files the task; one `correlation_id` ties the +whole exchange together: + +```bash +mempalace logstream append --type task.request --stream project/myapp \ + --room delegation --from-agent mac-claude --to-agent windows-codex \ + --correlation-id task_fix_ranking_7f3a --status open \ + --branch fix/ranking --base-commit abc1234 \ + --body "Fix the search ranking regression. Done = uv run pytest tests/test_searcher.py passes." \ + --json | jq -r .id # -> evt_... (the request event) +``` + +**2. Claim** — `windows-codex` long-polls its inbox, then acks so no other +agent duplicates the work: + +```bash +mempalace logstream wait --to-agent windows-codex --type task.request \ + --timeout-ms 300000 --json # request event id is .events[0].id +mempalace logstream ack evt_... --from-agent windows-codex --status claimed +``` + +**3. Deliver** — after doing the work on the stated branch, it stores the +diff byte-exactly and announces `patch.ready` referencing the artifact (over +MCP, `mempalace_patch_submit` does both in one call): + +```bash +git diff | mempalace artifact put --kind patch --created-by windows-codex \ + --json # note .id (art_...) and .sha256 +mempalace logstream append --type patch.ready --stream project/myapp \ + --room patches --from-agent windows-codex --to-agent mac-claude \ + --correlation-id task_fix_ranking_7f3a --status ready \ + --artifact-id art_... --body "Ranking fixed; tests green on Windows." +``` + +Pushing a branch is **not** a handoff — the event is. + +**4. Verify and apply** — `mac-claude` has been waiting on the correlation +id. The `patch.ready` event carries only the artifact id; the hash to check +is the artifact's own `sha256`, returned by `artifact put` and +`artifact get`. Fetch the expected hash, compute the actual one, and only +then apply — `artifact get` prints exact bytes on stdout, so it pipes +straight into `git apply`: + +```bash +mempalace logstream wait --correlation-id task_fix_ranking_7f3a \ + --type patch.ready --to-agent mac-claude --timeout-ms 300000 --json + +mempalace artifact get art_... --json | jq -r .sha256 # expected +mempalace artifact get art_... | shasum -a 256 # actual — must match + +mempalace artifact get art_... | git apply --3way # explicit, user-visible +uv run pytest tests/test_searcher.py -q +``` + +**5. Close the loop** — ack with the result, then file the outcome as a +drawer so the decision is searchable without replaying the event trail: + +```bash +mempalace logstream ack evt_... --from-agent mac-claude --status applied \ + --body "Patch applied on abc1234; tests/test_searcher.py green." +``` + +`wait` is a long-poll: default timeout 60000 ms, capped at 300000 ms. On +timeout the CLI exits `2` instead of erroring, so agents loop on it, passing +`--since-event-id` of the last event seen; over MCP a timeout returns +`{timed_out: true, events: []}`. If the worker can't produce a patch, it +still replies — `task.reply` with `status=blocked` or `failed` and verbatim +notes. Silence is the only unrecoverable failure. + +## Fleet roles and cadence + +Not every agent should do every job. Lessons the fleet reported from its own +first delegations: + +- **Route work by agent type.** CLI agents with persistent terminals + (Claude Code, Codex) take builds, test runs, and patch production. Desktop + assistants take quick recall lookups, hash verifications, status + summaries, and filing outcomes — short, synchronous actions that survive + the user closing the window mid-session. Don't delegate a test suite to + an agent whose session can vanish at any moment. +- **The desktop assistant is the user's gateway.** Users don't read the + logstream; they ask their assistant. Its most valuable fleet role is + translation — turning `patch.ready` events into a plain-language summary, + and turning conversational intent into well-formed coordination events. +- **Match inbox cadence to agent shape.** A daemon-adjacent CLI agent can + long-poll continuously; an ad hoc assistant should check at session start + and before long tasks, and no more. +- **Watch your whole inbox, not just known tasks.** A watcher filtered on + one `correlation_id` misses unsolicited requests and broadcasts. Poll + `to_agent=` (which also matches `*`) with `--since-event-id` as the + cursor. +- **Keep memory writes user-visible.** File a drawer when a durable decision + is made — and say so ("saving this decision to the shared brain") rather + than filing silently. Transparency is what makes a fleet the user cannot + directly inspect trustworthy. + +## Hard rules + +The same non-negotiables that govern memory govern coordination: + +- **Append-only.** Events are immutable. Corrections are new events + (`status=superseded`) referencing the old one — never edits or deletes. + Even `logstream ack` appends an `event.ack` event; it never mutates the + original. +- **Verbatim payloads.** Bodies and artifacts are exact — no summarized + diffs, no truncated logs. Too big for a body? Store it as an artifact. +- **Close every loop.** Every claimed `task.request` ends in `applied`, + `failed`, or `blocked`. No dangling `open` tasks. +- **Never apply a patch silently.** Fetching an artifact is free; applying + it is an explicit local decision, stated to the user. +- **Verify hashes.** An artifact's `sha256` must match its content before + you act on it. +- **Store diffs byte-exactly.** A patch stored without its final newline has + its last hunk line truncated — `git apply` rejects it as corrupt — and + CRLF line endings are often rejected too. Pipe `git diff` straight into + `artifact put` rather than copy-pasting; the store warns at store time on + both problems (CLI warnings go to stderr, so `--json | jq` stays clean). + Treat a warning as a broken handoff and re-store the diff. +- **File the outcome.** When a delegation concludes, write one drawer + recording what was decided, so the result is searchable without replaying + the event trail. + +## Operating the shared brain + +- **Upgrades**: the hub process serves the tool list, so new tools (or a new + MemPalace version) appear fleet-wide after a **hub restart**. Stdio + proxies re-check for a live hub on every request, so clients follow a + restarted hub — even on a new port — with no restart or reconfiguration. +- **Monitoring**: `GET /healthz` is a token-free liveness probe. + `GET /statusz` (follows the bearer-token policy) returns JSON with + version, uptime, request counters, SQLite integrity, writer mode, and + recently observed MCP clients — a quick way to confirm every agent in the + fleet is actually connected. +- **Coordination survives index work**: the logstream lives in its own + `logstream.sqlite3` next to the palace and opens no vector-index handles. + Delegations keep flowing while the palace is being mined, repaired, or + rebuilt. +- **Read-only observers**: a hub started with `--read-only` exposes recall + plus `event_list`, `event_wait`, and `artifact_get`; mutating tools — + including `event_append`, `event_ack`, `artifact_put`, and + `patch_submit` — are hidden and refused. Useful for a dashboard or an + agent that should watch the fleet but never write. + +## See also + +- [Agent Logstream](/concepts/agent-logstream) — the event/artifact model in depth +- [Remote / Team Server](/guide/remote-server) — full hub deployment: tokens, TLS, backends, Docker/systemd +- [MCP Integration](/guide/mcp-integration) — the memory tools every connected agent gets +- [CLI Reference](/reference/cli#mempalace-logstream) — `mempalace logstream` and `mempalace artifact` flags