Windows CI intermittently returns zero hybrid hits right after a fast
seed write (same class as "Nothing found on disk" on tiny collections).
Close the palace client after seeding so the next open re-reads flushed
segments, retry search once if empty, and assert non-empty results with
a clear message instead of IndexError.
Move Unreleased entries into the 3.7.0 section, add the Wave 2 headlines
(Hermes core, source adapters, stale-library MCP gate, date search,
openai-compat embeddings, integrity/encoding fixes), and set the release
date to 2026-08-11.
The #2200 reap tests only monkeypatched HOME. On Windows expanduser("~")
reads USERPROFILE, so the reaper scanned the real home and the suite
failed on test-windows after Wave 2. Share _isolate_home() that sets both.
MemoryStack/Layer1 opened a second PersistentClient on the same palace
while the Hermes provider already held one for live filing. Concurrent
access corrupted local Chroma SQLite (disk I/O / Failed to get segments)
and failed CI on develop after #1915.
Wake-up L1 now scans the long-lived collection under the collection lock,
and filing holds that lock for the full upsert. Also rewrite the RFC 001
section-4.4 docstring to avoid the internal §N jargon guard.
Close the last open items on #743 before merge:
- Conformance: document two isolation arms (cross-id for all backends;
same-id/different-namespace for supports_namespace_isolation advertisers).
- No silent drop: non-advertising backends must raise UnsupportedCapabilityError
when PalaceRef.namespace is set, rather than accept-and-ignore.
- Wire require_namespace_support() into chroma/sqlite_exact; add conformance test.
- Refresh implementation-status banner now that #1727/#1731/#1732/#1734 landed.
#757 landed mtime/inode cache invalidation and mempalace_reconnect
in mcp_server._get_client(). Both are Chroma-specific (stat of
chroma.sqlite3). They should migrate into ChromaBackend.get_collection
and ChromaBackend.close_palace during the §10 cleanup so the freshness
contract lives inside the backend, not in the caller.
Addresses the actual spec defects flagged in #743 review, ignoring
operator-UX asks that are not plugin-contract concerns.
- Goal #3: 'without data loss' → mirrors §8.2's capability-conditional
lossless-vs-reembed framing. No more overpromise.
- §1.5: `server_embedder` is no longer an implicit escape hatch from
identity/dimension rules. Such backends MUST expose an effective
identity via `effective_embedder_identity()` and are bound by the
same three-state check.
- §7.3: adds `maintenance_kinds: ClassVar[frozenset[str]]` advertisement
mechanism. `run_maintenance(kind)` must raise
UnsupportedMaintenanceKindError for unadvertised kinds. Benchmark
harness reads this set rather than guessing kind names. Reserves
`analyze`/`compact`/`reindex` as well-defined names.
- §1.2: adds `update()` as optional method with a default get+merge+
upsert implementation. §2.1: `supports_update` redefined to gate
atomic single-round-trip semantics (not mere capability), since the
default impl already supports partial updates.
Operator asks explicitly NOT adopted (diplomatic shims, not contract
defects): `.to_dict()` compat on typed results, migration progress
reporting, `BaseBackend.repair()` separate from `run_maintenance`,
per-palace capability variance, identity recording on read-only ops.
Incorporates review feedback from skuznetsov (Postgres, #665) and
dekoza (Lance, #574) on issue #737:
- §1.5: split 'accepts embeddings=' (signature compliance) from
'persists embeddings as-is' (correctness). Adds
supports_embeddings_passthrough capability; the former is universal,
the latter is required to label a migration lossless.
- §1.5: model identity check becomes a three-state machine
(known_match / known_mismatch / unknown) so legacy palaces without
recorded identity don't hard-fail on upgrade.
- §1.4: makes explicit that supports_contains_fast is the ONLY
performance floor the spec promises; without it callers MUST assume
O(n). $contains is a correctness requirement, not a performance one.
- §3.3: clarifies auto-detect is an upgrade-compat path only, never
the selection mechanism for new palaces.
- §8.2: migrate CLI refuses to run against a target lacking
supports_embeddings_passthrough unless --accept-re-embed is passed;
migration record now captures lossless status and model identities.
Copilot review flagged back-references in §1.4 and §6 that still used
the pre-skuznetsov-rename names (`$contains_fast`, `sync_capable`,
`change_feed`). Updated to the `supports_*` prefix used in the §2.1
capability table.
Formalizes the BaseCollection/BaseBackend contract introduced as a seam
in #413 into an interchangeability spec that third-party backends can
build to. Driven by six in-flight backend PRs (#574, #643, #665, #697,
#700, #381) each implementing the interface differently.
Key decisions captured: entry-point distribution, typed QueryResult/
GetResult replacing Chroma dict shape, daemon-first multi-palace model
via PalaceRef, required where-clause subset (incl. $contains),
mandatory embedder injection with model-identity validation, capability
tokens, shared pytest conformance suite, and a backend-neutral
migrate/verify CLI.
After rebasing #1671 onto current develop:
- Opt test_embedding_api out of conftest's stable EF mock so the
get_embedding_function selection tests exercise the real factory.
- Move the CHANGELOG entry from released 3.7.0 Performance into
Unreleased Features (rebase context had drifted).
The section only ever documented `docker build`, so every reader
compiled the image locally even though `ghcr.io/mempalace/mempalace` is
published multi-arch. Worse, a clone builds `develop` (the default
branch), not the release — a build and a pull could hand you different
versions with no hint that they differ.
Lead with `docker pull`, and cover the things that actually cost people
an evening:
- the container only sees what you mount, so the MCP client config now
mounts a transcripts directory; without it the server starts fine and
every mine finds nothing
- paths become container paths after that, and `~` / `$HOME` are not
expanded by every MCP client
- the first embedding call downloads ~80 MB (minilm) or ~300 MB
(embeddinggemma) into /data, which reads as a hung container
- bind mounts keep host ownership and the image runs as uid 1000, so a
0700 directory fails with a bare PermissionError on Linux; Docker
Desktop's uid mapping hides this on macOS and Windows. `--user` is
called out as the wrong fix — /data is mode 700 owned by uid 1000, so
another uid cannot write the palace at all
- mining never writes to its source, so the examples mount it read-only
- the GPU image is x86_64-only; onnxruntime-gpu has no aarch64 Linux
wheels, so that build fails on Apple Silicon
Every command in the section was run as written against the published
image, and each claim checked rather than assumed: the uid and mode of
/data read out of the image, `--user` confirmed to break a *fresh*
volume, the read-only mount confirmed to still mine, and the aarch64 GPU
failure reproduced.
The first CI run failed at step 3 with
PermissionError: [Errno 13] Permission denied: '/work/mempalace.yaml'
`mktemp -d` creates the fixture 0700 owned by the runner user. Bind
mounts carry host ownership through unchanged, and the image runs as
uid 1000, so the container could not stat inside /work. Docker
Desktop's uid mapping hides this on macOS, which is why it passed
locally and only failed on Linux.
Model an ordinary project checkout instead — 0755 dir, 0644 file — which
is the shape that makes the README's `-v /path/to/project:/work` work
against a normal repo.
The Docker workflow built both images and never started a container,
and never parsed a Compose file. A green run therefore only meant the
Dockerfile compiled. Two defects that break the very first documented
command shipped past it: `docker-compose.yml` carried a bare
`environment:` key that made Compose reject the file outright (#2188),
and `_embed_texts` handed chromadb `np.float32` scalars so `mine`
aborted on the first drawer (#2187).
Add `scripts/docker-smoke.sh`, which exercises what the README tells
users to run:
1. `compose config` on docker-compose.yml and the server compose file
2. entrypoint dispatch for both `cli ...` and bare passthrough
3. `mine` a mounted directory, asserting a drawer is filed
4. `search` from a *separate* container, asserting the stored text
comes back verbatim — this is the assertion that matters, since
storing user words exactly is the promise the palace makes
5. a real MCP stdio JSON-RPC handshake: initialize, tools/list, and a
mempalace_search call whose result must contain the drawer
It asserts on returned content, not just exit codes, and lives in a
script rather than inline YAML so it runs identically on a laptop:
`scripts/docker-smoke.sh <image>`.
The new `smoke` job builds amd64 natively with `load: true` (buildx
cannot load a multi-arch manifest) and reads the publish job's cache
while writing its own scope, so an amd64-only export never lands on top
of the multi-arch one. `build` now needs it, so a failing smoke test
blocks publication rather than being noticed afterwards.
Verified by reintroducing each defect against a real build: the compose
regression fails at step 1, the embedding regression at step 3, and the
current tree passes all five. Failure output is clipped to 500 columns
because a rejected embedding batch otherwise prints a whole 384-dim
vector on one line and buries the message.
`environment:` was present with nothing but comments beneath it, so YAML
parsed it as null and Compose refused the whole file:
services.mcp.environment must be a mapping
That is every documented Compose command — `docker compose build`,
`docker compose run --rm mcp`, `docker compose down` — failing before
anything starts, on any machine. The README points at this file.
Comment the key out along with its example entries, and say in the file
why it cannot be left bare. The examples switch to mapping syntax so
uncommenting them yields a valid block.
Verified with `docker compose config` on the shipped file (now valid)
and by running the documented flow against a build of this tree: mine a
mounted directory through `docker compose run --rm mcp cli mine`, then
read it back with `... cli search`, which returns the drawer verbatim.
`deploy/docker-compose.server.yml` was checked for the same defect and
is unaffected.
Chroma declares `requires_explicit_embeddings`, so every write on the
default backend routes through `EmbeddingCollection`. `_embed_texts`
built its rows with `list(v)`, and `v` is a float32 `np.ndarray` — that
unpacks into `np.float32` *scalars*, which chromadb's
`normalize_embeddings` rejects:
ValueError: Expected embeddings to be a list of floats or ints, a
list of lists, a numpy array, or a list of numpy arrays
`mine` aborted on the first drawer, as did every other write against a
default palace. Convert with `.tolist()` (C-speed), keeping a
`float(x)` branch for embedders that already return plain sequences.
The suite could not see this. conftest's autouse
`_stable_embedding_function_for_tests` monkeypatches
`embedding_wrapper._embed_texts` itself for every module outside
`_REAL_EMBEDDING_TEST_MODULES`, so the defective function was never
executed under test. The regression tests therefore go in
`test_embedding.py`, which is exempt from that stub: one asserts the
returned elements are builtin floats, one drives a real Chroma
collection through `EmbeddingCollection.upsert` and reads the document
back. Both fail against the previous line with the production
ValueError.
Verified end to end outside the suite: mining a project and searching
it back returns the drawer verbatim, on the host and in the container
image built from this tree.
`mempalace_diary_write` returns an `entry_id` for every diary entry, but
for entries large enough to be chunked that id was unusable: get_drawer,
update_drawer and delete_drawer all answered "Drawer not found", and
list_drawers showed the entry as N unrelated chunk rows.
Two metadata conventions never met. The diary chunking path stamped
`parent_entry_id` on each chunk, while the logical-id read paths added in
#1782 query only `parent_drawer_id`. Both keys mean the same thing --
"physical chunk of this logical drawer" -- so chunk groups written by
diary_write were invisible to logical-id resolution. Same bug class as
#1763, which #1782 fixed for `add_drawer` drawers only.
Read paths now resolve either key via `_PARENT_ID_KEYS`:
- `_logical_chunk_group()` matches both with an `$or` (fixes get /
update / delete). All four backends support `$or`.
- `_collapse_drawer_rows()` groups on either (fixes list_drawers, which
the `$or` alone does not cover).
- `searcher._result_drawer_id()` resolves either, so a hit on a chunked
diary entry reports the id that fetches the whole entry rather than
the single chunk that matched.
New diary writes also stamp `parent_drawer_id` alongside
`parent_entry_id` so the two conventions converge going forward. Because
the read paths still accept the `parent_entry_id`-only shape, palaces
written before this fix are repaired with no data migration.
Diary chunks are written without `source_file`, so neighbor expansion
(#1580) returns early on them and is unaffected by the added key.
Also drops the comment telling callers to iterate `chunk_ids` (it
documented the bug as intended behavior) and a stale claim that search
rejoins chunks via `parent_entry_id` -- no search code read that key.
The dependabot bump moved pyproject's ruff pin to 0.16.1, but nothing else
followed it, so the repo asked for three different versions at once:
- pyproject.toml said 0.16.1 (the bump)
- uv.lock still resolved 0.15.20 (dependabot did not update it)
- .github/workflows/ci.yml installed 0.15.14 by its own literal pin
The lint job never reads pyproject, so CI kept linting with 0.15.14 and
reported this PR green without 0.16.1 ever running. The ci.yml pin had
already drifted from pyproject before this bump, under a comment saying to
keep them identical.
- ci.yml: pin 0.16.1 to match pyproject.
- uv.lock: regenerated so the locked resolution agrees.
- test_ruff_pins_match: assert ci.yml and pyproject stay equal, so the next
bump that touches only one of them fails loudly instead of passing blind.
- extend-exclude '*.md': 0.16 began formatting Python inside markdown
fences, taking the formatter from 199 files to 295 and reflowing
hand-aligned example code in docs/rfcs/002 and three website pages.
Excluding docs keeps this a version bump rather than a silent
documentation reflow, and restores the exact file scope the project has
always formatted.
Verified with 0.16.1 actually installed: ruff check and ruff format --check
both clean over the same 199 files, full suite green (3669 passed).
Python's text layer applies universal-newline translation: CRLF becomes LF
on read, and on Windows LF expands back to CRLF on write. Every CLI path
that carried artifact and event content went through it, so a patch piped
in from a Windows agent was stored as different bytes than the ones git
produced — under a sha256 of the mangled content, which then failed to
match on the machine that applied it. For a store whose contract is
verbatim bytes addressed by digest, that is silent corruption.
It also disarmed the CRLF warning added for kind=patch: put_artifact looks
for a carriage return that the read path had already stripped, so the one
check meant to catch unappliable diffs could never fire on the platform
that produces them.
- _read_stdin_exact() / _write_stdout_exact(): go through sys.stdin.buffer
and sys.stdout.buffer, falling back to the text layer when a replaced
stream has no .buffer.
- --file / --body-file: read_bytes().decode() instead of read_text().
- artifact get --out: write_bytes() instead of write_text().
Verified through real pipes: a 63-byte CRLF patch round-trips at 63 bytes
with an unchanged sha256 via stdin, --out, and stdout, and the CRLF warning
now fires. Regression coverage in TestVerbatimNewlines, confirmed failing
against the old behavior first.
Config reads (peers.json, replica.json, server registry, token file) are
left on read_text: JSON is newline-insensitive and none of them carry user
content.
Reported by Copilot on the pull request.
Narrows the 3.7.0 slice to the feature users actually get: RFC 003 agent
coordination (events, artifacts, patch handoffs) plus the multi-master
logstream sync that makes it work across machines.
Removed, deferred with the rest of RFC 004:
- replica_sync.py / vector_cache.py and the `mempalace replica` CLI
- the /snapshot/* hub endpoints they backed
- website/concepts/replicated-palace.md
Memory read replicas were a leaf on the dependency graph (nothing in the
logstream path imports them), and shipping them half-done meant documenting
a mesh whose memory does not actually converge. Dropping them lets the docs
say one true thing instead of two hedged ones: coordination syncs, memory
stays local, point every agent at one hub if you want shared recall.
Fixes a bug found by running it: _start_peer_sync_thread() read peers.json
once at startup and returned early when absent, so a hub started before
peers.json was written never synced — silently, forever. That is the order
the guide tells users to follow. Membership is now re-read every round.
Verified on two live hubs: delegation loop end-to-end, verbatim patch
round-trip by sha256, bidirectional sync, CLI sync alongside a live hub,
and automatic convergence 15s after writing peers.json with no restart.
The website pages arrived from the full RFC 004 branch and documented the
memory op-log, `mempalace oplog`, and automatic bidirectional memory
convergence. None of that is in this slice: the CLI has logstream, artifact,
and replica, and memory moves by one-way `replica pull`.
- cli.md: drop the `mempalace oplog` section (the command does not exist);
note that full convergence is staged.
- replicated-palace.md: split the sync table into shipping (coordination,
multi-master) vs next (memory ops, bidirectional); say plainly that two
machines capturing independently do not merge yet.
- shared-brain.md: mark step 3 a pull you re-run, not a converging loop.
- RFC 004: record which steps landed.
The rebase resolution had taken the branch version of both files wholesale,
dropping develop's backend table, version badge, mempalace_kg_supersede
section, and the checkpoint added_by row. Rebuilt as develop's content plus
only the RFC 003 logstream additions; tool count is 43 (36 + 7 logstream).
Identity becomes the triple host:agent:project — the granularity at which an
actor actually shares filesystem, local config, and knowledge. Two sessions in
the same project on the same box are the SAME actor with the SAME identity;
session/PID is event metadata, never identity. Fixes the live collision where
two windows-claude sessions interleaved inboxes, claims, and diary under one
flat name.
Two decisions resolved (both live in the backend/hub):
- Matcher: NO change. Route the tuple as an opaque string on RFC 003's exact +
'*' broadcast; colons already pass _sanitize_routing. Hierarchical/glob
routing is deferred — it's a real index/correctness surface and no concrete
need has landed. This ships as a convention + one renderer change, zero risk
to the routing hot path.
- Order: host:agent:project (host-first), matching how mac-*/windows-*/blade-*
names already read; migration is a pure suffix append, not a rename.
Claim safety under a shared identity needs no lease server: natural mutex
(don't claim what your identity already claimed) plus RFC 004 R3's earliest-HLC
tiebreak for the residual race. The durable fix is one place — the renderer of
the mempalace-shared-brain block emits the tuple, so every box re-derives on
next sync instead of being hand-edited. Deferred behind the RFC 004 write-flip;
touches no storage or merge semantics.
(cherry picked from commit b043c29fd6a65ea32f92252b8afab1b62bc95bf5)
Event bodies are stored verbatim and fleet status updates run to several
KB, so listing many events full-body is a large payload — agents hit it
catching up on the stream. New preview=true truncates each body to a
200-char excerpt (marking body_truncated + body_length) while keeping
every routing/metadata field, so an agent can scan and then re-fetch
only the events it cares about in full via a targeted since_event_id.
Default (full verbatim) is unchanged; the verbatim-on-demand guarantee
is intact.
(cherry picked from commit dcf43984b84baea49838b2c665d62e572d259615)
The estate is observability — logstream + sync state + peers.json, no
FTS5 dependency; the profile's drawer count already degrades
gracefully. A damaged palace is exactly when mesh visibility matters
most. Caught live on the third replica: its FTS5 index went malformed
and the integrity gate refused the estate tool along with the palace
tools.
(cherry picked from commit c694a8401cb2efd0ca482344503956e4b4c5bad3)
Every hub now advertises a self-derived node profile: roles
(replica/agents/compute), resolved accelerator + embedder, live drawer
count, hardware string, advertised_at. Every field is derivation from
what the daemon observes about itself — never configuration, so mesh
dashboards render what each machine reported, not what a UI guessed.
Profiles ride the existing sync surfaces: /sync/version_vector gains
additive profile/profiles fields, each round's stats carry them, and
carriers relay profiles for origins they only know transitively —
/sync/peers now serves per-peer profiles plus origin_profiles keyed by
replica id. Unreachable peers keep their last advertised profile.
New MCP tool mempalace_mesh_peers returns exactly the /sync/peers
payload from the same producer function, so the desktop app consumes
the estate through the existing bridge — one integration, one token
path, and the tool and endpoint can never drift. Tokens are never in
the payload, asserted in tests.
(cherry picked from commit f31d0904167f5463285ae78de335bcea304ed71a)
Called out explicitly after proving it live: a pre-mesh palace joining
the replicated palace keeps everything (nothing re-mined, nothing lost),
serves its history to peers via one embed-cache + pull, and its drawers
are first-class mesh citizens. This is the path most existing users will
take, so the guarantee belongs in the guide, not in an RFC appendix.
(cherry picked from commit 1023368eed16f16e29544ba44f562872d26df101)
The docs ended at the single-hub logstream era; the mesh has since gone
from RFC to production. This brings the site current:
- NEW concepts/replicated-palace.md: one logical palace across N
machines — the availability invariant, the three layers, ops/HLC/
version vectors, gossip convergence, snapshot bootstrap + peers.json,
distributed embedding (sync the facts, derive the senses), the
/sync/peers estate surface, and the trust story (bearer tokens today,
mesh identity next). Status-honest: the memory op-log is documented
AS the dual-write shadow it currently is, with oplog verify named as
the cutover gate.
- reference/cli.md: mempalace replica (pull/embed-cache), mempalace
oplog (status/sync/fold/verify), logstream sync — with the
operational rules (hub-stopped pulls, quiescent origins,
MEMPALACE_SYNC_HTTP_TIMEOUT for big bootstraps).
- guide/shared-brain.md: new 'From hub to mesh' section — the
three-step join recipe and why the mesh removes the hub as a single
point of failure; See-also links updated.
- Sidebar: Replicated Palace under Concepts.
Example hostnames throughout; vitepress build clean.
(cherry picked from commit 63c2c3c20fcd350b8aca6b3249dd1a08ba5ecc22)
Layer 2 promised it would see peers only through a transport interface;
until now that seam existed in the RFC but not in the code. This lands
it: mempalace/transport.py defines the full contract (self_id / peers /
request now; open_stream / on_presence_change / on_inbound documented
and explicitly deferred to the MeshGuard binding with loud
NotImplementedError), and today's shipping behavior — tailnet HTTPS with
peers.json bearer tokens — becomes its first implementation,
HttpsBearerTransport, moved wholesale below the seam (load_peers,
PEERS_FILENAME, the wire primitive, the timeout knob).
get_transport(palace_path) is the single swap point: MEMPALACE_TRANSPORT
defaults to https; 'meshguard' is reserved and fails loudly until the
binding ships — a user who asked for mesh-identity auth must never
silently run on the bearer-token model it replaces. sync_all now
iterates the transport's membership snapshot (injectable for tests and
for the binding).
Compat surface preserved exactly: logsync re-exports load_peers /
PEERS_FILENAME / _peer_get, and SyncPeerError IS TransportError (same
class), so every existing importer and except-clause keeps working.
Context: three-replica mesh went live tonight on bearer tokens; joining
the third node took ~10 manual credential operations for 2 of 3 edges —
the N-squared out-of-band token relay this seam exists to retire.
MeshGuard trust-path sweep (RFC 004 pre-integration gate a) running in
parallel; the FFI binding (gate b) builds on this seam next.
tests: seam contract, https impl wiring, factory swap point incl.
reserved-name loudness, compat surface. Full suite: 3496 passed.
(cherry picked from commit 45b0aa335375af2e7c93eb9b4b805bcc6877b219)
PalaceMind's mesh view had no data source: the hub exposed version
vectors and op ranges but nothing that says which replicas exist, what
they're called, whether they were reachable last round, or how far
behind they are. A third replica joining the mesh was invisible to the
UI even though every version vector already carried its origin.
GET /sync/peers (bearer, lock-free like the rest of /sync) now serves
the RFC 004 estate data from this replica's point of view:
- self: replica_id, hostname, local version vector
- peers: configured peers.json entries (name + url — the token is
NEVER included) merged with last-round sync state recorded by the
anti-entropy loop: peer replica_id, reachable, last_success_at /
last_error(_at), last pulled counts, and the peer's remote version
vector (drift = remote vector vs self vector, computed by consumers)
- unnamed_origins: origins present in the local log but not configured
as peers — replicas known only transitively through a carrier
- sync_interval_s so consumers can pace their polling
sync_with_peer now returns remote_version_vector; the sync loop folds
each round's outcome into _PEER_SYNC_STATE (whole-entry replacement; an
error round preserves the last known good state so the UI can show
"unreachable since X, last seen at vector V").
tests: estate payload over real HTTP incl. token-privacy assertion and
a transitively-known origin; error-round state preservation.
Full suite: 3484 passed.
(cherry picked from commit ffb8f81ef41b2af32747f58f42b4f354b56d5f0b)
The hardcoded 30s per-request timeout failed in production during the
third-replica bootstrap: a deep-offset snapshot page (Chroma get(offset=N)
scans N rows) behind the mac's tailscale-serve TLS proxy exceeded 30s at
offset 27500 of a 31k-drawer origin. The pull is resumable so nothing was
lost, but bootstrap pulls need a higher ceiling. Default unchanged at 30s;
malformed values degrade to the default. Longer-term fix (tracked): id-
cursor snapshot pagination so page cost stops growing with offset.
(cherry picked from commit 30715f438eb0427cb6783f2a55e0330f20652b8b)
RFC 004 distributed derivation, first brick (asked by Igor: 'could the
mesh become the user's personal GPU cluster?'). A vector is a pure
function of (content, embedder identity); with a pinned fleet identity it
becomes a portable fact computable on whichever machine has the
horsepower.
vector_cache.py: sidecar vector_cache.sqlite3 keyed (drawer_id,
embedder), packed float32; build_cache() bulk-embeds local documents —
read-only on the content store (quiescence-safe), idempotent and
resumable. CLI: mempalace replica embed-cache [--model] [--batch]
[--all].
Serving: /snapshot/drawers?vectors=<model> joins the cache and ships
base64 float32 per item; manifest advertises per-model cache counts.
Fold: mempalace replica pull --with-vectors verifies the served identity
against the LOCAL embedder identity (loud refusal — a foreign-model
vector poisons the index silently), upserts with explicit embeddings,
falls back to local embedding per missing item; stats report
vectors_used vs locally_embedded.
Six new tests incl. the strict one: a fully-vectored fold must invoke
zero local embedding. Full suite 3450 green. Immediate use: the 156k
Windows merge — origin embeds on the 9950X3D under 'minilm', the mac
folds insert-only, hub window drops from ~an hour to minutes.
(cherry picked from commit a134f50cef950affb4888e9fed36489e433ec824)
--no-kg was only honored with an explicit --peer; pull_from_peers now
accepts and forwards it. Caught during the first production reverse pull
(Windows pilot batch, 2,235 drawers folded into the live mac palace in
31.7s including local embedding).
(cherry picked from commit a609e32fc60ca6f63101a2782c91124c1df61d0d)
Raised by windows-claude before its conversation-mining run (option C on
correlation windows_conversation_mining_20260702): with snapshots serving
the full collection, a reverse pull would re-import the puller's own
drawers re-stamped as peer-origin — an echo loop corrupting provenance and
delete-reconciliation scoping. Snapshots now serve only what a replica
AUTHORS (drawers without a replica_origin stamp), making each machine the
origin of its own history; pull pairs converge with no echo. Pages expose
next_offset (raw scan cursor) since filtered pages can be sparse; the pull
engine follows it.
Also adds pull_kg=False / CLI --no-kg: KG rows carry no origin stamp yet,
so a reverse pull could overwrite newer local KG state with the peer's
stale replicated copy — disabled for reverse pulls until step 2a gives KG
facts op provenance. New test: two-origin bidirectional pull, no echo,
clean provenance both sides. Full suite 3444 green.
(cherry picked from commit cafe7fba34868bd86280f8d0c2efbc18a3f50ae0)
Revisions kept forever (no GC path; search surfaces heads only); v4
migration staged on a copy before live-with-backup; step 2 ships as 2a
(drawers+KG ops) then 2b (registry/hallways); mac origin dual-writes
during a shadow period before cutover; remote write-flip (step 3) gated
on the local-capture promotion validating end-to-end.
(cherry picked from commit 34a264e12bdc3ba6f40877972e9435e3f912ee6d)
Driven by a real customer: the Windows machine holds un-mined projects and
conversations. Capture must happen where the data lives (mining reads local
disk), so step-1 replicas may mine locally now — unstamped drawers are
untouchable by reconciliation — and step 2 commits to a one-time promotion
pass turning them into drawer.add ops under the replica's identity.
Content-pure v4 ids double as the cross-machine dedup: identical content
mined on two origins collapses to one drawer at merge.
(cherry picked from commit d18b2a03519a1cfbe9e10bbf12c902c307195c15)
Recall becomes local on every machine. The hub serves the palace's FACTS
over GET /snapshot/manifest|drawers|ids|kg (bearer, paged): verbatim drawer
documents + metadata and raw KG rows — never the vector index, which each
replica derives with its own embedder (RFC 004 layer 3: sync the facts,
derive the senses).
replica_sync.py folds pulls idempotently: drawer upserts by id with ONE
additive provenance key (replica_origin = origin replica id), which scopes
delete reconciliation strictly to copies from that origin — locally-authored
drawers are untouchable. KG rows fold INSERT OR REPLACE by id, so
invalidations converge on re-pull; replication never deletes KG rows.
KnowledgeGraph gains dump_rows/apply_row for rowid-paged replication.
CLI: mempalace replica pull [--peer URL --token T] [--no-reconcile]
[--json]; defaults to peers.json. Step-1 boundary is explicit: writes still
belong to the origin (multi-writer memory is step 3); freshness is re-pull
until the step-2 op-log gives precise tails.
10 new tests over the production HTTP server with the sqlite_exact backend:
endpoint pagination, full fold + local search of pulled facts, idempotent
re-pull, reconcile that deletes origin copies but spares local drawers, KG
invalidation convergence, CLI. Full suite 3443 green.
(cherry picked from commit ebd772b00f73c35e571df2c2caebc6177853bb4a)
The pilot of the replicated palace: per-origin append-only op sync between
peer replicas, pull-based anti-entropy, no coordinator, no new deps.
Core: replica identity minted per palace (replica.json, upgradeable to the
mesh identity later via alias); hybrid logical clock (fixed-width sortable
stamps, restart-seeded from the log, absorbs remote instants); events gain
origin_replica/origin_seq/hlc — additive to the viewer compat surface, with
seq remaining the LOCAL arrival cursor per the RFC. Pre-replication logs
migrate in place: existing rows backfill as authored by this replica with
their original order preserved.
Sync: version vectors diff into per-origin pull ranges; artifacts fold
before the events that reference them (never a dangling id) with sha256
verified on receive; every apply is idempotent, so crash mid-round just
re-pulls the tail. Hub serves GET /sync/version_vector|ops|artifact
(bearer-authenticated, lock-free) and runs an optional background
anti-entropy loop when peers.json exists (MEMPALACE_SYNC_INTERVAL, default
15s). CLI: mempalace logstream sync [--peer URL --token T] [--json].
22 new tests: HLC ordering/regression/observe, identity mint, migration
backfill, idempotent apply + hash rejection + verbatim copy, bidirectional
two-replica convergence, partition with duplicate claims resolving
deterministically by HLC (R3), endpoints + CLI over real HTTP. Full suite
3433 green.
(cherry picked from commit 340b529e7c6a087adab190023e72230c8311f878)
Transport/lifecycle sections (§5), transport-side security (§10.1, §10.3),
and the PalaceMind appendix authored by windows-claude and delivered as a
patch artifact over the logstream (correlation
rfc004_replicated_palace_position, artifact art_20260702T081251_76aef8fb111a,
sha256 verified, applied at ab6f362). §10.2 at-rest encryption drafted by
mac-claude on apply: per-replica data keys that never traverse the mesh,
two-tier design (FDE baseline + SQLCipher-style page encryption over the
canonical stores, OS-keystore wrapped), replica-local rotation, R7
composition with the revocation ceremony.
The seam contract (§5.1) verifies against the Layer 2 design: request/stream
map to anti-entropy pull + SSE push-notify, and ReplicaId doubles as
origin_replica so provenance and authentication are one fact. RFC 004 is now
fully drafted and awaiting Igor's review.
(cherry picked from commit 9f3333998673fb196664325396c104036f771d1f)
One logical palace per human, fully replicated per machine: agents always
local, services converge via provenance-stamped ops over an encrypted
mesh. Judged against offline as the default posture (R0-R8).
Storage sections drafted (mac-claude ownership per the logstream debate on
correlation rfc004_replicated_palace_position): canonical op-log with HLC +
per-origin version vectors, complete merge-semantics table, v4 content-pure
id migration (verified mutable-state inventory with file refs), backends
demoted to fold-and-index consumers, provenance/source-bound maintenance,
4-step sequencing starting with logstream multi-master. Transport (§5),
transport-side security (§10), and the PalaceMind appendix are reserved
stubs for windows-claude, delivered as patch artifacts to keep a single
committer per repo.
(cherry picked from commit ab6f362af7a2ce947c2c37c11910fa8aeb6b983d)
GET /logstream/stream on the hub HTTP transport: bearer-authenticated
Server-Sent Events with the event_list filter set, since_event_id /
Last-Event-ID replay-then-tail resume (live tail only without a cursor,
via new Logstream.latest_event_id), ~15s heartbeat comments, and a
bounded client pool (MEMPALACE_SSE_MAX_CLIENTS, default 8; 503 +
Retry-After on cap). Frames carry the exact event_list JSON envelope —
the compat surface promised to the PalaceMind desktop viewer.
Also fixes a latent hub-starvation hazard the first SSE consumer would
have tripped: do_POST serialized every request under the global HTTP
lock, so one agent's five-minute event_wait long-poll blocked the whole
fleet. Logstream tools (own WAL db, no Chroma/KG state) now dispatch
lock-free via _http_dispatch; everything else keeps the global lock.
Requested by claude-fable-5-windows for the PalaceMind live viewer
(correlation palacemind_desktop_logstream_viewer). Full suite: 3411
passed.
(cherry picked from commit a7c720507974acc2f02957ac24b7045503ad74da)
Patch authored by windows-codex via logstream (task_win_hubforward_fix,
artifact art_20260702T044935_18f200c38ece, sha verified). Root causes
were test portability assumptions, not product bugs: the 0o600 mode
assertion is POSIX-only (Windows chmod reports 0o666 — now guarded), and
the _disown_record helper stamped PID 1 as a 'live foreign process',
which is dead on Windows, so read_live_serverinfo correctly rejected the
record and the proxy fell back local. Now uses the live parent PID with
liveness asserts. Verified 30/30 on Windows 11 (by author) and 30/30 on
macOS (this apply).
(cherry picked from commit 4006113dd6aa89448d898146b21ba01bbf367f80)
From windows-codex's full-suite verification reply over the logstream:
debug hub connections with curl before blaming agent config, client tool
caches masquerade as hub problems, the event body is the work order, and
cross-platform worker gotchas (CRLF, quoting, platform-guarded tests).
(cherry picked from commit c843d396cc915aa90f5392fd08cbfa1b91483cac)
Igor Lins e Silva