Move Unreleased entries into the 3.7.0 section, add the Wave 2 headlines
(Hermes core, source adapters, stale-library MCP gate, date search,
openai-compat embeddings, integrity/encoding fixes), and set the release
date to 2026-08-11.
_cleanup_mine_lock_file reclaims a lock correctly on the happy path (see
its own docstring for the flock-based rendezvous safety it already
handles) — but only for the specific lock a mine_lock context manager
just released. A process that dies before reaching its own finally
block (SIGKILL, force-quit, host crash) never runs that cleanup, and
nothing else in the codebase later revisits that lock file.
Found in the wild: one long-lived installation had 5,636 stale lock
files in ~/.mempalace/locks/, the oldest several months old, none held
by any live process (confirmed via lsof before cleanup). This is
distinct from the #1264 lock-holder-diagnostics fix (identifies who
holds a live lock) and the #1299 mcp_server embedding-function fix
(unrelated code path) — neither addresses orphan reclamation, and the
2026-07-10 outage postmortem comment in mcp_server.py's stdio loop
covers graceful client disconnection, not abrupt process death.
Adds reap_stale_mine_locks(), which reuses _cleanup_mine_lock_file
itself for the actual removal — same nonblocking-flock-reacquire safety
mechanism, same Windows/POSIX handling already tested in this file, no
duplicated locking logic. A lock is only ever removed after this
process re-acquires it, so anything genuinely held by a live process is
left untouched regardless of age. Wired into mine_lock() via a
throttled opportunistic call (_maybe_reap_stale_mine_locks, at most
once per 15 minutes) rather than a new background thread, scheduled
task, or CLI surface — it piggybacks on the natural cadence of mining
rather than adding new infrastructure.
mine_palace_*.lock (the newer per-palace lock added for the #974/#965
fan-out fix) is explicitly skipped — it has its own lifecycle and
holder-identity tracking and doesn't have this failure mode.
Tests: 6 new cases in test_palace_locks.py covering removal of a
genuinely stale+unheld lock, preservation of a young lock regardless of
hold state, the core safety property (a lock held by another process is
never removed even when backdated past the age threshold), skipping
mine_palace_*-prefixed locks, a missing-lock-dir no-op, and the
throttle itself. Full existing test_palace_locks.py suite (19 tests)
passes unchanged. Broader tests/ -k 'palace or mine' run clean (730
passed) aside from two pre-existing failures confirmed unrelated and
present on an unmodified checkout (test_hnsw_capacity.py SQLite WAL
signature caching, test_repair.py FTS5 shadow-table write restriction —
both environment/SQLite-build-specific, neither touches locking).
After rebasing #1671 onto current develop:
- Opt test_embedding_api out of conftest's stable EF mock so the
get_embedding_function selection tests exercise the real factory.
- Move the CHANGELOG entry from released 3.7.0 Performance into
Unreleased Features (rebase context had drifted).
Add an `embedding_model: "openai-compat"` option that computes embeddings
via any OpenAI-compatible `/v1/embeddings` server (LM Studio, llama.cpp,
vLLM, Ollama's OpenAI shim, self-hosted) instead of a local ONNX model.
- New OpenAICompatEmbeddingFunction (stdlib urllib, no new dependency):
batches requests, asks for `encoding_format: "float"` and a custom
User-Agent (avoids Cloudflare 403, see #1570), validates the response
(contiguous 0..n-1 indices + well-formed vectors) before use, and
L2-normalizes for the cosine collection. Exposes `embed_query` (ChromaDB
1.5 dispatches query embedding through it, not `__call__`). `name()`
encodes the model id so switching it forces `mempalace repair
rebuild-index`. Failures raise a module-specific `EmbeddingAPIError`.
- Endpoint settings resolved by MempalaceConfig as a single source of truth:
`embedding_api_url` / `embedding_api_model` / `embedding_api_key`, each
overridable via the matching `MEMPALACE_EMBEDDING_API_*` env var.
Whitespace-only values are treated as unset; the EF cache key fingerprints
the key so a token rotation is picked up.
- The miner/MCP `Device:` header reports `openai-compat (<url>)` instead of a
misleading local accelerator label when this backend is active.
- Opt-in; default stays minilm. Mirrors the existing `openai-compat` LLM
provider naming; stays local when the endpoint is on the machine/LAN.
- Tests: tests/test_embedding_api.py (no server / no network required).
- Docs: README requirement note, module docstring, CHANGELOG.
Refs #1559.
`repair` in its default mode and `migrate` both back up the whole palace
directory with `shutil.copytree` before they overwrite it. `copytree` cannot
duplicate a Unix domain socket or a named pipe: it copies everything else and
then raises `shutil.Error`, so one such entry anywhere under the palace killed
the command at the backup step, before any rebuild ran. Re-running only
repeated it, because the backup already written held a complete
`chroma.sqlite3`, so the next run deleted it and died in the same place.
Both call sites now go through `copy_palace_dir`, which hands `copytree` an
`ignore` callback. The predicate is an allowlist, matching how the rest of the
package treats directory entries: only regular files and directories are
copied, and anything else is named and left out. Device nodes are skipped for
the opposite reason to sockets, since the copy does not fail on them but
dereferences them; a link to `/dev/zero` wrote 493 MB in 8 seconds here before
it was stopped.
A failed `stat` is never a reason to skip. It says the entry cannot be
resolved right now, not that it holds no data, and no errno separates the two:
a symlink into a volume that is not mounted fails exactly like one whose
target was deleted, and on Windows an unmapped drive letter and an unreachable
network share both arrive as `ENOENT` as well. Such an entry is left for the
copy to fail on, exactly as before this change, because the backup is the
safety net for the rebuild that overwrites the palace next. Five tests pin
that boundary, each asserting `shutil.Error` still comes out.
What was skipped is printed for the operator, and printed for a failed copy
too, since that is when knowing what the backup lacks matters most. The report
can fail on its own, because both callers pass `print`: an stdout that cannot
encode an entry's name, or a report long enough to flush partway through onto
a device that refuses the write. One such line costs neither the lines after
it nor, on the failure path, the copy's own exception, which a report that
raises would otherwise replace. On the success path it is not suppressed, so a
caller that passed something other than a working logger hears about it.
Co-Authored-By: marcoaperez <25101951+marcoaperez@users.noreply.github.com>
The default (legacy) repair path ignored --dry-run and ran the real
rebuild: it deleted any existing <palace>.backup, copied the live palace
over it, re-filed the drawers collection through a staged temp copy, then
rebuilt FTS5 and VACUUMed. #2095 and #2133 fixed this for
--mode from-sqlite only.
The preview now returns before ChromaBackend() is constructed. Opening a
chromadb client is itself a write to chroma.sqlite3, so a preview that
reached one could not be inert; the row count comes from
sqlite_drawer_count instead, the read-only SQLite ground truth
check_extraction_safety already trusts. Staying off the chromadb layer
also keeps a dry run clear of the layer repair is separately reported to
segfault in on a large palace (#2113).
resolve_repair_preflight_errors() decides what a dry run does about the
FTS5 autoheal. The autoheal is a write, so a preview must not run it, but
skipping it routed an isolated inverted-index error into the abort banner
and exit 1 - telling the operator to run offline sqlite3 .recover on a
palace a real run heals by itself (#1596). The dry run now classifies the
errors with the same _errors_are_isolated_fts5 predicate the real path
gates on and continues; broader corruption still aborts in both modes.
It is worded as an attempt rather than a promise, because the real heal
still gives up when another process holds the mine lock, when the rebuild
raises, or when quick_check is still dirty afterwards.
The plan describes the real run in execution order. It names the
live-collection delete the rebuild performs, since "re-file via a staged
temp copy" alone reads as additive; it warns when an existing backup
would be deleted; and it reports a no-op instead of a rebuild when the
collection holds no rows. It states the #1208 truncation guard that can
abort the run, and reports that guard as disabled when
--confirm-truncation-ok is set, because check_extraction_safety returns
immediately then and the promised abort would not happen. An unreadable
count fails closed with a non-zero exit, for parity with the from-sqlite
preview.
A real ChatGPT conversations.json is a top-level array of conversation
objects. _try_chatgpt_json requires a single dict with a "mapping" key, and
no other parser in the chain claims the array either, so the export fell
through to the plain-text path: mine --mode convos reported success while
filing serialized JSON sliced at arbitrary offsets.
Unwrap the array and normalize each conversation separately, mirroring
_try_claude_ai_json_split, so a freshly downloaded export sitting next to the
previous one files only the conversations that are actually new.
The ChatGPT parser is now reached for every element of every top-level JSON
array under the mined tree, so type-check its nested shapes: ten of the
twelve malformed payloads now covered by tests raised AttributeError,
KeyError or TypeError, which would abort a whole mine run since convo_miner
catches only OSError and ValueError.
The tokenizer pads every row of a sub-batch to the longest sequence in it,
and attention costs batch x heads x length^2 per layer, so arrival order
decided what an embeddinggemma run cost: one long verbatim message dragged
its whole sub-batch up to its own length.
Sort by UTF-8 byte length before the split and scatter each row back to its
input index. An input that fits one sub-batch is left alone, since every row
pads to the same width either way.
Over 43,157 sweep drawers from 160 transcripts, padded token slots drop
39.7% and the quadratic attention term 45.0%. Wall clock on 52 messages with
one model instance: 1169.70 s to 490.83 s, control baseline 1286.23 s. Row
values move by at most one float32 ULP (1.2e-07, cosine 0.99999992), which
is reduction-order rounding, not a change of meaning.
Only palaces configured for embeddinggemma are affected; the default MiniLM
embedder pads to a fixed width.
from-sqlite ignored --dry-run and performed the real archive+rebuild
(#2095, #2133). Preview after source validation, skip the destructive
confirm, never take the mine-lock or rename the palace, and fail closed
when SQLite row counts are unreadable instead of inventing zeros.
Bump package, plugins, lock, and README badge to 3.7.0. Promote the
post-3.6.0 integrity spine into CHANGELOG: single-writer ownership,
HNSW write defaults and preflight, safer repair/re-mine, MCP/daemon
lifecycle hardening, entity ReDoS guard, and hook write-routing.
Also ruff-format two test files that drifted during conflict merges.
Local validation: ruff check/format clean; 3497 passed, 31 skipped.
Bump version to 3.5.0 across version.py, pyproject.toml, the Claude/Codex
plugin manifests, the README badge, and uv.lock. Refresh the "N MCP tools"
prose from 34 to 35 (delete_by_source #1729 and checkpoint #1851 each added a
tool). Add the 3.5.0 CHANGELOG entry.
Address review feedback surfaced on the 3.4.1 release promotion (#1810).
Bug fix — `date -r FILE` is GNU-only. On BSD/macOS `date -r` expects
epoch seconds, not a path, so the staleness/throttle checks in the new
Cursor and Antigravity hooks silently failed on macOS: the state GC
swept on every fire and the pending-save guard was skipped. Replace
with a portable `os.path.getmtime` one-liner via the already-resolved
$MEMPAL_PYTHON_BIN (cursor/lib, antigravity/lib, antigravity save hook).
This restores the "bash 3.2.57 / macOS default" compatibility the
Antigravity changelog claims.
Docs:
- Correct the MCP tool count to 33 (was 19/29/31 in 21 places across
plugin manifests, READMEs, and website docs — all drifted from the
TOOLS dict / mcp-tools.md reference, which both have 33).
- Fix broken CHANGELOG link to the Cursor skill (skills/, not
.cursor-plugin/skills/).
- Fix one-too-many `../` in skills/mempalace/SKILL.md's cursor-hooks
link (resolved above the repo root).
- Add the required `mcpServers` wrapper to the mcp.json example in
.cursor-plugin/README.md so copy-paste yields a valid Cursor config.
Left intentionally unchanged: the os.dup2 fd-1 redirect in
mcp_server.py is deliberate (#225 keeps JSON-RPC off fd 1).
Bump version across all sources (version.py, pyproject.toml, both
Claude plugin manifests, Codex plugin manifest, README badge, uv.lock)
and promote the Unreleased changelog to 3.4.1.
Shipping: Cursor IDE plugin + hooks, first-class Antigravity IDE
support (with zero-config interpreter resolution), embeddinggemma
bulk re-embed OOM fix, and backup-retention pruning.
Also rebuilds the CHANGELOG compare-link block, which had been left
at v3.2.0: adds the full 3.3.0-3.4.1 chain plus the previously
undocumented 3.4.0, and points Unreleased at v3.4.1...HEAD. Every
version header now resolves to a compare link.
Two threads sharing a cold EmbeddinggemmaONNX via _EF_CACHE could each
build a full model session, and two factory callers could each keep a
private instance. The load is now double-check locked with the session
published last, and the factory cache has an atomic check-then-construct
behind a lock-free fast path. __call__ wraps a bare string, returns []
for None and empty input before the lazy download, and its annotation
matches the accepted types.
One session.run over a repair-scale batch (5000 docs) allocates
attention buffers far beyond available RAM and the kernel kills the
process. Mirror chromadb's ONNXMiniLM_L6_V2 and embed in sub-batches
of 32; per-chunk padding also stops one long doc inflating the whole
batch.
Co-Authored-By: mojie5 <262519016+mojie5@users.noreply.github.com>
mempalace migrate (.pre-migrate.* full-palace copies) and mempalace repair
max-seq-id (chroma.sqlite3.max-seq-id-backup-* DB copies) each wrote a fresh,
full-size, timestamped backup every run and never deleted the old ones. On a
machine that mines or repairs on a schedule, those copies could silently
accumulate until they filled the disk.
Add a configurable max_backups setting (default 10; env MEMPALACE_MAX_BACKUPS
or config.json) and a shared prune_backups helper that trims the oldest copies
after each new backup is written. Pruning is keyed by filesystem mtime, scoped
strictly to each backup's own naming pattern so live data is never touched, and
best-effort so a deletion failure can never abort the migrate/repair that just
succeeded. Set max_backups to 0 to keep every backup.
`uv tool install mempalace` / `pipx install` place the mempalace
console scripts in an isolated environment whose interpreter is not
the system python3. mempal_resolve_python previously resolved
`command -v python3`, landing on a Python that cannot import
mempalace: the `-m mempalace --version` probe failed and mining
silently never fired (hit by a real user on PR #1633).
Resolution now derives the interpreter from the mempalace-mcp /
mempalace console-script shebang on PATH (the same script the MCP
server launches) before falling back to python3. It is pure shebang
parsing + stat — no Python subprocess at source time — so the hook
performance budget is preserved. An env-style `#!/usr/bin/env python`
shebang and a non-executable interpreter are both rejected and fall
through. MEMPAL_PYTHON remains the explicit override.
Adds 6 resolver regression tests, documents resolution + MEMPAL_PYTHON
in the guide and hooks README (fixing the stale `command -v mempalace`
note), and a CHANGELOG entry.
Co-authored-by: Cursor <cursoragent@cursor.com>
Resolves the maintainer review on the Cursor IDE support PR. Cursor-only
scope; cross-IDE items (wing-naming convention, shared-file merge order)
are coordinated on the separate Antigravity branch.
followup_message default (the one "decide before merge" item):
- Keep the stop-hook followup ON by default. Cursor's transcript format
is undocumented and mempalace/normalize.py has no Cursor parser, so the
background `mempalace mine --mode convos` is best-effort only and does
not yet yield clean verbatim drawers. The followup is therefore the
load-bearing verbatim-capture path; defaulting it off would leave a
default Cursor install capturing nothing.
- Add an opt-out (MEMPAL_CURSOR_SILENT=1, or MEMPAL_VERBOSE=false) for
users who want the Claude-style "zero tokens in chat" behaviour. The
hook still mines and keeps its counters/markers when silenced.
- Correct the misleading "background mine captures it" comments in the
save and precompact hooks; update hooks/cursor/README.md and the guide.
Hygiene fixes:
- Drop the hardcoded "version" field from .cursor-plugin/plugin.json and
marketplace.json (mempalace/version.py is the single source of truth);
tests now assert the field stays absent.
- Remove the committed .cursor-plugin/{commands,skills} symlinks (they
break on Windows clones with core.symlinks=false and were redundant
with the real repo-root components that `source: "."` already serves);
add a guard test that no symlinks exist under .cursor-plugin/.
- Document the preCompact synchronous-mine timeout tradeoff and that an
incremental/append-only mine is recoverable if killed (no corruption).
- Add a Cursor-namespaced, daily-throttled TTL sweep (MEMPAL_STATE_TTL_DAYS,
default 30) to lib/common.sh that GCs stale cursor_*.count/.pending only,
after the kill-switch check; shared logs and antigravity_* are untouched.
Verification: full suite green (2424 passed, 3 skipped), ruff check +
format clean, bash -n clean on all cursor scripts. +30 Cursor tests
(followup opt-out, state GC, TTL validation, no-symlink/version guards).
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds first-class integration with Google's Antigravity IDE
(https://antigravity.google/) as a third sibling to the existing
Claude Code and Codex hook integrations. Strictly additive — no
existing files in main are restructured.
What ships
----------
* `.antigravity-plugin/` — verified-minimal plugin package:
* `plugin.json` with `{"name": "mempalace"}` (no fabricated fields)
* `mcp_config.json` registering the `mempalace-mcp` stdio server
* `hooks.json.tmpl` templated with `__PLUGIN_DIR__` substitution
* `skills/mempalace/SKILL.md` (real file — no symlinks)
* `hooks/antigravity/`:
* `lib/common.sh` — shared bash 3.2.57-compatible helpers with
sentinel-guarded camelCase JSON parser, antigravity_*-namespaced
state files, every existing kill switch, `MEMPAL_SAVE_INTERVAL >= 1`
floor (no /0), and fail-open emitters
* `mempal_save_hook_antigravity.sh` — Stop event handler:
increments per-conversation counter, defers when fullyIdle=False
or terminationReason=error, validates transcriptPath against
`..` traversal, spawns `mempalace mine --mode convos` in a
detached subprocess with a per-conversation pending marker,
ALWAYS emits `{}` (never `{"decision":"continue"}` — that would
force an infinite agent loop)
* `mempal_wake_hook_antigravity.sh` — PreInvocation handler gated
to invocationNum==1 with an atomic mkdir loop guard, runs
`mempalace wake-up` with a 500ms hard timeout, emits verbatim
output as `{"injectSteps":[{"ephemeralMessage":"..."}]}` or
`{}` on any failure
* `install.sh` — idempotent installer with cmp-gated copies,
`__PLUGIN_DIR__` substitution, relative path absolutization,
`--dry-run`, and basename-guarded `--uninstall` (refuses to
wipe a directory whose basename isn't `mempalace`)
* `INVESTIGATION.md` — verbatim quotes + URLs + dates from the
five official Antigravity doc pages, recording every surface
shipped and every surface deliberately omitted
(PreCompact equivalent, slash-commands, rules/, plugin
permissions field — the latter is third-party fabrication)
* `STDIN_SHAPE.md` — exact stdin/stdout contract per event with
worked examples
* `README.md` — local hook docs + troubleshooting
* `examples/antigravity/{hooks.json,mcp_config.json,README.md}` —
standalone configs for users who don't want the full installer
* `website/guide/antigravity.md` + sidebar entry — VitePress guide
* Updates to `README.md`, `CHANGELOG.md` (Unreleased), `hooks/README.md`
Tests (56 new, all passing)
---------------------------
* `tests/test_antigravity_plugin_manifest.py` (11 tests) — schema
contract on the in-repo `.antigravity-plugin/` directory, including
guards against re-introducing the fabricated `permissions` field
and against any symlink leak.
* `tests/test_antigravity_hooks_shell.py` (31 tests) — invokes the
bash hooks via subprocess with synthetic camelCase stdin, asserts
`{}` on every failure path, kill-switch coverage (env vars +
config.json + palace nuke), divide-by-zero floor, transcript
traversal rejection, namespacing, wing inference, and the hard
refusal to ever emit `decision=continue` from the Stop hook.
* `tests/test_antigravity_hooks_install.py` (14 tests) — `--dry-run`
side-effect-free, real install layout, executable bits preserved,
byte-identical idempotent re-runs (md5 + filecmp), basename-match
uninstall safety, refusal when plugin.json is missing or names a
different plugin, relative path absolutization. Skipped on Windows.
Verification
------------
* `uv run pytest tests/ --ignore=tests/benchmarks -v` → 2314 passed,
3 skipped (Windows), 1 unrelated warning
* `uv run ruff check .` → all checks passed
* `uv run ruff format --check .` → 139 files already formatted
* `bash -n` clean on common.sh, both hook scripts, install.sh
* Local install at `~/.gemini/config/plugins/mempalace/` verified end-
to-end: layout correct, paths absolutized in hooks.json, both hooks
fire with realistic camelCase JSON in <1s, wing inference picks
`wing_mempalace` from workspacePaths[0], state files all
`antigravity_*`-namespaced, second `install.sh` run produces
byte-identical output (md5 snapshots match), uninstall removes
only the mempalace plugin and leaves all 6 sibling Google plugins
untouched.
Constraints honoured
--------------------
bash 3.2.57 (no mapfile / readarray / declare -A / `${var^^}`),
verbatim guarantee on all wake injections, hooks <500ms / startup
injection <100ms target (kill-switch path returns in <1.5s in CI),
zero new runtime dependencies, no telemetry, no external API,
strictly additive (existing Claude/Codex hooks unchanged).
Refs: hooks/antigravity/INVESTIGATION.md for the full audit.
Adds first-class Cursor IDE integration alongside the existing Claude
Code and Codex hook flows, so Cursor users get the same automatic
diary saves, pre-compaction transcript capture, and session-start
memory recall — without changing any default behaviour for existing
users.
What's included
---------------
Cursor hook scripts (hooks/cursor/):
- mempal_save_hook_cursor.sh — Stop event, counter +
loop_count guard, pending-save marker consumption, background
mempalace mine, followup_message emission.
- mempal_precompact_hook_cursor.sh — synchronous mine before
compaction, drops a pending_save marker, returns user_message.
- mempal_wake_hook_cursor.sh — sessionStart event,
wing-scoped recall guidance via additional_context.
- lib/common.sh — shared parsing + state helpers
(bash 3.2 safe, no heredoc-in-subshell traps).
- install.sh — idempotent installer with
--scope, --variant, --dry-run, --uninstall. Recognises existing
entries by basename so re-installs across paths work.
- STDIN_SHAPE.md, README.md — payload schemas + quick
reference.
Cursor plugin (.cursor-plugin/ + repo-root components):
- plugin.json, marketplace.json, README.md.
- skills/mempalace/SKILL.md — model-invocable skill mirroring the
Claude plugin's skill surface.
- commands/mempalace-{help,init,mine,search,status}.md — slash
commands for marketplace-published installs (filename = slug).
- mcp.json — auto-registers the mempalace MCP
server, wrapped under the documented mcpServers key.
Examples + docs:
- examples/cursor/hooks.json, hooks.minimal.json + README.
- website/guide/cursor-hooks.md + sidebar entry.
- README.md and CHANGELOG.md updates.
Tests (129 new, all green):
- tests/test_cursor_hooks_shell.py — 75 behavioural tests for
the three hook scripts: kill switches, input parsing, counter
logic, loop prevention, pending markers, wing inference, logging.
- tests/test_cursor_hooks_install.py — 19 contract tests for the
installer: dry-run, idempotent merge, basename-matched uninstall,
refusal to overwrite malformed JSON.
- tests/test_cursor_plugin_manifest.py — 35 contract tests for the
plugin: manifest validity, version sync with mempalace.version,
mcp.json shape, skill/command frontmatter, default-discovery
layout invariants.
Design notes
------------
- Local-first and zero-API by default; hooks never call external
services. Same privacy model as the existing Claude Code hooks.
- Fail-open: hook scripts deliberately do not use set -e so a broken
hook can never block the user's conversation.
- Cursor preCompact cannot block + return a followup, so we
synchronously mine the transcript and drop a pending_save marker
that the next stop hook consumes — guarantees verbatim capture
before context window compression.
- Cursor's default plugin discovery requires real commands/, skills/,
and mcp.json at the plugin root (verified against the cached
cloudflare plugin); .cursor-plugin/{commands,skills} are convenience
symlinks back to those canonical locations.
- bash 3.2 compatibility throughout: avoids heredoc-in-command-
substitution parser bugs; uses python -c for JSON parsing;
basename-matched entry recognition in install.sh.
- All changes are additive. No existing files are removed, no
existing hooks change behaviour, and no new runtime dependencies
are introduced.
Co-authored-by: Cursor <cursoragent@cursor.com>
The two pre-existing entries for #1467 (tunnels.json path) and #1468
(create_tunnel endpoint validation) were sitting at the bottom of the
[Unreleased] block before this release-prep PR. Inserting the new
Performance section between the freshly-backfilled Bug Fixes and these
two pre-existing entries put them under Performance, which is wrong —
they're bug fixes. Moves them back ahead of Performance.
Resolves conflicts in CHANGELOG.md and pyproject.toml by combining
the multilingual-embedder additions (huggingface_hub/tokenizers/numpy
core deps, [multilingual] alias, Features section) with develop's
additions (python-dateutil core dep, [extract] extra, tunnel Bug
Fixes and Internal sections).
Prepares PR #1483 for merge into v3.3.6.
Onboarding now asks the user once, on first run, whether to use the
multilingual embedding model. The default answer is yes — defaulting to
English-only made the recall promise effectively unreachable for any
non-English content (cross-lingual cos ~0.35 vs ~0.88 for the multilingual
model). The choice is written to config.json so subsequent runs pick the
right EF without re-prompting; existing installs that never set the env
var or ran onboarding stay on minilm for back-compat. MEMPALACE_EMBEDDING_MODEL
still overrides both.
Multilingual deps (huggingface_hub, tokenizers, numpy) move from the
[multilingual] extra into core. The extra is kept as a no-op alias so
existing install scripts keep working. The 300 MB ONNX model is still
lazy-downloaded on first use, not at install time.
`quick_setup` (the programmatic non-interactive path) grows an optional
`embedding_model` arg so tests and benchmark scripts can pick a model
without writing config.json by accident.
EmbeddinggemmaONNX's "missing deps" error now points at the right
recovery path (reinstall mempalace, since the deps are core) rather
than the obsolete pip install mempalace[multilingual] hint.
Tests: 9 new (3 _ask_embedding_model variants + 2 run_onboarding
persistence + 2 quick_setup + 2 set_embedding_model round-trips). The
existing 2 run_onboarding tests now patch _ask_embedding_model so they
don't print to stdout.
Three follow-ups bundled for the embeddinggemma EF added in 51702e9:
1. Offline tests for EmbeddinggemmaONNX (10 tests, 0.08s, no network).
Mocks huggingface_hub.hf_hub_download, tokenizers.Tokenizer.from_file,
and onnxruntime.InferenceSession so CI never pulls the 300 MB model.
Guarded with pytest.importorskip so the file is skipped when the
multilingual extra isn't installed. Covers: stable name(), lazy-load
runs exactly once, output shape (n, 384) after MRL truncation, L2
normalization, sim prefix applied, dispatch from
get_embedding_function(model="embeddinggemma"), cache key separates
models, helpful ImportError when deps missing, env override.
2. Friendlier ChromaDB EF-name-mismatch error. Switching
MEMPALACE_EMBEDDING_MODEL on an existing palace previously surfaced
ChromaDB's bare "Embedding function conflict: new: X vs persisted: Y"
ValueError. Now ChromaBackend.get_collection() wraps that error and
points users at the two recovery paths: revert the env var, or run
`mempalace repair rebuild-index --palace <path>`. New
_explain_ef_mismatch helper + 3 tests (unit + end-to-end).
3. Docs: CHANGELOG [Unreleased] entry covers both the new EF and the
error wrapper. README Requirements section mentions the multilingual
extra and points at the embedding.py docstring for the migration note.
Copilot review on PR #1434 caught that the existing 3.3.5 entry
described the validator as it was authored under #1167 — accepting
``YYYY``/``YYYY-MM``/``YYYY-MM-DD`` and rejecting ISO datetimes — but
PR #1417 (closes#1374) merged into develop on 2026-05-10 and
inverted that: ``sanitize_iso_temporal()`` now rejects partial dates
and accepts canonical UTC datetimes (``YYYY-MM-DDTHH:MM:SSZ`` /
``+00:00``). ``sanitize_iso_date()`` is kept as a backwards-compat
wrapper.
Update the bullet to describe the *shipped* behavior, name both
functions, list both accepted and rejected forms, and call out the
3.3.4 → 3.3.5 behavior change for partial-date inputs that now error.
Reference both #1167 (original) and #1374/#1417 (the expansion).
The 3.3.4 release shipped 2026-05-01 (per GitHub release v3.3.4) but
its CHANGELOG header was never flipped from ``unreleased`` to the
release date. Backfill while we're already touching CHANGELOG for
the 3.3.5 cut.
Bumps version 3.3.4 → 3.3.5 across pyproject.toml, version.py, plugin
manifests, README badge, and uv.lock. Flips CHANGELOG.md from
``[3.3.5] — unreleased`` to ``[3.3.5] — 2026-05-09`` and adds entries
for the four PRs that landed after the bug-fix block was authored:
- Bug Fixes: #1396 (tool_search retry on transient HNSW flush)
- Documentation: #1385 (CONTRIBUTING git-identity guidance, closes#1317)
- Internal: #1431 (test multiprocessing fork → spawn)
- Internal: #1430 (test sqlite connection lifecycle via contextlib.closing)
The four open issues remaining on the v3.3.5 milestone (#1266, #1253,
#1092, #1082) have been moved to v3.4 — they form the concurrent-writer
/ HNSW corruption cluster that needs deeper work than this cycle could
absorb.
Both `--mode legacy` and the inline `cli.cmd_repair` rebuild path
call `Collection.count()` as their first read — the same call that
raises `chromadb.errors.InternalError: Failed to apply logs to the
hnsw segment writer` on the corruption class reported in #1308.
Repair would print "Cannot recover — palace may need to be re-mined
from source files" even though the underlying SQLite tables were
fully intact.
The new `--mode from-sqlite` reads `(id, document, metadata)` rows
directly from `chroma.sqlite3` via `segments` → `embeddings` →
`embedding_metadata` joins, never opens a chromadb client against
the corrupt palace, and re-upserts everything into a fresh palace.
- `--source PATH` extracts from a corrupt palace already moved aside
- `--archive-existing` handles the in-place case by renaming the
existing palace to `<palace>.pre-rebuild-<timestamp>` first
- Partial-rebuild failures raise `RebuildPartialError` with the
archive path so users can recover; CLI exits non-zero
- In-place mode calls `SharedSystemClient.clear_system_cache()` to
drop chromadb's process-wide System registry (cross-palace use
does not, to limit blast radius for library callers)
- Source validation runs before any destructive moves
Verified end-to-end recovering a 52,300-row real-world corrupt
palace.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bundled CHANGELOG entries for the seven Tier-1 PRs merged today, including
the behavior-change call-out for #1167 (KG date validators now reject
non-ISO inputs that previously produced silent empty results).
`tool_diary_write` stored the `agent` metadata verbatim after `sanitize_name`
(which preserves case), while `tool_diary_read` filtered by exact match —
so writing as "Claude" and reading as "claude" silently returned zero rows.
Both endpoints now lowercase `agent_name` immediately after sanitization.
The default per-agent wing slug is also stable across casings since it's
derived from the same normalized form.
Behavior change: entries written prior to this fix under mixed-case agent
names will not match the new lowercase filter; documented under v3.3.5
in CHANGELOG with a `mempalace repair` pointer.
Adds a regression test (`test_diary_read_case_insensitive_agent`) and
updates the existing `test_diary_write_and_read` to assert the new
lowercase agent identity.
Closes#1243
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the user removes ~/.mempalace/ (a strong "do not auto-capture"
signal), the next hook fire would silently recreate the entire dir
hierarchy and ingest existing transcripts:
1. _log() at hooks_cli.py:148 unconditionally calls
STATE_DIR.mkdir(parents=True, exist_ok=True), so the act of
writing the hook log line recreated ~/.mempalace/hook_state/
2. With no config file present, hook_stop_auto_save and
hook_precompact_auto_save defaulted to True (no override to read)
3. The full save path then ran, materializing palace/, wal/,
knowledge_graph.sqlite3, and N drawers from existing transcripts
in ~/.claude/projects/*.jsonl
All four entry points (hook_stop, hook_precompact, hook_session_start,
and _log itself) now check a new PALACE_ROOT = Path.home() / ".mempalace"
constant first and short-circuit (returning {} on stdout, never logging)
when the dir is absent. The user-removable directory is now a kill-switch.
Five unit tests in tests/test_hooks_cli.py cover: hook_stop /
hook_precompact / hook_session_start do not create the dir when absent;
_log() does not create it when absent; existing dir proceeds normally
(regression).
Caught in the wild on a downstream fork: ~146 drawers materialized in
under a second after a deliberate `rm -rf ~/.mempalace/`, into a planning
session that was explicitly not meant to be captured.
Igor Lins e Silva
mvalentsev
Offbeat-Breed
maximilize
Grace Gettert
margaretjgu
undeadindustries
Stephen Coogan
Brian potter
lcatlett