The dependabot bump moved pyproject's ruff pin to 0.16.1, but nothing else
followed it, so the repo asked for three different versions at once:
- pyproject.toml said 0.16.1 (the bump)
- uv.lock still resolved 0.15.20 (dependabot did not update it)
- .github/workflows/ci.yml installed 0.15.14 by its own literal pin
The lint job never reads pyproject, so CI kept linting with 0.15.14 and
reported this PR green without 0.16.1 ever running. The ci.yml pin had
already drifted from pyproject before this bump, under a comment saying to
keep them identical.
- ci.yml: pin 0.16.1 to match pyproject.
- uv.lock: regenerated so the locked resolution agrees.
- test_ruff_pins_match: assert ci.yml and pyproject stay equal, so the next
bump that touches only one of them fails loudly instead of passing blind.
- extend-exclude '*.md': 0.16 began formatting Python inside markdown
fences, taking the formatter from 199 files to 295 and reflowing
hand-aligned example code in docs/rfcs/002 and three website pages.
Excluding docs keeps this a version bump rather than a silent
documentation reflow, and restores the exact file scope the project has
always formatted.
Verified with 0.16.1 actually installed: ruff check and ruff format --check
both clean over the same 199 files, full suite green (3669 passed).
Bump package, plugins, lock, and README badge to 3.7.0. Promote the
post-3.6.0 integrity spine into CHANGELOG: single-writer ownership,
HNSW write defaults and preflight, safer repair/re-mine, MCP/daemon
lifecycle hardening, entity ReDoS guard, and hook write-routing.
Also ruff-format two test files that drifted during conflict merges.
Local validation: ruff check/format clean; 3497 passed, 31 skipped.
Bump version to 3.5.0 across version.py, pyproject.toml, the Claude/Codex
plugin manifests, the README badge, and uv.lock. Refresh the "N MCP tools"
prose from 34 to 35 (delete_by_source #1729 and checkpoint #1851 each added a
tool). Add the 3.5.0 CHANGELOG entry.
ChromaDB's rust HNSW core intermittently fails compaction on Windows with
"Failed to apply logs to the hnsw segment writer" during add/update — a
long-standing, non-reproducible-on-Linux/macOS flake that hits different tests
(test_migrate_wings, test_closets) across unrelated commits and has been
turning otherwise-green release/CI runs red at random.
Add pytest-rerunfailures and wire `--reruns 2 --only-rerun "Failed to apply
logs to the hnsw segment writer"` into the test-windows job only. The
--only-rerun scope means a real, deterministic failure still fails on the first
run; only this specific transient native-dependency error is retried. The
Linux and macOS jobs deliberately keep zero reruns so genuine regressions
surface there loudly.
Bump version across all sources (version.py, pyproject.toml, both
Claude plugin manifests, Codex plugin manifest, README badge, uv.lock)
and promote the Unreleased changelog to 3.4.1.
Shipping: Cursor IDE plugin + hooks, first-class Antigravity IDE
support (with zero-config interpreter resolution), embeddinggemma
bulk re-embed OOM fix, and backup-retention pruning.
Also rebuilds the CHANGELOG compare-link block, which had been left
at v3.2.0: adds the full 3.3.0-3.4.1 chain plus the previously
undocumented 3.4.0, and points Unreleased at v3.4.1...HEAD. Every
version header now resolves to a compare link.
Adds a second external storage backend (Postgres/pgvector) alongside Qdrant
to prove the BaseBackend/BaseCollection contract generalizes across substrates
(SQL + JSONB containment filters + pgvector `<=>` ranking vs Qdrant's REST/dict
model), and addresses the review feedback on PR #1679.
Backend (mempalace/backends/pgvector.py):
- table-per-(namespace, palace, collection) isolation; advertises
supports_namespace_isolation
- JSONB filter pushdown for the containment subset, local-exact fallback for
$or/$contains/comparisons/where_document
- BM25 lexical search; marker-based mismatch protection
- optional psycopg dependency (lazy import), in-memory fake for CI, live test
gated on MEMPALACE_PGVECTOR_LIVE_URL
- registered in registry/__init__/pyproject entry point + [pgvector] extra;
MEMPALACE_PGVECTOR_DSN / MEMPALACE_PGVECTOR_NAMESPACE config; README docs
Isolation contract (RFC 001):
- PalaceRef/BaseBackend document the per-id MUST and the cross-namespace MUST,
gated on the new supports_namespace_isolation capability token
- runnable conformance suite (tests/_backend_conformance.py,
tests/test_backend_conformance.py); qdrant + pgvector run it via their fakes
Marker fail-loud guard:
- qdrant and pgvector now refuse get_collection when local_path is None instead
of silently opening a remote collection with no mismatch protection
Review fixes:
- palace._open_collection_or_explain handles unknown-backend KeyError as a CLI
state message instead of an escaping stack trace
- dedup.py docstring no longer claims "No API calls" unconditionally (false for
remote backends)
Resolve conflicts:
- backends/chroma.py: keep both new except handlers in get_collection
(CollectionNotInitializedError from develop + EF-mismatch helper from
this branch), ordered _ChromaNotFoundError before ValueError to match
the create-branch handler order.
- uv.lock: regenerated from merged pyproject.toml.
Fix lint: ruff format mempalace/embedding.py + tests/test_embeddinggemma.py
(CI now pins ruff==0.15.9 via develop's workflow update).
Full suite: 1923 passed, 1 skipped.
Addresses Copilot review: the ruff pin was incomplete.
- .pre-commit-config.yaml stayed on ruff-pre-commit v0.4.10, so
contributors running pre-commit would reintroduce the exact 0.4-vs-0.5
formatter drift this PR removes. Bumped rev to v0.15.9 and rewrote the
lock-step comment to point at pyproject as the source of truth.
- uv.lock still constrained dev ruff at >=0.4.0, so `uv sync --extra dev`
(the documented setup) would not honor the new exact pin. Regenerated;
the dev specifier is now ==0.15.9 in both lock entries.
ruff check/format still pass; no formatting delta from these files.
Onboarding now asks the user once, on first run, whether to use the
multilingual embedding model. The default answer is yes — defaulting to
English-only made the recall promise effectively unreachable for any
non-English content (cross-lingual cos ~0.35 vs ~0.88 for the multilingual
model). The choice is written to config.json so subsequent runs pick the
right EF without re-prompting; existing installs that never set the env
var or ran onboarding stay on minilm for back-compat. MEMPALACE_EMBEDDING_MODEL
still overrides both.
Multilingual deps (huggingface_hub, tokenizers, numpy) move from the
[multilingual] extra into core. The extra is kept as a no-op alias so
existing install scripts keep working. The 300 MB ONNX model is still
lazy-downloaded on first use, not at install time.
`quick_setup` (the programmatic non-interactive path) grows an optional
`embedding_model` arg so tests and benchmark scripts can pick a model
without writing config.json by accident.
EmbeddinggemmaONNX's "missing deps" error now points at the right
recovery path (reinstall mempalace, since the deps are core) rather
than the obsolete pip install mempalace[multilingual] hint.
Tests: 9 new (3 _ask_embedding_model variants + 2 run_onboarding
persistence + 2 quick_setup + 2 set_embedding_model round-trips). The
existing 2 run_onboarding tests now patch _ask_embedding_model so they
don't print to stdout.
MemPalace's default embedder (all-MiniLM-L6-v2) is English-only-trained.
Cross-lingual cosine similarity on parallel-translated text averages 0.35
across DE/FR/HI/IT/KO/RU — vs 0.88 for embeddinggemma-300m ONNX (q8) with
the semantic-similarity prefix. RU is the worst at 0.17, meaning a Russian
memory and its identical English translation embed to nearly orthogonal
vectors. Multilingual users effectively cannot retrieve their own memories.
This commit adds embeddinggemma-300m as an opt-in alternative:
* New EmbeddinggemmaONNX class implementing ChromaDB's EF protocol.
Lazy-downloads model_quantized.onnx (~300 MB) via huggingface_hub on
first use; cached under ~/.cache/huggingface/. Applies the sim prefix,
runs onnxruntime inference, truncates to 384 dims via Matryoshka
(MRL), L2-normalizes.
* MRL truncation to 384d is intentional: matches MiniLM's vector width
so collection schemas don't change, and validation showed 384d MRL
actually outperforms full 768d on these similarity tasks (0.893 vs
0.881 avg) — known property of MRL training.
* MEMPALACE_EMBEDDING_MODEL env (default "minilm" for back-compat).
Switching models on an existing palace requires re-embedding —
ChromaDB rejects reads with a mismatched EF name. Run
`mempalace repair rebuild-index` after changing the value.
* New optional dep group: pip install mempalace[multilingual]
Adds huggingface_hub + tokenizers + numpy. Core deps unchanged.
ONNX q8 validated lossless vs the Ollama gguf benchmarked previously
(max delta 0.002 cos across 240 parallel pairs).
Bumps version 3.3.4 → 3.3.5 across pyproject.toml, version.py, plugin
manifests, README badge, and uv.lock. Flips CHANGELOG.md from
``[3.3.5] — unreleased`` to ``[3.3.5] — 2026-05-09`` and adds entries
for the four PRs that landed after the bug-fix block was authored:
- Bug Fixes: #1396 (tool_search retry on transient HNSW flush)
- Documentation: #1385 (CONTRIBUTING git-identity guidance, closes#1317)
- Internal: #1431 (test multiprocessing fork → spawn)
- Internal: #1430 (test sqlite connection lifecycle via contextlib.closing)
The four open issues remaining on the v3.3.5 milestone (#1266, #1253,
#1092, #1082) have been moved to v3.4 — they form the concurrent-writer
/ HNSW corruption cluster that needs deeper work than this cycle could
absorb.
The v3.3.4 release prep landed on main but was never merged back into
develop, leaving every version-bearing file one release behind. Bumps
pyproject.toml, mempalace/version.py, both plugin manifests, the
marketplace entry, the README badge, and the lockfile to 3.3.4 to match
the tagged release.
The miner upserted one drawer per ChromaDB call, paying tokenizer +
ONNX session setup per chunk. The embedding device was CPU-only because
no EmbeddingFunction was ever wired through the backend.
Two changes, each a speedup in its own right; stacked they give ~10x
end-to-end on a medium corpus (20 files, 568 drawers):
1. Batched upsert. `process_file` and `_file_chunks_locked` now collect
all chunks of a file into a single `collection.upsert(...)` so the
embedding model runs one forward pass per file instead of N.
2. Hardware-accelerated embedding function. New `mempalace/embedding.py`
wraps `ONNXMiniLM_L6_V2` with configurable `preferred_providers`.
`MEMPALACE_EMBEDDING_DEVICE` (or `embedding_device` in config.json)
selects auto / cpu / cuda / coreml / dml. Unavailable accelerators
log a warning and fall back to CPU.
The factory subclasses `ONNXMiniLM_L6_V2` and spoofs its `name()` to
`"default"` so the persisted EF identity matches existing palaces
created with ChromaDB's bare `DefaultEmbeddingFunction` -- same
model, same 384-dim vectors, no rebuild needed when turning GPU on.
`ChromaBackend.get_collection` / `create_collection` now pass the
resolved EF on every call so miner writes and searcher reads agree.
Benchmarks (i9-12900KF + RTX 3090, medium scenario, 568 drawers):
per-chunk + CPU 19.77s · 29 drw/s (baseline)
batched + CPU 8.07s · 70 drw/s (2.4x)
batched + CUDA 2.15s · 264 drw/s (9.2x)
Reproducible via `benchmarks/mine_bench.py`.
Install paths:
pip install mempalace[gpu] # NVIDIA CUDA
pip install mempalace[dml] # DirectML (Windows)
pip install mempalace[coreml] # macOS Neural Engine
Mine header now prints `Device: cpu|cuda|...` so users can confirm the
accelerator engaged.
Extends the init orchestrator to consume two new signal sources:
1. Claude Code conversation dirs: when the target is a
`~/.claude/projects/` root, convo_scanner contributes ProjectInfo
entries alongside the git/manifest projects. Dedup is by name,
preferring the entry with more user-authored activity.
2. Optional LLM refinement: when --llm is passed, discover_entities
constructs the provider, validates availability, and runs
llm_refine.refine_entities on the merged candidates. Status
summary (reclassified / dropped / cancelled / batch errors)
prints to stderr.
New init flags (opt-in, default remains zero-API):
- --llm: enable refinement
- --llm-provider: ollama (default) | openai-compat | anthropic
- --llm-model: default gemma4:e4b for Ollama
- --llm-endpoint: URL (required for openai-compat)
- --llm-api-key: falls back to env ($ANTHROPIC_API_KEY or
$OPENAI_API_KEY depending on provider)
Provider check_available runs before the scan, so the user sees an
immediate error ("Run: ollama pull <model>" or "ANTHROPIC_API_KEY not
set") rather than a mid-scan failure.
`mempalace init` previously leaned entirely on regex-based entity
extraction from prose. That path works for text-only folders but wastes
signal in any codebase: the project's own name is already in
`package.json` / `pyproject.toml` / `Cargo.toml` / `go.mod`, and the
people who worked on it are in `git log`.
This adds `project_scanner.py`, which becomes the primary signal source
when real signal is available, with the regex detector preserved as the
fallback for prose-only folders (diaries, research notes, writing).
What it does:
- Walks the target directory, parses manifests for canonical project
names, and detects git repos by the presence of a `.git` directory.
- For each repo, reads `git log` for authors and filters obvious bots
(`[bot]`, `dependabot`, `renovate`, `github-actions`, names ending in
`bot`, `-autoroll`). Importantly does NOT filter
`@users.noreply.github.com` - that's GitHub's privacy-protected human
email, used by real contributors.
- Resolves author aliases with a union-find: commits that share a name
OR an email collapse into one person. Picks the most-frequent
real-name variant as display, ignoring handles and single-token
usernames.
- Flags "mine" projects: user is top-5 committer OR has >=10% of
commits OR >=20 commits. Ordered by user_commits in the UX.
- `discover_entities()` merges scanner results with the regex detector
case-insensitively (so `mempalace` from pyproject absorbs `MemPalace`
from docs), and suppresses the regex `uncertain` bucket when real
signal is already found - the user doesn't need to adjudicate prose
noise when the answer is already in git.
Integration: `cmd_init` now calls `discover_entities` instead of
running the regex detector directly. Same output shape, so
`confirm_entities` works unchanged.
Ships with 39 new tests covering manifest parsing, bot filtering,
union-find dedup, git repo discovery, scan integration, and
merge/fallback behavior. Existing 56 regex-detector tests all pass.
Version bumps across pyproject.toml, mempalace/version.py, README badge,
uv.lock, and plugin manifests (.claude-plugin/*, .codex-plugin/*).
CHANGELOG aligned with main (post-3.3.1) and a new [3.3.2] section added
covering the 11 PRs merged on develop since v3.3.1 — silent-transcript-drop
fix + tandem sweeper (#998), None-metadata guards (#999, #1013),
chromadb ≥1.5.4 for Py 3.13/3.14 (#1010), Windows Unicode (#681),
HNSW quarantine recovery (#1000), PID stacking guard (#1023), doc-path
cleanup (#996, #1012), and RFC 001/002 internal scaffolding (#995, #1014, #990).
Prerequisite for RFC 001 (plugin spec, #743). Removes every direct
`import chromadb` outside the ChromaDB backend itself so the core
modules depend only on the backend abstraction layer.
Extends ChromaBackend with make_client, get_or_create_collection,
delete_collection, create_collection, and backend_version. Adds
update() to the BaseCollection contract. Non-backend callers
(mcp_server, dedup, repair, migrate, cli) now go through the
abstraction; tests patch ChromaBackend instead of chromadb.
With this landed, the RFC 001 spec can be enforced and PalaceStore
(#643) can ship as a plugin without touching core modules.
Prepare develop for the 3.3.0 release cycle.
Version bumps:
- mempalace/version.py: 3.2.0 -> 3.3.0
- pyproject.toml: 3.2.0 -> 3.3.0
- README.md: pyproject.toml label and shields.io badge
- uv.lock: mempalace 3.0.0 -> 3.3.0 (also fills in resolved dev/extras)
CHANGELOG.md:
- Close out the stale [Unreleased] section as [3.2.0] - 2026-04-12
(v3.2.0 was tagged on that date but the release flip was never made)
- Add a fresh [Unreleased] - v3.3.0 section covering the 49 commits
since v3.2.0: closet layer, BM25 hybrid search, entity metadata,
diary ingest, cross-wing tunnels, drawer-grep, offline fact checker,
LLM-based closet regen, hall detection, cosine-distance fix,
multi-agent locking, README audit, etc.
- Adopt Keep a Changelog + SemVer framing
- Add version compare reference links at the bottom
- Fix stale milla-jovovich/mempalace preamble URL to MemPalace/mempalace
Igor Lins e Silva
Cheney Zhang
Legion345