hnsw_capacity_status() (chroma.py) exists precisely to preflight the
#1222 SIGSEGV/pyo3-panic class before anything touches the HNSW
segment, but repo-wide it was wired into only 4 call sites while raw
count()/collection.count() is called at 20+ others -- a bare
except Exception around count() cannot catch a native crash, since
the process dies regardless of any Python try/except. This wires the
existing, already-tested probe into the 7 remaining call sites the
audit identified as CRITICAL:
- #89 palace.py::_enforce_embedder_identity -- the universal
get_collection() chokepoint every tool passes through, previously
guarded only by except Exception. Highest leverage: skips this
bookkeeping-only check on divergence instead of risking count().
- #90 migrate.py::migrate -- routes straight to the same
SQLite-extraction fallback the except branch already used, instead
of ever reaching col.count() when diverged.
- #91 repair.py::scan_palace / prune_corrupt -- both abort with the
existing from-sqlite recovery guidance instead of opening the
collection.
- #10 repair.py::rebuild_index -- preflights divergence alongside its
existing sqlite-integrity and poisoned-max-seq-id preflights, before
opening the collection.
- #13 repair.py::rebuild_index never rebuilt or reported on the
closets collection -- now warns when closets is still diverged
after a drawers-only rebuild, pointing at --mode from-sqlite instead
of letting 'Repair complete' stand unqualified.
- #92 dedup.py::get_source_groups -- takes an optional palace_path
(threaded from both callers) to preflight before count(); omitted by
existing tests, which keep their pre-existing behavior.
- #93 miner.py::status -- preflights before the ChromaDB-client
fallback path (used when the direct sqlite read is unavailable).
7 new regression tests, each confirmed failing against the pre-fix
code (via git stash of the source files only) and passing after the
fix. One existing dedup.py test updated for the new palace_path kwarg
in its call-signature assertion. Full suite: 3154 passed, 1 unrelated
pre-existing flake (test_mcp_server.py peer-writer-lock module-global
state leaking across test files in full-suite ordering -- this diff
never touches mcp_server.py).
Adds end-to-end regression coverage for the migration swap path where os.replace hits EXDEV, the shutil.move fallback fails, and the original palace must be restored from the rename-aside copy.
mempalace migrate (.pre-migrate.* full-palace copies) and mempalace repair
max-seq-id (chroma.sqlite3.max-seq-id-backup-* DB copies) each wrote a fresh,
full-size, timestamped backup every run and never deleted the old ones. On a
machine that mines or repairs on a schedule, those copies could silently
accumulate until they filled the disk.
Add a configurable max_backups setting (default 10; env MEMPALACE_MAX_BACKUPS
or config.json) and a shared prune_backups helper that trims the oldest copies
after each new backup is written. Pruning is keyed by filesystem mtime, scoped
strictly to each backup's own naming pattern so live data is never touched, and
best-effort so a deletion failure can never abort the migrate/repair that just
succeeded. Set max_backups to 0 to keep every backup.
Two resource leaks in mempalace.migrate that bite hardest on Windows:
1. extract_drawers_from_sqlite() opened a sqlite3 connection at the
top and only closed it in the happy path. Any exception during
query/iteration leaked the handle, leaving a file lock on
chroma.sqlite3 that prevented the rest of the migration from
touching the palace directory. Wrap with contextlib.closing().
2. migrate() called tempfile.mkdtemp() and never cleaned it up if a
later step (chromadb open, batch import, count, swap) raised.
The orphaned palace under the system temp root could be 100s of
MB and never gets reclaimed. Wrap the import-and-swap dance in
try/finally and rmtree the temp dir if it still exists at the
end (os.replace consumes it on the happy path so the existence
guard makes that case a no-op).
Tests cover the happy-path extraction and verify mkdtemp's directory
is removed when ChromaBackend.get_or_create_collection() raises.
shutil.move() can partially create palace_path before raising, which would
trip a bare os.replace(stale_path, palace_path) rollback (dest exists).
- Switch the primary swap to os.replace so same-filesystem moves stay atomic
- Branch on errno.EXDEV before falling back to shutil.move, so real errors
(permissions, EIO) surface instead of silently attempting a slow copy
- Extract rollback into _restore_stale_palace which clears any partial
destination and, if the restore itself fails, logs both stale_path and
palace_path so the operator can recover by hand
Adds three regression tests covering clean rollback, partial-copy cleanup,
and logged failure on rollback-failure.
Flagged by the Qodo reviewer on #935.
KeilerHirsch