_get_kg now accepts an optional canonical_path argument and uses it
verbatim when given. _call_kg passes its top-of-function captured key
through, so the dict insertion that _get_kg performs and the eviction
lookup _call_kg performs always agree on the key -- even if env state
(MEMPALACE_PALACE_PATH rotation) or FS state (symlink remap) mutates
between attempts. Without the pass-through, _get_kg's internal call
to _canonicalize_kg_path(_resolve_kg_path()) could land on a different
key than the captured path, leaving a closed handle in the cache under
a stale key the eviction lookup no longer matches.
Tests:
test_call_kg_passes_captured_path_through_resolve_drift -- patches
_resolve_kg_path with a shifting iterator + identity canonicalize,
pre-seeds the closed handle under the captured key; asserts both
_get_kg calls receive the captured key and the closed handle is
evicted by retry.
test_call_kg_oserror_at_top_propagates_unmasked -- patches
_canonicalize_kg_path to raise OSError; asserts it propagates without
conversion (the rationale for capturing at the top before the retry
loop -- prevents an OSError mid-handler from masking a
sqlite3.ProgrammingError it can't retry through).
_patch_mcp_server test helper updated to a varargs lambda so existing
tests that patch _get_kg as a no-arg fixture stay compatible with the
new optional-arg signature.
Move _canonicalize_kg_path call from inside the except branch to the
top of _call_kg so OSError from realpath (e.g. transient Windows
junction loss) propagates cleanly instead of masking the original
sqlite3.ProgrammingError mid-handler. Also keeps the eviction key
identical to the insertion key if FS state mutates between attempts,
preventing a closed handle from leaking into the cache under a stale
key the lookup no longer matches.
Patch both realpath and normcase with sentinel wrappers and assert the
result composes them as normcase(realpath(p)). Swapping the order would
leave Windows symlinks under the original case, defeating the dedup.
The previous assertion (result == seen[-1].lower()) was tautological --
the fake normcase already returned p.lower(), so any implementation
that lowercased anything would pass.
Switch _kg_by_path lookups in _get_kg and _call_kg from os.path.abspath
to a new _canonicalize_kg_path helper applying realpath (collapses
symlink aliases) + normcase (Windows drive-letter casing). Two tenants
pointing at /srv/A and /srv/link-to-A no longer open duplicate
sqlite3.Connection handles over the same file.
Addresses review-bot feedback on #1519:
- gemini: JSON `Infinity`/`1e1000` round-trips to float('inf'); int(inf)
raises OverflowError, which _try_coerce_int did not catch — bad config
could still crash ingest. Catch it alongside TypeError/ValueError and
fall back like any other garbage value. Regression test added.
- Copilot: the 30-char convo floor was described as "stricter" than the
50-char project default. It is the opposite — lower, i.e. more
permissive (keeps short exchanges the project default would drop).
Reworded the docstring and comments to state this accurately.
The #1024 merge added `import pytest` to tests/test_convo_miner_unit.py
while a pytest import already existed from another merged change. The
collision was textually clean so git didn't flag it, but CI ruff
reports F811 (redefinition of unused `pytest`), turning lint red on
develop and every branch cut from it.
Drop the duplicate import. `ruff check .` (0.4.x, CI pin) clean;
test_convo_miner_unit 23 passed.
#1024 routed min_chunk_size for the project miner through the repairing
MempalaceConfig validation, but the convo path read the raw, unvalidated
_file_config["min_chunk_size"]. A garbage/negative/bool config value
would TypeError the convo length gate or ValueError out of
chunk_exchanges and abort the whole convo mine — breaking the PR's own
"a bad config.json key never takes ingest down" guarantee for one path
only.
Extract _try_coerce_int helper (shared with _coerce_config_int) and add
a public MempalaceConfig.min_chunk_size_explicit accessor that returns
the validated value when the user set it, else None. convo_miner uses
the None sentinel to keep its stricter 30-char floor for untuned users
while honoring a valid override, with no raw _file_config reach.
Adds accessor + convo-fallback regression tests.
Unknown kwargs (wrong parameter name, e.g. text= instead of content=)
were silently dropped by the schema-filter, surfacing only indirectly
as a later "Missing required 'X'". Emit -32602 naming the offending
kwarg instead — symmetric with #1500's missing-required path. Gated by
the existing accepts_var_keyword check; wait_for_previous excluded.
Closes#1512.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Onboarding now asks the user once, on first run, whether to use the
multilingual embedding model. The default answer is yes — defaulting to
English-only made the recall promise effectively unreachable for any
non-English content (cross-lingual cos ~0.35 vs ~0.88 for the multilingual
model). The choice is written to config.json so subsequent runs pick the
right EF without re-prompting; existing installs that never set the env
var or ran onboarding stay on minilm for back-compat. MEMPALACE_EMBEDDING_MODEL
still overrides both.
Multilingual deps (huggingface_hub, tokenizers, numpy) move from the
[multilingual] extra into core. The extra is kept as a no-op alias so
existing install scripts keep working. The 300 MB ONNX model is still
lazy-downloaded on first use, not at install time.
`quick_setup` (the programmatic non-interactive path) grows an optional
`embedding_model` arg so tests and benchmark scripts can pick a model
without writing config.json by accident.
EmbeddinggemmaONNX's "missing deps" error now points at the right
recovery path (reinstall mempalace, since the deps are core) rather
than the obsolete pip install mempalace[multilingual] hint.
Tests: 9 new (3 _ask_embedding_model variants + 2 run_onboarding
persistence + 2 quick_setup + 2 set_embedding_model round-trips). The
existing 2 run_onboarding tests now patch _ask_embedding_model so they
don't print to stdout.
Three follow-ups bundled for the embeddinggemma EF added in 51702e9:
1. Offline tests for EmbeddinggemmaONNX (10 tests, 0.08s, no network).
Mocks huggingface_hub.hf_hub_download, tokenizers.Tokenizer.from_file,
and onnxruntime.InferenceSession so CI never pulls the 300 MB model.
Guarded with pytest.importorskip so the file is skipped when the
multilingual extra isn't installed. Covers: stable name(), lazy-load
runs exactly once, output shape (n, 384) after MRL truncation, L2
normalization, sim prefix applied, dispatch from
get_embedding_function(model="embeddinggemma"), cache key separates
models, helpful ImportError when deps missing, env override.
2. Friendlier ChromaDB EF-name-mismatch error. Switching
MEMPALACE_EMBEDDING_MODEL on an existing palace previously surfaced
ChromaDB's bare "Embedding function conflict: new: X vs persisted: Y"
ValueError. Now ChromaBackend.get_collection() wraps that error and
points users at the two recovery paths: revert the env var, or run
`mempalace repair rebuild-index --palace <path>`. New
_explain_ef_mismatch helper + 3 tests (unit + end-to-end).
3. Docs: CHANGELOG [Unreleased] entry covers both the new EF and the
error wrapper. README Requirements section mentions the multilingual
extra and points at the embedding.py docstring for the migration note.
MemPalace's default embedder (all-MiniLM-L6-v2) is English-only-trained.
Cross-lingual cosine similarity on parallel-translated text averages 0.35
across DE/FR/HI/IT/KO/RU — vs 0.88 for embeddinggemma-300m ONNX (q8) with
the semantic-similarity prefix. RU is the worst at 0.17, meaning a Russian
memory and its identical English translation embed to nearly orthogonal
vectors. Multilingual users effectively cannot retrieve their own memories.
This commit adds embeddinggemma-300m as an opt-in alternative:
* New EmbeddinggemmaONNX class implementing ChromaDB's EF protocol.
Lazy-downloads model_quantized.onnx (~300 MB) via huggingface_hub on
first use; cached under ~/.cache/huggingface/. Applies the sim prefix,
runs onnxruntime inference, truncates to 384 dims via Matryoshka
(MRL), L2-normalizes.
* MRL truncation to 384d is intentional: matches MiniLM's vector width
so collection schemas don't change, and validation showed 384d MRL
actually outperforms full 768d on these similarity tasks (0.893 vs
0.881 avg) — known property of MRL training.
* MEMPALACE_EMBEDDING_MODEL env (default "minilm" for back-compat).
Switching models on an existing palace requires re-embedding —
ChromaDB rejects reads with a mismatched EF name. Run
`mempalace repair rebuild-index` after changing the value.
* New optional dep group: pip install mempalace[multilingual]
Adds huggingface_hub + tokenizers + numpy. Core deps unchanged.
ONNX q8 validated lossless vs the Ollama gguf benchmarked previously
(max delta 0.002 cos across 240 parallel pairs).
Without an explicit num_ctx, each candidate ran at its Modelfile default
(32k for the Gemma4 variants, larger for qwen3), so VRAM and latency
weren't comparable across families — a 32k-default model pre-allocates
KV cache a 4k-default model doesn't. The flag's own docstring promised
"apples-to-apples" but defaulted to None, defeating the intent.
All current benchmark prompts fit comfortably under 4k tokens
(memory_extraction is the longest at ~500). Users with longer prompts
can still pass --num-ctx <larger>.
Adds a methodology note to the 2026-05-13 multilingual report so its
VRAM/latency numbers aren't conflated with future runs at the new default.
Addresses the review feedback from igorls, gemini-code-assist, and Copilot.
HIGH:
- orchestrator: --output single-file mode now shares ONE (fh, writer) across
all languages instead of opening N handles to the same path. The old code
caused interleaved buffer corruption: first language opened "w", subsequent
ones opened "a", and writes from independent file offsets could overwrite
each other. Verified with a multi-language --output smoke test (4 rows
written, all distinct).
- 19 untranslated/empty samples re-translated:
- dataset.de.jsonl: cal_017
- dataset.hi.jsonl entity_extraction: ent_020, ent_025, ent_032, ent_038
- dataset.hi.jsonl room_classification: rc_017, rc_026, rc_028, rc_040,
rc_064, rc_089, rc_091
- dataset.ko.jsonl room_classification: rc_027, rc_067
- dataset.it.jsonl room_classification: rc_029, rc_030, rc_031, rc_032,
rc_053 (previously empty strings)
- labels.ko.jsonl: restored all proper nouns to English (Doreth, Saela, Ivora,
Ren Solanke, Pol Krisat, Pell Halloran, Bramble, Hollowmounts Institute,
Wendelsea, Bridgewater Community Garden, Wends, Drukar, Aerwyn cycle,
Jaccard, Mason bee, Markdown). Also fixed mistranslation 유전자 사과
(genetic apple) → 재래종 사과 (heirloom apple).
MEDIUM:
- runner.py: refactored label-resolution one-liner into 3 readable lines
and added an info log when falling back to English ground truth, so
readers don't misread "score collapse" as model failure.
LOW:
- orchestrator: moved `import socket` to module top (PEP 8); removed
unused `out_path` from the unpacking tuple.
- translate_datasets.py: renamed loop variable `l` → `code` (ruff E741);
made the _translate_one fallback return path explicit instead of relying
on for-loop fall-through; added a privacy warning in the docstring
flagging that the default `kimi-k2.6:cloud` sends prose to a remote
endpoint and should not be used over real palace data.
- 2026-05-13-multilingual.md: converted analytical paragraphs from
Portuguese to English to match the existing repo convention.
Adds 6 new language datasets (German, French, Hindi, Italian, Korean, Russian)
across all 4 benchmark tasks (calibration, entity_extraction, memory_extraction,
room_classification) — 630 samples total, same conventions as the existing
pt-BR/es/zh datasets: inputs translated, labels/ground-truth stay English
except where noted.
Changes:
- 24 new dataset.{de,fr,hi,it,ko,ru}.jsonl files across all 4 tasks
- labels.ko.jsonl for memory_extraction: Korean ground-truth so the scorer
compares Korean model output against Korean expected content instead of
English (fixes ~20pp score gap identified during testing — see report)
- runner.py: loads labels.{lang}.jsonl when present, falls back to labels.jsonl
- orchestrator.py: adds --output-dir (writes <dir>/<lang>/YYYY-MM-DD-<host>.csv
per language); --output single-file mode unchanged
- candidates.yaml: adds community tier (igorls classifier variants, heretic)
and local tier (gemma4:e4b)
- translate_datasets.py: script used to generate the translations via Ollama;
included so contributors can extend to new languages without manual work
- reports/2026-05-13-multilingual.md: 210-run benchmark report across
6 models × 7 languages × 5 tasks on RTX 3080 Laptop 8 GB
Split the bare except in handle_request tools/call: TypeError matching
Python's signature-mismatch shape (gated on handler __qualname__) returns
-32602 with parsed parameter name(s); handler-internal TypeError and
non-TypeError exceptions stay on the existing generic -32000 path so
internals never leak. Adds TestParamShapeDiagnostics with 6 cases.
Closes#1351
Co-authored-by: anastasiiaanfimova <271004602+anastasiiaanfimova@users.noreply.github.com>
A direct ``hook["type"]`` subscript raised ``KeyError`` before the
assert evaluated when the ``"type"`` key was missing, so the
diagnostic message naming the event and the unexpected value was
never shown. ``hook.get("type")`` lets the assert fail gracefully
with the intended feedback.
A duplicate hook entry inside `entries[0]["hooks"]` (or a second
entry under `Stop`/`PreCompact`) silently passes the per-hook
bounds check and would double-fire the hook. Pin `len(entries) == 1`
and `len(sub_hooks) == 1` so cardinality drift fails loudly with a
diagnostic message naming the event and the unexpected count.
The `.claude-plugin/hooks/hooks.json` Stop and PreCompact entries
shipped without an explicit `timeout`, so a hung `mempalace hook
run` (reported on Windows under CLI/plugin version drift) blocks
the Claude Code UI for the full 600s `"type": "command"` default.
Set explicit bounds: Stop = 30s (fire-and-forget save path with
synchronous chromadb diary write, detached mine via _spawn_mine),
PreCompact = 90s (60s inner _mine_sync subprocess timeout in
mempalace/hooks_cli.py plus a 30s slack so legitimate mines for
users with MEMPAL_DIR are not truncated mid-flight).
Schema test in tests/test_claude_plugin_hook_config.py pins both
per-event (floor, ceiling) bounds and catches future events added
to the plugin config without a registered bounds entry.
* Move SKIP log from stdout to sys.stderr, matching the existing
warning-channel pattern (the "No mempalace.yaml" notice and
topic-tunnel warnings). Stdout stays clean for "Files: N" /
"Drawers filed: N" markers that downstream callers parse.
* Show the path relative to the scan root via
Path.relative_to(...).as_posix() instead of the leaf filename, so
a nested symlink in a deep subdirectory is unambiguous and the log
renders with forward slashes on every platform.
* Wrap the print in try/except OSError, mirroring the existing guard
around filepath.stat() below; a broken pipe or closed stderr no
longer aborts the scan mid-walk.
* Document the stderr side effect in both scan_convos and
scan_project docstrings.
* Gate the new tests with @pytest.mark.skipif(sys.platform == "win32",
...) because Path.symlink_to requires SeCreateSymbolicLinkPrivilege
on Windows.
* Add dangling-symlink and nested-subdirectory tests for both miners,
locking the two-space " SKIP:" prefix, the "(symlink)" reason
marker, and the full relative path format.
Both scan_convos() and scan_project() silently drop symlinked files,
leaving callers staging a temp directory of symlinks with no diagnostic
for the "Files: 0" outcome. Keep the existing skip-symlinks behavior
and add a " SKIP: {name} (symlink)" log per skipped file, matching
the convention already used by split_mega_files.py for the
oversized-file skip case.
Three-way assertions in test_cli and test_mcp_server now also assert
SENTINEL_IN_PATH is False at import, so a regression in the package
sys.path filter is caught by the same test that catches the env pop
regression. SystemExit catch is narrowed to exit codes (0, None) so a
future argparse change surfaces as a failure instead of being swallowed.
cli.py:main and mcp_server.py:main docstrings document the PYTHONPATH
pop side effect for callers that invoke main() programmatically.
Refs #1423.
Split the leaked-PYTHONPATH workaround: __init__.py now only filters
sys.path (fixes the ABI crash from #1423 at import time), while
os.environ.pop("PYTHONPATH") moves to cli:main() and mcp_server:main()
where the strip affects only subprocesses we spawn ourselves.
Host applications that import mempalace as a library and rely on
PYTHONPATH for their own child subprocesses are no longer affected.
Refs #1423.
When PYTHONPATH is `.`, both the leaked entry and the implicit
empty-string CWD marker on sys.path normalize to `.`. The previous
filter would remove the CWD marker too, silently changing import
resolution. The sys.path filter now preserves any empty-string entry
unconditionally.
Also drop the asymmetric `.strip()` on PYTHONPATH tokens. CPython
does not strip whitespace from PYTHONPATH before populating sys.path,
so stripping on one side only could let whitespace-padded leaks
through.
Test extended with a `dot` parametrize case and a dedicated
test_init_preserves_cwd_marker_when_pythonpath_collides. Test child
code now asserts on the sentinel-prefix substring directly instead of
re-implementing the production normalization, so future changes to
_norm cannot silently mask test regressions.
Previous commit compared sys.path entries to PYTHONPATH tokens by
byte-exact string equality, which let leaks slip through on Windows
(case-insensitive paths) and across trailing-separator or whitespace
quirks (a leading or trailing space in PYTHONPATH would not match the
trimmed sys.path entry the interpreter actually populated).
Both sides of the comparison now run through os.path.normcase +
os.path.normpath so case differences and trailing separators no longer
defeat the filter, and PYTHONPATH tokens are trimmed before the
match.
Test extended to 6 parametrized cases (single, multi-path, trailing
separator, leading pathsep, empty, unset) with explicit ids and
stderr included in assertion diagnostics. Synthetic leak placeholder
replaces /tmp so the test is portable across CI runners.
Popping PYTHONPATH from os.environ alone is insufficient: the interpreter
populates sys.path from PYTHONPATH at startup, before package init runs.
Transitive imports of compiled extensions (pydantic_core,
chromadb_rust_bindings) would still resolve from the leaked path even
after the env var is cleared.
Now also remove the matching entries from sys.path so the import search
path no longer contains the wrong Python's site-packages.
Test extended with a multi-path PYTHONPATH case and assertions on both
os.environ and sys.path after import.
Venvs do not strip PYTHONPATH from inheritance. On multi-Python systems
where PYTHONPATH points at a different Python's site-packages, transitive
imports of compiled extensions (pydantic_core, chromadb_rust_bindings)
load the wrong ABI and crash with ModuleNotFoundError on
pydantic_core._pydantic_core.
Place the pop in mempalace/__init__.py (instead of cli.py as the reporter
proposed) so all entry points are covered: the mempalace shell command,
python -m mempalace, and import mempalace from a library.
Tests: tests/test_init.py runs the strip in a subprocess, parametrized
over set / unset PYTHONPATH.
Closes#1423.
Documents the recovery procedure for the chromadb index-metadata corruption
shape filed at chroma-core/chroma#6949 and reproduced by mempalace's
rebuild_index code path (#1492).
Symptom: mempalace integrity gate quarantines a segment dir with
"labels present but dimensionality is missing or invalid (None)" at
startup, vector search drops to BM25-only fallback, recall gap appears.
Recovery: patch the dimensionality field back into the index metadata
file (the rest of the segment state is intact). ~90 seconds end-to-end
on a 183k-drawer palace; restored 99.97% of recall.
The "delete the metadata file entirely" workaround from chroma-core/chroma#6949
loses the id_to_label and label_to_id mappings; the patch approach
documented here preserves them.
Companion content:
- docs/recovery/index-metadata-recovery.md (this file)
- Related issues #1492 (producer-side fix) and #1493 (auto-recover
proposal for the integrity gate)
- External: jphein/palace-daemon docs/recovery/chromadb-metadata-dict-patch.md
has the same procedure from a palace-daemon HTTP operator's
perspective, plus tests/test_chromadb_metadata_recovery.py with a
regression test that builds a real palace + corrupts + recovers.
Previously rebuild_index() printed "Staged N/M" / "Re-filed N/M" lines
with no time information, leaving operators to do mental math against
wall-clock to estimate completion on a large palace. On a 183k-drawer
rebuild, it's the difference between "this is a 20-min job" and "this
is an 8-hour job" — non-trivial scheduling information that the
function already has and could easily share.
This patch:
1. Adds a `progress: Optional[Callable[[str], None]] = None` parameter
to `rebuild_index()`. Defaults to a new `_DefaultProgress` class
when omitted (backward-compatible default behavior — same content
plus ETA decoration).
2. `_DefaultProgress.__call__` recognizes `Staged N/M` and `Re-filed N/M`
lines via regex, computes elapsed/rate/ETA, and appends them:
Staged 5000/182953 drawers... (elapsed 7m, rate 11.3/s, ETA 4h)
Non-progress lines (e.g. "Backing up chroma.sqlite3...") pass
through unchanged.
3. The clock + baseline counter reset at the stage→refile transition
so refile-phase rate isn't muddied by the slower stage phase
(refile re-embeds the same drawers and may run at different
throughput).
4. Replaces the in-function `print(...)` status calls with `progress(...)`
so a custom callable receives the full status stream — not just the
batch progress lines.
5. Passes `progress=progress` (instead of hardcoded `progress=print`)
into `_rebuild_collection_via_temp`.
Use cases for a custom `progress` callable:
- Daemon-side capture for HTTP `/repair/status` (this is the motivating
case — palace-daemon was tailing the output to surface progress to
operators via the status endpoint; a callback avoids stdout capture
trickery)
- Test silence: `progress=lambda *_: None`
- Custom formatting: structured logging, JSON event stream, etc.
The default behavior is unchanged for callers that omit the parameter;
they just get a nicer message format.
Closes#1485.
Addresses Copilot + gemini-code-assist review on #1483.
1. Path-traversal guard for --language. The value is interpolated into
the dataset filename (`dataset.{language}.jsonl`), so unvalidated
input could escape `task_dir`. Now:
- regex `^[A-Za-z][A-Za-z0-9]*(?:[_-][A-Za-z0-9]+)?$` accepts en,
pt-BR, zh-CN, fr_CA, etc. and rejects anything with path separators
or `..`
- belt-and-suspenders `Path.resolve().is_relative_to(task_dir)` check
before opening the file
2. --embed-endpoint now defaults to None and is resolved after parsing:
uses --endpoint when --llm-provider=ollama (so remote benchmark
runs score against the same host), else http://localhost:11434.
Help text now matches behavior. runner.py's CLI was also missing the
flag entirely — added so single-task runs honor remote endpoints.
Enables shipping decisions for non-English users and fair comparison across
candidates whose Modelfile defaults disagree.
- --language / --languages: load dataset.{lang}.jsonl alongside the base
dataset.jsonl. CSV gains a language column. Synthesized candidate
entries let ad-hoc model tags run without editing candidates.yaml.
- --num-ctx: force Ollama options.num_ctx per request, overriding the
model's Modelfile default. Required for apples-to-apples VRAM/TPS
(qwen3:4b-q8 defaults to 32k = 9.7 GB resident; at 8k it's 5.6 GB).
- --embed-model: thread the semantic-similarity embedding model through
scoring. Default flips to embeddinggemma (was nomic-embed-text v1).
Reason: v1 cosine on EN<->PT-BR same-meaning pairs sits at ~0.607
(right at the 0.6 match threshold), so any phrasing drift collapses
to false-negative. embeddinggemma lands ~0.766 with 2.7x the
signal/noise spread. PT-BR memory_extraction recovered 0.15 -> 0.85
on the same outputs after the swap.
Datasets: 12 new files (pt-BR/es/zh x 4 tasks, 633 samples). Input text
translated; proper nouns and labels stay English so cross-lingual
scoring against the existing labels.jsonl works without re-translation.
Pass the GA Measurement ID from the GitHub Actions repo variable into the
docs build so the published site at mempalaceofficial.com actually emits
the gtag tags. Also escape the ID via encodeURIComponent / JSON.stringify
so a malformed value can't break the page.