Five fixes for issues called out by the gemini-code-assist[bot] review.
Each gets a regression test that locks in the correction.
1. CRITICAL: marker-cleanup watcher used POSIX `wait` on a sibling pid
(save hook). bash `wait` only works on direct children of the
calling shell — the `( wait $MINE_PID ... ) &` subshell runs as a
sibling of MINE_PID, so wait fails immediately and the pending
marker is deleted within milliseconds, defeating the concurrency
guard. Replace with `while kill -0 $MINE_PID; do sleep 1; done`,
which queries pid existence regardless of parent-child relationship.
Test: test_save_hook_marker_watcher_uses_kill_polling.
2. Bare `mempalace` console-script invocation in the save hook fails
when the venv's bin/ is not on the hook's PATH (e.g. uv tool
install in some configurations, manually managed virtualenvs).
Switch to `"$MEMPAL_PYTHON_BIN" -m mempalace mine ...` so the
resolved interpreter runs the package directly via
mempalace/__main__.py. Tests:
test_save_hook_uses_python_module_invocation,
test_save_hook_missing_mempalace_python_module_does_not_crash.
3. Same issue in the wake hook's inner Python helper. Switch
`['mempalace', 'wake-up', ...]` to `[sys.executable, '-m',
'mempalace', 'wake-up', ...]` — sys.executable is the same
interpreter that resolved MEMPAL_PYTHON in lib/common.sh.
Test: test_wake_hook_uses_sys_executable_module_invocation.
4. The Python parser in lib/common.sh wrapped `json.load` in
`try/except` and silently fell back to `data = {}`. The script
then printed the `__MEMPAL_PARSE_OK__` sentinel even on parse
failure, so the bash sentinel-check on the caller side
(`[ "$_marker" != "__MEMPAL_PARSE_OK__" ]`) never triggered the
defense-in-depth `input parse failed` branch. Remove the
try/except so the exception propagates, Python exits non-zero,
and the sentinel is omitted on bad JSON. The traceback still
lands in antigravity_last_python_err.log for debugging.
Test: test_common_sh_parser_omits_sentinel_on_malformed_json.
5. `mempal_save_interval()` failed to strip leading zeros from
MEMPAL_SAVE_INTERVAL. Values like "08" or "09" then crashed the
modulo step `$((COUNT % INTERVAL))` because bash arithmetic
parses tokens starting with `0` as octal, and 8/9 are not valid
octal digits ("value too great for base"). Strip leading zeros
while preserving the literal "0" (which is then floored to 15).
Test: test_save_hook_handles_leading_zero_save_interval (4 cases).
Plus one cosmetic fix in install.sh: removed a no-op `(cd "$OLDPWD"
2>/dev/null || cd .) >/dev/null 2>&1` line in mempal_absolutize().
The subshell cd doesn't affect the parent shell, and the installer
never cd's in the main shell anyway, so $PWD is already correct.
Verification:
* 9 new regression tests, all 65 antigravity tests pass
* full repo: 2323 passed (was 2314), 3 skipped, 1 unrelated warning
* ruff check + ruff format --check both clean across 139 files
* bash -n clean on all four shell files
* clean reinstall to ~/.gemini/config/plugins/mempalace/ succeeds
* idempotent re-run produces zero file writes (cmp-gated)
* both hooks return {} exit 0 with synthetic camelCase stdin
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds first-class integration with Google's Antigravity IDE
(https://antigravity.google/) as a third sibling to the existing
Claude Code and Codex hook integrations. Strictly additive — no
existing files in main are restructured.
What ships
----------
* `.antigravity-plugin/` — verified-minimal plugin package:
* `plugin.json` with `{"name": "mempalace"}` (no fabricated fields)
* `mcp_config.json` registering the `mempalace-mcp` stdio server
* `hooks.json.tmpl` templated with `__PLUGIN_DIR__` substitution
* `skills/mempalace/SKILL.md` (real file — no symlinks)
* `hooks/antigravity/`:
* `lib/common.sh` — shared bash 3.2.57-compatible helpers with
sentinel-guarded camelCase JSON parser, antigravity_*-namespaced
state files, every existing kill switch, `MEMPAL_SAVE_INTERVAL >= 1`
floor (no /0), and fail-open emitters
* `mempal_save_hook_antigravity.sh` — Stop event handler:
increments per-conversation counter, defers when fullyIdle=False
or terminationReason=error, validates transcriptPath against
`..` traversal, spawns `mempalace mine --mode convos` in a
detached subprocess with a per-conversation pending marker,
ALWAYS emits `{}` (never `{"decision":"continue"}` — that would
force an infinite agent loop)
* `mempal_wake_hook_antigravity.sh` — PreInvocation handler gated
to invocationNum==1 with an atomic mkdir loop guard, runs
`mempalace wake-up` with a 500ms hard timeout, emits verbatim
output as `{"injectSteps":[{"ephemeralMessage":"..."}]}` or
`{}` on any failure
* `install.sh` — idempotent installer with cmp-gated copies,
`__PLUGIN_DIR__` substitution, relative path absolutization,
`--dry-run`, and basename-guarded `--uninstall` (refuses to
wipe a directory whose basename isn't `mempalace`)
* `INVESTIGATION.md` — verbatim quotes + URLs + dates from the
five official Antigravity doc pages, recording every surface
shipped and every surface deliberately omitted
(PreCompact equivalent, slash-commands, rules/, plugin
permissions field — the latter is third-party fabrication)
* `STDIN_SHAPE.md` — exact stdin/stdout contract per event with
worked examples
* `README.md` — local hook docs + troubleshooting
* `examples/antigravity/{hooks.json,mcp_config.json,README.md}` —
standalone configs for users who don't want the full installer
* `website/guide/antigravity.md` + sidebar entry — VitePress guide
* Updates to `README.md`, `CHANGELOG.md` (Unreleased), `hooks/README.md`
Tests (56 new, all passing)
---------------------------
* `tests/test_antigravity_plugin_manifest.py` (11 tests) — schema
contract on the in-repo `.antigravity-plugin/` directory, including
guards against re-introducing the fabricated `permissions` field
and against any symlink leak.
* `tests/test_antigravity_hooks_shell.py` (31 tests) — invokes the
bash hooks via subprocess with synthetic camelCase stdin, asserts
`{}` on every failure path, kill-switch coverage (env vars +
config.json + palace nuke), divide-by-zero floor, transcript
traversal rejection, namespacing, wing inference, and the hard
refusal to ever emit `decision=continue` from the Stop hook.
* `tests/test_antigravity_hooks_install.py` (14 tests) — `--dry-run`
side-effect-free, real install layout, executable bits preserved,
byte-identical idempotent re-runs (md5 + filecmp), basename-match
uninstall safety, refusal when plugin.json is missing or names a
different plugin, relative path absolutization. Skipped on Windows.
Verification
------------
* `uv run pytest tests/ --ignore=tests/benchmarks -v` → 2314 passed,
3 skipped (Windows), 1 unrelated warning
* `uv run ruff check .` → all checks passed
* `uv run ruff format --check .` → 139 files already formatted
* `bash -n` clean on common.sh, both hook scripts, install.sh
* Local install at `~/.gemini/config/plugins/mempalace/` verified end-
to-end: layout correct, paths absolutized in hooks.json, both hooks
fire with realistic camelCase JSON in <1s, wing inference picks
`wing_mempalace` from workspacePaths[0], state files all
`antigravity_*`-namespaced, second `install.sh` run produces
byte-identical output (md5 snapshots match), uninstall removes
only the mempalace plugin and leaves all 6 sibling Google plugins
untouched.
Constraints honoured
--------------------
bash 3.2.57 (no mapfile / readarray / declare -A / `${var^^}`),
verbatim guarantee on all wake injections, hooks <500ms / startup
injection <100ms target (kill-switch path returns in <1.5s in CI),
zero new runtime dependencies, no telemetry, no external API,
strictly additive (existing Claude/Codex hooks unchanged).
Refs: hooks/antigravity/INVESTIGATION.md for the full audit.
Five fixes from the Gemini Code Assist review on
https://github.com/MemPalace/mempalace/pull/1632 — three real bugs,
two cleanups, all consistent with the bash-3.2-compatibility
contract documented in the original commit.
Bug fixes (high)
----------------
1. hooks/cursor/lib/common.sh — config.json kill-switch check used
a `python3 - <<'PYEOF' ... PYEOF` heredoc inside a `$(...)`
command substitution. The heredoc body contains parens which
trips the macOS bash 3.2.57 parser bug. Replaced with a
`python -c '...'` call passing the config path as argv[1]. Matches
the pattern already used in mempal_parse_stdin in the same file.
2. hooks/cursor/install.sh — a relative `--install-dir` was written
verbatim into hooks.json. Cursor invokes hook commands from its
own working directory (typically the project root), so a relative
command path would silently fail to launch the hook. Now resolved
to an absolute path against `$PWD` before being baked in.
3. hooks/cursor/mempal_save_hook_cursor.sh — `MEMPAL_SAVE_INTERVAL=0`
would crash bash on `$((NEXT % 0))` (division by zero). Extended
the existing sanitiser case to coerce 0 to the default interval
alongside empty / non-numeric values.
Cleanups (medium)
-----------------
4. hooks/cursor/install.sh — the EMPTY_CHECK_PY temp file is now
inlined as `python -c '...'`. Removes a small leak window
(tmpfile would linger if the script were interrupted between
mktemp and rm -f) and shortens the script.
5. hooks/cursor/install.sh — `mktemp -t prefix` has subtly different
semantics on BSD (macOS) vs GNU mktemp. Switched to the
portable absolute-template form `mktemp "${TMPDIR:-/tmp}/...XXXXXX"`
which behaves identically on both.
Regression tests
----------------
- tests/test_cursor_hooks_shell.py
test_save_interval_zero_is_coerced_to_default — guards fix#3.
- tests/test_cursor_hooks_install.py — new TestInstallDirAbsolutePath
class:
test_relative_install_dir_is_absolutized_in_hooks_json — guards
fix#2 against regression.
test_absolute_install_dir_is_preserved_verbatim — guards that
the relative-to-absolute resolution does not mangle paths
that were already absolute.
Verification
------------
- bash -n on all three edited scripts: clean.
- uv run pytest tests/test_cursor_hooks_*.py tests/test_cursor_plugin_manifest.py: 132 passed (was 129; +3 regression tests).
- uv run pytest tests/ --ignore=tests/benchmarks: 2399 passed,
3 skipped (pre-existing).
- uv run ruff check . / ruff format --check .: clean.
Co-authored-by: Cursor <cursoragent@cursor.com>
Adds first-class Cursor IDE integration alongside the existing Claude
Code and Codex hook flows, so Cursor users get the same automatic
diary saves, pre-compaction transcript capture, and session-start
memory recall — without changing any default behaviour for existing
users.
What's included
---------------
Cursor hook scripts (hooks/cursor/):
- mempal_save_hook_cursor.sh — Stop event, counter +
loop_count guard, pending-save marker consumption, background
mempalace mine, followup_message emission.
- mempal_precompact_hook_cursor.sh — synchronous mine before
compaction, drops a pending_save marker, returns user_message.
- mempal_wake_hook_cursor.sh — sessionStart event,
wing-scoped recall guidance via additional_context.
- lib/common.sh — shared parsing + state helpers
(bash 3.2 safe, no heredoc-in-subshell traps).
- install.sh — idempotent installer with
--scope, --variant, --dry-run, --uninstall. Recognises existing
entries by basename so re-installs across paths work.
- STDIN_SHAPE.md, README.md — payload schemas + quick
reference.
Cursor plugin (.cursor-plugin/ + repo-root components):
- plugin.json, marketplace.json, README.md.
- skills/mempalace/SKILL.md — model-invocable skill mirroring the
Claude plugin's skill surface.
- commands/mempalace-{help,init,mine,search,status}.md — slash
commands for marketplace-published installs (filename = slug).
- mcp.json — auto-registers the mempalace MCP
server, wrapped under the documented mcpServers key.
Examples + docs:
- examples/cursor/hooks.json, hooks.minimal.json + README.
- website/guide/cursor-hooks.md + sidebar entry.
- README.md and CHANGELOG.md updates.
Tests (129 new, all green):
- tests/test_cursor_hooks_shell.py — 75 behavioural tests for
the three hook scripts: kill switches, input parsing, counter
logic, loop prevention, pending markers, wing inference, logging.
- tests/test_cursor_hooks_install.py — 19 contract tests for the
installer: dry-run, idempotent merge, basename-matched uninstall,
refusal to overwrite malformed JSON.
- tests/test_cursor_plugin_manifest.py — 35 contract tests for the
plugin: manifest validity, version sync with mempalace.version,
mcp.json shape, skill/command frontmatter, default-discovery
layout invariants.
Design notes
------------
- Local-first and zero-API by default; hooks never call external
services. Same privacy model as the existing Claude Code hooks.
- Fail-open: hook scripts deliberately do not use set -e so a broken
hook can never block the user's conversation.
- Cursor preCompact cannot block + return a followup, so we
synchronously mine the transcript and drop a pending_save marker
that the next stop hook consumes — guarantees verbatim capture
before context window compression.
- Cursor's default plugin discovery requires real commands/, skills/,
and mcp.json at the plugin root (verified against the cached
cloudflare plugin); .cursor-plugin/{commands,skills} are convenience
symlinks back to those canonical locations.
- bash 3.2 compatibility throughout: avoids heredoc-in-command-
substitution parser bugs; uses python -c for JSON parsing;
basename-matched entry recognition in install.sh.
- All changes are additive. No existing files are removed, no
existing hooks change behaviour, and no new runtime dependencies
are introduced.
Co-authored-by: Cursor <cursoragent@cursor.com>
ChromaDB 1.5.x calls embedding_function.embed_query(input=...) via
keyword argument during collection.query(). EmbeddinggemmaONNX lacked
both embed_query and embed_documents methods, causing:
TypeError: embed_query() got an unexpected keyword argument 'input'
whenever semantic search was triggered.
This patch adds the two methods required by the ChromaDB EF protocol,
using (the ChromaDB kwarg name, noqa A002) so that palace
search works correctly with the embeddinggemma model.
Also downloads the companion ONNX file alongside the main model
to prevent runtime InferenceSession failures.
Fixes silent search failures when is set to
embeddinggemma.
Address review feedback: `with sqlite3.connect() as conn:` only
manages transactions, it does not close the connection. An open
connection before PersistentClient instantiation can leave WAL state.
Use explicit `try...finally: conn.close()` matching the read-only
helpers elsewhere in the module.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chromadb <= 1.5.8 writes config_json_str = '{}' (empty JSON) when
creating collections. chromadb 1.5.9 introduced a strict _type check
in the collection config deserialization path -- its absence raises
KeyError: '_type' on palace open. Since the pin allows >=1.5.4,<2,
any upgrade pulls 1.5.9 and breaks every existing palace.
Add a fourth pre-open migration step (_fix_missing_collection_type)
that injects "_type": "CollectionConfigurationInternal" into
collections.config_json_str rows that lack it. Same lifecycle and
marker-file pattern as the existing _fix_blob_seq_ids.
Co-Authored-By: nautis <nautis@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Brings in the three commits that landed on main during the v3.3.6
release cycle but not on develop:
- 62a555c fix(release): align ruff pin to 0.15.14 + hoist COCA
imports out of hot paths (#1614)
- a1cf052 Merge pull request #1614 from MemPalace/fix/release-3.3.6-ruff-pin-align
- db1fbe8 Merge pull request #1610 from MemPalace/release/3.3.6
Resolves conflicts in miner.py and palace.py where develop's #1613
(known-systems lexicon, Tier 3) had added _apply_known_systems_prepass
to module-top imports + the prepass call in build_closet_lines. Main
side had only the COCA hoist from #1614. Resolution: keep develop's
side (the prepass additions).
Mirrors the v3.3.5 sync-back pattern (#1442).
Resolves conflicts in miner.py and palace.py against the COCA-hoist
work from #1612 just merged. Both files had local imports for
_get_coca_filter / _candidate_entity_words in the function bodies;
after #1612 those moved to module top. Resolution: hoist
_apply_known_systems_prepass alongside _get_coca_filter at module
top in both files, drop the now-empty local imports.
No behavior change; just removes the local-import duplication that
would have been left in place if the merge auto-resolved naively.
Two release-blocking fixes for v3.3.6:
1. CI ruff pin drift
.github/workflows/ci.yml installed ruff==0.15.9 while pyproject.toml
[dev] extras and .pre-commit-config.yaml both pin 0.15.14. Ruff's
formatter output can change between minor versions, so a contributor
running `pip install -e ".[dev]"` and formatting locally with 0.15.14
would produce output the 0.15.9 lint job rejects. Same failure mode
that surfaced on PR #1579 (2026-05-22). Aligning CI to 0.15.14 keeps
the three pin sites in lock-step.
2. COCA filter imports inside per-drawer hot paths
PR #1605 (COCA content-word filter, shipping in 3.3.6) introduced
`from .entity_detector import _get_coca_filter` and
`from .palace import _candidate_entity_words` inside
_extract_entities_for_metadata (called per drawer) and
build_closet_lines (called per closet). Python caches module imports
so the runtime cost after the first call is small, but the import
machinery still runs Python bytecode every invocation — gemini
flagged this on the original PR. Hoisting to module-level removes
the per-call import overhead.
The hoist is identical to PR #1612, which targets develop. Folding
it into the release so 3.3.6 doesn't ship the perf regression that
3.3.7 would immediately have to fix.
Verification: ruff check + format clean on 0.15.14, full pytest
(2258 passed / 12 skipped) on Linux Py 3.9 / 3.11 / 3.13 via
`pip install -e ".[dev]"` (CI-matching).
Addresses gemini-code-assist MEDIUM finding on PR #1613: the previous
implementation of _apply_known_systems_prepass compiled a regex pattern
for every compound on every call, repeating the work on every drawer
mined and every closet built. With 59 compounds × N drawers, that's
59N re.compile() calls for a workload where the patterns never change.
The fix moves compilation into _get_known_systems (already lru_cache'd
to size=1), which now returns tuple[tuple[str, re.Pattern], ...] —
pairs of (canonical name, pre-compiled case-insensitive word-bounded
regex). _apply_known_systems_prepass consumes the cached tuple and
does zero compilation in the hot path.
Behavior is identical: same word boundaries, same case-insensitive
matching, same longest-first ordering, same graceful-degrade on
malformed json. All 74 entity_detector tests still pass on macOS plus
the full 2258-test suite on Linux Py 3.9 / 3.11 / 3.13.
Adds a curated list of multi-word product/system names ("Claude Code",
"GitHub Copilot", "Visual Studio Code", "GPT-4", …) and a compound
pre-pass that detects them atomically before the existing single-word
extraction runs. Without this, the regex-based detector decomposes
"Claude Code" into "Claude" + "Code" — and the COCA filter (shipped
in v3.3.6) then drops "Code" as a content word, leaving "Claude" alone
with the wrong attribution.
What ships
- mempalace/data/known_systems.json — 59 curated compounds covering
common AI assistants, IDEs, model names, cloud platforms, and
Office/Google apps. Each entry is multi-word or hyphenated;
single-word product names ("ChatGPT", "Cursor") have no
decomposition risk and stay handled by the existing regex.
- mempalace/entity_detector.py — _get_known_systems() (cached loader,
mirrors _get_coca_filter from Tier 2) and _apply_known_systems_prepass
which scans for each compound case-insensitively with word boundaries,
counts occurrences, and returns the masked text + count dict so the
subsequent single-word + multi-word loops don't re-decompose.
- mempalace/miner.py and mempalace/palace.py — same pre-pass wired
into _extract_entities_for_metadata (per-drawer tagger) and
build_closet_lines (closet pointer construction). Without these,
the new behavior would only apply at init-time and per-drawer
metadata would still decompose compounds.
How it interacts with Tier 2
Tier 2 (COCA filter) blocks single-word content nouns like "Code"
and "Brutal". Tier 3 protects multi-word product names so they
don't get decomposed in the first place. They complement each
other: the compound pre-pass runs FIRST and masks compounds out
of the text; the COCA filter then runs on the remaining
single-word candidates.
Behavior verification
Before this PR, mining a document containing "Claude Code wrote the
patch" three times emitted entities:
Claude;Claude Code;Code (Code filtered by COCA);
After this PR, the same document emits:
Claude Code
The standalone "Claude" no longer appears (it never actually appeared
alone in the source) and decomposition stops at the compound boundary.
Tests
Nine new tests in tests/test_entity_detector.py covering:
- "Claude Code" detected as atomic compound at extract_candidates
- "Claude" alone NOT in results when only mentioned as part of compound
- Case-insensitive compound matching (claude code, CLAUDE CODE, etc.)
- Single-word "Code" still filtered by COCA (no Tier 2 regression)
- Single-word real name "Aya" still detected (no regression on names)
- Multiple distinct compounds in one text both detected
- Unknown two-word phrase still detected via existing multi-word regex
- known_systems.json ships with expected schema (>=20 entries, all multi-token)
- known_systems.json contains expected high-value entries
Verification
Full pytest 2267 passed / 3 skipped on macOS, coverage 85.34%.
Linux Py 3.9 / 3.11 / 3.13 via CI-matching pip install -e ".[dev]":
2258 passed each. End-to-end mine of a compound-rich corpus
confirms chromadb entities metadata now shows compounds atomic
(Claude Code, GPT-4, GitHub Copilot, Visual Studio Code) with no
decomposition.
The COCA content-word filter shipped in PR #1605 imported
`_get_coca_filter` and `_candidate_entity_words` locally inside two
hot paths:
- `palace.build_closet_lines` — runs per source file during mine
- `miner._extract_entities_for_metadata` — runs per drawer during mine
Both imports are now at module top, where they're resolved once at
import time instead of on every per-drawer call. Module-top imports
also make the dependency graph visible to static analysis (pylint's
C0415 was flagging the locals).
No behavior change. The `_get_coca_filter()` call is unchanged — only
the import statement moved. End-to-end mining produces identical
chromadb output. Addresses the MEDIUM finding gemini-code-assist
raised on PR #1605 review.
Verification: full pytest 2258 passed / 3 skipped / coverage 85.35%.
ruff check + format clean. Linux Py 3.9 / 3.11 / 3.13 via CI-matching
`pip install -e ".[dev]"`: 2249 passed each. End-to-end mine of a
test corpus produces the expected drawer + closet pointer.
The two pre-existing entries for #1467 (tunnels.json path) and #1468
(create_tunnel endpoint validation) were sitting at the bottom of the
[Unreleased] block before this release-prep PR. Inserting the new
Performance section between the freshly-backfilled Bug Fixes and these
two pre-existing entries put them under Performance, which is wrong —
they're bug fixes. Moves them back ahead of Performance.
The scam-alert and Claude-Code-retention admonitions were the first
content visitors saw on the repo page — louder than the project
introduction. Moves both below the logo/title/badges so the project
identity reads first, and softens the scam block (drops the H1
"CRITICAL SECURITY WARNING" + all-caps shouting + redundant emoji)
to a single-paragraph CAUTION. All factual content preserved:
impostor-domain warning, official sources, malware caveat, link to
docs/HISTORY.md.
Resolves conflicts in CHANGELOG.md and pyproject.toml by combining
the multilingual-embedder additions (huggingface_hub/tokenizers/numpy
core deps, [multilingual] alias, Features section) with develop's
additions (python-dateutil core dep, [extract] extra, tunnel Bug
Fixes and Internal sections).
Prepares PR #1483 for merge into v3.3.6.
Push the surrogate-cleaning behaviour from per-call-site _clean() helpers
into sanitize_content, sanitize_kg_value, and sanitize_query so every
caller (existing and future) gets the fix automatically. New MCP tools
no longer need to remember to call a separate helper.
- Add strip_lone_surrogates() in mempalace/config.py as the single
regex-based implementation (one U+FFFD per surrogate, not three).
- Wire it into sanitize_content and sanitize_kg_value.
- Wire it into sanitize_query so embedding lookups can't crash either.
- Drop the _clean() helper from mcp_server.py and the per-site calls;
retain a direct strip_lone_surrogates() for source_file/added_by
metadata which doesn't route through any sanitizer.
- Move the ONNX model-cache patch out of the test module and into
conftest.py so it's session-scoped instead of duplicated locally.
- Update tests to assert one U+FFFD per surrogate and exercise the
sanitizer-level entry points directly.
Add tests/test_clean_lone_surrogates.py covering:
Unit tests (TestCleanLoneSurrogates, 11 cases):
- _clean() passes normal ASCII and CJK strings unchanged
- lone surrogates (high/low, single/multiple) are replaced with U+FFFD
- real emoji (\U0001f600 astral code points) pass through unchanged
- empty string, all-surrogate string, SHA-256-hash-after-clean
- the specific \udcad surrogate observed in WorkBuddy production logs
Integration tests (TestLoneSurrogateCleaning, 6 cases):
- tool_add_drawer: surrogate in content and in metadata fields
- tool_check_duplicate: surrogate in query
- tool_search: surrogate in search query
- tool_update_drawer: surrogate in updated content
- tool_diary_write: surrogate in diary entry
Fix test environment issue:
- conftest.py redirects HOME to a temp dir, causing chromadb's
ONNXMiniLM_L6_V2 to look for its ONNX model in the wrong location
and trigger a 79 MB network download on every run.
- Fix: at module import time, recover the real USERPROFILE from
conftest._original_env and patch ONNXMiniLM_L6_V2.DOWNLOAD_PATH
before any ChromaDB collection fixture is invoked.
- Fix _clean() docstring (paired surrogate description was misleading)
- Apply _clean() to source_file and added_by metadata fields
- Remove redundant surrogatepass from SHA256 hashes (content already cleaned)
- Apply _clean() to content in tool_check_duplicate
- Apply _clean() to new_doc in tool_update_drawer
- Apply _clean() to sanitized query in tool_search
Addresses feedback from gemini-code-assist[bot] on PR #1422.
MCP clients can emit lone surrogates (\udc00–\udfff) that
cause Python's str.encode('utf-8') to raise UnicodeEncodeError,
which bubbles up as -32000 Internal Error from ChromaDB.
Add _clean(text) helper that uses 'surrogatepass'/'replace' to
remove lone surrogates before the string reaches ChromaDB.
Apply it in tool_add_drawer and tool_diary_write, and use
'surrogatepass' error handler on the SHA256 hash inputs for
defensive idempotency.
Wires _validate_palace_fts5_after_mine into all three mine entry
points so corrupted-FTS5 palaces cannot silently exit 0 from any
of them:
- _mine_impl (mempalace/miner.py) — project file miner
- mine_convos (mempalace/convo_miner.py) — conversation exports
- mine_formats (mempalace/format_miner.py) — binary office documents
via --mode extract, introduced on develop by #1555 (3.3.6 release)
between this PR's open date and its rebase
cmd_mine surfaces MineValidationError as exit 1 + the same
print_sqlite_integrity_abort banner cmd_repair already prints,
appended with a mine-specific stderr note that hedges attribution
(quick_check cannot tell pre-existing corruption from corruption
this mine produced). 17 tests in tests/test_miner_fts5_validation.py
cover the helper, the three call sites, dry-run / KeyboardInterrupt
skip semantics, and the MineValidationError constructor invariants.
Closes#1537.
Co-authored-by: Caleb Wells <15988028+calebcwells@users.noreply.github.com>
The previous commit landed the code referencing mempalace/data/coca_content_words.json
but the data file itself was never actually staged (likely a multi-line paste mishap
during the initial commit). CI on every platform failed because the file isn't in
the repo. This commit ships the wordlist + adds TypeError to the COCA loader's
graceful-degradation exception tuple per gemini-code-assist review on PR #1605.
Common English content words ("Code", "Brutal", "Phase", "Line", "Note",
"Planning", "Chat", ...) frequently appear capitalized at sentence
start, in headings, or in markdown — and the existing regex-based entity
detector treats every capitalized word appearing 3+ times as a proper
noun candidate. This produces false positives that pollute
known_entities.json, the per-drawer entities metadata, and closet
pointer entity lists.
Adds a curated list of ~1000 common English content words (nouns,
adjectives, verbs — no proper nouns, no known product names) at
mempalace/data/coca_content_words.json, derived from the COCA top-2000
frequency list and from observed false positives in real palaces.
The list is loaded once at module import via _get_coca_filter() (cached
frozenset) and consulted at three sites:
- entity_detector.extract_candidates — init-time entity detection
- palace.build_closet_lines — closet pointer construction
- miner._extract_entities_for_metadata — per-drawer entity tagging
Matching is case-insensitive: the candidate is lowercased before
membership check, so "Code", "CODE", and "code" are all blocked.
Only single-word candidates are filtered; multi-word phrases (e.g.
"Claude Code") are NOT filtered, so legitimate compound names continue
to be detected. A future PR will add a known-systems lexicon that
explicitly protects compound names — for now, the multi-word pass
provides the implicit protection.
Adds 8 new tests in tests/test_entity_detector.py covering:
- "Code" appearing 5x is filtered
- All known false positives from real palaces are filtered
- Case-insensitive matching works
- Real names ("Aya", "Riley") are NOT filtered
- Multi-word phrases like "Claude Code" are NOT filtered
- The data file ships in the wheel with the expected schema
- The data file contains every known false positive
Note: this filter prevents NEW false positives at detection time. It
does NOT retroactively scrub existing entries from a user's
~/.mempalace/known_entities.json. Stale entries from previous runs
must be removed via a future entity-management CLI (planned).
Verification: pytest 2191 passed / 3 skipped / 0 failed on macOS.
ruff check + format --check both clean. Cross-platform via
pip install -e ".[dev]" on python:3.9-slim / 3.11-slim / 3.13-slim
containers, each running 65 entity-detector tests including the new
data-file-presence test that imports from the installed package.
End-to-end pipeline test mining a deterministic corpus into an
isolated palace confirmed entity metadata is empty (no false
positives surface in real chromadb-backed storage).
test_precompact_hook_enabled_by_default asserted
result["decision"] == "block", but hook_precompact has never emitted
decision — it mines synchronously and returns {}. Assertion was
copy-pasted from the stop-hook test. Fix to assert result == {} with
_mine_sync mocked so the test verifies the real contract (enabled →
mine + pass through) without actually mining.
Plus ruff format on 6 test files the CI pin flagged.
- Add parenthetical hints to block reasons so AI knows what each tool
saves (session summary, quotes/decisions/code)
- Remove dead config file creation from test_stop_hook_disabled_by_config
- Add missing test for MEMPALACE_HOOKS_AUTO_SAVE=no env var
- Replace bare except: with except Exception: in shell scripts
Add a clean opt-out for auto-save hook blocking (closes#494).
- New `hooks.auto_save` config option (default: true) in
~/.mempalace/config.json and MEMPALACE_HOOKS_AUTO_SAVE env var
- When disabled, stop and precompact hooks pass through without blocking
- Shorten block reason text from 6-line instructions to single-line
prompts — reduces UI noise while keeping tool names explicit
- Both Python (hooks_cli.py) and standalone shell scripts respect the
toggle via config file or env var
Lock-step with pyproject.toml `[project.optional-dependencies].dev`
(ruff bumped from 0.15.9 → 0.15.14 via PR #1583). The
`.pre-commit-config.yaml` header explicitly requires this rev to
match the pyproject pin — without this bump, contributors who run
`pre-commit install` will hit the same version-mismatch debacle this
PR was opened to prevent.
Adds two test-discipline tools to the dev extras and closes a real
contributor-onboarding gap in CONTRIBUTING.md that allowed PR #1579's
2026-05-22 4 AM ruff-version-mismatch lint failure.
Adds to ``[project.optional-dependencies].dev`` (and matching
``[dependency-groups].dev`` for uv users):
- **``hypothesis>=6.0``** — property-based testing framework.
Generates hundreds of random inputs per test and shrinks failing
cases to a minimal counterexample. Used opt-in (sprinkle
``@given(...)`` on a test); zero runtime cost on tests that don't
use it. Would have caught the Tier 6a dateutil-fuzzy hallucination
on PR #1584 with one property test.
- **``pre-commit>=3.0``** — the pre-commit framework itself.
``.pre-commit-config.yaml`` already lives in the repo (committed
by @igorls on 2026-05-18, pinned to ruff 0.15.9 in lockstep with
CI). What was missing was making the framework a declared dev
dependency so ``pip install -e .[dev]`` / ``uv sync --extra dev``
actually pulls it in.
Adds a ``pre-commit install`` line to the Getting Started bash block
plus a short paragraph explaining why this step is required (the
hook file at ``.git/hooks/pre-commit`` is per-machine and NOT tracked
by git, so the repo's ``.pre-commit-config.yaml`` only takes effect
after each developer runs ``pre-commit install`` once locally).
Adds an optional "Property-based tests" subsection under Running
Tests showing the minimal ``@given(...)`` pattern, so contributors
who want to reach for the new tool know it's available.
On 2026-05-22 at 4:30 AM, PR #1579 (Tier 6a) hit a CI lint failure
caused by a ruff version mismatch: my local machine had ruff 0.4.10,
CI runs ruff 0.15.9 (pinned in pyproject.toml). The two versions
produce different ``ruff format`` output for the same code. The repo
HAD ``.pre-commit-config.yaml`` pinning ruff 0.15.9 — but the local
git hook had never been wired on my machine because nothing in
CONTRIBUTING.md said to run ``pre-commit install``. The protection
existed at the project layer; the activation gap was at the
contributor-onboarding layer.
This commit closes that gap structurally. Anyone cloning the repo
from now on sees ``pre-commit install`` as part of the Getting Started
flow and is protected from the same failure.
- **No ``mutmut`` in dev deps.** Mutation testing is heavier (runs
the whole test suite per mutation) and useful periodically rather
than every commit. Contributors who want to run it can install
manually. Adding it to dev deps would bloat the install footprint
for every contributor when most will never use it.
- **No new property tests.** This PR ships the TOOL, not new test
coverage. Property tests should land alongside the specific
functions they cover, in their own PRs.
- **No changes to ``.pre-commit-config.yaml``.** That file is correct
as Igor wrote it. The fix here is purely making the framework
installable + documenting the activation step.
ruff check .
→ All checks passed.
pre-commit run --all-files (locally)
→ ruff (legacy alias): Passed
→ ruff format: Passed
OrbStack triple-Linux verify (Py 3.9 / 3.11 / 3.13)
→ ``pip install -e .[dev]`` resolves cleanly; hypothesis +
pre_commit import successfully; existing test suite unaffected.