Short sessions that exit cleanly below SAVE_INTERVAL and without a PreCompact
were never saved. Add a SessionEnd hook that takes one final flush.
Claude Code budgets SessionEnd hooks at 1.5s and a plugin-provided timeout
cannot raise it, and a cold mempalace start exceeds that, so the wrapper
backgrounds the work and returns immediately; the detached child completes the
transcript ingest, project mine, and diary checkpoint after the session exits.
The handler validates transcript_path through _validate_transcript_path before
any ingest or diary write, so a traversal or wrong-suffix path is rejected while
the independent project mine still runs.
Adds hook_session_end, both shell wrappers, the plugin hooks.json entry, the
session-end CLI choice, and focused tests.
(cherry picked from commit 10e1450e04fc7cec72984ab7442d3b4fca1490e8)
HTTPServer.server_bind() resolves server_name via socket.getfqdn(host). For the
daemon's 127.0.0.1 bind that lookup is pointless, and on a host with slow or
absent reverse DNS it blocks startup until the resolver times out (~30s) — which
looks exactly like the daemon never coming up. This is why the first daemon
HTTP-lifecycle test timed out on the macOS CI runner (httpd_bound=False after
30s) while every later one bound in seconds once the OS had cached the negative
lookup. Bind via TCPServer directly and set server_name from the literal host.
_pid_alive used os.kill(pid, 0) as an existence check. On Windows signal 0
is signal.CTRL_C_EVENT, so Python routes it to GenerateConsoleCtrlEvent and
sends a console Ctrl-C to the target's process group rather than probing the
pid. DaemonClient polls a same-process endpoint during startup, so on a CI
runner with an attached console that Ctrl-C was delivered back to the
interpreter as a spurious KeyboardInterrupt — the Windows CI hang that
interrupted the suite at the first daemon HTTP-lifecycle test (socket.py
recv). Probe via the Win32 OpenProcess/WaitForSingleObject handle API instead,
which has no signalling side effects. This is also a real Windows production
bug, not just a test artifact.
Tests:
- Skip the two owner-only (0600) permission tests on Windows: os.chmod cannot
represent POSIX mode bits there (files report 0o666); the daemon relies on
user-profile ACLs on Windows.
- _start_server now captures and re-surfaces a run_server thread crash instead
of spinning for 30s and failing with a bare assert (diagnoses the macOS
startup flake).
- Add a regression test asserting _pid_alive is correct and emits no console
control event when hammered like the poll loop.
- Remove the temporary win32 exit-hang diagnostic fixture from conftest now
that the root cause is fixed.
The Windows CI run passes all 666 tests then hangs at interpreter shutdown
(KeyboardInterrupt at socket.py:723) until the runner kills it. All daemon
lifecycle tests assert their server threads died, so the hang is a different
non-daemon thread blocked on a socket — not the daemon server thread. CI
round-trips can't show which thread it is.
Add a win32-only session fixture that:
- arms faulthandler.dump_traceback_later(130s) to print every thread's stack
to stderr once the hang has run a while, and
- prints every live thread (name + daemon flag) at session teardown — a
non-daemon thread present there is the shutdown blocker.
Gated to sys.platform == 'win32' so Linux/macOS CI see no extra output. Remove
once the Windows hang is fixed.
DaemonClient.request now uses a no-proxy opener (build_opener(ProxyHandler({})))
instead of urllib.urlopen. The daemon is always on 127.0.0.1, so a request must
never go through an HTTP proxy — this is the correct production choice. It also
bypasses urllib's proxy discovery (macOS _scproxy via SystemConfiguration), which
runs on the first request to any host and is NOT bounded by the per-request
timeout: on a CI runner with no network it hangs for tens of seconds, which looked
exactly like the daemon never came up (test_daemon_http_lifecycle_executes_job
timed out at 30.18s). With the no-proxy opener the lifecycle runs in 0.78s and no
server thread is leaked — which also removes the timing skew that made the
sqlite_exact concurrent-connection test flake on macOS CI.
The leaked server thread was also the Windows exit-hang root cause: a slow/failed
client.shutdown() POST left serve_forever running, and the interpreter blocked on
the open listening socket at process exit. Tests now capture the httpd run_server
creates (by subclassing daemon.ThreadingHTTPServer) and force httpd.shutdown() +
server_close() from the test thread if the normal shutdown path leaves the thread
alive, asserting the thread died so a leak becomes a visible failure instead of
a silent exit hang.
CI was red on all three platforms for the daemon-mode draft PR. Root causes
and fixes:
- Linux 3.9 collection error: `_submit_daemon_job`'s `dedupe_key: str | None`
parameter annotation is evaluated at def time, and hooks_cli.py has no
`from __future__ import annotations` — `str | None` raises TypeError on 3.9.
Reverted to `dedupe_key: str = None` (the original, 3.9-safe). The other
`int | None` in the file is a function-local annotation, which is never
evaluated, so it was never the problem.
- macOS/Windows daemon lifecycle flakes: the 3 HTTP-lifecycle tests failed at
the 10s readiness deadline on contended CI runners (localhost bind is
sub-second locally but took ~5s when it passed on the macOS fleet, >10s when
it didn't), and because the server thread never shuts down on timeout,
run_server's `os.environ["MEMPALACE_PALACE_PATH"]` + `os.umask(0o077)`
mutations leaked into the rest of the suite — poisoning every later test that
reads MempalaceConfig().palace_path (the 60+ test_mcp_server cascade on macOS;
the at-exit socket hang → SIGINT on Windows). Bumped the readiness deadline
to 30s and added a module-scoped snapshot + autouse fixture in test_daemon.py
that force-restores the env + umask to the pre-suite baseline after every
daemon test, so a leaked server thread can't poison other test files.
Gemini review comments (fixed in code, no thread replies per convention):
- daemon.py `_connect()` was a bare `sqlite3.connect` whose `with`-block only
managed the transaction, not the connection — an unbounded FD leak in a
long-lived daemon running thousands of jobs (also the source of the Windows
"unclosed database" ResourceWarning noise). Converted to a closing
@contextlib.contextmanager.
- `QueueStore.finish()` gained `only_if_running`; `_safe_finish` passes it so a
late worker finish can't overwrite a shutdown-cancelled job back to
succeeded/failed — removes the reliance on process-exit timing.
- `DaemonClient.request` wraps the final `json.loads` in try/except
JSONDecodeError → DaemonError, so a non-JSON 2xx response surfaces as a
structured error instead of a bare JSONDecodeError.
- test_sync.py: removed the module-level `import mempalace.mcp_server` and moved
the stdout-rebinding side effect into an autouse fixture scoped to
TestServiceRunSyncReport, so the embedder/Chroma import chain is no longer
forced at collection time for the existing sync tests.
Coverage: added focused happy-path tests for service.run_sync early-returns,
run_mine backend application + invalid mode, execute_job kind dispatch,
run_diary_write arg forwarding, run_mcp_tool write-tool dispatch, and
print_job_result — lifts service.py from 57% to 85% so the new files
(service 85%, daemon 80%) don't drag the total below the 80% CI gate now that
the daemon tests complete and the gate is actually evaluated.
- New mempalace/daemon.py: long-lived localhost HTTP server (127.0.0.1) with a
SQLite WAL job queue, single worker thread, bearer-token auth, and owner-only
file perms (0600/0700) on queue DB, token, endpoint, and log.
- New mempalace/service.py: transport-neutral job execution surface shared by the
daemon, with per-job env isolation so one job's backend/palace switch cannot
leak into the next. mcp_tool is allowlisted to write-classified tools only.
- Crash recovery re-queues jobs left 'running' by a killed daemon; jobs that
already exhausted MAX_ATTEMPTS are dead-lettered to 'failed' instead of being
retried (non-idempotent diary_write would otherwise duplicate verbatim
content on every restart).
- Bounded retention prunes terminal jobs older than 7 days
(MEMPALACE_DAEMON_RETENTION_DAYS); queued/running jobs are never touched so a
crash mid-prune cannot drop in-flight work.
- CLI: --daemon/--background on mine/sync submit to the queue; new
`mempalace daemon {start,stop,status,jobs,wait}` subcommand. Strictly opt-in:
no flag, env, or config means no daemon and no behavior change.
- Hooks opt in via MEMPALACE_HOOKS_DAEMON or config hooks.daemon; when the daemon
is not already running, hooks fall back to the existing direct/spawn path so
the 500ms hook budget is preserved (hooks never auto-start the daemon).
- service.run_sync renders the same operator-facing report shape as the direct
CLI sync path (no_source, out_of_scope, by_source, Re-run/Removed hints) and
drops the old KeyError-prone 'deleted' read.
Address Gemini review on #1729:
- normalize source_file with strip_lone_surrogates so exact matching hits
rows mined from non-ASCII paths via cp1252 stdin (#1488), mirroring
tool_add_drawer's ingestion-side normalization
- isinstance(str) guard so a non-string source_file returns a clean error
instead of AttributeError
- default missing wing/room to "" in the dry-run sample, consistent with
the rest of the file
- add tests: non-string rejection + surrogate-normalization match
Adds an MCP tool to remove every drawer mined from a given source_file
exact match, for cleaning up benchmark/test data accidentally mined into
a user wing (ShareGPT dumps, results_mempal_*.jsonl, language config
JSON) that drowns out real memories in semantic search.
Matching is pushed to the backend via delete(where={"source_file": ...})
the same idiom the miner and diary-ingest paths already use so it is not
subject to the SQLite variable limit regardless of how many drawers share
the source. Defaults to a dry run reporting match count and a sample;
dry_run=false commits. Absent source is an idempotent no-op, not an error.
Adds end-to-end regression coverage for the migration swap path where os.replace hits EXDEV, the shutil.move fallback fails, and the original palace must be restored from the rename-aside copy.
The snippet has no shell interpolation — the path arrives via argv, not
string interpolation — so single quotes are correct and make it
unambiguous that nothing is shell-expanded. Behavior is identical:
`sys.argv[1]` contains no `$`, so it was never expanded (verified
empirically). Matches the single-quoted `python -c` blocks already in
hooks/cursor/lib/common.sh. No functional change.
Address review feedback surfaced on the 3.4.1 release promotion (#1810).
Bug fix — `date -r FILE` is GNU-only. On BSD/macOS `date -r` expects
epoch seconds, not a path, so the staleness/throttle checks in the new
Cursor and Antigravity hooks silently failed on macOS: the state GC
swept on every fire and the pending-save guard was skipped. Replace
with a portable `os.path.getmtime` one-liner via the already-resolved
$MEMPAL_PYTHON_BIN (cursor/lib, antigravity/lib, antigravity save hook).
This restores the "bash 3.2.57 / macOS default" compatibility the
Antigravity changelog claims.
Docs:
- Correct the MCP tool count to 33 (was 19/29/31 in 21 places across
plugin manifests, READMEs, and website docs — all drifted from the
TOOLS dict / mcp-tools.md reference, which both have 33).
- Fix broken CHANGELOG link to the Cursor skill (skills/, not
.cursor-plugin/skills/).
- Fix one-too-many `../` in skills/mempalace/SKILL.md's cursor-hooks
link (resolved above the repo root).
- Add the required `mcpServers` wrapper to the mcp.json example in
.cursor-plugin/README.md so copy-paste yields a valid Cursor config.
Left intentionally unchanged: the os.dup2 fd-1 redirect in
mcp_server.py is deliberate (#225 keeps JSON-RPC off fd 1).
Bump version across all sources (version.py, pyproject.toml, both
Claude plugin manifests, Codex plugin manifest, README badge, uv.lock)
and promote the Unreleased changelog to 3.4.1.
Shipping: Cursor IDE plugin + hooks, first-class Antigravity IDE
support (with zero-config interpreter resolution), embeddinggemma
bulk re-embed OOM fix, and backup-retention pruning.
Also rebuilds the CHANGELOG compare-link block, which had been left
at v3.2.0: adds the full 3.3.0-3.4.1 chain plus the previously
undocumented 3.4.0, and points Unreleased at v3.4.1...HEAD. Every
version header now resolves to a compare link.