Commit Graph
1588 Commits
Author SHA1 Message Date
fatkobra 393107e4ac fix(claude-plugin): resolve SessionEnd merge semantics 2026-06-19 13:44:52 +00:00
mvalentsev d09392c11a feat(hooks): add a budget-safe SessionEnd save hook for clean exits (#1341)
Short sessions that exit cleanly below SAVE_INTERVAL and without a PreCompact
were never saved. Add a SessionEnd hook that takes one final flush.

Claude Code budgets SessionEnd hooks at 1.5s and a plugin-provided timeout
cannot raise it, and a cold mempalace start exceeds that, so the wrapper
backgrounds the work and returns immediately; the detached child completes the
transcript ingest, project mine, and diary checkpoint after the session exits.

The handler validates transcript_path through _validate_transcript_path before
any ingest or diary write, so a traversal or wrong-suffix path is rejected while
the independent project mine still runs.

Adds hook_session_end, both shell wrappers, the plugin hooks.json entry, the
session-end CLI choice, and focused tests.

(cherry picked from commit 10e1450e04fc7cec72984ab7442d3b4fca1490e8)
2026-06-19 13:00:14 +00:00
Igor Lins e Silva fe0391ab56 fix(daemon): skip reverse-DNS in server_bind so startup can't block ~30s
HTTPServer.server_bind() resolves server_name via socket.getfqdn(host). For the
daemon's 127.0.0.1 bind that lookup is pointless, and on a host with slow or
absent reverse DNS it blocks startup until the resolver times out (~30s) — which
looks exactly like the daemon never coming up. This is why the first daemon
HTTP-lifecycle test timed out on the macOS CI runner (httpd_bound=False after
30s) while every later one bound in seconds once the OS had cached the negative
lookup. Bind via TCPServer directly and set server_name from the literal host.
2026-06-19 09:49:44 -03:00
Igor Lins e Silva f868ee78d4 fix(daemon): Windows-safe pid liveness probe; finalize cross-platform daemon tests
_pid_alive used os.kill(pid, 0) as an existence check. On Windows signal 0
is signal.CTRL_C_EVENT, so Python routes it to GenerateConsoleCtrlEvent and
sends a console Ctrl-C to the target's process group rather than probing the
pid. DaemonClient polls a same-process endpoint during startup, so on a CI
runner with an attached console that Ctrl-C was delivered back to the
interpreter as a spurious KeyboardInterrupt — the Windows CI hang that
interrupted the suite at the first daemon HTTP-lifecycle test (socket.py
recv). Probe via the Win32 OpenProcess/WaitForSingleObject handle API instead,
which has no signalling side effects. This is also a real Windows production
bug, not just a test artifact.

Tests:
- Skip the two owner-only (0600) permission tests on Windows: os.chmod cannot
  represent POSIX mode bits there (files report 0o666); the daemon relies on
  user-profile ACLs on Windows.
- _start_server now captures and re-surfaces a run_server thread crash instead
  of spinning for 30s and failing with a bare assert (diagnoses the macOS
  startup flake).
- Add a regression test asserting _pid_alive is correct and emits no console
  control event when hammered like the poll loop.
- Remove the temporary win32 exit-hang diagnostic fixture from conftest now
  that the root cause is fixed.
2026-06-19 09:38:34 -03:00
Igor Lins e Silva 95211e84cb test: win32-only diagnostic for daemon process-exit hang
The Windows CI run passes all 666 tests then hangs at interpreter shutdown
(KeyboardInterrupt at socket.py:723) until the runner kills it. All daemon
lifecycle tests assert their server threads died, so the hang is a different
non-daemon thread blocked on a socket — not the daemon server thread. CI
round-trips can't show which thread it is.

Add a win32-only session fixture that:
  - arms faulthandler.dump_traceback_later(130s) to print every thread's stack
    to stderr once the hang has run a while, and
  - prints every live thread (name + daemon flag) at session teardown — a
    non-daemon thread present there is the shutdown blocker.

Gated to sys.platform == 'win32' so Linux/macOS CI see no extra output. Remove
once the Windows hang is fixed.
2026-06-19 09:01:50 -03:00
Igor Lins e Silva d859d5a565 fix: daemon client bypasses proxy discovery; tests force-shutdown server thread
DaemonClient.request now uses a no-proxy opener (build_opener(ProxyHandler({})))
instead of urllib.urlopen. The daemon is always on 127.0.0.1, so a request must
never go through an HTTP proxy — this is the correct production choice. It also
bypasses urllib's proxy discovery (macOS _scproxy via SystemConfiguration), which
runs on the first request to any host and is NOT bounded by the per-request
timeout: on a CI runner with no network it hangs for tens of seconds, which looked
exactly like the daemon never came up (test_daemon_http_lifecycle_executes_job
timed out at 30.18s). With the no-proxy opener the lifecycle runs in 0.78s and no
server thread is leaked — which also removes the timing skew that made the
sqlite_exact concurrent-connection test flake on macOS CI.

The leaked server thread was also the Windows exit-hang root cause: a slow/failed
client.shutdown() POST left serve_forever running, and the interpreter blocked on
the open listening socket at process exit. Tests now capture the httpd run_server
creates (by subclassing daemon.ThreadingHTTPServer) and force httpd.shutdown() +
server_close() from the test thread if the normal shutdown path leaves the thread
alive, asserting the thread died so a leak becomes a visible failure instead of
a silent exit hang.
2026-06-19 08:52:34 -03:00
Igor Lins e Silva 5f58cdbfa8 fix: unblock daemon PR CI + address review comments
CI was red on all three platforms for the daemon-mode draft PR. Root causes
and fixes:

- Linux 3.9 collection error: `_submit_daemon_job`'s `dedupe_key: str | None`
  parameter annotation is evaluated at def time, and hooks_cli.py has no
  `from __future__ import annotations` — `str | None` raises TypeError on 3.9.
  Reverted to `dedupe_key: str = None` (the original, 3.9-safe). The other
  `int | None` in the file is a function-local annotation, which is never
  evaluated, so it was never the problem.

- macOS/Windows daemon lifecycle flakes: the 3 HTTP-lifecycle tests failed at
  the 10s readiness deadline on contended CI runners (localhost bind is
  sub-second locally but took ~5s when it passed on the macOS fleet, >10s when
  it didn't), and because the server thread never shuts down on timeout,
  run_server's `os.environ["MEMPALACE_PALACE_PATH"]` + `os.umask(0o077)`
  mutations leaked into the rest of the suite — poisoning every later test that
  reads MempalaceConfig().palace_path (the 60+ test_mcp_server cascade on macOS;
  the at-exit socket hang → SIGINT on Windows). Bumped the readiness deadline
  to 30s and added a module-scoped snapshot + autouse fixture in test_daemon.py
  that force-restores the env + umask to the pre-suite baseline after every
  daemon test, so a leaked server thread can't poison other test files.

Gemini review comments (fixed in code, no thread replies per convention):

- daemon.py `_connect()` was a bare `sqlite3.connect` whose `with`-block only
  managed the transaction, not the connection — an unbounded FD leak in a
  long-lived daemon running thousands of jobs (also the source of the Windows
  "unclosed database" ResourceWarning noise). Converted to a closing
  @contextlib.contextmanager.
- `QueueStore.finish()` gained `only_if_running`; `_safe_finish` passes it so a
  late worker finish can't overwrite a shutdown-cancelled job back to
  succeeded/failed — removes the reliance on process-exit timing.
- `DaemonClient.request` wraps the final `json.loads` in try/except
  JSONDecodeError → DaemonError, so a non-JSON 2xx response surfaces as a
  structured error instead of a bare JSONDecodeError.
- test_sync.py: removed the module-level `import mempalace.mcp_server` and moved
  the stdout-rebinding side effect into an autouse fixture scoped to
  TestServiceRunSyncReport, so the embedder/Chroma import chain is no longer
  forced at collection time for the existing sync tests.

Coverage: added focused happy-path tests for service.run_sync early-returns,
run_mine backend application + invalid mode, execute_job kind dispatch,
run_diary_write arg forwarding, run_mcp_tool write-tool dispatch, and
print_job_result — lifts service.py from 57% to 85% so the new files
(service 85%, daemon 80%) don't drag the total below the 80% CI gate now that
the daemon tests complete and the gate is actually evaluated.
2026-06-19 08:33:37 -03:00
fatkobra 3c7808e1bc fix(mcp): guard remaining None palace_path in _mcp_sqlite_integrity_refusal. Added one regression test calling the function directly with palace_path=None. 2026-06-19 09:51:02 +00:00
fatkobra 4c46290b80 fix(mcp): applied 3 of the 4 reviewer suggestions 2026-06-19 08:59:25 +00:00
dependabot[bot] 6bccf82a8c chore(deps-dev): bump ruff from 0.15.15 to 0.15.18
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.15 to 0.15.18.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.15...0.15.18)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.15.18
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-06-19 07:54:00 +00:00
fatkobra 63a382fc6d fix(mcp): gate startup on sqlite integrity failures 2026-06-19 06:42:36 +00:00
Igor Lins e Silva aa96bb5623 feat: add opt-in local daemon for queued MemPalace writes
- New mempalace/daemon.py: long-lived localhost HTTP server (127.0.0.1) with a
  SQLite WAL job queue, single worker thread, bearer-token auth, and owner-only
  file perms (0600/0700) on queue DB, token, endpoint, and log.
- New mempalace/service.py: transport-neutral job execution surface shared by the
  daemon, with per-job env isolation so one job's backend/palace switch cannot
  leak into the next. mcp_tool is allowlisted to write-classified tools only.
- Crash recovery re-queues jobs left 'running' by a killed daemon; jobs that
  already exhausted MAX_ATTEMPTS are dead-lettered to 'failed' instead of being
  retried (non-idempotent diary_write would otherwise duplicate verbatim
  content on every restart).
- Bounded retention prunes terminal jobs older than 7 days
  (MEMPALACE_DAEMON_RETENTION_DAYS); queued/running jobs are never touched so a
  crash mid-prune cannot drop in-flight work.
- CLI: --daemon/--background on mine/sync submit to the queue; new
  `mempalace daemon {start,stop,status,jobs,wait}` subcommand. Strictly opt-in:
  no flag, env, or config means no daemon and no behavior change.
- Hooks opt in via MEMPALACE_HOOKS_DAEMON or config hooks.daemon; when the daemon
  is not already running, hooks fall back to the existing direct/spawn path so
  the 500ms hook budget is preserved (hooks never auto-start the daemon).
- service.run_sync renders the same operator-facing report shape as the direct
  CLI sync path (no_source, out_of_scope, by_source, Re-run/Removed hints) and
  drops the old KeyError-prone 'deleted' read.
2026-06-18 23:46:11 -03:00
Igor Lins e Silva afa749c141 Merge pull request #952 from Davez69gto/feat/csharp-extensions
feat(miner): add C# and .NET file extensions to READABLE_EXTENSIONS
2026-06-18 14:28:59 -03:00
Igor Lins e Silva f0e76c4363 Merge pull request #169 from adv3nt3/feat/pi-cli-normalizer
feat: add Pi agent JSONL session normalizer
2026-06-18 13:20:25 -03:00
Igor Lins e Silva 5935dacc98 Merge remote-tracking branch 'origin/develop' into HEAD
# Conflicts:
#	mempalace/miner.py
2026-06-18 13:19:48 -03:00
Igor Lins e Silva 349485b6b6 Merge remote-tracking branch 'origin/develop' into HEAD
# Conflicts:
#	mempalace/normalize.py
2026-06-18 13:19:48 -03:00
Igor Lins e Silva 78a32c69c1 Merge pull request #204 from FBISiri/feat/gemini-cli-import
feat: add Gemini CLI / AI Studio session import support
2026-06-18 13:04:14 -03:00
Igor Lins e Silva 22cf8afd1e Merge remote-tracking branch 'origin/develop' into HEAD
# Conflicts:
#	mempalace/normalize.py
2026-06-18 12:42:26 -03:00
Igor Lins e Silva cbf6cbe851 Merge pull request #731 from sjhddh/feat/continue-dev-parser
feat(normalize): add Continue.dev session parser
2026-06-18 12:38:15 -03:00
Igor Lins e Silva 301aa335e1 Merge pull request #1720 from jsiu93/feat/java-project-scanner
fix: detect Java project manifests
2026-06-18 12:38:00 -03:00
Igor Lins e Silva d4391abb59 Merge pull request #1368 from EVSalomon/feat/add-support-swift-kotlin
feat(miner): add support for Swift and Kotlin file extensions
2026-06-18 12:37:40 -03:00
Igor Lins e Silva 617aa0c3f3 feat(miner): add PHP ecosystem file extensions (#1819) 2026-06-18 12:37:26 -03:00
fatkobra f0b5cbf374 fix(mcp): cache writer lock setup failures 2026-06-18 12:47:17 +00:00
fatkobra e594bad9d6 fix(mcp): refuse second writer for same palace 2026-06-18 11:24:18 +00:00
fatkobra ea71c5234e fix reviewer feedback for chroma and tests 2026-06-18 11:03:06 +00:00
fatkobra d704433701 fix(chroma): route stale hnsw divergence to sqlite fallback 2026-06-18 10:37:58 +00:00
fatkobra d8d88f4d21 fix reviewer feedback: To prevent a KeyError and provide a clear, actionable assertion failure message if PreCompact is ever missing 2026-06-18 09:20:41 +00:00
fatkobra ad5e7dfe2c fix(claude-plugin): run final mine on SessionEnd 2026-06-18 09:07:06 +00:00
ManuelReschke c2f42cd57e feat(miner): add PHP ecosystem file extensions 2026-06-18 10:31:18 +02:00
Arnold Wender f56e3067eb fix(mcp): harden delete_by_source per review — strip surrogates + type guard
Address Gemini review on #1729:
- normalize source_file with strip_lone_surrogates so exact matching hits
  rows mined from non-ASCII paths via cp1252 stdin (#1488), mirroring
  tool_add_drawer's ingestion-side normalization
- isinstance(str) guard so a non-string source_file returns a clean error
  instead of AttributeError
- default missing wing/room to "" in the dry-run sample, consistent with
  the rest of the file
- add tests: non-string rejection + surrogate-normalization match
2026-06-17 15:08:29 +02:00
Arnold Wender d486aef083 feat(mcp): add mempalace_delete_by_source bulk-cleanup tool (#1722)
Adds an MCP tool to remove every drawer mined from a given source_file
exact match, for cleaning up benchmark/test data accidentally mined into
a user wing (ShareGPT dumps, results_mempal_*.jsonl, language config
JSON) that drowns out real memories in semantic search.

Matching is pushed to the backend via delete(where={"source_file": ...})
the same idiom the miner and diary-ingest paths already use so it is not
subject to the SQLite variable limit regardless of how many drawers share
the source. Defaults to a dry run reporting match count and a sample;
dry_run=false commits. Absent source is an idempotent no-op, not an error.
2026-06-17 15:08:07 +02:00
Eldar Shlomi 9f434e0bfd test(migrate): cover swap-failure rollback
Adds end-to-end regression coverage for the migration swap path where os.replace hits EXDEV, the shutil.move fallback fails, and the original palace must be restored from the rename-aside copy.
2026-06-15 10:40:24 -03:00
Igor Lins e Silva bc9c052b51 Merge pull request #1810 from MemPalace/develop
Release v3.4.1 — promote develop to main
v3.4.1
2026-06-15 06:11:35 -03:00
Igor Lins e Silva db6e4f0654 Merge pull request #1811 from MemPalace/fix/3.4.1-review-followups
fix(hooks): portable mtime in macOS hook throttles; doc cleanup
2026-06-15 06:00:18 -03:00
Igor Lins e Silva 3f20305eda style(hooks): single-quote the static python -c mtime snippet
The snippet has no shell interpolation — the path arrives via argv, not
string interpolation — so single quotes are correct and make it
unambiguous that nothing is shell-expanded. Behavior is identical:
`sys.argv[1]` contains no `$`, so it was never expanded (verified
empirically). Matches the single-quoted `python -c` blocks already in
hooks/cursor/lib/common.sh. No functional change.
2026-06-15 05:50:08 -03:00
Igor Lins e Silva 868b4c9b39 fix(hooks): portable mtime in macOS hook throttles; doc cleanup
Address review feedback surfaced on the 3.4.1 release promotion (#1810).

Bug fix — `date -r FILE` is GNU-only. On BSD/macOS `date -r` expects
epoch seconds, not a path, so the staleness/throttle checks in the new
Cursor and Antigravity hooks silently failed on macOS: the state GC
swept on every fire and the pending-save guard was skipped. Replace
with a portable `os.path.getmtime` one-liner via the already-resolved
$MEMPAL_PYTHON_BIN (cursor/lib, antigravity/lib, antigravity save hook).
This restores the "bash 3.2.57 / macOS default" compatibility the
Antigravity changelog claims.

Docs:
- Correct the MCP tool count to 33 (was 19/29/31 in 21 places across
  plugin manifests, READMEs, and website docs — all drifted from the
  TOOLS dict / mcp-tools.md reference, which both have 33).
- Fix broken CHANGELOG link to the Cursor skill (skills/, not
  .cursor-plugin/skills/).
- Fix one-too-many `../` in skills/mempalace/SKILL.md's cursor-hooks
  link (resolved above the repo root).
- Add the required `mcpServers` wrapper to the mcp.json example in
  .cursor-plugin/README.md so copy-paste yields a valid Cursor config.

Left intentionally unchanged: the os.dup2 fd-1 redirect in
mcp_server.py is deliberate (#225 keeps JSON-RPC off fd 1).
2026-06-14 19:54:16 -03:00
Igor Lins e Silva 327da58c09 Merge pull request #1809 from MemPalace/chore/release-3.4.1
chore(release): 3.4.1
2026-06-14 17:56:22 -03:00
Igor Lins e Silva b5c79a1eea chore(release): 3.4.1
Bump version across all sources (version.py, pyproject.toml, both
Claude plugin manifests, Codex plugin manifest, README badge, uv.lock)
and promote the Unreleased changelog to 3.4.1.

Shipping: Cursor IDE plugin + hooks, first-class Antigravity IDE
support (with zero-config interpreter resolution), embeddinggemma
bulk re-embed OOM fix, and backup-retention pruning.

Also rebuilds the CHANGELOG compare-link block, which had been left
at v3.2.0: adds the full 3.3.0-3.4.1 chain plus the previously
undocumented 3.4.0, and points Unreleased at v3.4.1...HEAD. Every
version header now resolves to a compare link.
2026-06-14 17:47:09 -03:00
Igor Lins e Silva 68a971b8a1 Merge pull request #1807 from MemPalace/codex/develop-341-windows-closet-boost-fix
test: stabilize closet boost fixture on Windows
2026-06-14 15:24:07 -03:00
Igor Lins e Silva 160a852bdb test: stabilize closet boost fixture on Windows 2026-06-14 15:17:01 -03:00
Igor Lins e Silva 7f5bbd8be8 Merge pull request #1803 from MemPalace/codex/1800-mine-lock-cleanup
fix(palace): clean source mine locks safely
2026-06-14 15:06:21 -03:00
Igor Lins e Silva 0cb45084bc Merge pull request #1624 from trek-e/fix/search-retry-preserve-collection
fix: preserve collection_name on MCP search retry
2026-06-14 15:03:38 -03:00
Igor Lins e Silva a40e0b7a47 fix: address mine lock review feedback 2026-06-14 14:58:20 -03:00
Igor Lins e Silva 58c45a9905 Merge pull request #1804 from MemPalace/codex/close-blob-seq-sqlite
fix: close blob seq sqlite migration connection
2026-06-14 14:50:50 -03:00
Igor Lins e Silva 9f16295f77 Merge pull request #1721 from thismilktea/fix/mcp-add-drawer-idempotency-fail-closed
fix(mcp): fail closed when add_drawer idempotency pre-check fails
2026-06-14 14:45:23 -03:00
Igor Lins e Silva 8e53959bb9 Merge pull request #1802 from MemPalace/codex/develop-341-ci-polish
test: stabilize release validation on develop
2026-06-14 14:44:59 -03:00
Igor Lins e Silva d860a008a1 fix: close blob seq sqlite migration connection 2026-06-14 13:12:00 -03:00
Igor Lins e Silva 5d5397bf52 fix(palace): clean source mine locks safely 2026-06-14 12:50:32 -03:00
Igor Lins e Silva f89bc08876 test: stabilize release validation on develop 2026-06-14 12:25:53 -03:00
Igor Lins e Silva 97ba05cfe3 fix(mcp): avoid Chroma open when cached DB disappears 2026-06-14 12:12:59 -03:00