Commit Graph
80 Commits
Author SHA1 Message Date
Igor Lins e SilvaandClaude Opus 4.7 3b6c7986c9 Merge branch 'develop' into fix/tunnel-file-config-and-endpoint-validation
Resolves conflicts from 60-commit divergence:

- tests/test_closets.py: assertion reformat — kept develop's ruff-format-
  preferred multi-line shape (functionally identical).
- tests/test_palace_graph_tunnels.py: both branches added a new test
  class at end-of-file (this PR's TestTunnelFileFollowsConfig + develop's
  TestEntityTunnels from #1564). Kept both, no overlap.

Local: ruff check / format pass; full pytest suite passes
(2113 passed, 3 skipped).

Co-Authored-By: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 22:31:41 -03:00
mvalentsev 6658a4d11f fix(audit): chunk and batch content before embedding upsert (#1539)
Four sites passed content to `collection.upsert(documents=[...])` or
`collection.add(documents=[...])` without per-drawer size cap, hitting
the same `RuntimeError: Invalid buffer size` crash class from the
embedding model's attention buffer:

- `general_extractor.extract_memories`: post-classification slicer
  that propagates `memory_type` to every slice. New `chunk_size`
  parameter defaulting to `DEFAULT_CHUNK_SIZE` and resolved from
  `MempalaceConfig.chunk_size` by the caller.
- `diary_ingest.ingest_diaries`: per-entry drawers via existing
  `_split_entries`, with character-chunk fallback for any single
  entry larger than `chunk_size`. New `_diary_drawer_id_entry`
  helper carries (entry_idx, entry_chunk_idx). `chunk_index` in
  metadata is a global counter across the file so
  `searcher._expand_with_neighbors` stitches sibling chunks
  regardless of entry boundary. Upsert is batched atomic per file
  (one call carrying every entry/chunk) so a mid-pass embedding
  failure cannot half-write the day. Auto-purge on full rebuild
  deletes prior-pass drawers via `where={"source_file": ...}`,
  which also migrates pre-#1539 legacy `drawer_diary_` IDs as a
  side effect of normal use.
- `mcp_server.tool_diary_write`: split oversized entries into
  bounded per-chunk drawers via a single batched `col.add` (atomic,
  no half-write on embedding failure). `col.add` is intentional:
  `entry_id` is timestamp-based with microsecond precision, so a
  duplicate is a same-microsecond clash that should surface rather
  than silently overwrite.
- `mcp_server.tool_add_drawer`: same crash class on the more common
  add-drawer surface (100 KB sanitize cap, 125x `CHUNK_SIZE`).
  Chunked path mirrors Site 3 with batched atomic `col.upsert`,
  per-chunk `parent_drawer_id` + `chunk_index` metadata, and a
  dual-id idempotency probe (last chunk for atomicity, legacy
  `drawer_id` for pre-#1539 single-row backwards-compat). Return
  shape additive: `chunks` always present, `chunk_ids` on the
  chunked path. `tool_get_drawer` / `tool_delete_drawer` against
  the logical handle report "not found" on the chunked path;
  callers iterate `chunk_ids` or query `parent_drawer_id`.

Chunk id width is `:06d` so even a single-digit `chunk_size`
config cannot lex-sort chunks out of order.
2026-05-22 00:21:25 +05:00
Igor Lins e SilvaandCopilot 1c818d5cdd fix: address stale hook review feedback
- Disable mine and MCP idle timeouts when env values are invalid.
- Use bare-PID slot file mtime as the compatibility timestamp instead of treating old slots as infinitely stale.
- Treat malformed PID-slot timestamps as stale without crashing hook execution.
- Use process-level idle watchdog termination so stale MCP servers actually release file handles.
- Restore tenant-isolation assertions accidentally removed from the KG cache test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-20 15:59:19 -03:00
Igor Lins e SilvaandCopilot 7bddb65cb4 fix: stale-PID timeout, MCP idle exit, structured errors (#1552)
- Add MEMPALACE_MINE_TIMEOUT_HOURS (default 2h): PID files now record
  '{pid} {unix_timestamp}'; _mine_already_running() treats alive-but-old
  processes as stale, unblocking queued mines after a ChromaDB hang.
  Backward-compatible: bare-PID files (old format) treated as stale.
- Add MEMPALACE_MCP_IDLE_HOURS (default 8h): daemon watchdog thread in
  mcp_server calls sys.exit(0) after the configured idle period, preventing
  accumulation of stale server processes holding ChromaDB/HNSW file handles.
  Set to 0 to disable.
- Enrich _internal_tool_error() with optional exc parameter: adds
  data: {error_class, message} to JSON-RPC error body so callers can
  distinguish lock contention, ChromaDB transients, and segfaults without
  scraping the message string. MineAlreadyRunning handler in tool_sync()
  adds error_class: 'LockHeldByOtherProcess' to the result dict.
- Update miner._cleanup_mine_pid_file() to parse first whitespace token
  as PID (handles both old and new PID file formats).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-20 15:31:57 -03:00
Igor Lins e Silva 3e92719710 fix(mcp): surface create_tunnel ValueError instead of masking it (#1473)
tool_create_tunnel sanitized names inside a try/except ValueError but
called create_tunnel() *outside* it, so any ValueError create_tunnel
raises (empty/non-string endpoints today, room-existence checks once
#1469 lands) escaped and the MCP framework wrapped it as the opaque
'Internal tool error'. Wrap the create_tunnel() call in the same
try/except and return {"error": str(e)}, mirroring sibling tools
(tool_add_drawer, tool_list_tunnels). Adds a regression test that
monkeypatches create_tunnel to raise and asserts the message is
surfaced verbatim.

Closes #1473
2026-05-18 17:25:09 -03:00
Igor Lins e Silva 2cab6f5a5b style: reformat tree with ruff 0.15.9
One-time mechanical reformat so `ruff format --check .` passes under the
newly pinned ruff. Layout only (assert-message parenthesization etc.),
no behavior change. 29 files: 28 under tests/ + 1 tools helper, no core
mempalace/ modules. Produced by `ruff format .`.
2026-05-17 23:20:01 -03:00
yonefive71 3171a1b4c1 fix(graph): resolve tunnel file from palace_path config (#1467)
palace_graph._TUNNEL_FILE was a module-level constant initialised from
os.path.expanduser("~") + "/.mempalace/tunnels.json", ignoring the
MempalaceConfig.palace_path config (and MEMPALACE_PALACE_PATH env var)
that drawers, KG, and every other piece of palace state honour. Under
any setup where $HOME and the configured palace diverge — subagent
profiles, sandboxes, multi-tenant hosts, container mounts moving the
palace to /srv/ — drawers landed in the configured palace while
tunnels silently landed in a different file invisible to other
processes touching the same palace.

Replace the constant with _get_tunnel_file(config=None) deriving the
path from a new MempalaceConfig.tunnel_file property (sibling of
palace_path). Default install unchanged because default palace_path
is ~/.mempalace/palace whose sibling tunnels.json is the legacy path.

Add a _legacy_tunnel_file() helper and a one-line WARNING in
_load_tunnels for the case where the configured tunnel file is missing
but the pre-fix hardcoded path has one. No auto-migration — silently
merging tunnel state across two locations risks clobbering newer data.

fix(graph): validate explicit-tunnel endpoints exist (#1468)

create_tunnel previously only validated that wing/room names were
non-empty strings; nothing queried chroma to confirm at least one
drawer carried matching {wing, room} metadata. Pointing an explicit
tunnel at a phantom room silently succeeded. Combined with #1467's
read-bubble, an agent could create_tunnel → list_tunnels and have both
calls return its own bogus write, self-confirming a tunnel that didn't
exist in the shared palace.

create_tunnel now calls _check_room_exists(wing, room, col) for both
endpoints before persisting an explicit tunnel; zero rows raises
ValueError naming the endpoint. Three deliberate carve-outs:

- kind != "explicit" skips validation because topic tunnels use
  synthetic topic:<name> room ids that don't correspond to real rooms
- _get_collection returning None (palace not yet created, transient
  failure, tests without backend) skips validation rather than
  fail-closed — matches tolerance pattern used throughout palace_graph
- Query exceptions are logged and treated as 'can't verify, allow' so
  a flaky index doesn't block legitimate writes

Behaviour change: callers that previously created tunnels pointing at
empty rooms (scaffolding before mining) will now raise. File the
drawer first, then create the tunnel.

Tests:
- _use_tmp_tunnel_file helper now also neutralises _get_collection so
  existing tests don't accidentally trip the new validation path when
  test-order pollution leaves a real chroma backend bound
- test_closets.py::TestTunnels setup/teardown updated to monkeypatch
  resolver functions instead of the removed constant; also neutralises
  _get_collection for the same reason
- Three tests in test_miner.py exercising compute_topic_tunnels are
  unchanged in intent — they monkeypatch the new resolvers and pass
  without stubbing _get_collection because kind=topic skips validation
- New TestTunnelFileFollowsConfig and TestCreateTunnelEndpointValidation
  classes cover the regression surface for both fixes
2026-05-17 20:19:56 +00:00
Igor Lins e Silva cabc3afb6e Merge pull request #1445 from yonefive71/fix/1426-none-metadata-mcp-handler
fix(mcp): tolerate None metadata cells (#1426)
2026-05-17 16:24:17 -03:00
Igor Lins e Silva 3c74ba6493 style(tests): apply ruff 0.4.x format to test_mcp_server (#1445)
CI pins ruff>=0.4.0,<0.5 (resolves 0.4.10); the PR's two new assert
blocks were laid out in ruff-0.5+ style, so `ruff format --check .`
failed. Reformat with ruff 0.4.10 (the exact CI version) — assertion
layout only, no semantic change. `ruff check .` + `ruff format --check .`
both pass under 0.4.10.
2026-05-17 16:09:45 -03:00
Igor Lins e Silva 8199d7287f Merge pull request #1529 from mvalentsev/fix/1504-tunnel-preserve-hyphenated-wing
fix(tunnels): preserve hyphenated wing slugs in create_tunnel write-path (#1504)
2026-05-17 12:22:19 -03:00
Igor Lins e Silva d114a594d6 Merge pull request #1383 from mvalentsev/fix/mcp-kg-cache-realpath
fix(mcp): canonicalize KG cache key via realpath + normcase (#1372)
2026-05-17 12:21:10 -03:00
mvalentsevandStarshipSuperjam cf96e68edf fix(mcp): cold-start embedder diagnostics + opt-in warmup (#1495)
MCP clients (Claude Code, Claude Desktop) surface the cold-load of the
ONNX/CoreML embedder + HNSW segment as -32000 "Internal tool error"
without a recoverable trace, because the first chromadb-touching tool
call after a fresh mempalace-mcp boot can exceed the client timeout.
Two opt-in env vars, both default off, both fail-soft:

- MEMPALACE_LOG_FILE: append-mode FileHandler so the server-side
  traceback that the MCP client never surfaces stays diagnosable. Invalid
  path falls back to stderr-only with a warning naming the env var, so
  it cannot itself become a server-start failure. basicConfig uses
  force=True so the FileHandler is not silently dropped when something
  already configured root logging. (OSError, ValueError) catch covers
  Windows NUL-byte paths in addition to POSIX missing-dir / EPERM.

- MEMPALACE_EAGER_WARMUP: pre-loads embedder + HNSW segment at the end
  of main() startup via _get_collection(create=False).query(...) — a
  single call drives ONNX _download_model_if_not_exists +
  InferenceSession AND HNSW data_level0.bin segment load (HNSW chunks
  are read lazily on the first chromadb collection op). Fresh-install
  pre-check on os.path.isfile(palace/chroma.sqlite3) skips the warmup
  before any chromadb client call so the chromadb constructor cannot
  materialize the palace scaffold before `mempalace init` (preserves
  CLAUDE.md "Incremental only" invariant). Truthy 1/true/yes/on; falsy
  0/false/no/off and empty are silent; unrecognized values warn so
  typos like "tru" do not silently disable the feature. All failure
  paths (collection-open-raise, returned-None, query-raise) log with
  palace path + device + error class so ops can disambiguate.

Empirical evidence (Linux, CPU-only ONNX, ~1k drawers): first
diary_write t_handoff = 7.60s baseline vs 0.17s with warmup
(-7.4s shifted off the first-call window into startup). CoreML cold-load
on macOS is reported at 5-30s in the issue's empirical record
(paideia#114); the CPU baseline above is the floor.

Tests (24 cases) cover both env vars across set/unset/empty/invalid/
garbage paths, warmup truthy/falsy/garbage parsing, fresh-install (real
integration, no monkeypatch — pins zero on-disk side effect),
collection-open-raise / returned-None / query-raise fail modes, and
two regression pins: warning-before-banner ordering (guards delay=True
regression) + truthy path actually invokes col.query (guards no-op
warmup regression). Probe text is a sentinel so it cannot semantically
match real drawer content.

Co-Authored-By: StarshipSuperjam <33380501+StarshipSuperjam@users.noreply.github.com>
2026-05-17 00:57:13 +05:00
mvalentsev fba6372f2c fix(tunnels): preserve hyphenated wing slugs in create_tunnel write-path (#1504)
create_tunnel no longer normalizes hyphen->underscore on write. Wings
created via "mempalace mine --wing my-wing" are stored verbatim in
tunnels.json. list_tunnels and follow_tunnels filters normalize both
the stored value and the query at compare time so previously-stored
underscore tunnels and post-fix verbatim tunnels both resolve via
either query form.

compute_topic_tunnels canonicalizes its wing keys via
normalize_wing_name so repeated mining runs with mixed slug forms
cannot accumulate parallel duplicate tunnels. topic_tunnels_for_wing
gains a symmetric lookup so a hyphenated argument finds an
underscore-keyed registry entry.

_normalize_wing now returns None for non-string inputs instead of
raising; read-path filters use the (t.get(k) or {}) idiom so
hand-edited records with "source": null no longer crash iteration.
2026-05-16 20:48:23 +05:00
mvalentsev 4dc3ccf26a fix(mcp): thread captured KG path through _get_kg to lock insert/evict keys
_get_kg now accepts an optional canonical_path argument and uses it
verbatim when given. _call_kg passes its top-of-function captured key
through, so the dict insertion that _get_kg performs and the eviction
lookup _call_kg performs always agree on the key -- even if env state
(MEMPALACE_PALACE_PATH rotation) or FS state (symlink remap) mutates
between attempts. Without the pass-through, _get_kg's internal call
to _canonicalize_kg_path(_resolve_kg_path()) could land on a different
key than the captured path, leaving a closed handle in the cache under
a stale key the eviction lookup no longer matches.

Tests:
  test_call_kg_passes_captured_path_through_resolve_drift -- patches
  _resolve_kg_path with a shifting iterator + identity canonicalize,
  pre-seeds the closed handle under the captured key; asserts both
  _get_kg calls receive the captured key and the closed handle is
  evicted by retry.

  test_call_kg_oserror_at_top_propagates_unmasked -- patches
  _canonicalize_kg_path to raise OSError; asserts it propagates without
  conversion (the rationale for capturing at the top before the retry
  loop -- prevents an OSError mid-handler from masking a
  sqlite3.ProgrammingError it can't retry through).

_patch_mcp_server test helper updated to a varargs lambda so existing
tests that patch _get_kg as a no-arg fixture stay compatible with the
new optional-arg signature.
2026-05-15 22:51:14 +05:00
mvalentsev 3ee158ecdd test(mcp): tighten _canonicalize_kg_path normcase wiring assertion
Patch both realpath and normcase with sentinel wrappers and assert the
result composes them as normcase(realpath(p)). Swapping the order would
leave Windows symlinks under the original case, defeating the dedup.
The previous assertion (result == seen[-1].lower()) was tautological --
the fake normcase already returned p.lower(), so any implementation
that lowercased anything would pass.
2026-05-15 22:51:14 +05:00
mvalentsev 6f9fe8055a fix(mcp): canonicalize KG cache key via realpath + normcase (#1372)
Switch _kg_by_path lookups in _get_kg and _call_kg from os.path.abspath
to a new _canonicalize_kg_path helper applying realpath (collapses
symlink aliases) + normcase (Windows drive-letter casing). Two tenants
pointing at /srv/A and /srv/link-to-A no longer open duplicate
sqlite3.Connection handles over the same file.
2026-05-15 22:51:14 +05:00
Anastasiia AnfimovaandClaude Opus 4.7 51c03a0c5c fix(mcp): structured -32602 for unknown parameter name (#1512)
Unknown kwargs (wrong parameter name, e.g. text= instead of content=)
were silently dropped by the schema-filter, surfacing only indirectly
as a later "Missing required 'X'". Emit -32602 naming the offending
kwarg instead — symmetric with #1500's missing-required path. Gated by
the existing accepts_var_keyword check; wait_for_previous excluded.

Closes #1512.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 13:48:29 +02:00
Igor Lins e Silva 9893100ca4 Merge pull request #1439 from mvalentsev/fix/init-strip-leaked-pythonpath
fix: drop leaked PYTHONPATH on package init (#1423)
2026-05-15 08:05:47 -03:00
mvalentsevandanastasiiaanfimova 982adcf09b fix(mcp): structured error for param-shape mismatch (#1351)
Split the bare except in handle_request tools/call: TypeError matching
Python's signature-mismatch shape (gated on handler __qualname__) returns
-32602 with parsed parameter name(s); handler-internal TypeError and
non-TypeError exceptions stay on the existing generic -32000 path so
internals never leak. Adds TestParamShapeDiagnostics with 6 cases.

Closes #1351

Co-authored-by: anastasiiaanfimova <271004602+anastasiiaanfimova@users.noreply.github.com>
2026-05-14 01:03:31 +05:00
mvalentsev 068930b710 test(init): probe sys.path filter in CLI/MCP env strip tests; docstring side effect
Three-way assertions in test_cli and test_mcp_server now also assert
SENTINEL_IN_PATH is False at import, so a regression in the package
sys.path filter is caught by the same test that catches the env pop
regression. SystemExit catch is narrowed to exit codes (0, None) so a
future argparse change surfaces as a failure instead of being swallowed.

cli.py:main and mcp_server.py:main docstrings document the PYTHONPATH
pop side effect for callers that invoke main() programmatically.

Refs #1423.
2026-05-13 22:53:36 +05:00
mvalentsev 4b1ab50750 fix(init): keep PYTHONPATH env for embedded callers, strip only in CLI entry points
Split the leaked-PYTHONPATH workaround: __init__.py now only filters
sys.path (fixes the ABI crash from #1423 at import time), while
os.environ.pop("PYTHONPATH") moves to cli:main() and mcp_server:main()
where the strip affects only subprocesses we spawn ourselves.

Host applications that import mempalace as a library and rely on
PYTHONPATH for their own child subprocesses are no longer affected.

Refs #1423.
2026-05-13 22:53:36 +05:00
yonefive71 afaedc68cf fix(mcp): tolerate None metadata cells (#1426)
ChromaDB's col.get() and col.query() can return None for the metadata
cell of a partially-flushed row, or for any row written without metadata
in older formats. The MCP handler then crashes with:

    AttributeError: 'NoneType' object has no attribute 'get'

This crashed the path before the embeddings_queue cleanup step, so the
queue grew without bound while writes kept appearing successful.

Add a small _safe_meta() boundary helper that coerces None (or any other
non-dict) to {}, and route every direct metadatas[i]/metadatas[0] read in
mcp_server through it: tool_get_drawer, tool_list_drawers,
tool_update_drawer, tool_delete_drawer (audit-log path),
tool_check_duplicate, and tool_diary_read.

The contract — *metadata is always a dict by the time it leaves the
boundary* — is documented on the helper and self-documents at each call
site.

Regression coverage: tests/test_mcp_server.py::TestNoneMetadataSafety
covers the helper and each affected handler with a stub collection that
returns None metadatas (Chroma rejects None at write time, so we can't
reproduce upstream state through the real backend).

Closes #1426.
2026-05-11 04:26:39 +00:00
Igor Lins e Silva 40524d5b04 Merge develop into main for v3.3.5 release
# Conflicts:
#	.claude-plugin/marketplace.json
#	.claude-plugin/plugin.json
#	.codex-plugin/plugin.json
#	CHANGELOG.md
#	README.md
#	mempalace/version.py
#	pyproject.toml
#	uv.lock
2026-05-10 01:05:36 -03:00
Igor Lins e Silva b2bab382f3 Merge remote-tracking branch 'origin/develop' into pr-1417-fix-kg-datetimes
# Conflicts:
#	tests/test_palace_locks.py
2026-05-09 21:12:40 -03:00
Igor Lins e Silva d9e60d84ad Merge pull request #1396 from JPdeB61/fix/1315-mcp-search-retry-on-hnsw-flush
fix(mcp): retry tool_search once on Chroma "Error finding   id" transient (#1315)
2026-05-09 19:32:20 -03:00
Igor Lins e Silva b2ce45d301 style: ruff format tests/test_mcp_server.py for ruff <0.5 2026-05-09 18:09:51 -03:00
fatkobra 4adc99f755 fix(kg): canonicalize temporal datetime support 2026-05-08 08:32:25 +00:00
fatkobra eebf48e975 fix(kg): accept ISO datetimes for temporal inputs 2026-05-08 06:29:36 +00:00
Igor Lins e Silva e9aee19433 fix(tests): apply ruff format after rebase resolution
The collection_name plumbing rebase produced a few unformatted blocks
in test_mcp_server.py and test_searcher.py; bringing them in line with
the 0.4.x CI pin so test-windows / lint stay green.
2026-05-07 09:10:22 -03:00
Mika Cohen ec6d2dde01 fix: use configured collection in recovery paths 2026-05-07 09:10:00 -03:00
JP de Boer knt be95ea7f33 fix(mcp): retry tool_search once on Chroma "Error finding id" transient (#1315)
After a bulk CLI mine, ChromaDB's HNSW segment metadata can be unflushed
  for ~30-60s. Wing-scoped MCP search hits "Internal error: Error finding id"
  during that window, and the existing inode/mtime cache invalidation isn't
  enough — tool_search routes via search_memories -> palace.get_collection
  -> _DEFAULT_BACKEND._client, which has its own per-palace cache.

  This wraps tool_search with a single retry that drops both the MCP-local
  cache and _DEFAULT_BACKEND._clients/_freshness for the palace, sleeps 2s,
  retries once, and tags successful retries with index_recovered=True.

  Does not address tool_check_duplicate or other index-touching tools, nor
  the underlying flush window — options 1-3 from #1315 (auto-flush after
  mine, fail-fast detection, SQLite-only fallback) are still on the table
  for a complete fix.

  Refs #1315
2026-05-07 02:11:07 +02:00
Igor Lins e SilvaandJeffrey Hein e334e257bf fix(mcp): retry _get_collection once on transient failure (#1286)
A transient chromadb exception inside `_get_collection` was swallowed by
the bare `except Exception: return None`, leaving every subsequent tool
call hitting the same poisoned cache silently. The fix wraps the body
in a `for attempt in range(2)` loop: on attempt 0 failure, log via
`logger.exception(...)` and clear `_client_cache` / `_collection_cache`
/ `_metadata_cache` so the next iteration forces `_get_client()` to
rebuild from scratch — that path now re-runs `quarantine_stale_hnsw`
(per #1322), so the second attempt heals the common stale-handle case
automatically. If both attempts fail, return `None` (matches the prior
contract for permanent failures).

Two new tests in `tests/test_mcp_server.py::TestCacheInvalidation`:
- `test_get_collection_retries_once_on_exception` — first attempt raises
  via a monkeypatched `_get_client`, second attempt succeeds; assert the
  caller gets the collection back, not None.
- `test_get_collection_returns_none_after_two_failures` — both attempts
  fail, assert we exhaust the loop and return None (no infinite retry).

Surgical extraction from PR #1286, which carried the same fix idea
(plus a fork-sync bundle that couldn't be merged); credit to the
original author below.

Co-authored-by: Jeffrey Hein <jp@jphein.com>
2026-05-06 04:52:18 -03:00
Igor Lins e Silva 869ab38095 style: ruff format mcp_server.py + test_mcp_server.py (CI lint) 2026-05-06 02:19:57 -03:00
Oleksii Pylypchuk a85d432b54 feat: add validation for missing name parameter in tools/call requests 2026-05-06 02:19:57 -03:00
Oleksii Pylypchuk 55d79dc8cd fix: include null id in JSON-RPC invalid request error responses and add validation tests 2026-05-06 02:19:57 -03:00
Igor Lins e Silva 67cda9d455 Merge pull request #1030 from eldar702/fix/none-metadata-residual-guards
fix: guard None metadata/doc in tool_check_duplicate and Layer1/Layer2
2026-05-06 01:51:24 -03:00
Igor Lins e Silva 53675dd194 Merge pull request #1160 from mvalentsev/fix/mcp-kg-lazy-per-path-cache
fix(mcp): lazy per-path KnowledgeGraph cache (#1136)
2026-05-06 01:33:47 -03:00
Igor Lins e Silva 7ede231da9 Merge pull request #1167 from arnoldwender/fix/kg-date-validation
fix(kg): validate ISO-8601 date formats at MCP boundary
2026-05-06 01:33:27 -03:00
mvalentsev b8816e0fe2 fix(mcp): retry KG handlers once on concurrent close race
Race scenario: a KG tool handler calls _get_kg() and gets a live
KnowledgeGraph; another thread fires tool_reconnect() between that
return and the handler's kg.add_triple()/kg.query_entity()/etc call.
tool_reconnect drains _kg_by_path and closes the underlying
sqlite3.Connection; the handler then raises sqlite3.ProgrammingError:
'Cannot operate on a closed database', which surfaces as a -32000
to the MCP client even though the user just asked for a reconnect.

New _call_kg(op) helper wraps each handler's kg call in a one-shot
retry: catch exactly sqlite3.ProgrammingError, evict the stale entry
(only if the cache slot still points at the closed instance — another
thread may have already replaced it), and rerun op against a fresh
_get_kg(). Beyond one retry give up so a sustained close-stream
surfaces clearly instead of looping.

All five KG handlers (tool_kg_query, tool_kg_add, tool_kg_invalidate,
tool_kg_timeline, tool_kg_stats) now route through _call_kg.

Tests pin the contract:
  * retries with a fresh KG and returns the second result
  * non-ProgrammingError exceptions propagate without retry
  * gives up after exactly one retry on sustained close
2026-05-03 21:43:51 +05:00
mvalentsev 0a62658051 fix(mcp): drain KG cache on tool_reconnect
tool_reconnect cleared ChromaDB caches but left _kg_by_path entries
intact. After an external replacement of knowledge_graph.sqlite3 the
server kept serving the old open sqlite3.Connection, returning stale
results.

Now iterate _kg_by_path under _kg_cache_lock, call close() best-effort,
and clear the dict so the next tool call reopens the KG from disk.
Two new tests in TestKGLazyCache verify cache invalidation and that a
failing close() does not block the clear.
2026-05-03 17:43:00 +05:00
mvalentsev 84f9726a39 test(mcp): fix Windows subprocess env in KG lazy-init test
Passing a stripped env dict without SYSTEMROOT/WINDIR breaks Python
bootstrap on Windows (_Py_HashRandomization_Init). Inherit the parent
env and strip MEMPAL* vars instead, then override HOME/USERPROFILE to
the tmp dir.
2026-05-03 17:43:00 +05:00
mvalentsev c69a622a18 test(mcp): add multi-tenant and lazy-init tests for KG (#1136)
TestKGLazyCache covers the scenarios behind the lazy per-path refactor:

- test_lazy_init_no_import_side_effect: a fresh subprocess import does
  not create ~/.mempalace/knowledge_graph.sqlite3 (what closed PR #167
  was aiming at).
- test_get_kg_returns_same_instance: two _get_kg() calls under the same
  resolved path return the same object, cache has one entry.
- test_get_kg_different_paths_different_instances: rotating env var
  produces distinct KGs.
- test_multi_tenant_env_switch: the exact scenario from #1136 — write
  under path A, query under path B returns empty, switching back to A
  sees the fact.
- test_cache_thread_safe: 16 threads racing _get_kg() end up with one
  shared instance and one cache entry.
2026-05-03 17:43:00 +05:00
mvalentsev 9e730098e9 test(mcp): migrate _kg monkeypatches to _get_kg (#1136)
Direct module-attribute patching of _kg is obsolete after the lazy
cache refactor. Switch test helpers to patch _get_kg instead so the
fixture KG replaces the factory rather than a now-missing singleton.

- tests/test_mcp_server.py: _patch_mcp_server helper
- tests/benchmarks/test_mcp_bench.py: _patch_mcp_config helper
- tests/benchmarks/test_memory_profile.py: inline patch in test_tool_status_repeated_calls
2026-05-03 17:43:00 +05:00
Igor Lins e Silva 5380189f82 Merge pull request #1320 from MemPalace/fix/1314-kg-temporal-params
fix(mcp): forward valid_to and source params in kg_add/kg_invalidate (#1314)
2026-05-03 03:51:29 -03:00
Igor Lins e Silva 3e6f6480c0 Merge pull request #1325 from MemPalace/security/mcp-omit-absolute-paths
fix(mcp): omit absolute filesystem paths from MCP tool responses
2026-05-03 03:20:11 -03:00
Igor Lins e Silva 7fc260f752 fix(mcp): basename source_file in tool_get_drawer responses
The MCP `mempalace_get_drawer` tool returned the entire raw drawer
metadata blob to any connected client, and the `source_file` field
in that blob is the absolute filesystem path written by the miners
(`miner.py`, `convo_miner.py` — `source_file = str(filepath)`). On
a single-user local deployment this is self-disclosure, but in
nested-agent or multi-server MCP topologies the client is a separate
trust domain and the host's directory layout has no documented
client-side use.

Mirror the mitigation that `searcher.search_memories()` already applies
on its own return path: reduce `source_file` to its basename via
`Path(source_file).name` before handing the metadata to the client.
Citations still work — the directory layout does not leak.

Companion to #1 (omit palace_path from tool_status). Same threat class,
different surface:

- mempalace_status — palace dir path     → fixed in #1
- mempalace_get_drawer — per-drawer source_file path → this PR

Other read tools were audited and do not leak host paths:
- mempalace_search    — already basenames source_file
- mempalace_list_drawers — returns wing/room/preview only
- mempalace_diary_read   — date/timestamp/topic/content only
- mempalace_reconnect    — success/message/drawers only
- mempalace_kg_*         — entity/predicate strings, counts
- mempalace_check_duplicate — wing/room/preview only

Changes:
- mempalace/mcp_server.py: tool_get_drawer() now basenames metadata.source_file
- tests/test_mcp_server.py: regression test asserting the absolute path
  and its parent directory do not appear anywhere in the response
- website/reference/mcp-tools.md: clarify the documented return shape
2026-05-03 05:58:46 -03:00
igorls 2397481158 style: ruff format tests/test_mcp_server.py (PR #1323) 2026-05-02 23:00:10 -03:00
igorls 6ffbf6ffc3 style: ruff format test_mcp_server.py (PR #1320) 2026-05-02 22:59:50 -03:00
Igor Lins e SilvaandClaude Opus 4.7 e9222b4c7b fix(mcp): case-insensitive agent name in diary_write/diary_read (#1243)
`tool_diary_write` stored the `agent` metadata verbatim after `sanitize_name`
(which preserves case), while `tool_diary_read` filtered by exact match —
so writing as "Claude" and reading as "claude" silently returned zero rows.

Both endpoints now lowercase `agent_name` immediately after sanitization.
The default per-agent wing slug is also stable across casings since it's
derived from the same normalized form.

Behavior change: entries written prior to this fix under mixed-case agent
names will not match the new lowercase filter; documented under v3.3.5
in CHANGELOG with a `mempalace repair` pointer.

Adds a regression test (`test_diary_read_case_insensitive_agent`) and
updates the existing `test_diary_write_and_read` to assert the new
lowercase agent identity.

Closes #1243

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 22:57:09 -03:00
Igor Lins e SilvaandClaude Opus 4.7 e4e25ed186 fix(mcp): forward valid_to and source params in kg_add/kg_invalidate (#1314)
`tool_kg_add` previously accepted only `valid_from` and `source_closet`,
silently dropping `valid_to`, `source_file`, and `source_drawer_id` at
the MCP boundary. Backfilling already-ended historical facts therefore
collapsed to "still current," and adapter provenance never reached
the SQLite layer even though `KnowledgeGraph.add_triple` already
supported every column.

`tool_kg_invalidate` returned the literal string `"today"` whenever the
caller omitted `ended`, hiding the actual stamped date from anyone trying
to verify what got persisted.

Changes:
- Extend `tool_kg_add` signature + MCP input_schema with `valid_to`,
  `source_file`, `source_drawer_id`; forward all of them to
  `_kg.add_triple` and to the WAL log.
- Resolve `ended` to `date.today().isoformat()` in `tool_kg_invalidate`
  before logging / returning, so the response always reports the actual
  date stored in `valid_to`.
- Add regression tests for valid_to round-trip, source_file /
  source_drawer_id provenance, and the resolved-ended-date contract.
- Leave TODO(#1283) markers so the open ISO-8601 validation PR can drop
  `validate_iso_date` over `valid_from` / `valid_to` / `ended` cleanly.

The underlying `KnowledgeGraph.add_triple` already accepted these
kwargs (RFC 002 §5.5) — only the MCP edge needed wiring up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 22:54:32 -03:00