Commit Graph
1588 Commits
Author SHA1 Message Date
dependabot[bot] 1bc5c508d5 chore(deps): bump actions/checkout from 6 to 7 (#1882)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-28 01:32:32 -03:00
5cbc37ee9b fix(cli): force UTF-8 when reading/writing .gitignore in init (#1648)
On Windows, Path.read_text() and open(path, 'a') use locale encoding
(GBK on Chinese-locale systems) before PEP 686 / Python 3.15. A valid
UTF-8 .gitignore with non-ASCII comments crashes
_ensure_mempalace_files_gitignored() with UnicodeDecodeError, which
aborts 'mempalace init' on Windows for any user whose .gitignore
contains non-ASCII text.

Force encoding='utf-8' on both read and append, with errors='replace'
on read as a defensive fallback for legacy mixed-encoding files.

Co-authored-by: ALaDingAhmad <16530935@qq.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
2026-06-28 01:31:19 -03:00
bacb935bb3 fix(layers): order L1 wake-up by recency so it surfaces the latest moments (#1630)
L1's generate() scored drawers by importance/emotional_weight/weight, and
the docstring promised "prefer high importance, recent filing". But no
ingest path (miner, convo_miner, diary, add_drawer) writes any of those
fields, so the sort collapsed to insertion order (oldest first) and
recency was never consulted. A scoped `wake-up --wing X` therefore
surfaced the *oldest* moments: the opposite of useful.

Add filed_at (present on every drawer, ISO-8601, lexically chronological)
as the secondary sort key. Importance stays primary for the day a scoring
pass populates it; filed_at is the effective signal today, making the
"recent filing" half of the promise true with data already present.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
2026-06-28 01:19:32 -03:00
Mikhail Valentsev c17d1aaf13 fix(mcp): stop clobbering host app root logger at import (#1860) (#1885)
* fix(mcp): stop clobbering host app root logger at import (#1860)

_init_logging() ran at import and called logging.basicConfig(force=True),
resetting the root logger's level, format, and handlers unconditionally. An
app that configured logging before importing mempalace.mcp_server lost its
setup: a host on DEBUG dropped to INFO, custom formatters and handlers were
replaced.

force=True existed (#1495) only to keep MEMPALACE_LOG_FILE working when root
already had handlers. This keeps that contract without the reset: configure
root only when it is unconfigured (standalone); otherwise attach a
mempalace-filtered file handler additively and leave the host's config alone.

Adds _MempalaceLogFilter so the file captures every mempalace logger (the
dotted mempalace.* family plus the flat mempalace_* names) and nothing else.

* fix(mcp): survive importlib.reload and pin file log format (#1860)

Addresses review on #1885.

Restore _logging_configured from globals() so the idempotency guard survives
importlib.reload: a reload re-executes the module body, and a plain reset would
let _init_logging() stack a duplicate file handler on root.

Set an explicit "%(message)s" formatter on the file handler so the embedded
path does not depend on logging's default formatter (which already renders the
same, but is now pinned and identical to the standalone path).

Adds a reload regression test and a format-pin assertion.
2026-06-28 01:02:59 -03:00
Igor Lins e Silva 292cf872a5 Merge pull request #1878 from MemPalace/fix/1596-fts5-autoheal
fix(repair): auto-heal isolated FTS5 inverted-index corruption (#1596)
2026-06-28 01:00:53 -03:00
Igor Lins e Silva b0a94bdef2 fix(repair): auto-heal isolated FTS5 inverted-index corruption (#1596)
Concurrent killed-mid-write mines can leave embedding_fulltext_search in a
malformed-inverted-index state that fails PRAGMA quick_check while the
underlying rows stay intact (integrity_check ok). The repair preflight then
hard-aborts before reaching the FTS5 rebuild step, so `mempalace repair`
refuses to run and full-text search stays broken — the exact loop #1596
reports. The MineValidationError banner even promises "repair --yes rebuilds
the FTS5 virtual table automatically," which the preflight abort made false.

Add maybe_autoheal_fts5_index(): when every quick_check error is an isolated
"malformed inverted index for FTS5 table" failure, rebuild the index in place
from the intact embedding_fulltext_search_content table
(INSERT ... VALUES('rebuild')) under mine_palace_lock, then re-run quick_check.
The rebuild touches no drawer rows. Wired into both repair preflights
(rebuild_index and cli cmd_repair). Any non-FTS5 error in the set, a lock held
by a live mine, or a rebuild that does not clear quick_check leaves the errors
unchanged so the caller still aborts with the recovery banner — broader
corruption is never silently rebuilt over.
2026-06-26 03:13:44 -03:00
Igor Lins e Silva 73e74bfb80 Merge pull request #1872 from MemPalace/fix/1716-all-layer-0-quarantine
fix(chroma): stop quarantining valid all-layer-0 HNSW segments (#1716)
2026-06-26 03:06:00 -03:00
Igor Lins e Silva dc433824fd fix(chroma): stop quarantining valid all-layer-0 HNSW segments (#1716)
An empty link_lists.bin is not corruption on its own: hnswlib stores the
layer-0 graph inside data_level0.bin and only writes link_lists.bin for
elements promoted to level > 0. A small/low-fanout index where every
element stays on layer 0 serializes an empty link_lists.bin and loads
fine. Flagging that shape as corrupt produced a self-perpetuating
quarantine loop — repair rebuilt the byte-identical all-layer-0 segment,
the next cold start re-quarantined it, accumulating drift dirs (221 MB in
the reported case) with no ingestion involved.

Use the persist-completion marker as the discriminator instead. ChromaDB
writes index_metadata.pickle last, so an intact pickle envelope proves the
flush finished and the empty link_lists.bin is the legitimate all-layer-0
shape. Only treat an empty link_lists.bin as a partial flush when there is
real payload AND no completion marker (absent or truncated pickle). The
#1457 partial-flush protection (real payload, no/truncated marker) is
preserved; the byte-sniff is factored into _hnsw_metadata_marker_intact
and reused by _segment_appears_healthy.

Also fixes the related single-writer stale-quarantine false positive
(#1564), which shares this all-layer-0 root cause.
2026-06-26 00:27:36 -03:00
Joe Sebastian 0f3f8ec652 fix: expand tilde in palace_path when read from config file\n\nMempalaceConfig.palace_path correctly called os.path.expanduser() for\nenv-var paths but not for paths read from config.json. If config.json\nstores palace_path as '~/.mempalace/palace' (the default written by\ninit), the tilde was returned unexpanded.\n\nDownstream callers such as cli.py cmd_mine did call expanduser when\n--palace was passed explicitly, but fell through to MempalaceConfig()\nwhen no flag was given, inheriting the unexpanded string. Python's\nos.makedirs and chromadb.PersistentClient treat a leading tilde as a\nliteral directory name rather than the home directory, so the palace\nwas silently written to a CWD-relative path such as\nmy_project/~/.mempalace/palace.\n\nThe fix is a single os.path.expanduser() call on line 343 of\nconfig.py, mirroring the existing env-var branch on line 342. Since\nDEFAULT_PALACE_PATH is already expanded at module load (line 197),\nexpanduser on an absolute path is a no-op, so the default case is\nunaffected.\n\nSymptoms: scattered {project}/~/.mempalace/palace directories, palace\nalways appears empty after mine, search returns Collection does not\nexist, launchd-driven nightly mine writes to a different location than\ninteractive mine.\n\nCo-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>n (#1865) 2026-06-26 00:09:51 -03:00
David FinkelsteinandDavid Finkelstein 5294d3118e fix(hooks): hide conhost window on Windows in _mine_sync non-daemon path (#1863)
The non-daemon synchronous mine fallback in _mine_sync() spawned the
mine subprocess without CREATE_NO_WINDOW, flashing a visible console
window on every PreCompact fire on Windows. The async paths
(_spawn_mine, _desktop_toast) already pass it via _detached_popen_kwargs();
this sync path was missed. getattr(..., 0) is a no-op off-Windows.

Fixes #1862

Co-authored-by: David Finkelstein <david@finkelstein.us>
2026-06-25 23:59:08 -03:00
Ivan AntsimonauandIvan Antsimonau aac947a5c5 Fix/wing slug special chars (#1852)
* fix: sanitize wing slug for project dirs with special characters

Project folders containing characters outside sanitize_name's set
(e.g. a leading '+') leaked into the derived wing name, producing names
like 'wing_+project' that config.sanitize_name rejects, silently
breaking diary auto-save for that project.

Add _safe_wing_slug(): collapse non-word runs to '_', trim, and fall
back to 'sessions' when a name reduces to nothing. Route the three
wing-derivation sites through it.

Tests: unit cases for the helper plus a hypothesis property test
asserting wing_<slug> always passes sanitize_name for any input.

* fix: preserve dots and apostrophes in wing slug for backward compatibility

The first pass collapsed every non-word character (including dot and
apostrophe) to underscore, renaming existing valid wings — e.g. my.app
became wing_my_app — which would orphan diary entries already filed under
the old name. Keep dot and apostrophe (both accepted by sanitize_name),
collapse consecutive dots to avoid the path-traversal rejection, and trim
edge separators.

Add backward-compatibility tests for previously-valid names plus a
double-dot collapse test.

* fix: cap wing slug length to stay within sanitize_name's limit

sanitize_name rejects names over 128 characters, so a very long project
directory name would produce a wing name that fails validation,
re-triggering the silent auto-save break this PR fixes. Truncate the slug
to 120 chars (the wing_ prefix keeps the total under 128).

Widen the hypothesis property test to max_size=300 so it exercises the
length path, and add an explicit truncation test.

Addresses gemini-code-assist review feedback on PR #1852.

---------

Co-authored-by: Ivan Antsimonau <ivan.antsimonau@katim.com>
2026-06-25 23:59:05 -03:00
skblue 86546348e4 fix(cli): add repair rebuild-index alias (#1670) 2026-06-25 23:59:02 -03:00
Eldar Shlomi d20ac0adf1 fix: spawn daemon with CREATE_NO_WINDOW to match hook miner (#1783) (#1857)
daemon.py:_detached_kwargs was the last production spawn site still using DETACHED_PROCESS. Swap it to CREATE_NO_WINDOW, matching the hook miner's _detached_popen_kwargs fixed in #1848 — the dedicated follow-up the review bot asked for. `grep -rn DETACHED_PROCESS mempalace/` now returns zero production hits.

Survivability is unchanged: CREATE_BREAKAWAY_FROM_JOB (escapes the parent Job Object's kill-on-close) plus the daemon never being attached to the launching console carry survive-terminal-close; CREATE_NEW_PROCESS_GROUP (also kept) isolates Ctrl-C/Break. CREATE_NO_WINDOW is ignored when OR'd with DETACHED_PROCESS, so this replaces the flag rather than adding it. The daemon already redirects stdout/stderr to daemon.log and reads no stdin, so it needs no console.

Adds the first tests for _detached_kwargs (posix + windows, cross-platform monkeypatch of the Windows-only flag constants, mirroring the #1848 hooks_cli tests).
2026-06-25 23:51:22 -03:00
Igor Lins e Silva d479d6f245 Merge pull request #1869 from arnoldwender/test/wal-error-paths
test(wal): cover WAL crash-safety, idempotent setup, and redaction edge paths
2026-06-25 23:46:50 -03:00
mvalentsev c4ce4c89d2 fix(mine): route SKIP to stderr and cover stat() OSError arm (#923)
The original commit printed SKIP for oversized files to stdout but the
sibling SKIP for symlinks in the same scan_project / scan_convos already
went to stderr. Align the new line with that convention.

Also adds a SKIP-with-error log for the except OSError arm right below the
size check. Files whose stat() raises (permission denied, racing delete,
broken symlink that survived the earlier is_symlink check) were the same
bug class as the silent oversize drop.

Tests switched from captured.out to .err and tightened to the full
template; new test covers the OSError arm via a selective Path.stat
monkeypatch with a follow_symlinks gate for Python 3.10+.
2026-06-25 22:20:03 +05:00
mvalentsev ca11a455f5 ci: retrigger after #1520 unblocked develop lint 2026-06-25 22:20:03 +05:00
mvalentsev 4700c44163 ci: retry macos chromadb pyo3 flake on test_missing_db_invalidates_cache 2026-06-25 22:20:03 +05:00
mvalentsev a426587d35 style: apply ruff format to print statements 2026-06-25 22:20:03 +05:00
mvalentsev c7428c3b71 fix(mine): log warning when files exceed MAX_FILE_SIZE (#923)
Both miner.py and convo_miner.py silently skip files larger than the
10 MB limit with a bare continue. This is especially painful for
conversation mining where long Claude/ChatGPT exports routinely
exceed 10 MB and vanish with no trace.

Print a SKIP warning per oversized file, matching the existing format
in split_mega_files.py.
2026-06-25 22:20:03 +05:00
Arnold Wender c80f6537ec test(wal): cover crash-safety, idempotent setup, and redaction edge paths
The write-ahead log gained its own module in v3.5.0 but sat at 82% coverage;
the uncovered lines were exactly the failure/guard branches that uphold its
contracts: the cache-hit early return, the restricted-FS chmod/mkdir swallow
paths, and the promise that a WAL write failure is logged and never crashes
the calling tool. Add five tests covering those branches plus the non-string
redaction marker, bringing mempalace/wal.py to 100% and locking the
crash-safety guarantees against regression. Test-only; no production change.
2026-06-24 22:58:08 +02:00
Igor Lins e Silva 6ea60c6043 Merge pull request #1864 from MemPalace/codex/security-bundle-guards
fix: tighten local guards and file handling
2026-06-24 02:16:42 -03:00
Igor Lins e Silva d59ca24647 fix: green up CI for hardened file handling
- ruff format llm_client.py and miner.py (lint job)
- _copy_file_no_follow: close src fd if the dst open fails (no leak), and
  route the rebuild restore through it so backup + restore share one
  no-follow/regular-file path
- update repair tests to assert the unified hardened copy instead of the
  removed shutil.copy2 calls; backup paths are now timestamped
- update normalize large-file test to stub fstat (size is checked on the
  open fd, not via a pre-open os.path.getsize)
2026-06-24 01:12:24 -03:00
Igor Lins e Silva e9746df6fd fix: restore convo miner scan indentation 2026-06-23 23:47:13 -03:00
Igor Lins e Silva 2366582fe3 fix: tighten local guards and file handling 2026-06-23 23:45:03 -03:00
Igor Lins e Silva 2c08bb251d Merge pull request #1855 from MemPalace/develop
Release v3.5.0 — promote develop to main
v3.5.0
2026-06-22 22:00:13 -03:00
Igor Lins e Silva 65fa1517a1 Merge pull request #1853 from MemPalace/chore/release-3.5.0
chore(release): 3.5.0
2026-06-22 16:48:16 -03:00
Igor Lins e Silva e8f96dd8b2 chore(release): 3.5.0
Bump version to 3.5.0 across version.py, pyproject.toml, the Claude/Codex
plugin manifests, the README badge, and uv.lock. Refresh the "N MCP tools"
prose from 34 to 35 (delete_by_source #1729 and checkpoint #1851 each added a
tool). Add the 3.5.0 CHANGELOG entry.
2026-06-22 16:39:37 -03:00
Igor Lins e Silva 270f1aede9 Merge pull request #1854 from MemPalace/ci/windows-chromadb-flake-retries
ci(test-windows): retry the transient ChromaDB HNSW compaction flake
2026-06-22 16:38:31 -03:00
Igor Lins e Silva 37194cf543 ci(test-windows): retry the transient ChromaDB HNSW compaction flake
ChromaDB's rust HNSW core intermittently fails compaction on Windows with
"Failed to apply logs to the hnsw segment writer" during add/update — a
long-standing, non-reproducible-on-Linux/macOS flake that hits different tests
(test_migrate_wings, test_closets) across unrelated commits and has been
turning otherwise-green release/CI runs red at random.

Add pytest-rerunfailures and wire `--reruns 2 --only-rerun "Failed to apply
logs to the hnsw segment writer"` into the test-windows job only. The
--only-rerun scope means a real, deterministic failure still fails on the first
run; only this specific transient native-dependency error is retried. The
Linux and macOS jobs deliberately keep zero reruns so genuine regressions
surface there loudly.
2026-06-22 16:01:08 -03:00
Igor Lins e Silva 58ea91b44d Merge pull request #1806 from fatkobra/fix/1801-http-transport-stdio
feat(mcp): add opt-in HTTP transport
2026-06-22 13:05:13 -03:00
Igor Lins e Silva 1095d684ff fix(mcp): harden HTTP transport — DNS-rebinding guard, optional token, real tests
The opt-in HTTP transport reuses the stdio dispatcher and binds loopback by
default, but /mcp was unauthenticated with no protection against a malicious
web page reaching a DNS-rebound localhost server, and its tests reached for
Starlette/uvicorn (not project deps) so they were silently skipped in CI —
the production _serve_http handler had zero coverage.

Hardening:
- Pin the Host header to loopback literals + the bound host on a loopback bind
  (DNS-rebinding defense); relaxed for a deliberately non-loopback bind, which
  is the operator's call and may sit behind a Host-rewriting proxy.
- Reject any browser Origin that isn't a loopback origin (rebinding/SSRF guard);
  non-browser MCP clients omit Origin and are unaffected.
- Optional bearer token via MEMPALACE_MCP_HTTP_TOKEN (constant-time compare);
  required on /mcp, never on /healthz so liveness probes work credential-free.
- Warn loudly when bound to a non-loopback host (palace reachable from network).

Testability:
- Split _build_http_server() out of _serve_http() so tests bind 127.0.0.1:0 and
  drive the real handler over a loopback socket via stdlib http.client.
- Replace the skipped Starlette reimplementation with 12 tests covering dispatch,
  initialize, /healthz, 404, parse-error, the 16 MiB cap, notification 202, and
  the Host/Origin/token rejections — no third-party deps.
2026-06-22 12:40:03 -03:00
fatkobra 32fad4844d fix(lint): resolve 7 ruff errors in test_mcp_http_transport 2026-06-22 12:40:03 -03:00
fatkobra eff887d12f fix(tests): rewrite test_mcp_http_transport for Python 3.9-3.13 + Windows 2026-06-22 12:40:03 -03:00
fatkobra ce00cf0718 fix reviewer: handling JSON-RPC 2026-06-22 12:40:03 -03:00
fatkobra 3e3fcf29fe fix(mcp): move _HTTP_REQUEST_LOCK 2026-06-22 12:40:02 -03:00
fatkobra 569beff342 fix(mcp): move _HTTP_REQUEST_LOCK and _HTTP_MAX_REQUEST_BYTES 2026-06-22 12:40:02 -03:00
fatkobra 3fbab55cab test(mcp): make HTTP transport tests network-free 2026-06-22 12:40:02 -03:00
fatkobra 1a47b78dcb test(mcp): make HTTP transport loopback tests proxy-free 2026-06-22 12:40:02 -03:00
fatkobra 960eaac851 test(mcp): bypass proxies in HTTP transport loopback tests 2026-06-22 12:40:02 -03:00
fatkobra 54e9f46027 test(mcp): avoid subprocess flakiness in HTTP transport tests 2026-06-22 12:40:02 -03:00
fatkobra 2e8ff64e3f test(mcp): keep HTTP transport tests Python 3.9 compatible 2026-06-22 12:40:02 -03:00
fatkobra 94519b91dd fix suggestion of reviewer to avoid a critical race condition and other fixes 2026-06-22 12:40:02 -03:00
fatkobra 2f92774eec feat(mcp): add opt-in HTTP transport 2026-06-22 12:40:02 -03:00
Igor Lins e Silva 9902e3a9a5 Merge pull request #1729 from arnoldwender/feat/mcp-delete-by-source
feat(mcp): add mempalace_delete_by_source bulk-cleanup tool (#1722)
2026-06-22 12:37:59 -03:00
Igor Lins e Silva a91f7e06c7 Merge pull request #1820 from fatkobra/fix/1814-session-end-hook
fix(claude-plugin): run final mine on SessionEnd
2026-06-22 12:37:19 -03:00
Igor Lins e Silva 971e5a75c8 Merge pull request #1851 from undeadindustries/feat/cursor-checkpoint-tool
feat: add mempalace_checkpoint batch save tool
2026-06-22 12:37:09 -03:00
Igor Lins e Silva 7392ab8a25 Merge pull request #1817 from mvalentsev/fix/1815-source-file-filter
feat(search): add an optional source_file filter to mempalace_search (#1815)
2026-06-22 12:36:58 -03:00
Igor Lins e Silva e38af8c2e9 Merge pull request #1823 from fatkobra/fix/1818-mcp-peer-writer
fix(mcp): refuse second writer for same palace
2026-06-22 12:36:33 -03:00
Igor Lins e Silva 70ec8e450f Merge pull request #1832 from fatkobra/fix/1796-qdrant-bulk-metadata-scroll
fix(backends): single-scroll bulk metadata fetch for Qdrant; bump scroll page size (#1796)
2026-06-22 12:36:15 -03:00
Igor Lins e Silva a11cf4b667 Merge pull request #1786 from MemPalace/dependabot/github_actions/docker/metadata-action-6
chore(deps): bump docker/metadata-action from 5 to 6
2026-06-22 12:36:08 -03:00