Commit Graph

60 Commits

Author SHA1 Message Date
maximilize d471a9e262 feat(embedding): add OpenAI-compatible /v1/embeddings backend
Add an `embedding_model: "openai-compat"` option that computes embeddings
via any OpenAI-compatible `/v1/embeddings` server (LM Studio, llama.cpp,
vLLM, Ollama's OpenAI shim, self-hosted) instead of a local ONNX model.

- New OpenAICompatEmbeddingFunction (stdlib urllib, no new dependency):
  batches requests, asks for `encoding_format: "float"` and a custom
  User-Agent (avoids Cloudflare 403, see #1570), validates the response
  (contiguous 0..n-1 indices + well-formed vectors) before use, and
  L2-normalizes for the cosine collection. Exposes `embed_query` (ChromaDB
  1.5 dispatches query embedding through it, not `__call__`). `name()`
  encodes the model id so switching it forces `mempalace repair
  rebuild-index`. Failures raise a module-specific `EmbeddingAPIError`.
- Endpoint settings resolved by MempalaceConfig as a single source of truth:
  `embedding_api_url` / `embedding_api_model` / `embedding_api_key`, each
  overridable via the matching `MEMPALACE_EMBEDDING_API_*` env var.
  Whitespace-only values are treated as unset; the EF cache key fingerprints
  the key so a token rotation is picked up.
- The miner/MCP `Device:` header reports `openai-compat (<url>)` instead of a
  misleading local accelerator label when this backend is active.
- Opt-in; default stays minilm. Mirrors the existing `openai-compat` LLM
  provider naming; stays local when the endpoint is on the machine/LAN.
- Tests: tests/test_embedding_api.py (no server / no network required).
- Docs: README requirement note, module docstring, CHANGELOG.

Refs #1559.
2026-08-11 07:07:13 -03:00
Igor Lins e Silva a85c1fd786 docs(readme): lead the Docker section with the published image
The section only ever documented `docker build`, so every reader
compiled the image locally even though `ghcr.io/mempalace/mempalace` is
published multi-arch. Worse, a clone builds `develop` (the default
branch), not the release — a build and a pull could hand you different
versions with no hint that they differ.

Lead with `docker pull`, and cover the things that actually cost people
an evening:

- the container only sees what you mount, so the MCP client config now
  mounts a transcripts directory; without it the server starts fine and
  every mine finds nothing
- paths become container paths after that, and `~` / `$HOME` are not
  expanded by every MCP client
- the first embedding call downloads ~80 MB (minilm) or ~300 MB
  (embeddinggemma) into /data, which reads as a hung container
- bind mounts keep host ownership and the image runs as uid 1000, so a
  0700 directory fails with a bare PermissionError on Linux; Docker
  Desktop's uid mapping hides this on macOS and Windows. `--user` is
  called out as the wrong fix — /data is mode 700 owned by uid 1000, so
  another uid cannot write the palace at all
- mining never writes to its source, so the examples mount it read-only
- the GPU image is x86_64-only; onnxruntime-gpu has no aarch64 Linux
  wheels, so that build fails on Apple Silicon

Every command in the section was run as written against the published
image, and each claim checked rather than assumed: the uid and mode of
/data read out of the image, `--user` confirmed to break a *fresh*
volume, the read-only mount confirmed to still mine, and the aarch64 GPU
failure reproduced.
2026-08-08 19:46:05 -03:00
Igor Lins e Silva 8a019919f4 docs: restore develop's README/mcp-tools content lost in the slice resolution
The rebase resolution had taken the branch version of both files wholesale,
dropping develop's backend table, version badge, mempalace_kg_supersede
section, and the checkpoint added_by row. Rebuilt as develop's content plus
only the RFC 003 logstream additions; tool count is 43 (36 + 7 logstream).
2026-08-05 23:54:56 -03:00
Igor Lins e Silva 77c8998174 docs(logstream): concept page, tool/CLI references, coordination protocol (RFC 003 phase 4)
- website/concepts/agent-logstream.md: event/artifact model, delegation
  loop, coordination-vs-memory table; added to the sidebar.
- website/reference/mcp-tools.md: schemas for the seven logstream tools;
  tool count 35 -> 42 (README updated to match).
- website/reference/cli.md: mempalace logstream / mempalace artifact.
- website/guide/remote-server.md: hub doubles as agent coordination bus.
- integrations/shared/coordination-protocol.md: canonical shared-brain
  protocol (identity, delegation loop, hard rules) with a copy-paste
  system-prompt snippet; linked from recall-protocol.md and the
  mempalace-recall skill so recall and coordination never blur.

(cherry picked from commit 1ff312511aef39abbe4638bdc23301266f87dbe6)
2026-08-05 23:50:18 -03:00
Igor Lins e Silva e1fc71a9b9 chore(release): 3.7.0
Bump package, plugins, lock, and README badge to 3.7.0. Promote the
post-3.6.0 integrity spine into CHANGELOG: single-writer ownership,
HNSW write defaults and preflight, safer repair/re-mine, MCP/daemon
lifecycle hardening, entity ReDoS guard, and hook write-routing.

Also ruff-format two test files that drifted during conflict merges.

Local validation: ruff check/format clean; 3497 passed, 31 skipped.
2026-08-02 05:34:29 -03:00
Igor Lins e Silva dbfeb82e68 chore(release): 3.6.0 2026-07-14 20:44:15 -03:00
Cheney Zhang 0104d86c53 fix: address Milvus backend review feedback
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
2026-07-01 07:10:39 +00:00
Cheney Zhang db6c3d6552 fix: refine Milvus backend consistency
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
2026-07-01 06:52:28 +00:00
Cheney Zhang 9b395fe9d5 feat: add Milvus backend
Signed-off-by: Cheney Zhang <chen.zhang@zilliz.com>
2026-07-01 06:50:27 +00:00
Igor Lins e Silva e8f96dd8b2 chore(release): 3.5.0
Bump version to 3.5.0 across version.py, pyproject.toml, the Claude/Codex
plugin manifests, the README badge, and uv.lock. Refresh the "N MCP tools"
prose from 34 to 35 (delete_by_source #1729 and checkpoint #1851 each added a
tool). Add the 3.5.0 CHANGELOG entry.
2026-06-22 16:39:37 -03:00
undeadindustries 4291fecfdf feat: add mempalace_checkpoint batch save tool
Collapse the Cursor auto-save sequence (check_duplicate Nx + add_drawer
Nx + diary_write 1x) into a single mempalace_checkpoint MCP call so the
host UI renders one tool-call card and keeps its spinner up for the whole
save. The new tool reuses the existing single-item handlers, so semantic
dedup, idempotency, and verbatim guarantees are unchanged.

- mcp_server.py: add tool_checkpoint + register mempalace_checkpoint
- service.py: classify mempalace_checkpoint as a write tool
- cursor save hook: followup now drives one mempalace_checkpoint call
- docs: new mcp-tools.md section, help.md entry, 33 -> 34 tool count sweep
- tests: checkpoint add/dedup/malformed/registry + classify_tool

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-22 13:41:15 +10:00
Igor Lins e Silva 868b4c9b39 fix(hooks): portable mtime in macOS hook throttles; doc cleanup
Address review feedback surfaced on the 3.4.1 release promotion (#1810).

Bug fix — `date -r FILE` is GNU-only. On BSD/macOS `date -r` expects
epoch seconds, not a path, so the staleness/throttle checks in the new
Cursor and Antigravity hooks silently failed on macOS: the state GC
swept on every fire and the pending-save guard was skipped. Replace
with a portable `os.path.getmtime` one-liner via the already-resolved
$MEMPAL_PYTHON_BIN (cursor/lib, antigravity/lib, antigravity save hook).
This restores the "bash 3.2.57 / macOS default" compatibility the
Antigravity changelog claims.

Docs:
- Correct the MCP tool count to 33 (was 19/29/31 in 21 places across
  plugin manifests, READMEs, and website docs — all drifted from the
  TOOLS dict / mcp-tools.md reference, which both have 33).
- Fix broken CHANGELOG link to the Cursor skill (skills/, not
  .cursor-plugin/skills/).
- Fix one-too-many `../` in skills/mempalace/SKILL.md's cursor-hooks
  link (resolved above the repo root).
- Add the required `mcpServers` wrapper to the mcp.json example in
  .cursor-plugin/README.md so copy-paste yields a valid Cursor config.

Left intentionally unchanged: the os.dup2 fd-1 redirect in
mcp_server.py is deliberate (#225 keeps JSON-RPC off fd 1).
2026-06-14 19:54:16 -03:00
Igor Lins e Silva b5c79a1eea chore(release): 3.4.1
Bump version across all sources (version.py, pyproject.toml, both
Claude plugin manifests, Codex plugin manifest, README badge, uv.lock)
and promote the Unreleased changelog to 3.4.1.

Shipping: Cursor IDE plugin + hooks, first-class Antigravity IDE
support (with zero-config interpreter resolution), embeddinggemma
bulk re-embed OOM fix, and backup-retention pruning.

Also rebuilds the CHANGELOG compare-link block, which had been left
at v3.2.0: adds the full 3.3.0-3.4.1 chain plus the previously
undocumented 3.4.0, and points Unreleased at v3.4.1...HEAD. Every
version header now resolves to a compare link.
2026-06-14 17:47:09 -03:00
Igor Lins e Silva f31db69e03 merge: resolve CHANGELOG conflict with develop 2026-06-11 04:28:42 -03:00
Igor Lins e Silva 692599aa94 merge: resolve CHANGELOG conflict with develop 2026-06-10 07:46:22 -03:00
Igor Lins e Silva 73d4e43b22 chore(release): bump README version badge to 3.4.0
Enforced by test_readme_claims.py::TestVersionBadge — a sixth version
location outside the version-guard 5-file set.
2026-06-06 03:41:49 -03:00
Igor Lins e Silva f88951e8d4 Merge remote-tracking branch 'origin/develop' into codex/pluggable-vector-backends-qdrant
# Conflicts:
#	README.md
2026-06-05 23:18:54 -03:00
Igor Lins e Silva ec5d1eb1c7 feat: add pgvector backend + namespace-isolation conformance contract
Adds a second external storage backend (Postgres/pgvector) alongside Qdrant
to prove the BaseBackend/BaseCollection contract generalizes across substrates
(SQL + JSONB containment filters + pgvector `<=>` ranking vs Qdrant's REST/dict
model), and addresses the review feedback on PR #1679.

Backend (mempalace/backends/pgvector.py):
- table-per-(namespace, palace, collection) isolation; advertises
  supports_namespace_isolation
- JSONB filter pushdown for the containment subset, local-exact fallback for
  $or/$contains/comparisons/where_document
- BM25 lexical search; marker-based mismatch protection
- optional psycopg dependency (lazy import), in-memory fake for CI, live test
  gated on MEMPALACE_PGVECTOR_LIVE_URL
- registered in registry/__init__/pyproject entry point + [pgvector] extra;
  MEMPALACE_PGVECTOR_DSN / MEMPALACE_PGVECTOR_NAMESPACE config; README docs

Isolation contract (RFC 001):
- PalaceRef/BaseBackend document the per-id MUST and the cross-namespace MUST,
  gated on the new supports_namespace_isolation capability token
- runnable conformance suite (tests/_backend_conformance.py,
  tests/test_backend_conformance.py); qdrant + pgvector run it via their fakes

Marker fail-loud guard:
- qdrant and pgvector now refuse get_collection when local_path is None instead
  of silently opening a remote collection with no mismatch protection

Review fixes:
- palace._open_collection_or_explain handles unknown-backend KeyError as a CLI
  state message instead of an escaping stack trace
- dedup.py docstring no longer claims "No API calls" unconditionally (false for
  remote backends)
2026-06-05 21:41:21 -03:00
plaza-s 8757ba2650 fix(docker): address gemini review on MR #1696 2026-06-05 13:35:36 +02:00
sebaplaza 717a57ff43 feat(docker): add container image for MCP server and CLI
Add a multi-stage, uv-based Dockerfile producing a CPU image (with the
extract + spellcheck extras), plus a CUDA variant (Dockerfile.gpu) for
onnxruntime-gpu accelerated embeddings.

A single flexible entrypoint dispatches to the MCP stdio server (default)
or the mempalace CLI. All state -- palace, config, and the lazily
downloaded embedding model -- persists under /data via HOME, runs as a
non-root user, and is exposed as a volume.

Also add a docker-compose.yml for convenience, a GHCR publish workflow,
and a Docker section in the README.
2026-06-05 12:32:33 +02:00
Igor Lins e Silva 6aa8e93bc9 feat: add pluggable vector backends 2026-06-02 21:38:53 -03:00
undeadindustries bf156fb010 feat: add Antigravity IDE support (plugin, MCP, skill, hooks, docs, tests)
Adds first-class integration with Google's Antigravity IDE
(https://antigravity.google/) as a third sibling to the existing
Claude Code and Codex hook integrations. Strictly additive — no
existing files in main are restructured.

What ships
----------

* `.antigravity-plugin/` — verified-minimal plugin package:
  * `plugin.json` with `{"name": "mempalace"}` (no fabricated fields)
  * `mcp_config.json` registering the `mempalace-mcp` stdio server
  * `hooks.json.tmpl` templated with `__PLUGIN_DIR__` substitution
  * `skills/mempalace/SKILL.md` (real file — no symlinks)
* `hooks/antigravity/`:
  * `lib/common.sh` — shared bash 3.2.57-compatible helpers with
    sentinel-guarded camelCase JSON parser, antigravity_*-namespaced
    state files, every existing kill switch, `MEMPAL_SAVE_INTERVAL >= 1`
    floor (no /0), and fail-open emitters
  * `mempal_save_hook_antigravity.sh` — Stop event handler:
    increments per-conversation counter, defers when fullyIdle=False
    or terminationReason=error, validates transcriptPath against
    `..` traversal, spawns `mempalace mine --mode convos` in a
    detached subprocess with a per-conversation pending marker,
    ALWAYS emits `{}` (never `{"decision":"continue"}` — that would
    force an infinite agent loop)
  * `mempal_wake_hook_antigravity.sh` — PreInvocation handler gated
    to invocationNum==1 with an atomic mkdir loop guard, runs
    `mempalace wake-up` with a 500ms hard timeout, emits verbatim
    output as `{"injectSteps":[{"ephemeralMessage":"..."}]}` or
    `{}` on any failure
  * `install.sh` — idempotent installer with cmp-gated copies,
    `__PLUGIN_DIR__` substitution, relative path absolutization,
    `--dry-run`, and basename-guarded `--uninstall` (refuses to
    wipe a directory whose basename isn't `mempalace`)
  * `INVESTIGATION.md` — verbatim quotes + URLs + dates from the
    five official Antigravity doc pages, recording every surface
    shipped and every surface deliberately omitted
    (PreCompact equivalent, slash-commands, rules/, plugin
    permissions field — the latter is third-party fabrication)
  * `STDIN_SHAPE.md` — exact stdin/stdout contract per event with
    worked examples
  * `README.md` — local hook docs + troubleshooting
* `examples/antigravity/{hooks.json,mcp_config.json,README.md}` —
  standalone configs for users who don't want the full installer
* `website/guide/antigravity.md` + sidebar entry — VitePress guide
* Updates to `README.md`, `CHANGELOG.md` (Unreleased), `hooks/README.md`

Tests (56 new, all passing)
---------------------------

* `tests/test_antigravity_plugin_manifest.py` (11 tests) — schema
  contract on the in-repo `.antigravity-plugin/` directory, including
  guards against re-introducing the fabricated `permissions` field
  and against any symlink leak.
* `tests/test_antigravity_hooks_shell.py` (31 tests) — invokes the
  bash hooks via subprocess with synthetic camelCase stdin, asserts
  `{}` on every failure path, kill-switch coverage (env vars +
  config.json + palace nuke), divide-by-zero floor, transcript
  traversal rejection, namespacing, wing inference, and the hard
  refusal to ever emit `decision=continue` from the Stop hook.
* `tests/test_antigravity_hooks_install.py` (14 tests) — `--dry-run`
  side-effect-free, real install layout, executable bits preserved,
  byte-identical idempotent re-runs (md5 + filecmp), basename-match
  uninstall safety, refusal when plugin.json is missing or names a
  different plugin, relative path absolutization. Skipped on Windows.

Verification
------------

* `uv run pytest tests/ --ignore=tests/benchmarks -v` → 2314 passed,
  3 skipped (Windows), 1 unrelated warning
* `uv run ruff check .` → all checks passed
* `uv run ruff format --check .` → 139 files already formatted
* `bash -n` clean on common.sh, both hook scripts, install.sh
* Local install at `~/.gemini/config/plugins/mempalace/` verified end-
  to-end: layout correct, paths absolutized in hooks.json, both hooks
  fire with realistic camelCase JSON in <1s, wing inference picks
  `wing_mempalace` from workspacePaths[0], state files all
  `antigravity_*`-namespaced, second `install.sh` run produces
  byte-identical output (md5 snapshots match), uninstall removes
  only the mempalace plugin and leaves all 6 sibling Google plugins
  untouched.

Constraints honoured
--------------------

bash 3.2.57 (no mapfile / readarray / declare -A / `${var^^}`),
verbatim guarantee on all wake injections, hooks <500ms / startup
injection <100ms target (kill-switch path returns in <1.5s in CI),
zero new runtime dependencies, no telemetry, no external API,
strictly additive (existing Claude/Codex hooks unchanged).

Refs: hooks/antigravity/INVESTIGATION.md for the full audit.
2026-05-27 19:19:17 +10:00
undeadindustries 071fa015e0 feat: add Cursor IDE support (hooks, plugin, skill, docs, tests)
Adds first-class Cursor IDE integration alongside the existing Claude
Code and Codex hook flows, so Cursor users get the same automatic
diary saves, pre-compaction transcript capture, and session-start
memory recall — without changing any default behaviour for existing
users.

What's included
---------------

Cursor hook scripts (hooks/cursor/):
  - mempal_save_hook_cursor.sh       — Stop event, counter +
    loop_count guard, pending-save marker consumption, background
    mempalace mine, followup_message emission.
  - mempal_precompact_hook_cursor.sh — synchronous mine before
    compaction, drops a pending_save marker, returns user_message.
  - mempal_wake_hook_cursor.sh       — sessionStart event,
    wing-scoped recall guidance via additional_context.
  - lib/common.sh                    — shared parsing + state helpers
    (bash 3.2 safe, no heredoc-in-subshell traps).
  - install.sh                       — idempotent installer with
    --scope, --variant, --dry-run, --uninstall. Recognises existing
    entries by basename so re-installs across paths work.
  - STDIN_SHAPE.md, README.md        — payload schemas + quick
    reference.

Cursor plugin (.cursor-plugin/ + repo-root components):
  - plugin.json, marketplace.json, README.md.
  - skills/mempalace/SKILL.md  — model-invocable skill mirroring the
    Claude plugin's skill surface.
  - commands/mempalace-{help,init,mine,search,status}.md  — slash
    commands for marketplace-published installs (filename = slug).
  - mcp.json                   — auto-registers the mempalace MCP
    server, wrapped under the documented mcpServers key.

Examples + docs:
  - examples/cursor/hooks.json, hooks.minimal.json + README.
  - website/guide/cursor-hooks.md + sidebar entry.
  - README.md and CHANGELOG.md updates.

Tests (129 new, all green):
  - tests/test_cursor_hooks_shell.py     — 75 behavioural tests for
    the three hook scripts: kill switches, input parsing, counter
    logic, loop prevention, pending markers, wing inference, logging.
  - tests/test_cursor_hooks_install.py   — 19 contract tests for the
    installer: dry-run, idempotent merge, basename-matched uninstall,
    refusal to overwrite malformed JSON.
  - tests/test_cursor_plugin_manifest.py — 35 contract tests for the
    plugin: manifest validity, version sync with mempalace.version,
    mcp.json shape, skill/command frontmatter, default-discovery
    layout invariants.

Design notes
------------

- Local-first and zero-API by default; hooks never call external
  services. Same privacy model as the existing Claude Code hooks.
- Fail-open: hook scripts deliberately do not use set -e so a broken
  hook can never block the user's conversation.
- Cursor preCompact cannot block + return a followup, so we
  synchronously mine the transcript and drop a pending_save marker
  that the next stop hook consumes — guarantees verbatim capture
  before context window compression.
- Cursor's default plugin discovery requires real commands/, skills/,
  and mcp.json at the plugin root (verified against the cached
  cloudflare plugin); .cursor-plugin/{commands,skills} are convenience
  symlinks back to those canonical locations.
- bash 3.2 compatibility throughout: avoids heredoc-in-command-
  substitution parser bugs; uses python -c for JSON parsing;
  basename-matched entry recognition in install.sh.
- All changes are additive. No existing files are removed, no
  existing hooks change behaviour, and no new runtime dependencies
  are introduced.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-27 14:05:40 +10:00
Igor Lins e Silva be819da146
Merge pull request #1608 from MemPalace/chore/readme-move-scam-alert-below-header
docs(readme): move scam/retention alerts below header, soften tone
2026-05-24 19:47:40 -03:00
Igor Lins e Silva 280e532724 chore(release): 3.3.6
Bumps version 3.3.5 → 3.3.6 across pyproject.toml, version.py, plugin
manifests (.claude-plugin/plugin.json, .claude-plugin/marketplace.json,
.codex-plugin/plugin.json), README badge, and uv.lock. Flips CHANGELOG.md
from ``[Unreleased]`` to ``[3.3.6] — 2026-05-24`` and backfills the
major user-facing entries that landed without changelog entries during
the cycle:

Features:
- #1555 office-document mining via --mode extract + virtual line numbers
- #1584 surgical closet pointers with date+line locators (Tier 6a)
- #1558 + #1560 within-wing hallways (entity co-occurrence graph)
- #1565 cross-wing tunnels auto-promoted from hallways
- #1578 Hebbian potentiation + Ebbinghaus decay on hallways/tunnels
- #1236 API-tool transcripts auto-route to wing_api
- #711 hooks.auto_save toggle for silent-mode sessions
- #1605 COCA content-word filter for entity detection
- #1557 case-insensitive entity matching at mine time
- #1483 multilingual embeddings (embeddinggemma-300m) by default

Bug Fixes (selected, user-visible):
- #1540 silent data loss in three unchunked upsert sites
- #1538 paragraph chunker oversized chunks
- #1554 per-file chunk cap too low for transcripts
- #1562 Windows hook subprocess/ChromaDB deadlock
- #1529 create_tunnel corrupted hyphenated wing names
- #1424 save-hook truncated hyphenated project folders
- #1383 KG cache duplicated graphs for symlinked/cased paths
- #1466 silent symlink skip now logged
- #1441 macOS stock-bash 3.2 hook compatibility
- #1500 / #1513 structured JSON-RPC errors on bad MCP input
- #1523 VACUUM + FTS5 rebuild after repair
- #1548 FTS5 validation at end of mine
- plus #1216, #1408, #1438, #1439, #1445, #1452, #1459, #1461, #1466,
  #1470, #1477, #1485, #1500, #1513, #1528, #1532, #1543, #1546, #1585

Performance:
- #1474 convo miner pre-fetches mined-set
- #1487 rebuild_index progress callback
- #1530 MCP cold-start diagnostics + opt-in warmup

Lint passes (ruff 0.15.14); mempalace-mcp entry point alignment
verified per RELEASING.md.
2026-05-24 14:17:41 -03:00
Igor Lins e Silva f51ecc1621 docs(readme): move CAUTION/IMPORTANT alerts below header, soften tone
The scam-alert and Claude-Code-retention admonitions were the first
content visitors saw on the repo page — louder than the project
introduction. Moves both below the logo/title/badges so the project
identity reads first, and softens the scam block (drops the H1
"CRITICAL SECURITY WARNING" + all-caps shouting + redundant emoji)
to a single-paragraph CAUTION. All factual content preserved:
impostor-domain warning, official sources, malware caveat, link to
docs/HISTORY.md.
2026-05-24 13:48:20 -03:00
Igor Lins e Silva 382409bd6b Merge origin/develop into feat/benchmark-multilingual
Resolves conflicts in CHANGELOG.md and pyproject.toml by combining
the multilingual-embedder additions (huggingface_hub/tokenizers/numpy
core deps, [multilingual] alias, Features section) with develop's
additions (python-dateutil core dep, [extract] extra, tunnel Bug
Fixes and Internal sections).

Prepares PR #1483 for merge into v3.3.6.
2026-05-24 13:18:08 -03:00
Igor Lins e Silva 8a6537d2fb feat(onboarding): multilingual embedder by default for new installs
Onboarding now asks the user once, on first run, whether to use the
multilingual embedding model. The default answer is yes — defaulting to
English-only made the recall promise effectively unreachable for any
non-English content (cross-lingual cos ~0.35 vs ~0.88 for the multilingual
model). The choice is written to config.json so subsequent runs pick the
right EF without re-prompting; existing installs that never set the env
var or ran onboarding stay on minilm for back-compat. MEMPALACE_EMBEDDING_MODEL
still overrides both.

Multilingual deps (huggingface_hub, tokenizers, numpy) move from the
[multilingual] extra into core. The extra is kept as a no-op alias so
existing install scripts keep working. The 300 MB ONNX model is still
lazy-downloaded on first use, not at install time.

`quick_setup` (the programmatic non-interactive path) grows an optional
`embedding_model` arg so tests and benchmark scripts can pick a model
without writing config.json by accident.

EmbeddinggemmaONNX's "missing deps" error now points at the right
recovery path (reinstall mempalace, since the deps are core) rather
than the obsolete pip install mempalace[multilingual] hint.

Tests: 9 new (3 _ask_embedding_model variants + 2 run_onboarding
persistence + 2 quick_setup + 2 set_embedding_model round-trips). The
existing 2 run_onboarding tests now patch _ask_embedding_model so they
don't print to stdout.
2026-05-14 06:40:27 -03:00
Igor Lins e Silva cef1c62fe7 feat(embedding): EF-mismatch error helper, offline tests, migration docs
Three follow-ups bundled for the embeddinggemma EF added in 51702e9:

1. Offline tests for EmbeddinggemmaONNX (10 tests, 0.08s, no network).
   Mocks huggingface_hub.hf_hub_download, tokenizers.Tokenizer.from_file,
   and onnxruntime.InferenceSession so CI never pulls the 300 MB model.
   Guarded with pytest.importorskip so the file is skipped when the
   multilingual extra isn't installed. Covers: stable name(), lazy-load
   runs exactly once, output shape (n, 384) after MRL truncation, L2
   normalization, sim prefix applied, dispatch from
   get_embedding_function(model="embeddinggemma"), cache key separates
   models, helpful ImportError when deps missing, env override.

2. Friendlier ChromaDB EF-name-mismatch error. Switching
   MEMPALACE_EMBEDDING_MODEL on an existing palace previously surfaced
   ChromaDB's bare "Embedding function conflict: new: X vs persisted: Y"
   ValueError. Now ChromaBackend.get_collection() wraps that error and
   points users at the two recovery paths: revert the env var, or run
   `mempalace repair rebuild-index --palace <path>`. New
   _explain_ef_mismatch helper + 3 tests (unit + end-to-end).

3. Docs: CHANGELOG [Unreleased] entry covers both the new EF and the
   error wrapper. README Requirements section mentions the multilingual
   extra and points at the embedding.py docstring for the migration note.
2026-05-14 05:00:48 -03:00
Igor Lins e Silva fa9b7e0525 chore(release): 3.3.5
Bumps version 3.3.4 → 3.3.5 across pyproject.toml, version.py, plugin
manifests, README badge, and uv.lock. Flips CHANGELOG.md from
``[3.3.5] — unreleased`` to ``[3.3.5] — 2026-05-09`` and adds entries
for the four PRs that landed after the bug-fix block was authored:

- Bug Fixes: #1396 (tool_search retry on transient HNSW flush)
- Documentation: #1385 (CONTRIBUTING git-identity guidance, closes #1317)
- Internal: #1431 (test multiprocessing fork → spawn)
- Internal: #1430 (test sqlite connection lifecycle via contextlib.closing)

The four open issues remaining on the v3.3.5 milestone (#1266, #1253,
#1092, #1082) have been moved to v3.4 — they form the concurrent-writer
/ HNSW corruption cluster that needs deeper work than this cycle could
absorb.
2026-05-09 21:11:13 -03:00
Igor Lins e Silva c35686c9e1 docs(install): recommend uv as the package manager
End-user installs now lead with `uv tool install mempalace`, with
`pip install mempalace` kept as a fallback. Dev/contributor docs lead
with `uv sync --extra dev` and `uv run` for tests/benchmarks/lint, with
the equivalent pip recipe kept inline. The shipped `/mempalace:init`
skill instructions (mempalace/instructions/init.md) try `uv tool install`
first when uv is on PATH, then fall back through the pip variants.

Adds a .python-version pin at 3.12 because the lockfile's
onnxruntime==1.24.3 only ships wheels for Python >=3.11; without the
pin, `uv sync` on a host where uv prefers 3.10 fails with no source
distribution available, which would make the documented command a
footgun. pyproject's `requires-python = ">=3.9"` is unchanged — pip
users on 3.9/3.10 are unaffected.

Files updated: README.md, CONTRIBUTING.md, CLAUDE.md, the gemini-cli
guide and example, the .claude-plugin / .codex-plugin READMEs, the
mempalace SKILL, the openclaw SKILL, tools/save.md, the three
benchmarks docs, and the corresponding website mirrors.
2026-05-08 01:38:00 -03:00
Igor Lins e Silva 25bfd37644 chore(release): sync 3.3.4 version bump back into develop
The v3.3.4 release prep landed on main but was never merged back into
develop, leaving every version-bearing file one release behind. Bumps
pyproject.toml, mempalace/version.py, both plugin manifests, the
marketplace entry, the README badge, and the lockfile to 3.3.4 to match
the tagged release.
2026-05-08 01:38:00 -03:00
MillaJ bddba59ae3 docs: add 30-day expiry callout + ship 4 auto-save tools
Adds a brief [!IMPORTANT] callout at the top of the README pointing
users to the urgent announcement at #1388. Claude Code auto-deletes
local JSONL transcripts after 30 days; users without the auto-save
hooks wired are losing transcript data off the rolling window.

Ships 4 small standalone tools at tools/:
- backup_claude_jsonls.sh — rsync ~/.claude/projects/ to a safe folder
- render_jsonl.py — convert JSONL transcripts to readable text
- find_orphan_claude_jsonls.sh — scan backup locations for orphan
  Claude Code transcripts (multi-line shape detection + topic preview)
- save.md — Claude Code slash command for manual /save into MemPalace

Tools verified by independent agent against v3.3.4 source.
Read-only on user data. POSIX bash + Python stdlib only.
2026-05-06 13:10:16 -07:00
Igor Lins e Silva fe56797762 fix(hooks): consolidate transcript ingest, harden shell parsers (#1231 review)
Address Copilot review on #1231:

1. Stop double-mining the transcript on the Python side. ``_get_mine_targets``
   now returns only the ``MEMPAL_DIR`` projects target — the convos target
   for the transcript dir is dropped because ``_ingest_transcript`` already
   handles it on every hook fire. The duplicate spawn was using
   ``sys.executable`` (vs ``_mempalace_python()``) and a different ``--wing``,
   so each Stop/PreCompact event was writing the same transcript into two
   wings under asymmetric interpreters and overwriting the single
   ``_MINE_PID_FILE`` lock.

2. ``_maybe_auto_ingest`` and ``_mine_sync`` now spawn via
   ``_mempalace_python()`` so the resolved interpreter matches the venv
   that owns mempalace (matters under GUI-launched harnesses where
   ``sys.executable`` may resolve to a system Python without chromadb).

3. Replace ``eval $(...)`` in both shell hooks with a ``mapfile``-based
   reader. Sanitized values are still emitted by the same Python parser,
   but the shell now does plain variable assignment instead of executing
   the parser's stdout — smaller blast radius if the sanitizer is ever
   bypassed.

4. Mirror ``_validate_transcript_path`` in the shell hooks via a
   ``is_valid_transcript_path`` helper — extension + traversal-segment
   rejection, parity with the Python validator. The convos mine in each
   shell hook is now gated on the validator instead of bare ``-f``.

5. Tighten the ``..`` traversal test that previously exercised the
   suffix gate by mistake (``../../etc/passwd`` lacks ``.json[l]``).
   Use ``.jsonl`` paths with traversal segments to actually hit the
   ``..`` rejection branch.

6. README: add a one-liner pointing at ``mempalace sweep`` for users
   who want per-message recall on top of the file-level chunks the
   hooks produce. The sweeper was undiscoverable previously.

Tests: 1418 passed, 1 skipped (full suite minus benchmarks).
2026-04-27 02:26:53 -03:00
bensig 4f799afd76 release(3.3.3): bump README version badge
test_readme_badge_matches_version_py compares the version in the
README badge URL against version.py. Missed it in the initial bump.
2026-04-23 16:51:41 -07:00
Igor Lins e Silva 5e9451407f release: v3.3.2
Version bumps across pyproject.toml, mempalace/version.py, README badge,
uv.lock, and plugin manifests (.claude-plugin/*, .codex-plugin/*).

CHANGELOG aligned with main (post-3.3.1) and a new [3.3.2] section added
covering the 11 PRs merged on develop since v3.3.1 — silent-transcript-drop
fix + tandem sweeper (#998), None-metadata guards (#999, #1013),
chromadb ≥1.5.4 for Py 3.13/3.14 (#1010), Windows Unicode (#681),
HNSW quarantine recovery (#1000), PID stacking guard (#1023), doc-path
cleanup (#996, #1012), and RFC 001/002 internal scaffolding (#995, #1014, #990).
2026-04-19 16:55:25 -03:00
bensig a2da0d6ef4 docs: use real ~/.claude/projects/ path in first-run help and README (#996)
The CLI help text and README told first-time users to mine from ~/chats/,
a path that doesn't exist on any machine. Real location where Claude
Code writes session JSONL is ~/.claude/projects/<escaped-project-path>/.

Updates three user-visible strings:
- mempalace/cli.py line 7 ("Two ways to ingest" block)
- mempalace/cli.py line 25 (Examples block)
- README.md line 58 (Quickstart)

Website guides (website/guide/mining.md, getting-started.md) still
reference ~/chats/ for ChatGPT/Slack export scenarios where that remains
a valid placeholder. Those can be a separate PR if the maintainers want
to tilt the website examples toward Claude Code specifically.

Fixes #996.
2026-04-18 13:30:50 -07:00
Igor Lins e Silva 65bf1ebda3 docs: slim README and move corrections/notices to docs/HISTORY.md
Addresses #875. The previous README was 755 lines mixing six purposes
(scam alert, hero, two mea-culpa notes, install guide, architecture
explainer, API reference, file map). Rework it as a pure entry point:
what MemPalace is, how to install, honest benchmark numbers, links to
the website for concept/architecture documentation.

Key content changes:
 - Drop the "highest-scoring AI memory system ever benchmarked" framing.
 - New tagline: "Local-first AI memory. Verbatim storage, pluggable
   backend, 96.6% R@5 raw on LongMemEval — zero API calls." Avoids
   naming a specific vector-store implementation since the backend is
   pluggable (see mempalace/backends/base.py).
 - Remove the cross-system comparison table. Retrieval recall (R@5)
   and end-to-end QA accuracy are different metrics and are not
   comparable; placing MemPalace's R@5 next to competitor QA accuracy
   under a single column header was a category error.
 - The "100%" LongMemEval headline is no longer the lead. The honest
   held-out figure is 98.4% R@5 on 450 unseen questions. The rerank
   pipeline reaches >=99% with any capable LLM (reproduced with
   Claude Haiku, Sonnet, and minimax-m2.7 via Ollama) — pipeline-level,
   not model-specific.
 - Benchmark reproduction commands now reference the correct repo
   (MemPalace/mempalace, not the defunct aya-thekeeper/mempal branch).

New file: docs/HISTORY.md as the canonical home for post-launch
corrections, public notices, and retractions. Contains verbatim:
 - 2026-04-14 note on this rewrite (links to #875)
 - 2026-04-11 impostor-domain notice (moved from README header)
 - 2026-04-07 "A Note from Milla & Ben" (moved from README body)

README keeps a one-line scam-alert callout that links to
docs/HISTORY.md for the full timeline.
2026-04-14 21:37:20 -03:00
Igor Lins e Silva 8630e7e3e0
Merge pull request #869 from gbhat618/fake-website-visibility-increase
Increase visibility of fake website caution
2026-04-14 14:12:16 -03:00
Igor Lins e Silva da89a11bda docs: name official domain and specific impostors in scam alert
Replace the blanket ban on .tech/.io/.com domains with an allowlist
of real MemPalace surfaces (GitHub repo, PyPI, mempalaceofficial.com)
and call out mempalace.tech as the reported impostor. The blanket
.com ban would have flagged mempalaceofficial.com as fake once DNS
resolves (CNAME shipped in #877).

Also update the April 11 follow-up section to match so the two
notices no longer contradict each other.
2026-04-14 14:11:05 -03:00
Guruprasad Bhat 1bff7d9907
Increase visibility of fake website caution
Noticed a URL 
```
hXXps://www.mempalace[.]tech/
```

Though the README currently warns, it is perhaps best to surface it at urgency level at the top of the README.
2026-04-14 17:08:06 +05:30
mvalentsev 001700ce72 fix: update stale org URLs in pyproject.toml and README (#787) 2026-04-14 09:07:39 +05:00
Igor Lins e Silva 41696df1ac chore(release): bump version strings to 3.3.0 and curate CHANGELOG
Prepare develop for the 3.3.0 release cycle.

Version bumps:
- mempalace/version.py: 3.2.0 -> 3.3.0
- pyproject.toml: 3.2.0 -> 3.3.0
- README.md: pyproject.toml label and shields.io badge
- uv.lock: mempalace 3.0.0 -> 3.3.0 (also fills in resolved dev/extras)

CHANGELOG.md:
- Close out the stale [Unreleased] section as [3.2.0] - 2026-04-12
  (v3.2.0 was tagged on that date but the release flip was never made)
- Add a fresh [Unreleased] - v3.3.0 section covering the 49 commits
  since v3.2.0: closet layer, BM25 hybrid search, entity metadata,
  diary ingest, cross-wing tunnels, drawer-grep, offline fact checker,
  LLM-based closet regen, hall detection, cosine-distance fix,
  multi-agent locking, README audit, etc.
- Adopt Keep a Changelog + SemVer framing
- Add version compare reference links at the bottom
- Fix stale milla-jovovich/mempalace preamble URL to MemPalace/mempalace
2026-04-13 21:36:56 -03:00
Milla J 62df24599e
fix: README audit — 42 TDD tests + hall detection + 7 claim fixes (#835)
* fix: README audit — match every claim to shipped code + add hall detection

TDD audit: wrote 42 tests verifying README claims against codebase.
Fixed all 7 failures:

1. Tool count: 19 → 29 (10 tools were undocumented)
2. Added tool table rows for tunnels, drawer management, system tools
3. Version badge: 3.1.0 → 3.2.0
4. dialect.py file reference: "30x lossless" → "AAAK index format for closet pointers"
5. Wake-up token cost: "~170 tokens" → "~600-900 tokens" (matches layers.py)
6. pyproject.toml version in project structure: v3.0.0 → v3.2.0
7. Hall detection: added detect_hall() to miner.py — drawers now tagged
   with hall metadata so palace_graph.py can build hall connections

New code:
- miner.py: detect_hall() — keyword scoring against config hall_keywords,
  writes hall field to every drawer's metadata
- tests/test_hall_detection.py — 12 TDD tests (written before code)
- tests/test_readme_claims.py — 42 TDD tests verifying README accuracy

859/859 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve ruff lint — unused imports and variables

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: ruff format with CI-pinned 0.4.x

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use conftest fixtures in hall tests for Windows compat

Windows CI fails with NotADirectoryError when ChromaDB tries to
write HNSW files in short-lived TemporaryDirectory. Use conftest
palace_path and tmp_dir fixtures instead — same pattern as all
other tests that touch ChromaDB.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Igor's review — convo_miner halls, cached config, markdown typo

TDD: wrote tests for convo_miner hall metadata and config caching
BEFORE verifying the code changes.

1. README markdown typo: extra ** in wake-up token row (line 195)
2. convo_miner.py: added _detect_hall_cached() — conversation
   drawers now get hall metadata (was missing, Igor caught it)
3. miner.py + convo_miner.py: cached hall_keywords at module level
   so config.json isn't re-read per drawer during bulk mine
4. New tests: TestConvoMinerWritesHalls, TestDetectHallCaching

861/861 tests pass. ruff clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 17:11:11 -07:00
clach04 c17cf079ad
fix #733: diagram misaligned (#734)
Converted to 7-bit clean us-ascii, which also aligns diagram.
Moved "hall" connection to be inline with Rooms in diagram.
2026-04-13 01:43:01 -03:00
Justin Clift b03ab482ef
docs: Add warning to the README about fake MemPalace websites 2026-04-11 14:25:16 +10:00
Kevin Pulikkottil 2981433535
fix: add mcp command with setup guidance (#315)
* fix: add mcp command with setup guidance

* fix: include --palace guidance in mcp command output

* fix: make mcp guidance commands copy-pastable

---------

Co-authored-by: Milla J <millaj1217@gmail.com>
2026-04-09 11:21:18 -07:00
Milla J 3919f13523
chore: bump version to 3.1.0 (#409)
PyPI release cut covering 39 merged PRs since v3.0.0 on 2026-04-06.
Highlights: Claude/Codex plugin packaging (#270), security hardening (#387),
honest AAAK stats + benchmark corrections (#147), Windows compatibility fixes,
Knowledge Graph WAL mode + batching, 10K limit safety caps, and much more.

See GitHub release notes for full changelog.

Co-authored-by: milla-jovovich <noreply@github.com>
2026-04-09 11:04:24 -07:00
Ben Sigman f40aed820f
Merge pull request #238 from neoneye/main
Fixed broken link reference for AAAK Dialect in README
2026-04-08 10:46:41 -07:00
Tal Muskal 9de302f881 feat: update README and CI configuration, add tests for hooks functionality 2026-04-08 20:40:03 +03:00