mirror of MemPalace/mempalace - best-benchmarked open-source AI memory system
Go to file
Josh cff43adb63
feat(convo): preserve authored timestamp from transcripts (#1890)
* feat(convo): preserve authored timestamp from transcripts

Conversation drawers only carried `filed_at` (ingest time), so a bulk
re-mine collapsed every drawer to a single instant and the chronological
signal was lost — even though each Claude Code / Codex JSONL line already
carries an ISO-8601 `timestamp`. The recency-window fallback and any
date-aware consumer then saw ingest order, not when content was written.

- convo_miner: derive `authored_at` (per-file max line `timestamp`) and
  store it as drawer metadata; falls back to `filed_at` when absent
- searcher: surface `authored_at` in search results, and break exact
  hybrid-score ties toward the more recently authored drawer (ISO strings
  sort chronologically; missing dates sort oldest) — benchmark-neutral as
  it only reorders exact ties
- tests: cover `_extract_authored_at` (latest wins, skips/tolerates lines
  without timestamps, non-jsonl/missing -> None) and the tie-break

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(search): surface authored_at in CLI + backfill for existing data

Completes the authored_at work so the field is visible end-to-end and
existing palaces can adopt it without re-mining.

- layers: CLI `search` output shows an `authored:` date line per result
  (peer of the existing date; markdown drawers fall back to filed_at)
- scripts/backfill_authored_at.py: in-place migration that stamps
  authored_at on convos drawers from their source transcripts — metadata
  only (no re-embedding), idempotent, dry-run by default
- docs/authored-at.md: documents created_at (ingest) vs authored_at
  (written) and both backfill paths (in-place / drop-and-recreate)
- tests: backfill integration tests over an ephemeral ChromaDB collection

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(search): address review — non-string timestamp guard + top-level authored_at tiebreak

Two correctness fixes from the PR review:

- _extract_authored_at: only compare when the parsed `timestamp` is a str.
  A non-string timestamp on a malformed/foreign JSONL line previously raised
  TypeError outside the try and could crash the mine.
- _hybrid_rank: the tie-break read `authored_at` only from nested `metadata`,
  but the search_memories path (MCP / Claude Code) carries it at the top level
  of each hit — so the tie-break silently no-op'd there. Read both shapes.
- tests: non-string timestamp cases, and a top-level-shape tie-break test
  (which fails before this fix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* style: apply ruff format to authored_at changes

CI ruff format --check flagged 4 files; ruff check already passed.
Formatting only — no behavior change.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Igor Lins e Silva <4753812+igorls@users.noreply.github.com>
2026-06-28 18:13:06 -03:00
.agents/plugins feat: add Codex plugin support with hooks, commands, and documentation 2026-04-08 19:10:44 +03:00
.antigravity-plugin fix(antigravity): sync recall skill protocol with canonical shared source 2026-06-11 13:19:24 +10:00
.claude-plugin chore(release): 3.5.0 2026-06-22 16:39:37 -03:00
.codex-plugin chore(release): 3.5.0 2026-06-22 16:39:37 -03:00
.cursor-plugin chore(release): 3.5.0 2026-06-22 16:39:37 -03:00
.devcontainer feat: add VSCode devcontainer matching CI environment 2026-04-14 15:10:23 -03:00
.github chore(deps): bump docker/setup-buildx-action from 3 to 4 (#1881) 2026-06-28 01:33:23 -03:00
assets
benchmarks Merge origin/develop into feat/benchmark-multilingual 2026-05-24 13:18:08 -03:00
commands feat: add Cursor IDE support (hooks, plugin, skill, docs, tests) 2026-05-27 14:05:40 +10:00
docs feat(convo): preserve authored timestamp from transcripts (#1890) 2026-06-28 18:13:06 -03:00
examples merge: resolve CHANGELOG conflict with develop 2026-06-11 04:28:42 -03:00
hooks fix: tighten local guards and file handling 2026-06-23 23:45:03 -03:00
integrations docs: add Windows backup alternative to corrupt-index recovery (#1843) 2026-06-22 10:57:19 +10:00
landing new landing page 2026-04-16 21:46:03 -03:00
mempalace feat(convo): preserve authored timestamp from transcripts (#1890) 2026-06-28 18:13:06 -03:00
rules feat: add mempalace-recall skill and optional Cursor recall rule 2026-06-10 15:40:35 +10:00
scripts feat(convo): preserve authored timestamp from transcripts (#1890) 2026-06-28 18:13:06 -03:00
skills Merge pull request #1851 from undeadindustries/feat/cursor-checkpoint-tool 2026-06-22 12:37:09 -03:00
tests feat(convo): preserve authored timestamp from transcripts (#1890) 2026-06-28 18:13:06 -03:00
tools Merge origin/develop into feat/benchmark-multilingual 2026-05-24 13:18:08 -03:00
website chore(release): 3.5.0 2026-06-22 16:39:37 -03:00
.dockerignore feat(docker): add container image for MCP server and CLI 2026-06-05 12:32:33 +02:00
.gitignore Merge origin/develop into feat/benchmark-multilingual 2026-05-24 13:18:08 -03:00
.pre-commit-config.yaml Merge origin/develop into feat/benchmark-multilingual 2026-05-24 13:18:08 -03:00
.python-version docs(install): recommend uv as the package manager 2026-05-08 01:38:00 -03:00
AGENTS.md docs: add CLAUDE.md + mission/principles to AGENTS.md (#720) 2026-04-12 15:28:01 -07:00
CHANGELOG.md chore(release): 3.5.0 2026-06-22 16:39:37 -03:00
CLAUDE.md docs(install): recommend uv as the package manager 2026-05-08 01:38:00 -03:00
CONTRIBUTING.md Merge origin/develop into feat/benchmark-multilingual 2026-05-24 13:18:08 -03:00
Dockerfile ci(docker): fix latest/main publishing, add arm64 + GPU build check 2026-06-05 22:39:15 -03:00
Dockerfile.gpu fix(docker): address gemini review on MR #1696 2026-06-05 13:35:36 +02:00
LICENSE
MISSION.md docs: add CLAUDE.md + mission/principles to AGENTS.md (#720) 2026-04-12 15:28:01 -07:00
README.md chore(release): 3.5.0 2026-06-22 16:39:37 -03:00
ROADMAP.md docs: add ROADMAP.md — v3.1.1 stability patch and v4.0.0-alpha plan 2026-04-11 22:05:00 -07:00
SECURITY.md docs: tighten SECURITY.md with real version policy and GHPVR-only channel 2026-04-14 11:50:00 -03:00
docker-compose.yml ci(docker): fix latest/main publishing, add arm64 + GPU build check 2026-06-05 22:39:15 -03:00
docker-entrypoint.sh fix(docker): address gemini review on MR #1696 2026-06-05 13:35:36 +02:00
mcp.json feat: add Cursor IDE support (hooks, plugin, skill, docs, tests) 2026-05-27 14:05:40 +10:00
openarena-claim.txt chore: add OpenArena owner claim verification file 2026-04-24 23:19:29 -03:00
pyproject.toml chore(deps-dev): bump ruff from 0.15.18 to 0.15.20 (#1883) 2026-06-28 01:33:30 -03:00
uv.lock chore(release): 3.5.0 2026-06-22 16:39:37 -03:00

README.md

MemPalace

MemPalace

Local-first AI memory. Verbatim storage, pluggable backend, 96.6% R@5 raw on LongMemEval — zero API calls.

[!CAUTION] Beware of impostor sites. MemPalace has no other official websites. The only official sources are this GitHub repository, the PyPI package, and the docs at mempalaceofficial.com. Any other domain (including .tech, .net, or other .com variants) is an impostor and may distribute malware. Details and timeline: docs/HISTORY.md.

[!IMPORTANT] Claude Code sessions expire in 30 days without auto-save hooks wired. Read this →

Need the shortest recovery/setup path? Use the Claude Code retention setup checklist.


What it is

MemPalace stores your conversation history as verbatim text and retrieves it with semantic search. It does not summarize, extract, or paraphrase. The index is structured — people and projects become wings, topics become rooms, and original content lives in drawers — so searches can be scoped rather than run against a flat corpus.

The retrieval layer is pluggable. The current default is ChromaDB; the interface is defined in mempalace/backends/base.py and alternative backends can be dropped in without touching the rest of the system.

Nothing leaves your machine unless you opt in.

Architecture, concepts, and mining flows: mempalaceofficial.com/concepts/the-palace.


Install

MemPalace ships a CLI, so install it in an isolated environment to avoid PEP 668 errors on Debian/Ubuntu/Homebrew Pythons and to keep mempalace's deps (chromadb, numpy, grpcio, …) from conflicting with anything else in your global site-packages.

We recommend uvuv tool install puts the mempalace CLI in an isolated environment on your PATH:

uv tool install mempalace
mempalace init ~/projects/myapp

pipx works the same way if you prefer it: pipx install mempalace.

Prefer plain pip only inside an activated virtualenv where you explicitly want import mempalace available:

python -m venv .venv && source .venv/bin/activate
pip install mempalace

Docker

A container image is also available for running the MCP server or the CLI without a local Python toolchain. Everything persists under /data (palace, config, and the cached embedding model), so mount a volume there.

# Build the image (CPU; bundles the `extract` + `spellcheck` extras)
docker build -t mempalace .

# MCP server over stdio — note the `-i` flag (JSON-RPC needs stdin)
docker run -i --rm -v mempalace-data:/data mempalace

# Run any CLI command instead (mount the host directory you want to mine)
docker run --rm -v mempalace-data:/data -v /path/to/project:/work mempalace mine /work
docker run --rm -v mempalace-data:/data mempalace search "why GraphQL"

Wire it into an MCP client (e.g. Claude Code) as a stdio server:

{
  "mcpServers": {
    "mempalace": {
      "command": "docker",
      "args": ["run", "-i", "--rm", "-v", "mempalace-data:/data", "mempalace"]
    }
  }
}

docker compose run --rm mcp works too (see docker-compose.yml). For CUDA-accelerated embeddings, build the GPU variant with docker build -f Dockerfile.gpu -t mempalace:gpu . and run it with --gpus all. Customise the bundled extras at build time, e.g. docker build --build-arg EXTRAS="extract,spellcheck" -t mempalace ..

Storage backends

ChromaDB is the default. For the pluggable-backend preview, MemPalace also ships sqlite_exact for local exact-vector correctness checks, and two opt-in external service backends — qdrant (REST) and pgvector (Postgres). The two external backends exercise the storage contract on different substrates (a REST/dict store and a SQL/JSONB store), so it is not accidentally shaped around one vendor.

# local no-service backend
mempalace mine ~/projects/myapp --backend sqlite_exact

# Qdrant backend, defaulting to http://localhost:6333
MEMPALACE_QDRANT_URL=http://localhost:6333 \
  mempalace mine ~/projects/myapp --backend qdrant

# Postgres + pgvector backend, defaulting to postgresql://localhost:5432/mempalace
#   needs the optional driver: pip install mempalace[pgvector]
#   and the `vector` extension available on the server
MEMPALACE_PGVECTOR_DSN=postgresql://localhost:5432/mempalace \
  mempalace mine ~/projects/myapp --backend pgvector

Qdrant can also be configured with MEMPALACE_QDRANT_API_KEY, MEMPALACE_QDRANT_NAMESPACE, and MEMPALACE_QDRANT_TIMEOUT; pgvector with MEMPALACE_PGVECTOR_NAMESPACE. Both external backends isolate tenants by namespace (advertised via the supports_namespace_isolation capability) and write a local marker (qdrant_backend.json / pgvector_backend.json) to guard against silently opening a palace against the wrong server.

When MEMPALACE_QDRANT_URL or MEMPALACE_PGVECTOR_DSN points anywhere other than your own local or trusted self-hosted service, MemPalace will send and store verbatim drawer text and metadata there. That is an explicit opt-in backend choice, never the default.

Quickstart

# Mine content into the palace
mempalace mine ~/projects/myapp                    # project files
mempalace mine ~/.claude/projects/ --mode convos   # Claude Code sessions (scope with --wing per project)

# Search
mempalace search "why did we switch to GraphQL"

# Load context for a new session
mempalace wake-up

For Claude Code, Gemini CLI, Antigravity, MCP-compatible tools, and local models, see mempalaceofficial.com/guide/getting-started.


Benchmarks

All numbers below are reproducible from this repository with the commands in benchmarks/BENCHMARKS.md. Full per-question result files are committed under benchmarks/results_*.

LongMemEval — retrieval recall (R@5, 500 questions):

Mode R@5 LLM required
Raw (semantic search, no heuristics, no LLM) 96.6% None
Hybrid v4, held-out 450q (tuned on 50 dev, not seen during training) 98.4% None
Hybrid v4 + LLM rerank (full 500) ≥99% Any capable model

The raw 96.6% requires no API key, no cloud, and no LLM at any stage. The hybrid pipeline adds keyword boosting, temporal-proximity boosting, and preference-pattern extraction; the held-out 98.4% is the honest generalisable figure.

The rerank pipeline promotes the best candidate out of the top-20 retrieved sessions using an LLM reader. It works with any reasonably capable model — we have reproduced it with Claude Haiku, Claude Sonnet, and minimax-m2.7 via Ollama Cloud (no Anthropic dependency). The gap between raw and reranked is model-agnostic; we do not headline a "100%" number because the last 0.6% was reached by inspecting specific wrong answers, which benchmarks/BENCHMARKS.md flags as teaching to the test.

Other benchmarks (full results in benchmarks/BENCHMARKS.md):

Benchmark Metric Score Notes
LoCoMo (session, top-10, no rerank) R@10 60.3% 1,986 questions
LoCoMo (hybrid v5, top-10, no rerank) R@10 88.9% Same set
ConvoMem (all categories, 250 items) Avg recall 92.9% 50 per category
MemBench (ACL 2025, 8,500 items) R@5 80.3% All categories

We deliberately do not include a side-by-side comparison against Mem0, Mastra, Hindsight, Supermemory, or Zep. Those projects publish different metrics on different splits, and placing retrieval recall next to end-to-end QA accuracy is not an honest comparison. See each project's own research page for their published numbers.

Reproducing every result:

git clone https://github.com/MemPalace/mempalace.git
cd mempalace
uv sync --extra dev   # or: pip install -e ".[dev]"
# see benchmarks/README.md for dataset download commands
uv run python benchmarks/longmemeval_bench.py /path/to/longmemeval_s_cleaned.json

Knowledge graph

MemPalace includes a temporal entity-relationship graph with validity windows — add, query, invalidate, timeline — backed by local SQLite. Usage and tool reference: mempalaceofficial.com/concepts/knowledge-graph.

MCP server

35 MCP tools cover palace reads/writes, knowledge-graph operations, cross-wing navigation, drawer management, and agent diaries. Installation and the full tool list: mempalaceofficial.com/reference/mcp-tools.

Agents

Each specialist agent gets its own wing and diary in the palace. Discoverable at runtime via mempalace_list_agents — no bloat in your system prompt: mempalaceofficial.com/concepts/agents.

Auto-save hooks

Auto-save hooks for Claude Code, Codex CLI, and Cursor IDE save periodically and before context compression:

If you are installing under time pressure, start with the Claude Code retention setup checklist: wire the hooks, back up existing JSONL transcripts, and backfill them with mempalace mine ~/.claude/projects/ --mode convos.

For per-message recall on top of the file-level chunks the hooks produce, run mempalace sweep <transcript-dir> periodically — it stores one verbatim drawer per user/assistant message, idempotent and resume-safe.


Requirements

  • Python 3.9+
  • A vector-store backend (ChromaDB by default)
  • ~300 MB disk for the embedding model. Onboarding (python -m mempalace.onboarding) offers embeddinggemma-300m (multilingual, 100+ languages, recommended) or all-MiniLM-L6-v2 (English-only, ~30 MB). See the docstring at mempalace/embedding.py for details and migration notes.

No API key is required for the core benchmark path.

Docs

Contributing

PRs welcome. See CONTRIBUTING.md.

License

MIT — see LICENSE.