d256048a6d
* fix(cascade): per-kind prune staleness + rebuild safety Adversarial review of the review-response fixes found five real defects, all in code this PR introduced. Health signal (P1): prune staleness reported the time since the NEWEST successful prune across kinds, so on a multi-kind deployment (every real one) a single kind whose cleanup died was masked by the others pruning on schedule — /health stayed green while that table's index dir grew unbounded, the exact incident the signal exists to catch. Report the WORST kind instead and name it in the reason. The failure streak could not cover this either: an intervening light beat resets it, so it never reaches the threshold for a prune-only failure. Documented that split of duties. Spurious fallback rebuilds (P1): the benign-conflict carve-out excluded the heavy beat, justified by "runs under the write lock, so it can't hit this benignly" — but that lock is in-process only, so a second process (a long `cascade backfill`, a `cascade sync`) preempts prune's Rewrite commit. Those counted as real failures, and ~25min of cross-process churn reached the threshold and fired a fallback rebuild, which drops every index before recreating it; a rebuild that also lost the race was swallowed as a warning, leaving the table with no FTS index (every /search on that kind 500s) until the next 12h sweep. Treat commit conflicts as benign on both beats and let prune-staleness detect a prune that genuinely stops succeeding. cascade rebuild (P1 ×2 + P2): it drops and recreates tables with no guard while --help/docstrings advertised it as safe, so `rebuild --yes` against a live daemon corrupted the rebuild (the daemon keeps writing through cached handles). Refuse when the OME jobstore lock is held, reusing backfill's detection and its exit code 3. It also ran the pre-drop migration pass (`ensure_business_indexes`) against the damaged table, so on the corruption classes it exists to repair (missing column, un-alterable type) the recovery path died on the damage itself — skip it via `_runtime(ensure=False)`. Reset the queue BEFORE dropping so every crash window converges on "queue pending → re-index" instead of empty tables with a fully-done queue (a silently empty deployment), and handle Ctrl-C with exit 130 plus a resume hint. Recovery guidance (P1): the nullable-vector migration error still told users to wipe the index directory — which this PR's own runbook documents as the wrong recovery (queue stays done, index comes back empty). Point it at `everos cascade rebuild`. Dropped the schema-drift error's "restart first" step too: the startup migrations only alter nullability, never a name or type, so a name/type drift never self-heals. Also: backfill's post-write prune passed a zero retention window from a separate process, able to delete files under a daemon /search still holding that version — pass the daemon's window instead. Runbook gains the /health cascade block (thresholds, what flips healthy, why failed_permanent does not) and its quoted schema-drift error now matches the code. Tests: the three safety mechanisms this PR adds were unpinned — a one-line revert of any of them passed the suite. Added per-kind staleness, heavy-beat benign conflict, benign-filter negative case (an error whose message merely contains "retryable" must still count), prune recurrence across light beats (mutation-verified: hoisting the attempt-clock advance out of the heavy branch turns it red), the prune timeout releasing the write lock, timeout-below-cadence, the rebuild server guard, and a tier3 assertion that the /health cascade block is actually wired. Froze the last fabricated-monotonic test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Also: treat ``last_prune_attempt_at == 0.0`` as "never attempted" instead of comparing clocks. ``monotonic()`` is boot-relative, so ``now - 0 >= cadence`` is false for the first ~cadence of container uptime — the catch-up prune was skipped exactly when a fresh process most needs it (and made a test depend on the runner uptime, which CI caught). * fix(lancedb): bound every write-lock critical section run7 (1h at 2.5x rate, concurrent CLI maintenance, doubled fuzz) reproduced a table whose version cleanup stopped permanently: 150 versions retained, disk 11x live size, while the other two tables sat at 1 version each — and with no error logged anywhere, because nothing failed. It simply never returned. Three things combined. The maintenance scheduler allows one task per table (a LanceDB table takes one writer), so it skips a kind whose task is still in flight. The prune timeout sat *inside* the lock and covered only the cleanup call. And the other six critical sections on that lock — add, upsert, update, delete, delete_by_md_path, rebuild_indexes — had no deadline at all. So one operation stuck anywhere outside that narrow window wedged the table for good: every writer blocked on acquire, and every later heartbeat was turned away because the stuck task never finished. Make it structurally impossible instead of patching prune: all seven sections now go through `LanceRepoBase._locked(budget, op)`, where the deadline covers **acquisition and the body**. No path can wait for this lock, or hold it, indefinitely. Budgets are hang-catchers, not throughput limits: 120s for row writes, 600s for an index rebuild, the existing 60s for prune. Expiry raises `VectorStoreBusyError`, deliberately under `ExternalServiceError` so the cascade worker retries the row; under `VectorStoreError` a transient lock contention would be marked permanently failed and need a manual `cascade fix`. Tests: a stuck holder now makes a waiter fail its deadline and release (the lock is reusable afterwards), and the prune timeout is pinned as retryable. Verified by mutation — moving the timeout back inside the lock makes a waiter block until the enclosing observation window expires (1001ms vs 51ms), i.e. wait forever in production. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(lancedb): size write-lock budgets from measurements 120s for a row write was a guess, and a bad one: the budget doubles as the detection latency for a wedged table, so an over-slack value means minutes of blocked writers before anything surfaces — the failure this change exists to prevent. Measured the four locked write ops on a local SSD across table sizes and batch sizes (10k-100k rows, 50-500 rows per call): add 3-22ms, upsert (merge_insert, the read-modify-write one) 6-25ms, update 2-4ms, delete 2-3ms; worst observation 63ms, and flat in both dimensions since these are append-and-commit, not scans. So: writes 120s -> 15s (~240x the worst observation, enough for a contended disk and several waiters queued ahead — the deadline includes acquisition and asyncio.Lock is FIFO), rebuild 600s -> 300s (still the one genuinely slow section at ~0.3s per 50k rows per indexed column). Prune stays 60s. Test pins the sizing intent: writes stay in the tens of seconds, and rebuild > prune > write so the slowest section is not the most eagerly killed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(lancedb): record wait/hold time on write-lock critical sections A soak run stalled one table's writes for ~16s and the logs could not say why: maintenance beats only log at `debug`, so a section that is slow but still inside its deadline is invisible, and the timeout warning did not distinguish "never acquired the lock" from "acquired it and overran". `_locked` now carries that apart. The deadline warning gains `acquired`, `waited_seconds` and `held_seconds` — `acquired` alone answers whether a holder was slow or this operation was — and a completed section that held the lock for at least a second logs `lancedb_write_lock_slow_hold` at info, so a stall that never reaches a deadline still leaves a trace. Uses `time.monotonic` (elapsed measurement, not wall clock — the datetime discipline bans `time.time`). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(search): reject a mismatched query vector before it reaches LanceDB A soak run showed every slow search was a failing search: `search:vector` p50 251ms / p99 1.6s, but its 8 requests over 10s were exactly its 8 failures (13-14s each). The cause was a query vector whose width disagreed with the index. LanceDB only notices after the query is built and reports it as an opaque `ValueError: Invalid input, No vector column found to match…`, which escaped as an unhandled 500. Validate at `_embed_query` — the single point every query vector passes through — against the provider's declared `dim`. Microseconds instead of 13s, and a named `ConfigurationError` (500 + CONFIGURATION_ERROR) instead of an unhandled crash. Deliberately not `InvalidInputError`/422: callers only send query *text*, so a bad width is our provider's fault, not the caller's. Also cap traceback rendering. structlog's default is `RichTracebackFormatter(show_locals=True, max_frames=100, extra_lines=3)`, which on an async stack rendered 82 frames into 6423 log lines per exception — 85MB of server.log across 11 of them — at ~290ms of synchronous CPU each, and risks printing request payloads into logs. With locals off and 15 frames the same traceback is 103 lines and 10ms. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(changelog): record the storage-reliability work under Unreleased #379 merged without changelog entries, so this covers both it and the follow-up work in this branch: the maintenance split (compaction vs reclamation) that fixes unbounded index growth, bounded write-lock critical sections, the /health cascade readiness block and its alert contract, `cascade rebuild`, schema type-drift detection, the query-vector width check, and the traceback-rendering cap. Each entry states the operator-visible consequence, not just the change — `cascade rebuild` now refusing to run against a live server, benign-conflict warnings dropping in volume, and `/health` being able to report a stalled kind that was previously invisible are all behaviour changes someone will notice. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: zhanghui <zhanghui@shanda.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|---|---|---|
| .claude | ||
| .github | ||
| benchmarks | ||
| data | ||
| docs | ||
| examples/langfuse | ||
| scripts | ||
| src/everos | ||
| tests | ||
| use-cases | ||
| .env.example | ||
| .gitignore | ||
| .gitlint | ||
| .pre-commit-config.yaml | ||
| ACKNOWLEDGMENTS.md | ||
| CHANGELOG.md | ||
| CITATION.md | ||
| CLAUDE.md | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| LICENSE | ||
| Makefile | ||
| NOTICE | ||
| QUICKSTART.md | ||
| README.md | ||
| README.zh-CN.md | ||
| SECURITY.md | ||
| config.example.toml | ||
| pyproject.toml | ||
| uv.lock | ||
README.md
Why Ever OS
EverOS is a Python library and local-first memory runtime for agents and makers. It gives one portable memory layer across coding assistants, apps, devices, and workflows from day one. It stores conversations, files, and agent trajectories as readable Markdown, then syncs local SQLite and LanceDB indexes for fast retrieval and self-evolving reuse.
| Title | EverOS | Other Agent Memory Libraries |
|---|---|---|
| Markdown source of truth | ✅ Canonical .md files that are readable, editable, diffable, and Git-versioned |
❌ Usually API, vector, graph, dashboard, or database state |
| Direct file editing | ✅ Edit .md files; cascade watcher syncs |
❌ Usually SDK, API, dashboard, or backend update paths |
| Local three-part stack | ✅ Markdown + SQLite + LanceDB; no MongoDB, Elasticsearch, or Redis required | ❌ Often depends on managed services, vector DBs, graph DBs, or server stacks |
| User + agent tracks | ✅ User episodes/profile and agent cases/skills are separate first-class surfaces |
❌ Usually centered on chat history, profiles, entities, facts, or retrieval records |
| Orthogonal retrieval | ✅ Search by user_id, agent_id, app_id, project_id, and session_id |
❌ Usually app, namespace, tenant, thread, or graph scoped |
| Knowledge Wiki | ✅ Editable, source-backed Markdown knowledge pages with taxonomy, CRUD APIs, and topic search | ❌ Usually separate from memory, trapped in a dashboard, or not tied back to source files |
| Reflection | ✅ Offline memory evolution that merges episode clusters and refines profiles and skills between sessions | ❌ Usually retrieval-only memory with little background consolidation or long-horizon improvement |
Quick Start
Goal: play with the memory visualizer first, then start EverOS, write one real memory, and search it back.
0. Prerequisites
- Python 3.12+
- No API keys are needed for
everos demo. - To run the real server-backed memory flow, create two provider keys before
everos init:
| Capability | Provider | Used for | Fill these .env slots |
|---|---|---|---|
| Chat + multimodal | OpenRouter | LLM / MULTIMODAL |
EVEROS_LLM__API_KEY, EVEROS_MULTIMODAL__API_KEY |
| Embedding + rerank | DeepInfra | EMBEDDING / RERANK |
EVEROS_EMBEDDING__API_KEY, EVEROS_RERANK__API_KEY |
You can use other OpenAI-compatible providers by changing the matching
*__BASE_URL fields in .env.
1. Install
uv pip install everos
# or: pip install everos
2. Play With The Demo
Run this before configuring API keys or starting the server:
everos demo
The command asks for one memory and one recall question, then opens a full-screen terminal UI. This is an educational visualizer: it is hardcoded, local to the CLI, and does not connect to the EverOS server. Its job is to make the memory lifecycle visible: conversation -> memory sphere -> recall -> source proof -> confetti. See docs/everos-demo.md for the demo scope and TUI source layout.
The sphere moves through ingest, extraction, indexing, recall, source reveal,
and a confetti burst after the first memory lands. Press r to replay and q
to quit.
For the looping showroom view used in README media, run:
everos demo --cinematic
If your shell is not interactive, or you want a copyable preview, use:
everos demo --plain
3. Configure
Generate a starter .env file, then fill the four API key slots shown in the
generated comments. With the default setup, paste your OpenRouter key into the
LLM / MULTIMODAL slots and your DeepInfra key into the EMBEDDING /
RERANK slots.
everos init
# or, from a source checkout:
cp .env.example .env
everos init writes ./.env by default. Use everos init --xdg to
write ${XDG_CONFIG_HOME:-~/.config}/everos/.env instead.
4. Start EverOS
everos server start
Keep the server running, then open a second terminal and check it:
curl http://127.0.0.1:8000/health
Expected response:
{"status":"ok"}
everos server start searches for .env in this order: --env-file <path> →
./.env (cwd) → ${XDG_CONFIG_HOME:-~/.config}/everos/.env → ~/.everos/.env.
The endpoint stack is OpenAI-protocol compatible (OpenAI / OpenRouter / vLLM /
Ollama / DeepInfra) - override *__BASE_URL in the generated .env to point
at any of them.
Now make the demo real. In the second terminal, run:
everos demo --live
Live demo mode connects to the running server and performs the real
/health -> /api/v2/memory/add -> /api/v2/memory/flush ->
/api/v2/memory/search flow before opening the same memory sphere UI. Use
--server-url <url> if your server is not on http://127.0.0.1:8000.
5. Try Your First Memory
[!NOTE] Business endpoints live under
/api/v2. The older/api/v1prefix still resolves to the same handlers so existing integrations keep working, but it is a legacy alias that may be removed in a future major release — write new code against/api/v2.
Add a tiny conversation:
TS=$(($(date +%s)*1000))
curl -X POST http://127.0.0.1:8000/api/v2/memory/add \
-H 'Content-Type: application/json' \
-d "{
\"session_id\": \"demo-001\",
\"app_id\": \"default\",
\"project_id\": \"default\",
\"messages\": [
{\"sender_id\": \"alice\", \"role\": \"user\", \"timestamp\": $TS, \"content\": \"I love climbing in Yosemite every spring.\"},
{\"sender_id\": \"alice\", \"role\": \"user\", \"timestamp\": $((TS+10000)), \"content\": \"My favorite coffee shop is Blue Bottle in SOMA.\"}
]
}"
Force extraction for the local demo:
curl -X POST http://127.0.0.1:8000/api/v2/memory/flush \
-H 'Content-Type: application/json' \
-d '{"session_id":"demo-001","app_id":"default","project_id":"default"}'
Search it back:
curl -X POST http://127.0.0.1:8000/api/v2/memory/search \
-H 'Content-Type: application/json' \
-d '{
"user_id": "alice",
"app_id": "default",
"project_id": "default",
"query": "Where do I like to climb?",
"top_k": 5
}'
You should see the Yosemite memory in the response. If the result is empty on the first try, wait a moment and retry; Markdown is written synchronously, while the local index catches up in the background.
[!TIP] First memory unlocked. You just gave EverOS a fact, flushed it into durable Markdown-backed memory, and searched it back through the local index. That is the core loop. Want to see the source of truth? Open
~/.everosand inspect the generated Markdown files.
For annotated responses and the Markdown files EverOS creates, see QUICKSTART.md.
Optional: Ingest Multimodal Files
To ingest non-text content (image / pdf / audio / office documents)
through /api/v2/memory/add content items, install the optional
extra:
uv pip install 'everos[multimodal]' # or: pip install 'everos[multimodal]'
This pulls in everalgo-parser (with the [svg] bundle for SVG
support via cairosvg) and wires up the multimodal LLM client
(EVEROS_MULTIMODAL__* fields in .env, defaults to
google/gemini-3-flash-preview via OpenRouter).
Office document support requires LibreOffice as a system dependency.
The parser shells out to soffice (LibreOffice's headless renderer) to
convert .doc / .docx / .ppt / .pptx / .xls / .xlsx to PDF
before feeding the result into the multimodal LLM. Without LibreOffice,
office uploads return HTTP 415 with a clear error message; PDF / image
/ audio / HTML / email parsing is unaffected.
Install on the host before serving office documents:
brew install --cask libreoffice # macOS
sudo apt-get install -y libreoffice # Debian / Ubuntu
For Contributors
git clone https://github.com/EverMind-AI/EverOS.git
cd EverOS
uv sync # creates ./.venv and installs deps
source .venv/bin/activate # or prefix commands with `uv run`
everos demo --plain # try the local educational demo; no API keys needed
everos init # paste OpenRouter + DeepInfra keys into .env
everos --help
make test
Use Cases
Now that you have had your first successful EverOS moment, explore what people are building with persistent memory across agents, apps, and community integrations.
Use cases show what persistent memory makes possible in real products and workflows. Some examples are packaged in this repository; others point to external demos or integrations you can study and adapt.
Reunite - Find With EverOSParents describe what they remember. Children describe what they recall. Reunite uses semantic memory to surface the connections. |
Hive OrchestratorBrowser-native hive-mind for CLI coding agents - Claude Code, Codex, Gemini, and OpenCode collaborate as real PTY processes via a team protocol. |
AI Coding Assistants With EverOSUniversal long-term memory layer for AI coding assistants, powered by EverOS. |
AI Data TechnicianAn agentic AI system that learns from scientist interaction to inspect, analyze, and classify high-dimensional time series data - with persistent memory that improves across sessions. |
Rokid AI Assistant With EverOSConnect to EverOS within Rokid Glasses enabling long-term memory for all of your smart activities. Coming soon |
Creative Assistant With MemoryCreative assistant with long-term memory, so your creative context stays available across sessions. Coming soon |
|
|
Earth Online Memory GameEarth Online is a memory-aware productivity game that turns everyday planning into a living quest log. |
Multi-Agent Orchestration PlatformGolutra presents a multi-agent workforce for engineering teams, extending the IDE model from a single assistant to coordinated agents. |
Your Personal Tasting UniverseRecord, visualize, and explore your tasting journey through an immersive 3D star map. |
EverOS Open HerBuild AI that feels. Open-source persona engine - personality emerges from neural drives, not prompts. Inspired by Her. |
Browser Agent For Personal MemoryRuminer brings persistent memory to a browser agent so it can carry personal context across web tasks. |
EverMem Sync With EverOSOne command to connect any AI coding CLI to EverMemOS long-term memory. |
|
|
|
MCO - Orchestrate AI Coding AgentsMCO equips your primary agent with an agent team that can work together to solve complex tasks. |
Study Buddy With Self-Evolving MemoryStudy proactively with an agent that has self-evolving memory. |
Alzheimer's Memory AssistantEmpowering individuals with advanced memory support and daily assistance. |
Memory-Driven Multi-Agent NPC ExperienceAn iOS sci-fi mystery game where players explore and uncover the truth. |
Mobi CompanionAn iOS app where users create, nurture, and live with a personalized AI companion called Mobi. |
AI Wearable With MemoryA context-native AI wearable that listens to everyday life and converts conversations into memory. |
|
|
|
Legacy OpenClaw Agent MemoryArchived pre-1.0.0 plugin reference. New integrations should use the current EverOS API. |
Live2D Character With MemoryAdd long-term memory to a real-time Live2D character, powered by TEN Framework. |
Computer-Use With MemoryRun screenshot-based analysis with computer-use and store the results in memory. |
Game Of Thrones MemoriesA demonstration of AI memory infrastructure through an interactive Q&A experience with A Game of Thrones. |
Claude Code PluginPersistent memory for Claude Code. Automatically saves and recalls context from past coding sessions. |
Memory Graph VisualizationExplore stored entities and relationships in a graph interface. Frontend demo; backend integration is in progress. |
Documentation
- docs/everos-demo.md — Demo scope and TUI source layout
- docs/how-memory-works.md — Markdown, SQLite, LanceDB, and recall flow
- docs/use-cases.md — Full use-case gallery and integration examples
- docs/engineering.md — Contributor engineering reference: build, test, CI, conventions
- docs/migration-to-1.0.0.md — Legacy API migration notes
- CHANGELOG.md — Release notes
- CONTRIBUTING.md — How to contribute
EverMind Ecosystems
EverMind is an open-source ecosystem for long-term memory, self-evolving agents, AI-native interfaces, and memory evaluation.
| EverMind Open-Source Ecosystem | |
|---|---|
| Memory Runtime | EverOS - the local memory operating system and research-backed runtime for agent and user memory. |
| Self-Improving Agent Harness | Raven - the self-improving agent harness that brings memory, proactivity, context control, and skill evolution into terminal-native agents. |
| Algorithm Engine | EverAlgo - stateless extraction, ranking, parsing, and memory operators that power EverOS. |
| Hypergraph Memory | HyperMem - hypergraph memory for long-term conversations, with its own benchmark-backed topic -> episode -> fact retrieval method. |
| Benchmarks | EverMemBench · EvoAgentBench - evaluation suites for conversational memory and agent self-evolution. |
| Long-Context Research | MSA - Memory Sparse Attention for scalable latent memory and 100M-token contexts. |
| Personal Memory Layer | EverMe - CLI and agent plugin suite for cross-device, cross-agent personal memory. |
| Developer Integrations | evermem-claude-code · everos-plugins - plugins, skills, and migration tooling for AI coding agents. |
Together, these repositories form EverMind's research-to-runtime stack: new memory methods, reusable algorithms, benchmark evidence, and practical agent integrations.
Contributing
Contributions are welcome across the whole repository: memory methods, benchmark coverage, use-case examples, documentation, and bug fixes. Browse Issues to find a good entry point, then open a PR when you are ready.
[!TIP]
Welcome all kinds of contributions 🎉
Help make EverOS better. Code, documentation, benchmark reports, use-case write-ups, and integration examples are all valuable. Share your projects on social media to inspire others.
Connect with one of the EverOS maintainers @elliotchen200 on 𝕏 or @cyfyifanchen on GitHub for project updates, discussions, and collaboration opportunities.
Code Contributors
License
Apache License 2.0 — see NOTICE for third-party attributions.
Citation
If you use EverOS in research, see CITATION.md.