Follow-up to the live-trace audit — all on the enabled path:
- boundary detection LLM (everalgo detect_boundaries) ran with the
SPAN-typed request root as the current span, so its ~1.2k tokens were
dropped from cost. Wrap it in an everos.memcell.boundary GENERATION
span so Langfuse prices it.
- embedding calls stamped usage on the enclosing retriever span. Wrap
each /embeddings call in an everos.embedding EMBEDDING span so the type
is correct and pricing can apply.
- agentic recall emitted a duplicate, same-name everos.search.recall
(cluster_scoped wrapping hybrid_full, which owns the real recall span).
Drop the redundant outer span; hybrid_full keeps the one recall span
(also used standalone in round 2).
- search now captures the returned hit ids (episodes/cases/skills) as
observation output when capture_content is on — previously only the
query input was captured.
- add/flush spans carry request_id in metadata.
- persist captures the memory-root-relative .md path, not the host
absolute path (no host layout leak to the telemetry backend).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adversarial review of the merged OTel instrumentation (#352) surfaced
four issues, all on the enabled path (default-off, so no impact until
tracing is turned on):
- search LLM client was unwrapped, so hybrid/agentic token usage — the
heaviest LLM spend — never reached Langfuse. Wrap it with
UsageRecordingClient when observability is enabled, mirroring
get_llm_client(); graceful keyword-only degradation is preserved.
- set_generation_usage overwrote token counts, undercounting any span
that wraps more than one chat call (the now-wrapped agentic path).
Accumulate instead of replacing.
- recall_hit was emitted for uncalibrated methods (unbounded BM25 /
single-route vector), a near-constant always-hit signal that inflates
dashboards. Gate hit on calibrated methods (HYBRID/AGENTIC); keyword
and vector emit only the raw top_score.
- init_tracing / init_score_sink were not idempotent — a re-init without
an intervening shutdown orphaned the export thread + OTLP socket +
worker task. Tear down the previous instance first (init_score_sink is
now async).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every business endpoint (memory/*, ome/*, knowledge/*) is now served
under /api/v2, aligning the open-source API with the EverOS Cloud
contract. /api/v1 is retained as a permanent, backward-compatible alias:
the same router objects are mounted under both prefixes, so both resolve
to identical handlers and request/response contracts. Existing /api/v1
integrations keep working unchanged. Infra endpoints (/health, /metrics)
stay unversioned.
Fix the Prometheus request-metric label to build the path from the full
request URL (with path params folded) rather than the route's
router-relative path, so the version prefix is preserved and v1/v2
traffic stays distinguishable.
Docs (docs/api.md, docs/openapi.json), CHANGELOG, and route docstrings
updated to lead with /api/v2. Add test_api_versioning as the parity
guard: every v2 route has an identical v1 twin and vice versa.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Open spans at the memory hot paths (all no-op when tracing is off):
- add / flush (service.memorize), extract + persist.markdown (user pipeline).
- search: everos.memory.search retriever + a uniform recall / rank
decomposition across keyword / vector / hybrid / agentic (manager, agentic
modules, cross-encoder callbacks); query-embedding tokens land on recall.
- recall quality: top_score / hit on the search span, plus recall_top_score /
recall_hit pushed to Langfuse scores via the bounded-queue sink (method
tagged; off the request path).
- OME: everos.ome.<strategy> agent span + everos.reflect.consolidate
generation; a W3C traceparent captured at enqueue is threaded through the
APScheduler job and re-attached in the Runner, so strategies fanned out
from a request nest under that request's trace.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surface gen_ai.* model + token attributes onto the active span so Langfuse
can compute cost — without touching everalgo:
- UsageRecordingClient wraps the LLM client and records response.usage after
each chat(); get_llm_client composes it over the existing _LoggingLLMClient
only when observability is enabled (disabled default stays overhead-free).
- OpenAIEmbeddingProvider records its response.usage (input tokens) onto the
active span too.
Tokens land on the everos.extract / everos.reflect.consolidate generation
spans and the search embedding recall; no-op when tracing is off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(examples): make Langfuse wrapper degrade cleanly on a real server
The wrapper synthesized child spans (extraction, embedding, hybrid
recall, rerank, index sync, consolidation) from a mock-only `_detail`
field. Against a real EverOS server that field is absent, so those spans
rendered with placeholder data — hardcoded model names, token=0, fixed
sleep durations — and recall scores fell to 0.
Now the per-stage child spans are emitted only when `_detail` is present
(the mock, or future native in-core instrumentation). Against a live
server only the top-level span per operation is emitted, with real
latency and output — no fabricated data. Recall quality
(recall_top_score / recall_hit) is derived from the real search
response, which already carries a per-hit score, so it works against a
live server today, not just the mock.
Verified: mock path unchanged (full trace tree, real scores); real-ish
path (no `_detail`) emits only top-level spans plus a real recall score.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(examples): count empty recalls as a miss in Langfuse hit-rate
When a search returns nothing scored, record recall_hit=0 (span attribute +
Langfuse score) instead of omitting it, so genuine empty recalls still show
up in recall hit-rate. No top_score is emitted (there is no hit to score).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds examples/langfuse/ — a thin OpenTelemetry wrapper that traces EverOS
memory operations (add / flush+extract / search / reflection) into Langfuse,
with recall quality pushed as Langfuse scores. Pure OTel SDK, no Langfuse
package dependency; runs against a built-in mock or a real EverOS server
(EVEROS_BASE_URL). Additive only, no changes to EverOS core.
Referenced by the upcoming Langfuse docs integration cookbook.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
FTS indexes built with with_position=True crash lance's optimize/compaction
on lancedb >= 0.32 when merging an unindexed tail (Max offset exceeds length
of values; upstream lance-format/lance#7653). The crash aborts optimize()
including version cleanup, so the index dir grows unbounded until the disk
fills. everos recall is OR-mode BM25 and never does phrase queries, so
positions are never read -- disabling is lossless.
- base: default with_position=False
- infra: migrate_fts_indexes() rebuilds pre-fix indexes once at startup + reclaims orphans
- cascade worker: count consecutive optimize failures, escalate warning->error
Fixes#335.
Co-authored-by: zhanghui <zhanghui@shanda.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
compile_filters() unconditionally appended 'deprecated_by IS NULL' to
every query, but the deprecated_by column only exists on user-scoped
tables (episode, atomic_fact — Reflection V1). Agent tables
(agent_case, agent_skill) lack this column, causing a SQL error on
agent search/get queries.
Gate the clause behind owner_type == 'user' so agent queries no longer
reference a non-existent column.
Bump version to 1.1.2.
Co-authored-by: Jiayao Song <jiayao.song@shanda.com>
* docs: re-link orphaned docs and slim engineering.md
After #311 realigned the docs, index.md again omitted four files that
exist under docs/: everos-demo, use-cases, migration-to-1.0.0, and
release-notes-1.1.0. Restore them so every docs/*.md is reachable from
the index, and rewrite engineering.md for an external audience.
index.md:
- Re-add a Tutorials section: everos-demo, use-cases
- See also: + release-notes-1.1.0, + migration-to-1.0.0
- Reframe the Engineering section as contributor-facing (not "internal")
engineering.md (575 -> 113 lines):
- Drop internal-only material: the self-justifying scope rationale, the
Claude Code loading internals, the infra failure-impact table, the
roadmap, and the "investing in infrastructure" essay
- Fix claims that were false for this GitHub repo: GitLab-primary CI,
the dev/master branch model, and the Gitmoji commit convention
- Keep what helps a contributor: toolchain, local make targets, the CI
gates, and the main-branch + Conventional Commits workflow
Conventions now match the repo's own .gitlint and GitHub Actions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: align engineering.md references with the slimmed doc
Update the inbound descriptions of engineering.md now that it is a
contributor reference rather than an "infrastructure overview", and
repoint the one reference that named content the trim removed.
- CLAUDE.md, README, README.zh-CN, architecture.md: reword the link
text to "contributor engineering reference: build, test, CI, conventions"
- CLAUDE.md: the GitFlow Lite rationale pointer now targets
.claude/skills/new-branch/SKILL.md (engineering.md no longer carries it)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
index.md only linked 12 of the 19 docs under docs/. Readers entering
from the index missed configuration, multimodal, demo, use-cases,
benchmark, migration, and release notes.
- Add a Tutorials section (Diátaxis 4th quadrant): everos-demo, use-cases
- Reference: + configuration, + multimodal
- How-to: + locomo_benchmark, + migration-to-1.0.0
- See also: + release-notes-1.1.0
Every docs/*.md (excluding the openapi.json artifact) is now linked.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three internal documentation references pointed at non-existent targets:
- docs/api.md: MessageItem.content linked to #addmessage, which has no
heading or anchor; corrected to #messageitem (the slug used by every
other MessageItem cross-reference and matching the ### MessageItem
heading).
- docs/cascade_runbook.md: the FD-exhaustion cross-ref used a single
hyphen where the GitHub slug of "FD exhaustion (`os error 24` /
EMFILE)" has a double hyphen (from the ` / ` separator); corrected to
#fd-exhaustion-os-error-24--emfile.
- use-cases/claude-code-plugin/skills/memory-tools.md: the always-injected
skill named two tools (search_memories, get_memory) that the MCP server
never exposes; replaced with the real evermem_search tool and its
params (query required, limit default 10 / max 20).
Markdown-only; no runtime behavior change.
* docs(readme): refine repository watch section
Replace the Stay Tuned section with a clearer Watch EverOS call to action in both English and Chinese READMEs.
Remove the star GIF and keep star history as lighter social proof.
* docs(readme): improve highlights table spacing
Add spacing below each feature title in the EverOS highlights table.
Mirror the same table spacing in the Chinese README.
Rename the overview section to focus on why EverOS matters in both English and Chinese READMEs.
Simplify the EverMind ecosystem copy around EverOS as the core memory architecture and mirror that positioning in the Chinese README.
md-first memory extraction framework for AI agents.
Markdown is the single source of truth; SQLite holds state and LanceDB
provides the rebuildable vector + BM25 + scalar index. The codebase follows
a single-direction DDD layering (entrypoints -> service -> memory -> infra,
with component / core / config cross-cutting) enforced by import-linter.
Engineering surface:
- Coding conventions in .claude/rules/ (path-scoped) and workflows in
.claude/skills/ (/commit, /new-branch, /pr).
- GitHub Actions CI runs make lint + test + integration; pre-commit mirrors
the gates locally (ruff, hygiene hooks, gitlint commit-msg).
- Commit messages follow Conventional Commits, enforced by gitlint.
- make lint also enforces datetime two-zone discipline and OpenAPI drift.
zhanghui
zhanghui
Elliot Chen
Dani
Kendrick-Song
Elliot Chen
Yangtze-Seventh
0xVox