EverOS/examples/langfuse
Dani d3a9f9e394
docs(examples): make Langfuse wrapper degrade cleanly on a real server (#342)
* docs(examples): make Langfuse wrapper degrade cleanly on a real server

The wrapper synthesized child spans (extraction, embedding, hybrid
recall, rerank, index sync, consolidation) from a mock-only `_detail`
field. Against a real EverOS server that field is absent, so those spans
rendered with placeholder data — hardcoded model names, token=0, fixed
sleep durations — and recall scores fell to 0.

Now the per-stage child spans are emitted only when `_detail` is present
(the mock, or future native in-core instrumentation). Against a live
server only the top-level span per operation is emitted, with real
latency and output — no fabricated data. Recall quality
(recall_top_score / recall_hit) is derived from the real search
response, which already carries a per-hit score, so it works against a
live server today, not just the mock.

Verified: mock path unchanged (full trace tree, real scores); real-ish
path (no `_detail`) emits only top-level spans plus a real recall score.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(examples): count empty recalls as a miss in Langfuse hit-rate

When a search returns nothing scored, record recall_hit=0 (span attribute +
Langfuse score) instead of omitting it, so genuine empty recalls still show
up in recall hit-rate. No top_score is emitted (there is no hit to score).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 07:02:16 +08:00
..
.gitignore docs(examples): add Langfuse (OpenTelemetry) integration example (#339) 2026-07-14 07:46:29 +08:00
README.md docs(examples): add Langfuse (OpenTelemetry) integration example (#339) 2026-07-14 07:46:29 +08:00
demo.py docs(examples): add Langfuse (OpenTelemetry) integration example (#339) 2026-07-14 07:46:29 +08:00
everos_langfuse.py docs(examples): make Langfuse wrapper degrade cleanly on a real server (#342) 2026-07-15 07:02:16 +08:00

README.md

EverOS × Langfuse (OpenTelemetry)

Trace EverOS memory operations — writes, LLM extraction, recall with quality scores, and reflection — into Langfuse as OpenTelemetry spans, so an agent's memory layer becomes visible and evaluable next to the rest of its traces.

This is a thin, dependency-light wrapper (pure OpenTelemetry SDK, no Langfuse package dependency). The same spans work with Langfuse Cloud, self-hosted Langfuse, or any other OTLP backend.

Files

  • everos_langfuse.py — the instrumentation wrapper (init_tracing, InstrumentedEverOS, HTTPTransport, recall-score push).
  • demo.py — a runnable end-to-end example. Ships a mock transport, so it runs with no EverOS server required; set EVEROS_BASE_URL to trace a real one.

Span model

EverOS operation Langfuse observation
POST /api/v1/memory/add span everos.memory.add
POST /api/v1/memory/flush → extraction span + generation everos.extract (model + tokens)
markdown persistence span everos.persist.markdown
async index sync span everos.cascade.index (separate correlated trace)
POST /api/v1/memory/search retriever everos.memory.search
↳ embedding / hybrid recall / rerank embedding / retriever / span
POST /api/v1/ome/trigger agent everos.ome.<strategy> + generation

langfuse.session.id / langfuse.user.id are set on every span; recall quality is pushed as Langfuse scores (recall_top_score, recall_hit).

Run

pip install opentelemetry-sdk opentelemetry-exporter-otlp requests

export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="https://us.cloud.langfuse.com"   # EU: https://cloud.langfuse.com

python demo.py
  • With no keys set, demo.py still runs against the built-in mock and writes a local spans.jsonl (offline inspection) — nothing is sent anywhere.
  • With Langfuse keys set, the same spans and recall scores flow into your Langfuse project. Open Tracing → Traces (filter by tag everos / memory).
  • To trace a real deployment, set EVEROS_BASE_URL to a running EverOS server (see the EverOS quickstart); the instrumentation is identical.

Privacy

Spans carry non-sensitive metadata (latency, token counts, model names, scores) by default. Capturing raw query or memory content as span input/output is opt-in. The demo uses synthetic data, and its public_traces flag (safe only for synthetic data) marks the resulting traces as publicly shareable.

Learn more