Adversarial review of the merged OTel instrumentation (#352) surfaced
four issues, all on the enabled path (default-off, so no impact until
tracing is turned on):
- search LLM client was unwrapped, so hybrid/agentic token usage — the
heaviest LLM spend — never reached Langfuse. Wrap it with
UsageRecordingClient when observability is enabled, mirroring
get_llm_client(); graceful keyword-only degradation is preserved.
- set_generation_usage overwrote token counts, undercounting any span
that wraps more than one chat call (the now-wrapped agentic path).
Accumulate instead of replacing.
- recall_hit was emitted for uncalibrated methods (unbounded BM25 /
single-route vector), a near-constant always-hit signal that inflates
dashboards. Gate hit on calibrated methods (HYBRID/AGENTIC); keyword
and vector emit only the raw top_score.
- init_tracing / init_score_sink were not idempotent — a re-init without
an intervening shutdown orphaned the export thread + OTLP socket +
worker task. Tear down the previous instance first (init_score_sink is
now async).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>