* fix(lancedb): reclaim stale versions via write-locked prune The storage soak (48h, sustained churn + fuzz) proved the bundled lock-free `optimize(cleanup_older_than=...)` loses its commit-conflict race against concurrent writes — cleanup ran only 16 of ~250 times, so old dataset versions / FTS orphans piled up and the index dir grew to the 40G disk guardrail and never reclaimed under load. main still had that bundled call. Split the maintenance path: - `LanceRepoBase.optimize()` is now compact-only and lock-free (a commit conflict here is benign — the next beat retries, so it must not stall writers). - `LanceRepoBase.prune(older_than)` runs `cleanup_older_than + delete_unverified=True` **under the per-table write lock**, so no writer is in flight: the Rewrite has the manifest to itself (cleanup completes every beat) and aggressive deletion is safe. It also removes the empty `_indices/<uuid>/` husks cleanup leaves behind (soak: 13061 dirs, 98% empty), offloaded to a thread. - The cascade worker's heavy beat calls `prune()`; the light beat calls `optimize()`. A benign light-beat commit conflict is logged at debug and does not count toward the failure streak or trigger a rebuild. - Prune's retention window (`cleanup_older_than`) is decoupled from the prune cadence and defaulted short (60s). It runs under the write lock, so the window only needs to outlive an in-flight read; keeping it = cadence (300s) left ~2 cadences of superseded full-table copies on disk between beats (soak: transient ~15G/table peaks). 60s reclaims all but the last minute each beat — same live floor (~625MB/table), far lower transient footprint. Result on the re-run soak: disk sawtooths and reclaims to live-data size (~1.3G total) under active load — vs run1 stuck at 40G until writes stopped — with 0 crashes / 0 OOM / 0 stuck cleanups over 48h. Cascade projection health is now observable: - `CascadeOrchestrator.health()` -> `CascadeHealth`, combining the worker's in-memory signals (drain-loop failures, unrecoverable count, optimize streak, prune staleness) with the SQLite queue summary. - `GET /health` gains a typed `cascade` readiness block. `healthy` reflects operational health only (drain / optimize / prune); `failed_permanent` (files awaiting `cascade fix`) is a data-quality backlog reported as an informational count that does NOT flip `healthy` — otherwise the signal would sit red permanently. The scanner-side retry cap and the `_MAX_TOTAL_RETRIES` budget already on main handle re-enqueue storms, so no duplicate is added here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(deps): pin lancedb to >=0.34.0,<0.35.0 The previous open-ended `>=0.13.0` let any environment float to an untested release, including 0.32-0.34 which carry a compaction offset-overflow regression (lance-format/lance#7653) that stalls version cleanup and grows the index dir without bound. - Floor 0.34.0: the current resolved version; runs safely thanks to the with_position=False FTS workaround shipped in #336. Verified that data written by lancedb 0.32.0 (lance v6) reads correctly under 0.34.0 (lance v8), so existing deployments upgrade cleanly. Never widen the floor below 0.34 -- older lance cannot read v8-format data. - Ceiling <0.35.0: 0.35 embeds lance-rust v9 (large encoding jump, not yet stable-released); it must pass the soak harness before we allow it. Resolved version is unchanged (still 0.34.0); this only tightens the declared constraint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> (cherry picked from commit 94f9aa67d11a07c93c7d59d6b134e80858b60316) * fix(search): bridge agent-kind metadata into the agentic doc contract The agent AGENTIC path (`agent_case` / `agent_skill`) fed recall candidates straight into `aagentic_retrieve`, whose `_format_docs` (LLM sufficiency / multi-query prompt) reads `metadata["episode"]` as a `{subject, content}` dict plus a ms-epoch `timestamp`. Agent-kind rows carry their body in the recaller's `text_field` and time as a datetime, so `_format_docs` raised `TypeError: Candidate ... has no episode dict` and `POST /api/v*/memory/search` returned 500 for any `owner_type=agent` + `method=agentic` request. Mirror the episode path's bridge: reshape agent candidate metadata into the everalgo doc contract before `aagentic_retrieve`, and revert it before DTO shaping so the agent shapers still see a datetime timestamp. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit bb9a3a585e044643d3852b9d2810b42c05a7a47c) * test(search): regenerate search seed to current linkage + migrate e2e The committed `search_seed` fixture and several search e2e tests were written for the pre-1.5 memcell fact-linkage model. Current extraction links atomic_facts to episodes via `parent_id == episode.entry_id` (parent_type="episode"), and user_memory clusters store episode entry_id members — so the stale fixture made VECTOR/AGENTIC recall and the cluster-narrowing path find nothing, and stale assertions checked an old error code. - Regenerate `search_seed/*` from a fresh corpus in the current entry_id format; facts now bridge across multiple episodes (richer agentic / hierarchical-eviction coverage). - Fix `_dump_search_seed.py` sampling: pick episodes that host facts first and keep facts by episode entry_id, so re-dumps stay coherent. - Migrate e2e tests to the entry_id model (hierarchical-eviction, session/timestamp filters, cluster seeding helper) and update the filter-error assertion to the current `INVALID_INPUT` code. - Provision `ome.toml` in the full-app pipeline fixture (the OME config reloader requires it; strategies are code-registered so the packaged default suffices), unblocking corpus regeneration. Full search e2e suite now green (49/49). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> (cherry picked from commit 44cd9cecc07775cb9270104a5804f36e32cd788d) * fix(lancedb): detect schema type drift and add cascade rebuild recovery verify_business_schemas only compared column names, so a column whose on-disk Arrow type had drifted (name unchanged) slipped through and detonated later inside merge_insert as an opaque LanceError(IO) "Spill has sent an error" (#337). Now compare each shared column's Arrow type against schema.to_arrow_schema() — the exact schema get_table builds tables from, so a healthy table never false-positives. Reproduced #337 byte-identically: an episode.subject_vector column left as string or null by an older build, plus a real 1024-d vector on upsert, yields the exact crash. No lancedb version (0.13-0.34) renders Optional[Vector] as a non-vector type, so the startup guard is what should catch it — not the runtime. Add `everos cascade rebuild` as the safe recovery: it drops the business LanceDB tables and re-indexes from markdown, skipping the verify guard (which the drift would otherwise trip on startup). Unlike removing only .index/lancedb it re-populates already-done entries (reset_all clears the cascade queue); unlike removing all of .index it preserves unprocessed_buffer (messages not yet extracted). Fixes #337. * docs(cascade): document cascade rebuild and correct recovery guidance Add the `everos cascade rebuild` command to the runbook, CLI, and how-memory-works docs. Correct the old recovery guidance: a bare `rm -rf .index/lancedb` leaves md_change_state marked `done`, so the scanner skips those files and the index comes back empty — the runbook previously claimed a full repopulation that does not happen. `cascade rebuild` is the safe path (re-populates done entries, preserves unprocessed_buffer). Also document that verify now checks column types. * chore(rebase): adapt #354 integration test to soft-embedding main CascadeOrchestrator dropped the embedder param when embedding became a soft dependency (main); fold the schema-drift integration test onto it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(cascade): freeze monotonic clock in prune-staleness health tests Both prune-staleness health tests fabricated `_started_at = time.monotonic() - (ALERT + 100)`, assuming monotonic() is a large value. On a fresh CI runner monotonic() is only ~100-180s, so the subtraction went negative, the source clamped the baseline to 0, and staleness read back as ~130s < 900s — failing on CI while passing on long-lived dev boxes where monotonic() is huge. Freeze the monotonic clock via monkeypatch so staleness is deterministic regardless of the runner's boot uptime. Source logic is unchanged; only the tests are made hermetic. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(cascade): wait for stable terminal state in rename scenarios `_wait_path_done` reached a terminal status, slept a 0.1s settle window, then asserted the status was still terminal — which contradicted its own docstring ("absorb any last-second re-enqueue"). A rename's delete event or an atomic-replace echo can flip a done row back to `processing` inside that window, so on a slow CI runner the assert fired ("flipped back to processing after reaching done"), failing test_rename_cross_owner_keeps_frontmatter_owner intermittently (seen on the 3.13 job). `make integration` runs without `--reruns`, so a single flake fails the whole job. Wait for a terminal state that *survives* the settle window instead: absorb a transient re-enqueue by waiting for terminal again, still bounded by `deadline` so a row that never settles surfaces as a timeout. Pre- existing flake on main, unrelated to the prune change; the scenario's real assertions (row counts, frontmatter owner) are untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(cascade): cross-process prune safety + backfill reclaim fix Review of #379 found two P0s plus P1/P2s, all verified against the code: - P0-1: cascade backfill still called the removed optimize(cleanup_older_than=…) kwarg → TypeError swallowed by the best-effort try/except → backfill silently skipped all compaction + reclaim (the exact bloat this PR fixes). CI stayed green because the test double kept the stale signature. Fix: call optimize() + prune(0) at the call site; make the fake mirror the real signature so the drift can't hide again; pin prune in the backfill tests. - P0-2: prune ran delete_unverified=True guarded only by an in-process asyncio lock, but the runbook promises `cascade sync` is safe alongside a live server — and the CLI's first optimize beat does prune, in a separate process. It could delete files the daemon is mid-commit on. Fix: switch prune to delete_unverified=False. Measured to reclaim identically on churned tables (both collapse superseded versions ~97%); True only additionally deletes in-flight/dangling files — exactly the corruption vector. No cross-process lock needed; the write-lock/commit fix (the real reclaim win) is unchanged. - P1-3: /health called orch.health() (6 SQLite aggregates) with no guard → a locked/full/migrating DB would 500 the liveness probe and restart the container. Wrap it: unhealthy readiness + reason, HTTP stays 200. - P1-4: rebuild drops + recreates tables; a live daemon holds cached handles pointing at the dropped dataset. Runbook now says stop the server first — the one cascade command unsafe alongside a live server. - P2: narrow _is_benign_commit_conflict to the "commit conflict" phrase (a bare "retryable" swallowed unrelated recoverable errors); add a timeout around the prune cleanup so a hung lance call can't wedge the write lock; correct two stale docstrings. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: fix schema-recovery guidance + drop dead internal doc refs Follow-up to the review; two doc issues neither the review nor the fix commit caught: - The schema-drift startup error's docstrings (verify_business_schemas and LanceDBLifespanProvider) still described the recovery as `rm -rf ~/.everos/.index/lancedb` — which the runbook explicitly calls the WRONG recovery (it leaves the cascade queue `done`, so nothing re-indexes and the index comes back empty). The raised error already points to `everos cascade rebuild`; align the docstrings to match. - 15 dangling references to an internal numbered design-doc set (12_/13_/16_/17_*.md) that was never shipped to this repo. Point the schema-recovery ones at docs/cascade_runbook.md; drop the rest (pure provenance in table/component docstrings) while keeping the substance. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: align maintenance docstrings with split optimize/prune API Two docstring residuals from the #379 review's P2 list: - _run_optimize_once still described the pre-split bundled heavy beat ("same work plus cleanup_older_than ... older than one cadence"); the heavy beat now calls prune() under the write lock and the retention window is decoupled from the cadence (DEFAULT_OPTIMIZE_PRUNE_RETENTION_SECONDS). - _restore_shaper_metadata converts any numeric timestamp, wider than an exact inverse of the bridge; document that this is deliberate (the shaper contract requires a datetime either way). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(e2e): hoist fixture-body imports to conftest module top shutil / importlib.resources.files were imported inside the core_pipeline_runtime fixture body; move them to the module top to match the repo import style (#379 review P2). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cascade): back off a hung prune via a separate attempt clock (N1) The write-lock timeout on prune (review P2) didn't achieve its goal: it was 300s — equal to the prune cadence — and `last_prune_at` only advanced on success. So a hung lance cleanup timed out after 300s, `should_prune` was still true (clock never moved), and the next beat re-pruned ~10s later — pinning the per-table write lock ~97% of the time, the exact write-starvation the timeout was meant to prevent. A real cleanup is milliseconds even on a heavily churned table (measured ~40ms at 320k writes / 100 versions), so the timeout is a pure hang-catcher: lower it to 60s (~1500x headroom, never fires normally, well below the 300s cadence). Split the prune clock so a failed prune backs off without masking the health signal: - last_prune_attempt_at (new) gates scheduling, advanced before the call whether it succeeds or times out → a hung prune waits a full cadence before retrying (light lock-free compaction runs meanwhile), so the lock is held at most ~timeout/cadence ≈ 17% in the worst case. - last_prune_at advances only on success and still drives the prune-staleness health signal, so a persistently failing prune surfaces as degraded instead of being hidden by an advanced schedule clock. Regression test: a raising prune advances the attempt clock (next beat is light, no immediate re-prune) but not the success clock. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: zhanghui <zhanghui@shanda.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
297 lines
11 KiB
Python
297 lines
11 KiB
Python
"""Shared fixtures for ``tests/e2e/``.
|
|
|
|
Provides:
|
|
|
|
- ``core_pipeline_runtime``: tmp memory root + reset memorize singletons.
|
|
Uses the **real** LLM / embedding / rerank creds from ``.env`` per the
|
|
project test policy.
|
|
- ``async_client``: ``httpx.AsyncClient`` wired into ``create_app()`` with
|
|
the full lifespan stack (SQLite + LanceDB + Cascade + OME).
|
|
- ``cascade_done_poll``: wait until ``md_change_state`` queue is fully
|
|
drained (``pending`` rows == 0; includes the internal ``processing``).
|
|
- ``pipeline_done_poll``: composite drain — waits until OME strategy runs AND
|
|
``md_change_state`` queue both drain (use for tests that exercise the full
|
|
OME → md → cascade pipeline).
|
|
- ``buffer_count`` / ``memcell_count``: raw counts for buffer-delta and
|
|
memcell-growth assertions.
|
|
|
|
The ``long_conversation`` fixture (LoCoMo conv_0) lives in
|
|
:mod:`tests.conftest` so both ``tests/e2e/`` and
|
|
``tests/integration/search/`` can depend on it.
|
|
|
|
Conventions:
|
|
|
|
- ``.env`` is loaded at import time (before any everos module reads
|
|
settings) — overrides for ``EVEROS_ROOT`` happen per-test.
|
|
- This file does **not** define ``cascade_runtime`` — that name belongs
|
|
to ``tests/integration/test_cascade_integration.py``'s local fixture.
|
|
The pipeline test uses ``core_pipeline_runtime`` to avoid name
|
|
collision.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import importlib
|
|
import json
|
|
import shutil
|
|
from collections.abc import AsyncIterator, Awaitable, Callable
|
|
from importlib.resources import files
|
|
from pathlib import Path
|
|
|
|
import httpx
|
|
import pytest
|
|
import pytest_asyncio
|
|
from dotenv import load_dotenv
|
|
from sqlalchemy import text
|
|
|
|
# Load real .env creds before any everos import touches load_settings().
|
|
_PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
|
load_dotenv(_PROJECT_ROOT / ".env", override=False)
|
|
|
|
_FIXTURE_DIR = _PROJECT_ROOT / "tests" / "fixtures"
|
|
_SEARCH_SEED_DIR = _FIXTURE_DIR / "search_seed"
|
|
|
|
# Memorize service module-level singletons that survive across tests; we
|
|
# null them out so each test rebuilds against its own ``tmp_path``.
|
|
_MEMORIZE_SINGLETONS: tuple[str, ...] = (
|
|
"_episode_writer",
|
|
"_prompt_loader",
|
|
"_user_pipeline",
|
|
"_agent_pipeline",
|
|
"_ome_engine",
|
|
)
|
|
|
|
# OME strategy modules carry module-level lazy singletons (``_writer`` /
|
|
# ``_reader``) that capture ``MemoryRoot.resolve()`` at first call. They
|
|
# survive across tests, so the second test writes its output to the
|
|
# **first test's** tmp_path. Reset all of them per-test.
|
|
_STRATEGY_SINGLETONS: tuple[tuple[str, tuple[str, ...]], ...] = (
|
|
("everos.memory.strategies.extract_atomic_facts", ("_writer",)),
|
|
("everos.memory.strategies.extract_foresight", ("_writer",)),
|
|
("everos.memory.strategies.extract_user_profile", ("_writer", "_reader")),
|
|
("everos.memory.strategies.extract_agent_case", ("_writer",)),
|
|
("everos.memory.strategies.extract_agent_skill", ("_writer",)),
|
|
)
|
|
|
|
|
|
def _reset_strategy_singletons(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
"""Null every strategy ``_writer`` / ``_reader`` so the next test
|
|
rebuilds against its own ``MemoryRoot.resolve()`` (driven by the
|
|
fresh ``EVEROS_ROOT`` env var set by the calling fixture).
|
|
"""
|
|
for mod_name, attrs in _STRATEGY_SINGLETONS:
|
|
mod = importlib.import_module(mod_name)
|
|
for attr in attrs:
|
|
monkeypatch.setattr(mod, attr, None, raising=False)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Data fixture
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.fixture(scope="session")
|
|
def search_seed() -> dict[str, list[dict]]:
|
|
"""Load the search seed slice produced by ``_dump_search_seed.py``.
|
|
|
|
Returns a dict with four keys (``episode`` / ``atomic_fact`` /
|
|
``foresight`` / ``user_profile``); each value is a list of raw row
|
|
dicts ready to be fed into ``Model.model_validate`` for LanceDB.
|
|
|
|
Tests pick the subset they need and may mutate per-row fields
|
|
(e.g. set distinct ``session_id`` values to exercise filter DSL)
|
|
before instantiating the pydantic model.
|
|
"""
|
|
return {
|
|
name: json.loads((_SEARCH_SEED_DIR / f"{name}.json").read_text())
|
|
for name in ("episode", "atomic_fact", "foresight", "user_profile")
|
|
}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Runtime fixture: tmp memory root + singleton reset (no app lifespan)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def core_pipeline_runtime(
|
|
tmp_path: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> AsyncIterator[Path]:
|
|
"""Prepare clean memory root + reset memorize singletons.
|
|
|
|
Keeps real LLM / embedding settings from ``.env`` (do NOT overwrite
|
|
``EVEROS_LLM__*`` or ``EVEROS_EMBEDDING__*``).
|
|
"""
|
|
monkeypatch.setenv("EVEROS_ROOT", str(tmp_path))
|
|
|
|
from everos.config import load_settings
|
|
|
|
load_settings.cache_clear()
|
|
|
|
svc = importlib.import_module("everos.service.memorize")
|
|
client_mod = importlib.import_module("everos.component.llm.client")
|
|
|
|
for attr in _MEMORIZE_SINGLETONS:
|
|
monkeypatch.setattr(svc, attr, None, raising=False)
|
|
monkeypatch.setattr(client_mod, "_llm_client", None, raising=False)
|
|
_reset_strategy_singletons(monkeypatch)
|
|
|
|
# The full-app lifespan starts OME, whose config reloader requires an
|
|
# ``ome.toml`` in the memory root (normally created by ``everos init``).
|
|
# Provision the packaged default so the pipeline e2e can boot; OME
|
|
# strategies are code-registered (see api/lifespans/ome.py), so the
|
|
# default config is sufficient for extraction.
|
|
default_ome = files("everos.config") / "default_ome.toml"
|
|
shutil.copyfile(str(default_ome), str(tmp_path / "ome.toml"))
|
|
|
|
yield tmp_path
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Async client fixture (full app lifespan)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest_asyncio.fixture
|
|
async def async_client(
|
|
core_pipeline_runtime: Path,
|
|
) -> AsyncIterator[httpx.AsyncClient]:
|
|
"""Bring up the full everos app with lifespan, return an httpx client.
|
|
|
|
The lifespan starts: SQLite engine, LanceDB connection + business
|
|
indexes, Cascade orchestrator (watcher + scanner + worker), OME
|
|
engine. Teardown stops everything in reverse.
|
|
"""
|
|
from everos.entrypoints.api.app import create_app
|
|
|
|
app = create_app()
|
|
transport = httpx.ASGITransport(app=app)
|
|
|
|
# Drive starlette's lifespan_context explicitly — httpx.ASGITransport
|
|
# does not run startup / shutdown on its own.
|
|
async with (
|
|
app.router.lifespan_context(app),
|
|
httpx.AsyncClient(transport=transport, base_url="http://test") as client,
|
|
):
|
|
yield client
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Poll helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
async def _poll(
|
|
condition: Callable[[], Awaitable[bool]],
|
|
*,
|
|
deadline_seconds: float,
|
|
interval: float = 0.5,
|
|
) -> None:
|
|
"""Poll an async predicate until truthy; ``TimeoutError`` on deadline."""
|
|
async with asyncio.timeout(deadline_seconds):
|
|
while True:
|
|
if await condition():
|
|
return
|
|
await asyncio.sleep(interval)
|
|
|
|
|
|
@pytest.fixture
|
|
def cascade_done_poll() -> Callable[..., Awaitable[None]]:
|
|
"""Wait until ``md_change_state`` queue is drained (no pending/processing)."""
|
|
|
|
async def _wait(*, deadline_seconds: float = 180.0) -> None:
|
|
from everos.infra.persistence.sqlite import md_change_state_repo
|
|
|
|
async def _drained() -> bool:
|
|
summary = await md_change_state_repo.queue_summary()
|
|
# `pending` includes the internal `processing` rows (see QueueSummary).
|
|
return summary.pending == 0
|
|
|
|
await _poll(_drained, deadline_seconds=deadline_seconds)
|
|
|
|
return _wait
|
|
|
|
|
|
@pytest.fixture
|
|
def pipeline_done_poll() -> Callable[..., Awaitable[None]]:
|
|
"""Wait until OME strategy runs AND ``md_change_state`` queue both drain.
|
|
|
|
Composite drain — fixes the trap where :func:`cascade_done_poll`
|
|
alone returns immediately while a slow LLM-driven strategy is still
|
|
in flight (the strategy has not written md yet, so the cascade queue
|
|
is momentarily empty). Pipeline tests that touch the full async
|
|
chain (OME -> md -> cascade -> LanceDB) must use this instead of
|
|
``cascade_done_poll``.
|
|
"""
|
|
|
|
async def _wait(*, deadline_seconds: float = 180.0) -> None:
|
|
from everos.infra.persistence.sqlite import md_change_state_repo
|
|
from everos.service.memorize import _get_engine
|
|
|
|
engine = _get_engine()
|
|
|
|
async def _drained() -> bool:
|
|
# OME side first: cascade can only fire after a strategy
|
|
# writes md, so an in-flight run means the queue check below
|
|
# is premature.
|
|
if not await engine.wait_idle(timeout=0.5):
|
|
return False
|
|
# `pending` includes the internal `processing` rows (see
|
|
# QueueSummary).
|
|
summary = await md_change_state_repo.queue_summary()
|
|
return summary.pending == 0
|
|
|
|
await _poll(_drained, deadline_seconds=deadline_seconds)
|
|
|
|
return _wait
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Count helpers (used directly by tests for buffer-delta assertions)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@pytest.fixture
|
|
def buffer_count() -> Callable[[str], Awaitable[int]]:
|
|
"""Return an async callable: ``await buffer_count(session_id) -> int``."""
|
|
|
|
async def _count(session_id: str) -> int:
|
|
from everos.infra.persistence.sqlite import get_engine
|
|
|
|
engine = get_engine()
|
|
async with engine.connect() as conn:
|
|
result = await conn.execute(
|
|
text("SELECT COUNT(*) FROM unprocessed_buffer WHERE session_id = :sid"),
|
|
{"sid": session_id},
|
|
)
|
|
return int(result.scalar() or 0)
|
|
|
|
return _count
|
|
|
|
|
|
@pytest.fixture
|
|
def memcell_count() -> Callable[[str], Awaitable[int]]:
|
|
"""Return an async callable: ``await memcell_count(user_id_or_session) -> int``.
|
|
|
|
Counts memcell rows; pass session_id to count by session, or omit to
|
|
count all.
|
|
"""
|
|
|
|
async def _count(session_id: str | None = None) -> int:
|
|
from everos.infra.persistence.sqlite import get_engine
|
|
|
|
engine = get_engine()
|
|
async with engine.connect() as conn:
|
|
if session_id is None:
|
|
result = await conn.execute(text("SELECT COUNT(*) FROM memcell"))
|
|
else:
|
|
result = await conn.execute(
|
|
text("SELECT COUNT(*) FROM memcell WHERE session_id = :sid"),
|
|
{"sid": session_id},
|
|
)
|
|
return int(result.scalar() or 0)
|
|
|
|
return _count
|