* fix(knowledge): contain original-file write path (CWE-22)
The multipart upload filename was joined into ``_original/`` verbatim.
An attacker-controlled filename such as ``/tmp/pwned`` or
``../../.bashrc`` would let ``POST /knowledge/documents`` write outside
the document directory (the ``/`` operator discards the left operand
for absolute paths; ``..`` walks upward). The read side had the
symmetric issue.
Fix, mirroring the sender_id containment shipped in 1.0.1
(GHSA-c795-2g9c-j48m):
- Add ``_safe_original_filename`` reducing the untrusted filename to a
single POSIX/Windows-basename component; reject degenerate residuals
(``""``, ``"."``, ``".."``) with PathTraversalError.
- ``_write_original_file`` asserts ``target.resolve()`` stays inside
``original_dir.resolve()`` before any filesystem touch (mkdir/write).
- ``_resolve_original_file_path`` sanitises symmetrically so a stored
provenance label can never resolve to an out-of-directory file.
Four SEC regression tests cover: absolute filename, ``..`` traversal,
degenerate filename rejection, and read-side sanitisation.
Backport from GitLab release/v1.1.4 (commit 40f19de) — 1.1.4 shipped
this fix; the GitLab -> GitHub sync stopped at 1.1.3, so 1.2.0
regressed the containment. This commit alone restores the fix; the
1.2.1 release PR ships it to PyPI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cascade): retry classification, budget, and reconcile races
Backport the cascade reliability work that shipped in GitLab 1.1.4
(MR !49 / commit 95db2f5) — six interlocking changes the reviewer
should read in order:
1. Worker retry classification uses ExternalServiceError (embedding /
LLM / rerank transient failures) as the "retry inline" signal.
The legacy RecoverableError hierarchy under cascade/errors.py is
removed; the retry contract now lives in the domain error tree
(core/errors.py). Docstrings in handlers/base.py and
sqlite/tables/md_change_state.py updated to match.
2. Cross-cycle retry budget: _MAX_TOTAL_RETRIES = 12. Once total
attempts across scanner cycles exhaust the budget, the worker
marks retryable=False in place instead of looping forever on a
sustained upstream outage.
3. md_change_state upsert preserves retry_count on scanner
re-enqueue when mtime is unchanged (previously reset to 0 every
sweep, defeating the budget). mtime change (user edit) still
resets the counter.
4. Reconciler no longer re-enqueues pending / processing rows on
stable mtime — that was overwriting the worker's mark_done. It
also skips failed rows with retryable=False on stable mtime so
the entry-check demote path is stable.
5. mtime tolerance (10 ms, MTIME_TOLERANCE_SECONDS) absorbs the
SQLite REAL float precision loss that previously flapped the
reconcile decision when the same md was rewritten without a real
content change. The constant is defined once in the sqlite repo
and imported by the reconciler so both sides use the same tol.
6. Worker _run_rebuild_once carries an explicit
`state.task is not asyncio.current_task()` guard before awaiting
the optimize task — the previous contextlib.suppress was silently
swallowing self-await RuntimeError.
Kept intact from the GitHub 1.2.0 baseline:
- The `except FileNotFoundError → handle_deleted` branch in the
worker (delete/modify race — see
test_modified_event_for_vanished_file_is_processed_as_delete).
Test coverage added:
- test_retry_budget_exhausted_marks_unrecoverable
- test_external_service_error_at_budget_edge_demotes_in_place
- test_upsert_preserves_retry_count_for_failed_stable_mtime
- test_upsert_resets_retry_count_on_mtime_change
- test_optimize_fallback_rebuild_on_sustained_failure
- reconciler mtime-tolerance / stable-mtime skip suite
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(embedding): raise on empty API data; forward MRL dimensions
Backport from GitLab 1.1.4 (MR !49 / commit 95db2f5).
Two behavioural changes on ``OpenAIEmbeddingProvider._embed_chunk``:
1. ``response.data == []`` now raises ``EmbeddingServiceError`` instead
of returning an empty list. Some upstream providers (observed on
DeepInfra under load) return HTTP 200 with an empty ``data`` array;
the silent zero-vector path was corrupting search indexes without
any signal.
2. When ``[embedding] dimensions = N`` is set in ``everos.toml``, the
parameter is forwarded to the API so MRL-capable models
(OpenAI text-embedding-3-*, Qwen3-Embedding, ...) do server-side
truncation with proper re-normalization. Client-side truncation to
``dim`` remains as a fallback for backends that ignore the param.
``openai.NOT_GIVEN`` is used as the sentinel so the request omits
the field when the setting is left at the default ``None``.
Config plumbing:
- ``EmbeddingSettings.dimensions: int | None = None``
- factory forwards ``dimensions=settings.dimensions`` to the provider
The provider stays inside the existing ``memory_span`` OTel wrapper
and continues to report input-only tokens via ``set_generation_usage``
- both are GitHub 1.2.0 native tracing behaviours preserved intact.
Test coverage:
- test_empty_response_data_raises_embedding_error (new)
- test_usage_span._FakeEmbeddings.create signature updated to accept
``dimensions`` kwarg so the OTel token-recording tests still exercise
the same call path
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(extract): retry episode extraction on malformed LLM output
Backport from GitLab 1.1.4 (MR !49 / commit 95db2f5).
The ``/flush`` synchronous path called ``EpisodeExtractor.aextract``
exactly once. everalgo raises ``ValueError`` when the LLM returns
malformed JSON (observed with OpenRouter partial responses where
finish_reason=stop but the body is truncated) — the caller was
surfaced a 500 for a transient upstream hiccup.
``_extract_with_retry`` wraps the call with two extra attempts at 1s
and 2s backoff (final attempt propagates untouched), typed as
``AlgoEpisode`` so the caller path stays annotated. Retry stays inside
the existing GitHub 1.2.0 ``memory_span("everos.extract", ...)`` OTel
wrapper — the OTel token capture and the retry loop are orthogonal.
TODO in the code notes we should catch a typed everalgo
``ExtractionError`` once that type is introduced (currently ValueError
is broad).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: log non-stop finish_reason; bump everalgo-user-memory 0.3.2
Two loosely coupled changes from GitLab 1.1.4 (MR !49 / commit 95db2f5)
that arrive together as a housekeeping commit.
1. ``_LoggingLLMClient`` diagnostic wrapper (new, file-private).
Wraps the raw everalgo LLM client and, on every ``chat()``, warns
when ``resp.finish_reason != "stop"`` — logging the reason,
``content_len``, the last 200 chars of ``content``, and ``model``.
Aims at OpenRouter/DeepSeek truncation triage where the provider
silently caps output length and returns finish_reason=length /
filter / etc. Non-invasive: one branch per call, no config gate.
Wrapper stack in ``get_llm_client``:
LoggingLLMClient(UsageRecordingClient(build_client(...)))
LoggingLLMClient(build_client(...)) # observability off
``UsageRecordingClient`` (GitHub 1.2.0 native OTel token capture)
stays gated by ``settings.observability.enabled`` — this commit
preserves that. LoggingLLMClient is always outermost so the reason
it observes is exactly the reason the underlying provider reported.
2. ``everalgo-user-memory`` 0.3.1 -> 0.3.2 (pyproject + uv.lock).
Same bump the GitLab 1.1.4 release lane took; unblocks the
episode-extract retry work in commit 4 seeing the upstream
improvements. Verified via ``uv sync``.
No functional API changes.
Test coverage:
- test_returns_singleton_when_configured now asserts the outer
LoggingLLMClient wrapper.
- test_wraps_client_when_observability_enabled asserts the two-layer
Logging(UsageRecording(...)) stack.
- test_does_not_wrap_client_when_observability_disabled asserts
Logging still wraps when tracing is off.
- test_logging_wrapper_warns_on_non_stop_finish_reason (new).
- test_logging_wrapper_silent_on_stop_finish_reason (new).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(changelog): restore [1.1.4] section to match published sdist
The 1.1.4 changelog entry on this branch previously listed three
items (Langfuse example, delete/modify race, live-server telemetry).
The 1.1.4 sdist on PyPI, however, was built from the internal
release lane and includes the CWE-22 containment, the cascade
retry-budget / mtime-tolerance / reconcile-guard work, the embedding
empty-data raise, the episode-extract retry, MRL dimensions, and the
LLM finish_reason diagnostic — none of which were represented here
when the tag was cut.
Rewrite the [1.1.4] section so it matches the wheel a user actually
installs from PyPI:
- Add a header note explaining the retroactive restoration.
- Fixed: CWE-22, cascade reliability bundle, delete/modify race
(unchanged wording), embedding empty-data, episode extract retry,
Langfuse live-server telemetry (unchanged wording).
- Added: MRL dimensions, LLM finish_reason diagnostic, Langfuse
example (unchanged wording).
- Changed: everalgo-user-memory 0.3.1 -> 0.3.2.
The GitLab-side `.gitlab-ci.yml` in-house-runner entry is dropped —
open-source CI runs on GitHub Actions and the internal runner switch
is not visible to public users.
Date stays 2026-07-20 (the GitHub v1.1.4 tag date / PyPI upload
timestamp) rather than the internal 2026-07-23 code-freeze date, so
the timeline of what shipped where remains internally consistent.
The corresponding code fixes are all backported by earlier commits
in this PR; this commit only aligns the changelog surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
42629dfd4d