* fix(chroma): require SQLite magic header for ChromaBackend.detect() (#1893) Closes #1893. ChromaBackend.detect() was returning True for a 0-byte chroma.sqlite3 file because the check was just os.path.isfile(...). On a palace that has any other backend marker alongside a stale 0-byte chroma.sqlite3, resolve_backend_name then raises BackendMismatchError and the palace becomes unopenable until the user manually rm's the empty file. The 0-byte file appears as a side effect of any sqlite3.connect() on a missing path — Python creates the file immediately but writes the SQLite header only on the first statement. So any code path that touches the chroma.sqlite3 path with bare sqlite3.connect(), including chromadb's own PersistentClient lazy-init (see the comment at backends/chroma.py:2052), can leave a 0-byte artifact behind. Fix: detect() now reads the first 16 bytes and compares to the SQLite magic prefix b"SQLite format 3\x00" instead of relying on file presence alone. One extra open() + 16-byte read; detect() isn't a hot path. Properties: - Rejects 0-byte files (the symptom #1893 is about). - Rejects non-SQLite garbage at the canonical path (partial writes, etc.). - Doesn't false-negative on real chroma palaces: any chroma palace whose PersistentClient has done any work has the magic header on disk (verified — CREATE TABLE is enough to land the header). - Doesn't couple detect() to chroma's specific schema; the magic header is stable across chromadb releases. Test sweep: many test files used (chroma.sqlite3).touch() or .write_bytes(b"") as a "fake palace" shortcut, exploiting the loose isfile() check (one such site even had the comment "# pass the isfile guard"). After this change, those stand-ins no longer register as chroma palaces. Introduced tests/_chroma_palace_helper.py::make_minimal_chroma_sqlite following the existing _backend_conformance.py precedent, and updated 15 call sites across 8 test files to use it. The existing test_chroma_detect_matches_palace_with_chroma_sqlite (which encoded the buggy semantics with write_bytes(b"")) is renamed to test_chroma_detect_matches_palace_with_sqlite_header and now writes a real SQLite database via the helper. Added two new tests for the rejection paths (empty file, non-SQLite garbage). Full env-cleared suite: 3137 passed, 20 skipped, 0 failed. ruff check and ruff format --check both clean. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA * fix(sqlite_exact): require SQLite magic header for SQLiteExactBackend.detect() Per gemini-code-assist review on #1892 PR #1896: SQLiteExactBackend has the same os.path.isfile() detection pattern as ChromaBackend did, with the same 0-byte-file vulnerability. Mirrors the chroma fix for repo-wide consistency. - SQLiteExactBackend.detect() now does the same 16-byte SQLite magic-prefix check as ChromaBackend.detect(). - _chroma_palace_helper.py: factored its body into a private _write_minimal_sqlite_file() and gained a sibling make_minimal_sqlite_exact_sqlite() for the sqlite_exact filename. No churn to any existing chroma call sites. - test_sqlite_exact_backend.py:426 (the one site that wrote b"" for sqlite_exact.sqlite3) updated to use the new helper. - Three new tests in test_sqlite_exact_backend.py mirror the chroma trio: matches with valid header, rejects empty file, rejects non-SQLite garbage. Full env-cleared suite: 3140 passed, 20 skipped, 0 failed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KC5Qsknh2zFRtRvVyjXiTA --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
58 lines
1.9 KiB
Python
58 lines
1.9 KiB
Python
"""Shared helpers: create minimal valid palace-marker SQLite files for tests.
|
|
|
|
Many tests want to stand up "a chroma / sqlite_exact palace" cheaply —
|
|
historically they did this with ``(path / "<marker>.sqlite3").touch()`` or
|
|
``write_bytes(b"")``, relying on the backend ``detect()`` methods' old
|
|
``os.path.isfile()`` semantics. Post-#1893, both ``ChromaBackend.detect()``
|
|
and ``SQLiteExactBackend.detect()`` require a valid SQLite magic header, so
|
|
the empty stand-in no longer registers. These helpers create the minimum
|
|
required to make detection fire without standing up a full palace.
|
|
|
|
This module is intentionally not a ``test_*`` file: it ships utilities, not
|
|
tests.
|
|
"""
|
|
|
|
import sqlite3
|
|
from pathlib import Path
|
|
from typing import Union
|
|
|
|
|
|
def _write_minimal_sqlite_file(db_path: Path) -> None:
|
|
"""Write a valid SQLite magic header at ``db_path``.
|
|
|
|
Writing any statement is sufficient to land the 16-byte
|
|
``SQLite format 3\\x00`` magic prefix that the backend ``detect()``
|
|
methods check.
|
|
"""
|
|
|
|
conn = sqlite3.connect(db_path)
|
|
try:
|
|
conn.execute("CREATE TABLE _detect_smoke(x)")
|
|
conn.commit()
|
|
finally:
|
|
conn.close()
|
|
|
|
|
|
def make_minimal_chroma_sqlite(palace_path: Union[Path, str]) -> Path:
|
|
"""Create ``<palace_path>/chroma.sqlite3`` with a valid SQLite header.
|
|
|
|
Returns the path to the file. Backs
|
|
:py:meth:`mempalace.backends.chroma.ChromaBackend.detect`.
|
|
"""
|
|
|
|
db_path = Path(palace_path) / "chroma.sqlite3"
|
|
_write_minimal_sqlite_file(db_path)
|
|
return db_path
|
|
|
|
|
|
def make_minimal_sqlite_exact_sqlite(palace_path: Union[Path, str]) -> Path:
|
|
"""Create ``<palace_path>/sqlite_exact.sqlite3`` with a valid SQLite header.
|
|
|
|
Returns the path to the file. Backs
|
|
:py:meth:`mempalace.backends.sqlite_exact.SQLiteExactBackend.detect`.
|
|
"""
|
|
|
|
db_path = Path(palace_path) / "sqlite_exact.sqlite3"
|
|
_write_minimal_sqlite_file(db_path)
|
|
return db_path
|