diff --git a/docs/rfcs/001-storage-backend-plugin-spec.md b/docs/rfcs/001-storage-backend-plugin-spec.md index 4f7b66a..c2c8d52 100644 --- a/docs/rfcs/001-storage-backend-plugin-spec.md +++ b/docs/rfcs/001-storage-backend-plugin-spec.md @@ -6,15 +6,17 @@ - **Related:** [#266](https://github.com/MemPalace/mempalace/issues/266), [#574](https://github.com/MemPalace/mempalace/pull/574), [#643](https://github.com/MemPalace/mempalace/pull/643), [#665](https://github.com/MemPalace/mempalace/pull/665), [#697](https://github.com/MemPalace/mempalace/pull/697), [#700](https://github.com/MemPalace/mempalace/pull/700), [#381](https://github.com/MemPalace/mempalace/pull/381), [#1679](https://github.com/MemPalace/mempalace/pull/1679) - **Spec version:** `1.0` -> **Implementation status (2026-06-07).** The §1–2 contract surface (`PalaceRef`, +> **Implementation status (2026-08-11).** The §1–2 contract surface (`PalaceRef`, > typed results, `BaseBackend` / `BaseCollection`, capability tokens, and most of > the §10 seam cleanup) landed ahead of this merge via [#1679](https://github.com/MemPalace/mempalace/pull/1679), -> which ships in-tree `pgvector`, `qdrant`, and `sqlite_exact` backends. Three -> areas this spec defines are tracked as follow-up implementation work and are -> **not** yet built: the embedder-identity contract (§1.5 / §5), the maintenance -> hooks (§7.3), and search-path metric-awareness (§10, `searcher.py`). Each is -> flagged in its section. Accepting this RFC pins the contract those follow-ups -> target; it does not assert the implementation is complete. +> which ships in-tree `pgvector`, `qdrant`, and `sqlite_exact` backends. The three +> areas originally deferred as follow-ups have since landed: embedder-identity +> (§1.5 / §5, [#1731](https://github.com/MemPalace/mempalace/pull/1731) / +> [#1734](https://github.com/MemPalace/mempalace/pull/1734)), maintenance hooks +> (§7.3, [#1732](https://github.com/MemPalace/mempalace/pull/1732)), and searcher +> metric-awareness (§10, [#1727](https://github.com/MemPalace/mempalace/pull/1727)). +> Accepting this RFC pins the contract; remaining gaps are evolutionary, not +> blocking. ## Summary @@ -428,15 +430,22 @@ The `options` kwarg to `get_collection` is a free-form dict. Each backend docume Per-tenant collection-name prefixing is not a backend concern. It is handled by the resolver layer above backends: `PalaceRef.namespace` carries the tenant identifier. The `collection_prefix` concept from #697 dissolves into this model. -**Isolation contract.** `PalaceRef.id` is the *required* isolation key for every backend: within a single backend instance, a record written for one `id` MUST NOT be returned, modified, or deleted by an operation issued for a different `id`. Cross-palace access is a spec violation. +**Isolation contract.** `PalaceRef.id` is the *required* isolation key for every backend: within a single backend instance, a record written for one `id` MUST NOT be returned, modified, or deleted by an operation issued for a different `id`. Cross-palace access is a spec violation. This is the non-negotiable blanket MUST; multi-tenant deployments may cite it as their primary partition boundary even when they do not use `namespace`. `namespace` is *additional* partitioning. A backend that advertises `supports_namespace_isolation` (§2.1) MUST extend the same guarantee to namespaces: > A record written under one `namespace` MUST NOT be returned, modified, or deleted by an operation issued under a different `namespace` within the same backend instance. Cross-namespace access is a spec violation, not a caller misconfiguration. -This is what hosted multi-tenant deployments cite as the basis for tenant isolation. Authorization (which namespaces a given request may touch) stays on the deployment side; the backend's job is to guarantee no bleed *within* the instance once the namespace is fixed. +This is what hosted multi-tenant deployments cite as the basis for *namespace*-level tenant isolation. Authorization (which namespaces a given request may touch) stays on the deployment side; the backend's job is to guarantee no bleed *within* the instance once the namespace is fixed. -A backend that does **not** advertise `supports_namespace_isolation` (e.g. `sqlite_exact`, whose isolation is the on-disk path alone) MAY ignore `namespace` entirely; callers MUST NOT rely on `namespace` for tenant isolation on such backends. Making isolation a declared capability rather than a blanket `MUST` is deliberate: path-rooted local backends already isolate by `local_path`, and forcing them to re-implement namespace partitioning would be ceremony with no security gain. The conformance suite (§7.1) exercises the namespace arm only for backends that advertise the token. +Making namespace isolation a declared capability rather than a blanket `MUST` is deliberate: path-rooted local backends (e.g. `chroma`, `sqlite_exact`) already isolate by `local_path` / `id`, and forcing them to re-implement a second axis would be ceremony with no security gain. + +**Conformance arms.** Self-attestation is not enough. The isolation suite (`tests/_backend_conformance.py` / `assert_partition_isolation`) has two distinct arms: + +1. **Cross-`id` isolation** — required of every backend. Two `PalaceRef`s that differ only in `id` MUST NOT see each other's records. +2. **Same-`id` / different-`namespace` isolation** — required of every backend that advertises `supports_namespace_isolation`. Two refs that share `id` (and, when applicable, `local_path`) but differ in `namespace` MUST NOT see each other's records. Distinct from arm 1; a backend that only passes arm 1 does not get to claim the capability. + +**No silent drop.** A backend that does **not** advertise `supports_namespace_isolation` MUST NOT silently accept and ignore a populated `namespace` — that is a latent cross-namespace leak, same spirit as `UnsupportedFilterError` for unknown operators. Such backends MUST either raise (e.g. `UnsupportedCapabilityError`) when `PalaceRef.namespace` is non-`None`, or honor the namespace and advertise the capability. Callers targeting path-rooted backends MUST leave `namespace` as `None`. --- @@ -500,6 +509,9 @@ The suite covers: - Delete-then-query consistency - `close()` releases handles and further calls raise `BackendClosedError` - Concurrent `get_collection` across different palaces is safe +- Isolation arm 1: cross-`PalaceRef.id` (every backend) +- Isolation arm 2: same-`id` / different-`namespace` (only when `supports_namespace_isolation` is advertised; self-attestation without this arm is a conformance failure) +- Non-advertising backends raise on a populated `namespace` rather than silently accepting it (§4.4) ### 7.2 Parametrized core suite diff --git a/mempalace/backends/base.py b/mempalace/backends/base.py index 3f8aa51..2fbcee1 100644 --- a/mempalace/backends/base.py +++ b/mempalace/backends/base.py @@ -123,10 +123,13 @@ class PalaceRef: access is a spec violation. Backends that do not advertise ``supports_namespace_isolation`` (e.g. - ``sqlite_exact``, whose isolation is the on-disk path alone) MAY ignore - ``namespace`` entirely; callers MUST NOT rely on it for tenant isolation - on such backends. Any conforming backend can self-check both guarantees by - running the shared assertions in ``tests/_backend_conformance.py``. + path-rooted ``chroma`` / ``sqlite_exact``) MUST NOT silently accept and + ignore a populated ``namespace`` — they MUST raise + :class:`UnsupportedCapabilityError` (same spirit as + :class:`UnsupportedFilterError`). Callers targeting those backends MUST + leave ``namespace`` as ``None``. Isolation conformance lives in + ``tests/_backend_conformance.py`` (cross-id arm for every backend; + same-id / different-namespace arm for advertisers only). """ id: str @@ -604,8 +607,9 @@ class BaseBackend(ABC): ``PalaceRef.namespace`` (multi-tenant / hosted deployments) MUST advertise the ``supports_namespace_isolation`` capability token; doing so is a promise to satisfy the cross-namespace guarantee and to pass the namespace - arm of the conformance suite. Backends without the token MAY ignore - ``namespace``. + arm of the conformance suite. Backends without the token MUST raise + :class:`UnsupportedCapabilityError` when ``PalaceRef.namespace`` is + non-``None`` rather than silently accept-and-ignore (RFC 001 §4.4). """ name: ClassVar[str] @@ -626,6 +630,21 @@ class BaseBackend(ABC): #: for anything not listed here. maintenance_kinds: ClassVar[frozenset[str]] = frozenset() + def require_namespace_support(self, palace: PalaceRef) -> None: + """Raise if ``palace.namespace`` is set but this backend does not isolate by it. + + Call at the start of ``get_collection`` (and any other entry that + accepts a :class:`PalaceRef`) so non-advertising backends never + silently drop a tenant namespace (RFC 001 §4.4). + """ + if palace.namespace is None: + return + if "supports_namespace_isolation" not in self.capabilities: + raise UnsupportedCapabilityError( + f"{type(self).name} does not advertise supports_namespace_isolation; " + f"leave PalaceRef.namespace as None (got {palace.namespace!r})" + ) + @abstractmethod def get_collection( self, diff --git a/mempalace/backends/chroma.py b/mempalace/backends/chroma.py index 02ff7e8..5827d6c 100644 --- a/mempalace/backends/chroma.py +++ b/mempalace/backends/chroma.py @@ -2395,6 +2395,7 @@ class ChromaBackend(BaseBackend): — still used by callers not yet migrated. """ palace_ref, collection_name, create, options = _normalize_get_collection_args(args, kwargs) + self.require_namespace_support(palace_ref) palace_path = palace_ref.local_path if palace_path is None: diff --git a/mempalace/backends/sqlite_exact.py b/mempalace/backends/sqlite_exact.py index 7c946f2..80670ce 100644 --- a/mempalace/backends/sqlite_exact.py +++ b/mempalace/backends/sqlite_exact.py @@ -1078,6 +1078,7 @@ class SQLiteExactBackend(BaseBackend): **kwargs, ) -> SQLiteExactCollection: palace, collection_name, create, read_only = self._normalize_args(args, kwargs) + self.require_namespace_support(palace) palace_path = palace.local_path if palace_path is None: raise PalaceNotFoundError("SQLiteExactBackend requires PalaceRef.local_path") diff --git a/tests/test_backend_conformance.py b/tests/test_backend_conformance.py index a62e0ba..903d351 100644 --- a/tests/test_backend_conformance.py +++ b/tests/test_backend_conformance.py @@ -41,3 +41,20 @@ def test_local_backends_do_not_claim_namespace_isolation(): advertise the namespace-isolation capability (RFC 001 isolation contract).""" assert "supports_namespace_isolation" not in ChromaBackend.capabilities assert "supports_namespace_isolation" not in SQLiteExactBackend.capabilities + + +@pytest.mark.parametrize("backend_cls", _LOCAL_BACKENDS) +def test_local_backends_reject_populated_namespace(backend_cls, tmp_path): + """RFC 001 §4.4 no-silent-drop: non-advertising backends MUST raise when + PalaceRef.namespace is set rather than accept-and-ignore.""" + from mempalace.backends.base import UnsupportedCapabilityError + + backend = backend_cls() + try: + path = tmp_path / "palace" + path.mkdir() + ref = PalaceRef(id=str(path), local_path=str(path), namespace="tenant-a") + with pytest.raises(UnsupportedCapabilityError): + backend.get_collection(palace=ref, collection_name="mempalace_drawers", create=True) + finally: + backend.close()