docs(rfc-001): fold §4.4 review nits; accept storage-backend spec
Close the last open items on #743 before merge: - Conformance: document two isolation arms (cross-id for all backends; same-id/different-namespace for supports_namespace_isolation advertisers). - No silent drop: non-advertising backends must raise UnsupportedCapabilityError when PalaceRef.namespace is set, rather than accept-and-ignore. - Wire require_namespace_support() into chroma/sqlite_exact; add conformance test. - Refresh implementation-status banner now that #1727/#1731/#1732/#1734 landed.
This commit is contained in:
@@ -6,15 +6,17 @@
|
||||
- **Related:** [#266](https://github.com/MemPalace/mempalace/issues/266), [#574](https://github.com/MemPalace/mempalace/pull/574), [#643](https://github.com/MemPalace/mempalace/pull/643), [#665](https://github.com/MemPalace/mempalace/pull/665), [#697](https://github.com/MemPalace/mempalace/pull/697), [#700](https://github.com/MemPalace/mempalace/pull/700), [#381](https://github.com/MemPalace/mempalace/pull/381), [#1679](https://github.com/MemPalace/mempalace/pull/1679)
|
||||
- **Spec version:** `1.0`
|
||||
|
||||
> **Implementation status (2026-06-07).** The §1–2 contract surface (`PalaceRef`,
|
||||
> **Implementation status (2026-08-11).** The §1–2 contract surface (`PalaceRef`,
|
||||
> typed results, `BaseBackend` / `BaseCollection`, capability tokens, and most of
|
||||
> the §10 seam cleanup) landed ahead of this merge via [#1679](https://github.com/MemPalace/mempalace/pull/1679),
|
||||
> which ships in-tree `pgvector`, `qdrant`, and `sqlite_exact` backends. Three
|
||||
> areas this spec defines are tracked as follow-up implementation work and are
|
||||
> **not** yet built: the embedder-identity contract (§1.5 / §5), the maintenance
|
||||
> hooks (§7.3), and search-path metric-awareness (§10, `searcher.py`). Each is
|
||||
> flagged in its section. Accepting this RFC pins the contract those follow-ups
|
||||
> target; it does not assert the implementation is complete.
|
||||
> which ships in-tree `pgvector`, `qdrant`, and `sqlite_exact` backends. The three
|
||||
> areas originally deferred as follow-ups have since landed: embedder-identity
|
||||
> (§1.5 / §5, [#1731](https://github.com/MemPalace/mempalace/pull/1731) /
|
||||
> [#1734](https://github.com/MemPalace/mempalace/pull/1734)), maintenance hooks
|
||||
> (§7.3, [#1732](https://github.com/MemPalace/mempalace/pull/1732)), and searcher
|
||||
> metric-awareness (§10, [#1727](https://github.com/MemPalace/mempalace/pull/1727)).
|
||||
> Accepting this RFC pins the contract; remaining gaps are evolutionary, not
|
||||
> blocking.
|
||||
|
||||
## Summary
|
||||
|
||||
@@ -428,15 +430,22 @@ The `options` kwarg to `get_collection` is a free-form dict. Each backend docume
|
||||
|
||||
Per-tenant collection-name prefixing is not a backend concern. It is handled by the resolver layer above backends: `PalaceRef.namespace` carries the tenant identifier. The `collection_prefix` concept from #697 dissolves into this model.
|
||||
|
||||
**Isolation contract.** `PalaceRef.id` is the *required* isolation key for every backend: within a single backend instance, a record written for one `id` MUST NOT be returned, modified, or deleted by an operation issued for a different `id`. Cross-palace access is a spec violation.
|
||||
**Isolation contract.** `PalaceRef.id` is the *required* isolation key for every backend: within a single backend instance, a record written for one `id` MUST NOT be returned, modified, or deleted by an operation issued for a different `id`. Cross-palace access is a spec violation. This is the non-negotiable blanket MUST; multi-tenant deployments may cite it as their primary partition boundary even when they do not use `namespace`.
|
||||
|
||||
`namespace` is *additional* partitioning. A backend that advertises `supports_namespace_isolation` (§2.1) MUST extend the same guarantee to namespaces:
|
||||
|
||||
> A record written under one `namespace` MUST NOT be returned, modified, or deleted by an operation issued under a different `namespace` within the same backend instance. Cross-namespace access is a spec violation, not a caller misconfiguration.
|
||||
|
||||
This is what hosted multi-tenant deployments cite as the basis for tenant isolation. Authorization (which namespaces a given request may touch) stays on the deployment side; the backend's job is to guarantee no bleed *within* the instance once the namespace is fixed.
|
||||
This is what hosted multi-tenant deployments cite as the basis for *namespace*-level tenant isolation. Authorization (which namespaces a given request may touch) stays on the deployment side; the backend's job is to guarantee no bleed *within* the instance once the namespace is fixed.
|
||||
|
||||
A backend that does **not** advertise `supports_namespace_isolation` (e.g. `sqlite_exact`, whose isolation is the on-disk path alone) MAY ignore `namespace` entirely; callers MUST NOT rely on `namespace` for tenant isolation on such backends. Making isolation a declared capability rather than a blanket `MUST` is deliberate: path-rooted local backends already isolate by `local_path`, and forcing them to re-implement namespace partitioning would be ceremony with no security gain. The conformance suite (§7.1) exercises the namespace arm only for backends that advertise the token.
|
||||
Making namespace isolation a declared capability rather than a blanket `MUST` is deliberate: path-rooted local backends (e.g. `chroma`, `sqlite_exact`) already isolate by `local_path` / `id`, and forcing them to re-implement a second axis would be ceremony with no security gain.
|
||||
|
||||
**Conformance arms.** Self-attestation is not enough. The isolation suite (`tests/_backend_conformance.py` / `assert_partition_isolation`) has two distinct arms:
|
||||
|
||||
1. **Cross-`id` isolation** — required of every backend. Two `PalaceRef`s that differ only in `id` MUST NOT see each other's records.
|
||||
2. **Same-`id` / different-`namespace` isolation** — required of every backend that advertises `supports_namespace_isolation`. Two refs that share `id` (and, when applicable, `local_path`) but differ in `namespace` MUST NOT see each other's records. Distinct from arm 1; a backend that only passes arm 1 does not get to claim the capability.
|
||||
|
||||
**No silent drop.** A backend that does **not** advertise `supports_namespace_isolation` MUST NOT silently accept and ignore a populated `namespace` — that is a latent cross-namespace leak, same spirit as `UnsupportedFilterError` for unknown operators. Such backends MUST either raise (e.g. `UnsupportedCapabilityError`) when `PalaceRef.namespace` is non-`None`, or honor the namespace and advertise the capability. Callers targeting path-rooted backends MUST leave `namespace` as `None`.
|
||||
|
||||
---
|
||||
|
||||
@@ -500,6 +509,9 @@ The suite covers:
|
||||
- Delete-then-query consistency
|
||||
- `close()` releases handles and further calls raise `BackendClosedError`
|
||||
- Concurrent `get_collection` across different palaces is safe
|
||||
- Isolation arm 1: cross-`PalaceRef.id` (every backend)
|
||||
- Isolation arm 2: same-`id` / different-`namespace` (only when `supports_namespace_isolation` is advertised; self-attestation without this arm is a conformance failure)
|
||||
- Non-advertising backends raise on a populated `namespace` rather than silently accepting it (§4.4)
|
||||
|
||||
### 7.2 Parametrized core suite
|
||||
|
||||
|
||||
@@ -123,10 +123,13 @@ class PalaceRef:
|
||||
access is a spec violation.
|
||||
|
||||
Backends that do not advertise ``supports_namespace_isolation`` (e.g.
|
||||
``sqlite_exact``, whose isolation is the on-disk path alone) MAY ignore
|
||||
``namespace`` entirely; callers MUST NOT rely on it for tenant isolation
|
||||
on such backends. Any conforming backend can self-check both guarantees by
|
||||
running the shared assertions in ``tests/_backend_conformance.py``.
|
||||
path-rooted ``chroma`` / ``sqlite_exact``) MUST NOT silently accept and
|
||||
ignore a populated ``namespace`` — they MUST raise
|
||||
:class:`UnsupportedCapabilityError` (same spirit as
|
||||
:class:`UnsupportedFilterError`). Callers targeting those backends MUST
|
||||
leave ``namespace`` as ``None``. Isolation conformance lives in
|
||||
``tests/_backend_conformance.py`` (cross-id arm for every backend;
|
||||
same-id / different-namespace arm for advertisers only).
|
||||
"""
|
||||
|
||||
id: str
|
||||
@@ -604,8 +607,9 @@ class BaseBackend(ABC):
|
||||
``PalaceRef.namespace`` (multi-tenant / hosted deployments) MUST advertise
|
||||
the ``supports_namespace_isolation`` capability token; doing so is a
|
||||
promise to satisfy the cross-namespace guarantee and to pass the namespace
|
||||
arm of the conformance suite. Backends without the token MAY ignore
|
||||
``namespace``.
|
||||
arm of the conformance suite. Backends without the token MUST raise
|
||||
:class:`UnsupportedCapabilityError` when ``PalaceRef.namespace`` is
|
||||
non-``None`` rather than silently accept-and-ignore (RFC 001 §4.4).
|
||||
"""
|
||||
|
||||
name: ClassVar[str]
|
||||
@@ -626,6 +630,21 @@ class BaseBackend(ABC):
|
||||
#: for anything not listed here.
|
||||
maintenance_kinds: ClassVar[frozenset[str]] = frozenset()
|
||||
|
||||
def require_namespace_support(self, palace: PalaceRef) -> None:
|
||||
"""Raise if ``palace.namespace`` is set but this backend does not isolate by it.
|
||||
|
||||
Call at the start of ``get_collection`` (and any other entry that
|
||||
accepts a :class:`PalaceRef`) so non-advertising backends never
|
||||
silently drop a tenant namespace (RFC 001 §4.4).
|
||||
"""
|
||||
if palace.namespace is None:
|
||||
return
|
||||
if "supports_namespace_isolation" not in self.capabilities:
|
||||
raise UnsupportedCapabilityError(
|
||||
f"{type(self).name} does not advertise supports_namespace_isolation; "
|
||||
f"leave PalaceRef.namespace as None (got {palace.namespace!r})"
|
||||
)
|
||||
|
||||
@abstractmethod
|
||||
def get_collection(
|
||||
self,
|
||||
|
||||
@@ -2395,6 +2395,7 @@ class ChromaBackend(BaseBackend):
|
||||
— still used by callers not yet migrated.
|
||||
"""
|
||||
palace_ref, collection_name, create, options = _normalize_get_collection_args(args, kwargs)
|
||||
self.require_namespace_support(palace_ref)
|
||||
|
||||
palace_path = palace_ref.local_path
|
||||
if palace_path is None:
|
||||
|
||||
@@ -1078,6 +1078,7 @@ class SQLiteExactBackend(BaseBackend):
|
||||
**kwargs,
|
||||
) -> SQLiteExactCollection:
|
||||
palace, collection_name, create, read_only = self._normalize_args(args, kwargs)
|
||||
self.require_namespace_support(palace)
|
||||
palace_path = palace.local_path
|
||||
if palace_path is None:
|
||||
raise PalaceNotFoundError("SQLiteExactBackend requires PalaceRef.local_path")
|
||||
|
||||
@@ -41,3 +41,20 @@ def test_local_backends_do_not_claim_namespace_isolation():
|
||||
advertise the namespace-isolation capability (RFC 001 isolation contract)."""
|
||||
assert "supports_namespace_isolation" not in ChromaBackend.capabilities
|
||||
assert "supports_namespace_isolation" not in SQLiteExactBackend.capabilities
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend_cls", _LOCAL_BACKENDS)
|
||||
def test_local_backends_reject_populated_namespace(backend_cls, tmp_path):
|
||||
"""RFC 001 §4.4 no-silent-drop: non-advertising backends MUST raise when
|
||||
PalaceRef.namespace is set rather than accept-and-ignore."""
|
||||
from mempalace.backends.base import UnsupportedCapabilityError
|
||||
|
||||
backend = backend_cls()
|
||||
try:
|
||||
path = tmp_path / "palace"
|
||||
path.mkdir()
|
||||
ref = PalaceRef(id=str(path), local_path=str(path), namespace="tenant-a")
|
||||
with pytest.raises(UnsupportedCapabilityError):
|
||||
backend.get_collection(palace=ref, collection_name="mempalace_drawers", create=True)
|
||||
finally:
|
||||
backend.close()
|
||||
|
||||
Reference in New Issue
Block a user