docs(rfc-001): fold §4.4 review nits; accept storage-backend spec

Close the last open items on #743 before merge:

- Conformance: document two isolation arms (cross-id for all backends;
  same-id/different-namespace for supports_namespace_isolation advertisers).
- No silent drop: non-advertising backends must raise UnsupportedCapabilityError
  when PalaceRef.namespace is set, rather than accept-and-ignore.
- Wire require_namespace_support() into chroma/sqlite_exact; add conformance test.
- Refresh implementation-status banner now that #1727/#1731/#1732/#1734 landed.
This commit is contained in:
Igor Lins e Silva
2026-08-11 07:50:09 -03:00
parent 4416dbdff0
commit aeac794c1a
5 changed files with 66 additions and 16 deletions
+22 -10
View File
@@ -6,15 +6,17 @@
- **Related:** [#266](https://github.com/MemPalace/mempalace/issues/266), [#574](https://github.com/MemPalace/mempalace/pull/574), [#643](https://github.com/MemPalace/mempalace/pull/643), [#665](https://github.com/MemPalace/mempalace/pull/665), [#697](https://github.com/MemPalace/mempalace/pull/697), [#700](https://github.com/MemPalace/mempalace/pull/700), [#381](https://github.com/MemPalace/mempalace/pull/381), [#1679](https://github.com/MemPalace/mempalace/pull/1679)
- **Spec version:** `1.0`
> **Implementation status (2026-06-07).** The §1–2 contract surface (`PalaceRef`,
> **Implementation status (2026-08-11).** The §1–2 contract surface (`PalaceRef`,
> typed results, `BaseBackend` / `BaseCollection`, capability tokens, and most of
> the §10 seam cleanup) landed ahead of this merge via [#1679](https://github.com/MemPalace/mempalace/pull/1679),
> which ships in-tree `pgvector`, `qdrant`, and `sqlite_exact` backends. Three
> areas this spec defines are tracked as follow-up implementation work and are
> **not** yet built: the embedder-identity contract (§1.5 / §5), the maintenance
> hooks (§7.3), and search-path metric-awareness (§10, `searcher.py`). Each is
> flagged in its section. Accepting this RFC pins the contract those follow-ups
> target; it does not assert the implementation is complete.
> which ships in-tree `pgvector`, `qdrant`, and `sqlite_exact` backends. The three
> areas originally deferred as follow-ups have since landed: embedder-identity
> (§1.5 / §5, [#1731](https://github.com/MemPalace/mempalace/pull/1731) /
> [#1734](https://github.com/MemPalace/mempalace/pull/1734)), maintenance hooks
> (§7.3, [#1732](https://github.com/MemPalace/mempalace/pull/1732)), and searcher
> metric-awareness (§10, [#1727](https://github.com/MemPalace/mempalace/pull/1727)).
> Accepting this RFC pins the contract; remaining gaps are evolutionary, not
> blocking.
## Summary
@@ -428,15 +430,22 @@ The `options` kwarg to `get_collection` is a free-form dict. Each backend docume
Per-tenant collection-name prefixing is not a backend concern. It is handled by the resolver layer above backends: `PalaceRef.namespace` carries the tenant identifier. The `collection_prefix` concept from #697 dissolves into this model.
**Isolation contract.** `PalaceRef.id` is the *required* isolation key for every backend: within a single backend instance, a record written for one `id` MUST NOT be returned, modified, or deleted by an operation issued for a different `id`. Cross-palace access is a spec violation.
**Isolation contract.** `PalaceRef.id` is the *required* isolation key for every backend: within a single backend instance, a record written for one `id` MUST NOT be returned, modified, or deleted by an operation issued for a different `id`. Cross-palace access is a spec violation. This is the non-negotiable blanket MUST; multi-tenant deployments may cite it as their primary partition boundary even when they do not use `namespace`.
`namespace` is *additional* partitioning. A backend that advertises `supports_namespace_isolation` (§2.1) MUST extend the same guarantee to namespaces:
> A record written under one `namespace` MUST NOT be returned, modified, or deleted by an operation issued under a different `namespace` within the same backend instance. Cross-namespace access is a spec violation, not a caller misconfiguration.
This is what hosted multi-tenant deployments cite as the basis for tenant isolation. Authorization (which namespaces a given request may touch) stays on the deployment side; the backend's job is to guarantee no bleed *within* the instance once the namespace is fixed.
This is what hosted multi-tenant deployments cite as the basis for *namespace*-level tenant isolation. Authorization (which namespaces a given request may touch) stays on the deployment side; the backend's job is to guarantee no bleed *within* the instance once the namespace is fixed.
A backend that does **not** advertise `supports_namespace_isolation` (e.g. `sqlite_exact`, whose isolation is the on-disk path alone) MAY ignore `namespace` entirely; callers MUST NOT rely on `namespace` for tenant isolation on such backends. Making isolation a declared capability rather than a blanket `MUST` is deliberate: path-rooted local backends already isolate by `local_path`, and forcing them to re-implement namespace partitioning would be ceremony with no security gain. The conformance suite (§7.1) exercises the namespace arm only for backends that advertise the token.
Making namespace isolation a declared capability rather than a blanket `MUST` is deliberate: path-rooted local backends (e.g. `chroma`, `sqlite_exact`) already isolate by `local_path` / `id`, and forcing them to re-implement a second axis would be ceremony with no security gain.
**Conformance arms.** Self-attestation is not enough. The isolation suite (`tests/_backend_conformance.py` / `assert_partition_isolation`) has two distinct arms:
1. **Cross-`id` isolation** — required of every backend. Two `PalaceRef`s that differ only in `id` MUST NOT see each other's records.
2. **Same-`id` / different-`namespace` isolation** — required of every backend that advertises `supports_namespace_isolation`. Two refs that share `id` (and, when applicable, `local_path`) but differ in `namespace` MUST NOT see each other's records. Distinct from arm 1; a backend that only passes arm 1 does not get to claim the capability.
**No silent drop.** A backend that does **not** advertise `supports_namespace_isolation` MUST NOT silently accept and ignore a populated `namespace` — that is a latent cross-namespace leak, same spirit as `UnsupportedFilterError` for unknown operators. Such backends MUST either raise (e.g. `UnsupportedCapabilityError`) when `PalaceRef.namespace` is non-`None`, or honor the namespace and advertise the capability. Callers targeting path-rooted backends MUST leave `namespace` as `None`.
---
@@ -500,6 +509,9 @@ The suite covers:
- Delete-then-query consistency
- `close()` releases handles and further calls raise `BackendClosedError`
- Concurrent `get_collection` across different palaces is safe
- Isolation arm 1: cross-`PalaceRef.id` (every backend)
- Isolation arm 2: same-`id` / different-`namespace` (only when `supports_namespace_isolation` is advertised; self-attestation without this arm is a conformance failure)
- Non-advertising backends raise on a populated `namespace` rather than silently accepting it (§4.4)
### 7.2 Parametrized core suite
+25 -6
View File
@@ -123,10 +123,13 @@ class PalaceRef:
access is a spec violation.
Backends that do not advertise ``supports_namespace_isolation`` (e.g.
``sqlite_exact``, whose isolation is the on-disk path alone) MAY ignore
``namespace`` entirely; callers MUST NOT rely on it for tenant isolation
on such backends. Any conforming backend can self-check both guarantees by
running the shared assertions in ``tests/_backend_conformance.py``.
path-rooted ``chroma`` / ``sqlite_exact``) MUST NOT silently accept and
ignore a populated ``namespace`` — they MUST raise
:class:`UnsupportedCapabilityError` (same spirit as
:class:`UnsupportedFilterError`). Callers targeting those backends MUST
leave ``namespace`` as ``None``. Isolation conformance lives in
``tests/_backend_conformance.py`` (cross-id arm for every backend;
same-id / different-namespace arm for advertisers only).
"""
id: str
@@ -604,8 +607,9 @@ class BaseBackend(ABC):
``PalaceRef.namespace`` (multi-tenant / hosted deployments) MUST advertise
the ``supports_namespace_isolation`` capability token; doing so is a
promise to satisfy the cross-namespace guarantee and to pass the namespace
arm of the conformance suite. Backends without the token MAY ignore
``namespace``.
arm of the conformance suite. Backends without the token MUST raise
:class:`UnsupportedCapabilityError` when ``PalaceRef.namespace`` is
non-``None`` rather than silently accept-and-ignore (RFC 001 §4.4).
"""
name: ClassVar[str]
@@ -626,6 +630,21 @@ class BaseBackend(ABC):
#: for anything not listed here.
maintenance_kinds: ClassVar[frozenset[str]] = frozenset()
def require_namespace_support(self, palace: PalaceRef) -> None:
"""Raise if ``palace.namespace`` is set but this backend does not isolate by it.
Call at the start of ``get_collection`` (and any other entry that
accepts a :class:`PalaceRef`) so non-advertising backends never
silently drop a tenant namespace (RFC 001 §4.4).
"""
if palace.namespace is None:
return
if "supports_namespace_isolation" not in self.capabilities:
raise UnsupportedCapabilityError(
f"{type(self).name} does not advertise supports_namespace_isolation; "
f"leave PalaceRef.namespace as None (got {palace.namespace!r})"
)
@abstractmethod
def get_collection(
self,
+1
View File
@@ -2395,6 +2395,7 @@ class ChromaBackend(BaseBackend):
— still used by callers not yet migrated.
"""
palace_ref, collection_name, create, options = _normalize_get_collection_args(args, kwargs)
self.require_namespace_support(palace_ref)
palace_path = palace_ref.local_path
if palace_path is None:
+1
View File
@@ -1078,6 +1078,7 @@ class SQLiteExactBackend(BaseBackend):
**kwargs,
) -> SQLiteExactCollection:
palace, collection_name, create, read_only = self._normalize_args(args, kwargs)
self.require_namespace_support(palace)
palace_path = palace.local_path
if palace_path is None:
raise PalaceNotFoundError("SQLiteExactBackend requires PalaceRef.local_path")
+17
View File
@@ -41,3 +41,20 @@ def test_local_backends_do_not_claim_namespace_isolation():
advertise the namespace-isolation capability (RFC 001 isolation contract)."""
assert "supports_namespace_isolation" not in ChromaBackend.capabilities
assert "supports_namespace_isolation" not in SQLiteExactBackend.capabilities
@pytest.mark.parametrize("backend_cls", _LOCAL_BACKENDS)
def test_local_backends_reject_populated_namespace(backend_cls, tmp_path):
"""RFC 001 §4.4 no-silent-drop: non-advertising backends MUST raise when
PalaceRef.namespace is set rather than accept-and-ignore."""
from mempalace.backends.base import UnsupportedCapabilityError
backend = backend_cls()
try:
path = tmp_path / "palace"
path.mkdir()
ref = PalaceRef(id=str(path), local_path=str(path), namespace="tenant-a")
with pytest.raises(UnsupportedCapabilityError):
backend.get_collection(palace=ref, collection_name="mempalace_drawers", create=True)
finally:
backend.close()