a4fc26eed360357640501540f0059014ecab58ea
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Igor Lins e Silva
|
287d8f1b4d | fix: scope hallway reads and deletion to selected palace | ||
|
Igor Lins e Silva
|
f68cb8d682 | fix: scope derived graph state to explicit palace | ||
|
Grace Gettert
|
4fd12318df |
fixup(hallways): drop _HALLWAY_FILE back-compat shim, migrate existing tests to resolver
Replaces the back-compat shim in _load_hallways/_save_hallways (which honored direct monkey-patches of the _HALLWAY_FILE module constant) with a clean single-source-of-truth resolver, matching the palace_graph tunnel-file migration in 3.3.6. The three existing test sites (tests/test_hallways.py, tests/test_hallways_pagination.py, tests/test_mcp_server.py) now monkey-patch _get_hallway_file and _legacy_hallway_file directly, exactly mirroring the helper in tests/test_palace_graph_tunnels.py. Production code now has one branch through the path resolution instead of two. No behavior change. 269/269 hallway + tunnel + mcp-server tests pass on Python 3.11 and 3.12, ruff clean. |
||
|
Milla J
|
4d3daf96d3 |
fix(hallways): paginate drawer fetch to avoid SQLite variable overflow on large wings (#1619)
compute_hallways_for_wing fetched the whole wing in a single
col.get(where={"wing": wing}). ChromaDB binds one SQL variable per matched id,
so on a wing larger than SQLITE_MAX_VARIABLE_NUMBER (32766) the call raised
"too many SQL variables" inside chromadb. The exception was caught, so the mine
completed — but the wing's hallway graph silently never built, and the
cross-wing tunnels promoted from it were starved, on exactly the large wings
that benefit most from navigation. Confirmed threshold: a 42,062-drawer wing
crashed; a 29,629-drawer wing succeeded.
Replace the single where-get with the established pagination pattern: count()
+ get(limit=5000, offset=...) filtered to the wing client-side — matching
miner.status, palace.regenerate_closets, and palace_graph.build_graph, which
already paginate to dodge the same 32766 limit.
Tests:
- test_hallways_pagination: a collection whose where-get raises (simulating the
overflow) while count() + paginated get works — RED before, GREEN after.
- test_hallways: _fake_collection updated to the paginated API; existing
hallway tests are unchanged in behavior.
Closes #1619.
|
||
|
Milla J
|
f1aa409a94 |
fix(dynamics): address PR #1578 gemini review — canonicalize lookup key + idiomatic dict update
Addresses both inline comments left by gemini-code-assist on PR #1578. ## What this commit does 1. **HIGH PRIORITY (gemini, hallways.py:260) — canonicalize dynamics preservation lookup key.** The previous code built the lookup key as ``(h.get("entity_a"), h.get("entity_b"))``, but the ``pair_counts`` dict and ``_hallway_id`` both use sorted entity pairs for symmetric IDs. A persisted record with reversed entity order (Lumi as entity_a, Aya as entity_b) would silently miss the lookup and lose its accumulated dynamics on recompute. Now the lookup canonicalizes the same way ID generation does: key = tuple(sorted([h.get("entity_a"), h.get("entity_b")])) Failure mode is rare in practice (the algorithm normally writes sorted records), but any legacy data, manual JSON edit, or future import path that violates the ordering would have silently broken L7 preservation. Belt-and-suspenders fix. 2. **MEDIUM PRIORITY (gemini, palace_graph.py:512) — idiomatic field preservation in tunnel dedup path.** Refactored the manual for-loop into a dict-update with a comprehension. Field list lives in one place; future schema expansion can't accidentally drop a field. No behavior change. ## Test added (RED-first) test_recompute_preserves_dynamics_when_existing_record_has_reversed_entity_order Pre-populates the hallway store with a record whose ``entity_a`` / ``entity_b`` are in REVERSED (non-sorted) order, bumps its strength / stability / access_count to non-default values, then runs ``compute_hallways_for_wing``. Asserts the recomputation preserves the bumped values — proving the canonical lookup matches against arbitrarily-ordered persisted entity pairs. Reproduces the bug class gemini flagged: RED before this commit (strength=1.0 instead of 4.2), GREEN after. ## Verification pytest tests/test_hallways.py::TestHallwayDynamicsIntegration tests/test_palace_graph_tunnels.py::TestTunnelDynamicsIntegration tests/test_dynamics.py → 32 passed (was 31; added 1 new for the reverse-order case) pytest -q (full mempalace suite) → 2112 passed, 3 skipped, 0 regressions ruff check mempalace/dynamics.py mempalace/hallways.py mempalace/palace_graph.py tests/ → All checks passed! ruff format --check ... → 3 files already formatted (pinned 0.15.9) |
||
|
Milla J
|
a2ba1ccb05 |
feat(dynamics): Hebbian potentiation + Ebbinghaus decay for halls + tunnels
Adds the L7 living-connection dynamics layer for the hallway primitive (PR #1558) and the tunnel primitive (PR #1565). Connections now carry strength, stability, last_activated, and access_count fields; pure-math helpers in a new ``mempalace.dynamics`` module update these via Hebbian potentiation (Hebb 1949) on co-access and Ebbinghaus exponential decay (Ebbinghaus 1885) on time-since-use, with the Cepeda spacing effect (Cepeda et al. 2006) growing stability when reinforcement is distributed rather than massed. Existing palaces continue to work unchanged — the new fields populate via safe defaults (strength=1.0, stability=1.0, last_activated=created_at, access_count=0) lazily on any potentiate/decay call. No migration step required. No schema break. Old records work; new records ship with full L7 state from creation. ## What this commit does 1. New module ``mempalace/dynamics.py`` — pure math, no I/O, no chromadb. Three public helpers: - ``initialize_dynamics_fields(connection, *, now=None)`` Backfill helper. Populates missing strength / stability / last_activated / access_count fields. Existing fields are NOT overwritten. Safe for pre-L7 records. - ``potentiate(connection, *, increment=POTENTIATION_INCREMENT, now=None)`` Hebbian strengthening on a co-access event. Increments strength (capped at MAX_STRENGTH), updates last_activated, increments access_count. Grows stability by STABILITY_INCREMENT ONLY when the gap since the prior activation is at least SPACED_INTERVAL_HOURS — implements the Cepeda spacing effect (rapid bursts don't build durability; distributed practice does). - ``apply_decay(connection, *, now=None)`` Ebbinghaus exponential decay. New strength = old * exp(-days_since / stability), floored at STRENGTH_FLOOR (0.05) so connections never reach zero — only become dim. Idempotent at the same instant. Higher stability = slower decay. 2. Integration in ``mempalace/hallways.py:compute_hallways_for_wing`` — before persisting the new hallway list for a wing, looks up the existing wing's hallways and builds a (entity_a, entity_b) → dynamics lookup. Each new record gets the preserved dynamics if the entity pair existed before, OR default dynamics via ``initialize_dynamics_fields`` if it's a brand-new pair. Without this preservation, every mine would wipe accumulated connection weights — defeating the whole L7 layer. 3. Integration in ``mempalace/palace_graph.py:create_tunnel`` — the existing dedup-on-canonical-id path now also preserves the four dynamics fields from the existing record onto the recreated tunnel (in addition to the already-preserved ``created_at``). Brand-new tunnels and legacy records (created before L7) get default dynamics via ``initialize_dynamics_fields``. ## Tests (RED-first) Twenty-five new tests in ``tests/test_dynamics.py`` covering the pure math: field initialization defaults, potentiation strength increment + cap, last_activated update, access_count increment, spaced vs. rapid reinforcement (Cepeda spacing effect), decay rate (exp(-days/stability)), floor clamping, idempotency at same instant, higher-stability slower-decay behavior, no-time-passed no-op, backfill on missing fields, mutate-and- return-same-dict chaining. Plus integration scenarios: potentiation restoring decayed strength, repeated spaced reinforcement growing stability monotonically, burst reinforcement not growing stability. Six new integration tests: test_new_hallway_record_carries_all_dynamics_fields test_recompute_preserves_accumulated_strength test_recompute_initializes_dynamics_for_brand_new_pairs test_new_tunnel_carries_all_dynamics_fields test_recreate_tunnel_preserves_accumulated_dynamics test_recreate_tunnel_initializes_dynamics_for_legacy_records All RED-first (KeyError: 'strength' before the integration commits). All GREEN after. ## Backward compatibility Pre-existing palaces work unchanged: - Old hallway records with no dynamics fields are still readable; the next ``compute_hallways_for_wing`` recompute populates the missing fields via ``initialize_dynamics_fields``. - Old tunnel records similarly; the next ``create_tunnel`` recreate (or any future call passing the record through ``initialize_dynamics_fields``) populates the missing fields. - A ``test_recreate_tunnel_initializes_dynamics_for_legacy_records`` test pins this contract explicitly. No forced migration. No schema break. Old records work; new behavior kicks in on first touch. ## What this does NOT do (intentionally scoped out) - Spreading activation at search time — that's a search-layer change; bigger PR; separate work. - Lazy decay-on-read in ``list_hallways`` / ``list_tunnels`` — the math is here; wiring it into the read path is a follow-up. - Config exposure of the tunable constants — STRENGTH_FLOOR, MAX_STRENGTH, POTENTIATION_INCREMENT etc. are hardcoded module-level for v1. If empirical palace data shows the defaults need tuning, expose via MempalaceConfig in a follow-up. - Decay-aware ranking in search — the strength field is now there; the search ranker doesn't yet use it. Follow-up. ## Verification pytest tests/test_dynamics.py → 25 passed (RED before this commit; GREEN after) pytest tests/test_hallways.py::TestHallwayDynamicsIntegration → 3 passed (RED before; GREEN after) pytest tests/test_palace_graph_tunnels.py::TestTunnelDynamicsIntegration → 3 passed (RED before; GREEN after) pytest -q (full mempalace suite) → 2111 passed, 3 skipped, 0 regressions ruff check mempalace/dynamics.py mempalace/hallways.py mempalace/palace_graph.py tests/ → All checks passed! ruff format --check ... → 6 files already formatted (pinned 0.15.9) ## Research grounding - Hebb, D. O. (1949). The Organization of Behavior. Wiley. → "Neurons that fire together, wire together" → potentiate() - Ebbinghaus, H. (1885). Über das Gedächtnis. → Exponential forgetting curve → apply_decay() - Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354-380. → Spacing effect → stability growth on spaced reinforcement |
||
|
Milla J
|
07807ab8ab |
feat(hallways): within-wing entity-to-entity connector primitive
Introduces the missing primitive in mempalace's palace architecture.
## The architectural gap
Mempalace's data model today is Wing → Rooms → Tunnels. Tunnels connect
rooms ACROSS wings (via shared topic words). What's missing — and what
this commit adds — is the connector between ENTITIES (people, projects,
concepts, interests) inside one wing.
The intended full architecture is:
WING → has DRAWERS (each tagged with entities)
entities → connected to other entities by HALLWAYS
(within-wing, built from drawer co-occurrence)
hallways → are the primitive
tunnels → use hallways to spawn
cross-wing connections
A hallway is the structural fact of "these two entities travel together
inside this wing." If Aya and Lumi are mentioned together in 47 drawers
across diary, letters, and ideas rooms, that's a hallway between Aya
and Lumi. If Aya and "consciousness" co-occur in 19 drawers, that's
another hallway. Entities aren't only people — projects, concepts, and
interests share the same entity-tag substrate, so a hallway can connect
a person to a concept (Aya ↔ consciousness) just as easily as person to
person (Aya ↔ Lumi).
`palace_graph.py:253`'s docstring currently calls cross-wing tunnels
"hallways" — that conflation is fixed in a separate naming-disambiguation
follow-up. This commit just introduces the primitive.
## Public API
New module `mempalace/hallways.py`:
compute_hallways_for_wing(wing, col=None, min_count=2) -> list[dict]
Query drawers for wing. For each drawer with >=2 entities, every
unordered pair of entities is one co-occurrence; increment a counter
and track the room. For each entity pair with count >= min_count,
materialize a hallway record. Persists to ~/.mempalace/hallways.json.
Returns records created for this wing (records for other wings
already on disk are preserved).
list_hallways(wing=None) -> list[dict]
Return all hallways, optionally filtered by wing.
delete_hallway(hallway_id) -> bool
Remove one record by id. Returns True on success.
_load_hallways() / _save_hallways(state)
JSON persistence at ~/.mempalace/hallways.json with 0600 perms on
POSIX and atomic os.replace writes. Mirrors palace_graph's tunnel
persistence pattern exactly.
## Data model (hallways.json)
```jsonc
{
"schema_version": 1,
"hallways": [
{
"id": "hallway_<wing>_<entity_a>_<entity_b>_<sha8>",
"wing": "wing_aya",
"entity_a": "Aya",
"entity_b": "Lumi",
"co_occurrence_count": 47,
"rooms": ["diary", "letters", "ideas"],
"label": "Aya ↔ Lumi (co-occur in 47 drawers across 3 rooms: diary, letters, ideas)",
"created_at": "2026-05-20T03:30:00Z",
"created_by": "auto"
}
]
}
```
Hallway IDs are deterministic — `sha256(wing + sorted(entity_a, entity_b))`
truncated. Sorted-pair canonicalization means (Aya, Lumi) and (Lumi, Aya)
produce the same id, so re-runs upsert idempotently and the same drawer
appearing as "Aya;Lumi" or "Lumi;Aya" counts as one co-occurrence (not two).
## Tests
17 RED-first tests in `tests/test_hallways.py`:
Storage:
- _load_hallways returns [] for missing file
- _load_hallways returns [] for corrupt JSON
- _save_hallways + _load_hallways round-trips
Algorithm:
- returns [] for unknown wing
- returns [] when no drawer has >=2 entities
- creates hallway for entity pair when threshold met (Aya↔Lumi in 3 drawers)
- connects person to concept (Aya↔consciousness — entities aren't only people)
- respects min_count threshold (rejects 2 co-occurrences when min_count=3)
- creates deterministic id per entity pair (idempotent re-runs)
- entity pair is symmetric (Aya;Lumi and Lumi;Aya are the same pair)
- persists to JSON file
- tracks rooms across co-occurrences (rooms set vs. count distinction)
- skips sentinel drawers (file_already_mined bookkeeping noise)
Query API:
- list_hallways returns all when no filter
- list_hallways filters by wing
- delete_hallway removes record and persists
- delete_hallway unknown id returns False
## Verification
ruff check mempalace/hallways.py tests/test_hallways.py
→ All checks passed!
ruff format --check ...
→ 2 files already formatted (pinned 0.15.9)
pytest tests/test_hallways.py
→ 17 passed
pytest -q (full mempalace suite)
→ 1936 passed, 1 skipped, 0 regressions caused by this PR
Linux CI parity replicated locally via OrbStack containers
(Python 3.9, 3.11, 3.13): 17/17 hallway tests pass, ruff clean on all
three.
## Not in this commit (deferred)
This PR introduces the primitive — the module, the entity-centric API,
the storage, the tests. It does NOT yet:
- integrate compute_hallways_for_wing() into miner.py / convo_miner.py /
format_miner.py post-mine flow (next PR — small one-line addition
per miner alongside the existing _compute_topic_tunnels call)
- refactor _compute_topic_tunnels_for_wing to BUILD ON hallways
(the architecturally meaningful follow-up — own PR)
- rename palace_graph.py:253's misleading docstring that calls
cross-wing tunnels "hallways" (cleanup PR)
- expose hallways through the MCP server (when there's a clear UX
need for it)
Each follow-up is small enough to review in isolation. Shipping the
primitive first lets reviewers think about the API surface separately
from the integration choices.
Per the design sequence: Wing → Drawer-entities → Hallway → Tunnel.
Hallways come first. This is them.
|