4e1f543a1d
Replace incomplete/outdated install instructions with accurate, detailed steps grounded in production infrastructure. setup/install.md: - Step 3: run docker compose in-place (not via cp to ~/memory-os/) - Step 5: explicit ~/.hermes/rulebook.md path, create-if-missing, idempotency guard with Memory OS Additions v1 marker - Step 6: document VAULT_PATH, wiki directory structure, link to vault-curator as optional enrichment tool - Step 7: replace fictional crontab with real maintenance scripts and hermes cron create commands; add backfill prerequisite note and exempt-prefix env var docs setup/rulebook.md (NEW): - Generic template with 3 blocks: Memory Architecture, Memory OS infrastructure, Mandatory Verifications - Idempotency markers on every block modifications/soul-rulebook.md: - SOUL.md-only (rulebook sections moved to setup/rulebook.md) - Conditional instructions (add level 2 vs full hierarchy) - Conflict resolution table (4 rules) - Context injection convention with acting vs reasoning distinction scripts/: - 8 maintenance scripts updated with sanitized equivalents - decay_scanner.py, semantic_dedup.py: collection_ → DECAY/DEDUP_EXEMPT_PREFIXES env vars - reflection_trigger.py: hardcoded venv path → /usr/bin/env python3 - All shebangs normalized, all internal references removed |
||
|---|---|---|
| .. | ||
| README.md | ||
| backfill_decay_metadata.py | ||
| bulk_wiki_ingest.py | ||
| context_enhancer.py | ||
| decay_scanner.py | ||
| dlq_manager.py | ||
| pre_validator.py | ||
| reflection_trigger.py | ||
| semantic_dedup.py | ||
| wiki_continuous_ingest.py | ||
README.md
Memory OS — Scripts
Standalone Python scripts that maintain the Qdrant vector database and wiki pipeline.
Qdrant Maintenance
| Script | What it does | Run |
|---|---|---|
decay_scanner.py |
Archives low-importance, aged AI content based on half-life decay | Weekly cron |
backfill_decay_metadata.py |
Populates missing importance_score, last_accessed_at, confidence_score in Qdrant points |
Run once before enabling decay scanner |
semantic_dedup.py |
Merges near-duplicate points (cosine >0.92) | Monthly cron |
Context Injection
| Script | What it does | Used by |
|---|---|---|
context_enhancer.py |
Embedding pipeline: query → embed → search Qdrant (4-level fallback). Also provides BM25 sparse embedding via FastEmbed. | Icarus pre_llm_call hook |
Wiki Pipeline
| Script | What it does | Run |
|---|---|---|
wiki_continuous_ingest.py |
SHA-256 diff detection: finds new/modified wiki files, enqueues ARQ jobs in Redis | Hourly cron |
bulk_wiki_ingest.py |
One-shot bulk ingestion of all wiki files into Qdrant | After initial setup or collection rebuild |
Quality Control
| Script | What it does | Run |
|---|---|---|
pre_validator.py |
Pre-flight validation of wiki documents: YAML frontmatter, required fields, link targets | Before ingestion |
reflection_trigger.py |
Idle detection for ARQ worker — enqueues micro-reflection when queue is empty and within hourly budget | Every 5min cron |
Monitoring
| Script | What it does | Run |
|---|---|---|
dlq_manager.py |
Dead letter queue monitoring and reporting | Every 6h cron |
Environment variables
All scripts read configuration from environment variables. See .env.example in the project root for the full reference.
Key variables:
OPENROUTER_API_KEY— embeddings (required)WIKI_PATH— wiki root directory (default:~/vault/wiki)COLLECTION_NAME— Qdrant collection (default:knowledge_base)EMBEDDING_DIMS— vector dimensions (default: 4096)REDIS_PASSWORD— Redis auth (required for wiki ingest)QDRANT_URL— Qdrant endpoint (default:http://localhost:6333)