Commit Graph

12 Commits

Author SHA1 Message Date
@aaronjmars e654b9f5ee
fix(security): use Path.is_relative_to for WIKI_PATH containment in ingest_file
Replaces str.startswith() with Path.is_relative_to() to prevent sibling-prefix paths (e.g. /wiki-shared/x.md when WIKI_PATH=/wiki) from bypassing the entry guard. Includes regression test with 3 scenarios proving the bug exists pre-patch and is fixed post-patch.

Closes #26
 
Co-authored-by: aaronjmars <61592645+aaronjmars@users.noreply.github.com>
2026-06-10 07:20:07 -03:00
ClaudioDrews 7de2265c66 fix: remove stale knowledge_base_hybrid references, fix generic ENV_PATH
- file_ingestion.py: docstrings now reference 'configured collection'
  instead of hardcoded 'knowledge_base_hybrid' (code already uses
  COLLECTION_NAME env var, defaulting to knowledge_base)
- context_enhancer.py: same docstring fix
- wiki_continuous_ingest.py: ENV_PATH now points to memory-os/docker/.env
  instead of ai-stack/cognitive-agent/.env (production-specific path)
2026-06-07 14:05:31 -03:00
ClaudioDrews 16e394cd07 fix: align worker UID with host user (1000) for bind mount read access
Worker ran as appuser (UID 10001) but wiki files are owned by
host user hermes (UID 1000) with permissions 600. Changed
Dockerfile to create appuser with UID 1000 so the container
process can read bind-mounted wiki files.
2026-06-07 13:06:11 -03:00
ClaudioDrews 1422a6d583 fix: resolve 5 easy Phase 4 performance/resilience issues
V4.04: Add retry_transient() to dlq_manager — re-enables reported transient
  failures for retry with --retry CLI flag, increments retry_count/last_retry

V4.05: save_report() now truncates JSONL to MAX_REPORT_HISTORY entries
  after each append (best-effort rotation)

V4.06: Wrap datetime.fromisoformat() in try/except (ValueError, TypeError)
  in get_status_summary() — skips entries with malformed timestamps

V4.08: Replace body[:50] dedup key with hashlib.sha256(body).hexdigest()
  in fabric-retrieve.py deduplicate()

V4.09: Add OrderedDict LRU cache (256 entries) to embedding.py get_embedding()
  — avoids recomputing embeddings for repeated text
2026-06-03 11:53:18 -03:00
ClaudioDrews 384395b78d fix: resolve all 7 Phase 1 setup/infra issues
V1.01: Add fastembed, httpx, redis to requirements.txt (now 10 lines, 9 deps)
V1.03: Create setup/setup_db.py — idempotent SQLite schema for state.db + memory_store.db
V1.04: Create .dockerignore (21 patterns — .env, *.key, __pycache__, etc.)
V1.05: Dockerfile COPY --chown=appuser:appuser (after useradd, before USER switch)
V1.06: Add EXPOSE 8000 + HEALTHCHECK (Redis ping, 30s interval)
V1.07: Document database setup as Step 2 in install.md (pip install + setup_db.py + table listing)
V1.08: Remove change-me from REDIS_PASSWORD in .env.example
V1.09: Replace gcc with build-essential + python3-dev in Dockerfile
2026-06-03 11:31:04 -03:00
ClaudioDrews a3d644a5c9 fix: resolve 4 code-critical issues from Fase 2 audit
V2.07 (A01): Replace subprocess.run(['arq', ...]) with programmatic
  arq.worker.run_worker(WorkerSettings). Fixes signal handling,
  graceful shutdown, and logging unification.

V2.10 (D11): Fix filename → file_path in backfill_decay_metadata.py
  resolve_timestamp() now reads payload.file_path (consistent with
  what bulk_wiki_ingest.py writes). Replaced VAULT_ROOT path
  reconstruction with direct Path(file_path).

V2.11 (D05): Auto-create Qdrant collection in bulk_wiki_ingest.py
  If collection doesn't exist, creates it with hybrid schema
  (dense 4096d Cosine + sparse BM25) instead of sys.exit(1).

V2.12 (D27): Add sparse vector generation to bulk_wiki_ingest.py
  Optional fastembed BM25 sparse vectors alongside dense embeddings.
  Falls back to dense-only if fastembed not installed. Vectors stored
  as Qdrant named vectors: {'dense': ..., 'sparse': ...}
2026-06-03 10:54:53 -03:00
ClaudioDrews ff781f9824 feat: add EMBEDDING_API_KEY for authenticated non-OpenRouter endpoints (fixes #8)
embedding.py:
- Add EMBEDDING_API_KEY env var (no fallback — explicit per-branch)
- OpenRouter branch: uses OPENROUTER_API_KEY + vendor headers
- Generic branch: sends Authorization: Bearer <EMBEDDING_API_KEY> if set
- Unauth fallback: no header (localhost, no auth needed)
- Credential isolation: no cross-branch key leakage

.env.example + setup/install.md:
- Document EMBEDDING_API_KEY as optional for non-OpenRouter
  authenticated endpoints (vLLM --api-key, custom hosted services)
2026-06-03 09:37:38 -03:00
ClaudioDrews b33b8d4196 fix: SyntaxError on QDRANT_API_KEY placeholder (fixes #10)
Split the corrupted line 15 into two valid assignments:
- QDRANT_API_KEY = os.environ.get('QDRANT_API_KEY', '')
- EMBEDDING_DIMS = int(os.environ.get('EMBEDDING_DIMS', '4096'))

The merged line 'QDRANT_API_KEY=os.env...DIMS = ...' was a leftover
placeholder that prevented the ARQ worker from starting.

Reported-by: CG1up
2026-06-03 08:18:32 -03:00
brian-doherty 02018160d2
fix(worker): add missing import os, Qdrant auth, and env config (#6)
Fix 4 bugs that prevent the ARQ worker from booting with a
Qdrant instance that has an API key set:

1. reflection.py: add missing 'import os' (NameError crash at
   COLLECTION_NAME = os.environ.get(...))

2. local_qdrant.py: read QDRANT_API_KEY from environment

3. local_qdrant.py: pass api_key= to AsyncQdrantClient and set
   https=False to prevent SSL error when API key triggers
   auto-HTTPS against a local HTTP Qdrant instance

4. docker-compose.yml: pass QDRANT_API_KEY env var to the
   worker service so the key is available inside the container
2026-06-02 13:21:30 -03:00
Claudio Drews 8e8ea95458
fix(embedding): make OpenRouter auth and headers conditional — unblock 100% local usage (#7)
- Remove unconditional OPENROUTER_API_KEY check that blocked local providers
  (Ollama, vLLM, llama.cpp) from running without an API key
- Send OpenRouter-specific headers (HTTP-Referer, X-Title) only when
  EMBEDDING_API_BASE contains 'openrouter'
- Stop sending empty Authorization header to non-OpenRouter endpoints
- Update .env.example and setup/install.md to clarify that
  OPENROUTER_API_KEY is only required for OpenRouter

Closes #1

Co-authored-by: ClaudioDrews <claudio@drews.com.br>
2026-06-02 13:05:12 -03:00
Claudio Drews 1281d7a778 fix: make embedding provider-agnostic via env vars
- embedding.py: EMBEDDING_API_BASE and EMBEDDING_MODEL now read from env
  with OpenRouter defaults (preserves backward compatibility)
- .env.example: document embedding backend config with Qwen3 rationale
- setup/install.md: update prerequisite to reflect configurable backend
- layers/05-qdrant.md: add "Why Qwen3-Embedding-8B" section
  (multilingual, quality, speed, cost)

Closes #1
2026-06-02 09:45:31 -03:00
ClaudioDrews 0b32ffcfc2 Initial commit: Memory OS — 6-layer memory architecture for Hermes Agent 2026-05-31 16:50:37 -03:00