v0.2.0 — automated install, community infra, 20+ audit fixes
- setup.sh: one-command install (curl | bash), 10 phases, idempotent - smoke_test.sh + test_ingestion.py: post-install verification - .github/: issue templates, PR template, contributing guide - QUICKSTART.md: quick start with setup.sh as primary method - README: release notes, install links - install.md: banner redirecting to automated install - All docs translated to English
This commit is contained in:
ClaudioDrews
parent
67ceb78202
commit
6418ed30a1
|
|
@ -0,0 +1,29 @@
|
|||
# Contributing to Memory OS
|
||||
|
||||
We're in active development and value feedback.
|
||||
Whether you write code, docs, tests, or have ideas — every contribution is welcome.
|
||||
|
||||
## Ways to contribute
|
||||
|
||||
- **Code:** performance, provider-agnostic features, integrations
|
||||
- **Documentation:** tutorials, translations, corrections
|
||||
- **Testing:** test on different hardware (8GB RAM, ARM, macOS)
|
||||
- **Icarus Fabric skills:** new connectors, usage examples
|
||||
- **Ideas:** GitHub Discussions for features and improvements
|
||||
|
||||
## PR workflow
|
||||
|
||||
1. Open an issue first (for features) or go directly (for small fixes)
|
||||
2. Fork + branch
|
||||
3. Follow the existing code style
|
||||
4. Test locally
|
||||
5. Open the PR — we respond quickly
|
||||
|
||||
## Local setup
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ClaudioDrews/memory-os.git
|
||||
cd memory-os
|
||||
```
|
||||
|
||||
Follow the guide: [setup.sh](setup.sh) (one command) or [setup/install.md](setup/install.md) (manual)
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
name: Bug Report
|
||||
description: Report a bug in Memory OS
|
||||
labels: ["bug"]
|
||||
body:
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Describe the bug
|
||||
description: What happened? What did you expect to happen?
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: steps
|
||||
attributes:
|
||||
label: Steps to reproduce
|
||||
description: Exact commands or actions that trigger the bug
|
||||
placeholder: |
|
||||
1. Run `./setup.sh`
|
||||
2. Open Hermes
|
||||
3. ...
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: Memory OS version
|
||||
description: Commit SHA or tag (e.g. `v0.1.0`, `65029e1`)
|
||||
placeholder: v0.1.0
|
||||
- type: textarea
|
||||
id: environment
|
||||
attributes:
|
||||
label: Environment
|
||||
description: OS, Docker version, Hermes version, embedding backend
|
||||
placeholder: |
|
||||
- OS: Ubuntu 24.04
|
||||
- Docker: 27.x
|
||||
- Hermes: v0.15.1
|
||||
- Embedding: OpenRouter / Ollama
|
||||
- type: textarea
|
||||
id: logs
|
||||
attributes:
|
||||
label: Relevant logs
|
||||
description: Any error messages, stack traces, or log output
|
||||
render: shell
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: GitHub Discussions
|
||||
url: https://github.com/ClaudioDrews/memory-os/discussions
|
||||
about: Ask questions, share ideas, and discuss Memory OS
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
name: Feature Request
|
||||
description: Propose a new feature or improvement for Memory OS
|
||||
labels: ["enhancement"]
|
||||
body:
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Problem
|
||||
description: What problem does this solve? Why does it matter?
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: solution
|
||||
attributes:
|
||||
label: Proposed solution
|
||||
description: How would you like it to work? Be as specific as possible.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
attributes:
|
||||
label: Alternatives considered
|
||||
description: What else did you try or think about?
|
||||
- type: dropdown
|
||||
id: scope
|
||||
attributes:
|
||||
label: Scope
|
||||
description: How broad is the change?
|
||||
options:
|
||||
- Single file / script
|
||||
- Multiple files / layer
|
||||
- Cross-layer / architectural
|
||||
- New integration or provider
|
||||
validations:
|
||||
required: true
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
## Checklist
|
||||
|
||||
- [ ] Follows existing code style
|
||||
- [ ] No new hardcoded paths or API keys (use env vars or `.env.example`)
|
||||
- [ ] Backward compatible (or breaking change is documented)
|
||||
- [ ] If adding a new file: `python3 -m py_compile` passes
|
||||
- [ ] If changing behavior: updated relevant docs in `setup/` or `layers/`
|
||||
|
||||
## Description
|
||||
|
||||
<!-- Brief summary of what this PR does -->
|
||||
|
||||
## Type
|
||||
|
||||
- [ ] Bug fix
|
||||
- [ ] New feature
|
||||
- [ ] Performance improvement
|
||||
- [ ] Documentation
|
||||
- [ ] Refactor (no behavior change)
|
||||
- [ ] Security fix
|
||||
|
||||
## Related issues
|
||||
|
||||
<!-- e.g. Closes #14 -->
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
# Memory OS — Quick Start
|
||||
|
||||
## One-command install
|
||||
|
||||
```bash
|
||||
curl -sSL https://raw.githubusercontent.com/ClaudioDrews/memory-os/main/setup.sh | bash
|
||||
```
|
||||
|
||||
This installs everything: Docker stack (Redis + Qdrant + Worker), Icarus plugin, SQLite databases, wiki vault, and environment variables. Safe to re-run — all steps are idempotent.
|
||||
|
||||
**Requires:** Docker, Python 3.11+, Hermes Agent. The script auto-detects your OpenRouter key and prompts only if missing.
|
||||
|
||||
> Prefer manual control? Follow [setup/install.md](setup/install.md) — step-by-step guide with validation checkpoints.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Docker** (Docker Compose v2)
|
||||
- **Python 3.11+**
|
||||
- **Hermes Agent** (v0.14.0 or later)
|
||||
- **OpenRouter API key** (or local Ollama for embeddings)
|
||||
|
||||
## 1. Clone
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ClaudioDrews/memory-os.git
|
||||
cd memory-os
|
||||
```
|
||||
|
||||
## 2. Install
|
||||
|
||||
Follow [setup/install.md](setup/install.md) — step-by-step guide with validation checkpoints.
|
||||
|
||||
## 3. Verify
|
||||
|
||||
Once installed, confirm the stack is operational:
|
||||
|
||||
```bash
|
||||
# Docker services
|
||||
docker compose ps # qdrant + redis + worker should be "healthy"
|
||||
|
||||
# Qdrant
|
||||
curl -s http://localhost:6333/healthz # should return "ok"
|
||||
|
||||
# Icarus plugin
|
||||
hermes plugins list | grep icarus
|
||||
```
|
||||
|
||||
## 4. Use
|
||||
|
||||
Open Hermes. From the next session onward, it will:
|
||||
- Recall past decisions (Icarus Fabric)
|
||||
- Search your vault documents (Qdrant)
|
||||
- Cross-reference facts you've mentioned (fact_store)
|
||||
|
||||
## 5. Add content
|
||||
|
||||
```bash
|
||||
# Adjust to your vault path
|
||||
mkdir -p ~/vault/wiki/raw
|
||||
echo "# My notes" > ~/vault/wiki/raw/notes.md
|
||||
```
|
||||
|
||||
The worker detects and indexes new files automatically.
|
||||
|
||||
## Next steps
|
||||
|
||||
- Full install guide: [setup/install.md](setup/install.md)
|
||||
- Architecture: [layers/](layers/)
|
||||
- How to contribute: [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md)
|
||||
14
README.md
14
README.md
|
|
@ -13,6 +13,18 @@ Memory OS turns Hermes Agent into a real long-term collaborator — one that rem
|
|||
|
||||
---
|
||||
|
||||
## What's New in v0.2.0
|
||||
|
||||
**One-command install.** `curl -sSL https://raw.githubusercontent.com/ClaudioDrews/memory-os/main/setup.sh | bash` sets up the entire stack — Docker services, SQLite databases, Icarus plugin, environment — in one shot. The 10-step manual guide is now a fallback for troubleshooting.
|
||||
|
||||
**Community infrastructure.** Issue templates (bug report + feature request), PR checklist, and contributing guide. Project is ready for external contributors — and already has them.
|
||||
|
||||
**20+ fixes from systematic audit.** Community-driven review across setup, configuration, performance, and resilience. Highlights: provider-agnostic LLM extraction, O(1) path lookups, FTS5-powered session search, semantic dedup at scale, and idempotent database initialization.
|
||||
|
||||
**Installation verified on real hardware.** Smoke tests and ingestion tests ship with the repo. The automated installer has been tested end-to-end — including on modest machines where Docker build times exposed UX gaps that are now handled gracefully.
|
||||
|
||||
---
|
||||
|
||||
## The problem every serious Hermes user knows
|
||||
|
||||
You spend hours configuring the agent, teaching it your preferences, solving hard problems together — and in the next session it acts like it's meeting you for the first time.
|
||||
|
|
@ -160,6 +172,6 @@ If you're like me — tired of amnesiac agents — Memory OS was built for you.
|
|||
**Want to see the agent remember for real?**
|
||||
Clone it, run it, feel the difference.
|
||||
|
||||
→ [Setup guide](setup/install.md) · [Layer deep-dives](layers/) · [Infrastructure docs](infrastructure/architecture.md) · [Operational skills](skills/) · [License](LICENSE)
|
||||
→ [Quick install](setup.sh) · [Manual guide](setup/install.md) · [Layer deep-dives](layers/) · [Infrastructure docs](infrastructure/architecture.md) · [Operational skills](skills/) · [License](LICENSE)
|
||||
|
||||
MIT License · Built with obsession by someone who runs Hermes every single day.
|
||||
|
|
@ -0,0 +1,187 @@
|
|||
#!/usr/bin/env python3
|
||||
"""End-to-end ingestion test for Memory OS.
|
||||
|
||||
Verifies the full pipeline: enqueue → ARQ worker → embedding → Qdrant upsert.
|
||||
|
||||
Usage:
|
||||
python3 scripts/test_ingestion.py
|
||||
|
||||
Environment:
|
||||
REDIS_PASSWORD Redis password (required)
|
||||
QDRANT_API_KEY Qdrant API key (default: "")
|
||||
REDIS_HOST Redis host (default: localhost)
|
||||
REDIS_PORT Redis port (default: 6379)
|
||||
QDRANT_HOST Qdrant host (default: localhost)
|
||||
QDRANT_PORT Qdrant port (default: 6333)
|
||||
COLLECTION_NAME Qdrant collection (default: knowledge_base)
|
||||
|
||||
Returns exit code 0 on success, 1 on failure.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
import uuid
|
||||
|
||||
|
||||
# ── Config from env ──────────────────────────────────────────────────────────
|
||||
REDIS_HOST = os.environ.get("REDIS_HOST", "localhost")
|
||||
REDIS_PORT = int(os.environ.get("REDIS_PORT", "6379"))
|
||||
REDIS_PASSWORD = os.environ.get("REDIS_PASSWORD", "")
|
||||
QDRANT_HOST = os.environ.get("QDRANT_HOST", "localhost")
|
||||
QDRANT_PORT = int(os.environ.get("QDRANT_PORT", "6333"))
|
||||
QDRANT_API_KEY = os.environ.get("QDRANT_API_KEY", "")
|
||||
COLLECTION_NAME = os.environ.get("COLLECTION_NAME", "knowledge_base")
|
||||
TIMEOUT = 60 # seconds for ARQ job completion
|
||||
POLL_INTERVAL = 2 # seconds between polls
|
||||
|
||||
|
||||
def fail(msg: str) -> None:
|
||||
print(f"❌ {msg}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def ok(msg: str) -> None:
|
||||
print(f"✅ {msg}")
|
||||
|
||||
|
||||
# ── Test document ────────────────────────────────────────────────────────────
|
||||
TEST_ID = uuid.uuid4().hex[:8]
|
||||
TEST_TEXT = (
|
||||
f"Memory OS ingestion test document {TEST_ID}. "
|
||||
"This file was automatically generated by test_ingestion.py "
|
||||
"to verify the end-to-end pipeline: enqueue, worker, embedding, Qdrant upsert."
|
||||
)
|
||||
TEST_PATH = f"/wiki/raw/test/ingestion-test-{TEST_ID}.md"
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
print(f"=== Memory OS Ingestion Test (doc: {TEST_ID}) ===")
|
||||
print()
|
||||
|
||||
# ── 1. Create temp file for the worker to ingest ──────────────────────────
|
||||
print("1. Creating test document...")
|
||||
# The worker reads from the wiki volume mounted in Docker.
|
||||
# Write to /tmp as a fallback; the worker path depends on volume config.
|
||||
tmpdir = tempfile.mkdtemp(prefix="memoryos-test-")
|
||||
test_file = os.path.join(tmpdir, f"ingestion-test-{TEST_ID}.md")
|
||||
with open(test_file, "w") as f:
|
||||
f.write(f"# Test {TEST_ID}\n\n{TEST_TEXT}\n")
|
||||
ok(f"Created {test_file}")
|
||||
|
||||
# ── 2. Enqueue ARQ job ───────────────────────────────────────────────────
|
||||
print("2. Enqueuing ARQ job...")
|
||||
try:
|
||||
from arq import create_pool
|
||||
from arq.connections import RedisSettings
|
||||
except ImportError:
|
||||
fail("arq not installed — run: pip install arq")
|
||||
|
||||
redis_settings = RedisSettings(
|
||||
host=REDIS_HOST,
|
||||
port=REDIS_PORT,
|
||||
password=REDIS_PASSWORD or None,
|
||||
)
|
||||
|
||||
redis = await create_pool(redis_settings)
|
||||
job = await redis.enqueue_job("process_wiki_file", TEST_PATH)
|
||||
job_id = job.job_id
|
||||
ok(f"Job {job_id} enqueued")
|
||||
|
||||
# ── 3. Wait for completion ───────────────────────────────────────────────
|
||||
print(f"3. Waiting for worker (timeout={TIMEOUT}s)...")
|
||||
deadline = time.monotonic() + TIMEOUT
|
||||
result = None
|
||||
while time.monotonic() < deadline:
|
||||
job_info = await redis.get_job_result(job_id)
|
||||
if job_info is not None:
|
||||
result = job_info.result
|
||||
if job_info.success:
|
||||
ok(f"Job completed in {TIMEOUT - (deadline - time.monotonic()):.0f}s")
|
||||
break
|
||||
else:
|
||||
fail(f"Job failed: {job_info.result}")
|
||||
await asyncio.sleep(POLL_INTERVAL)
|
||||
|
||||
if result is None:
|
||||
fail(f"Job timed out after {TIMEOUT}s — is the ARQ worker running?")
|
||||
|
||||
# ── 4. Search Qdrant for the ingested point ──────────────────────────────
|
||||
print("4. Searching Qdrant for ingested point...")
|
||||
try:
|
||||
from qdrant_client import AsyncQdrantClient
|
||||
from qdrant_client.models import Filter, FieldCondition, MatchText
|
||||
except ImportError:
|
||||
fail("qdrant-client not installed — run: pip install qdrant-client")
|
||||
|
||||
client = AsyncQdrantClient(
|
||||
host=QDRANT_HOST,
|
||||
port=QDRANT_PORT,
|
||||
api_key=QDRANT_API_KEY or None,
|
||||
https=False,
|
||||
)
|
||||
|
||||
# Search by the unique test ID in the payload
|
||||
points, _ = await client.scroll(
|
||||
collection_name=COLLECTION_NAME,
|
||||
scroll_filter=Filter(
|
||||
must=[
|
||||
FieldCondition(
|
||||
key="source_file",
|
||||
match=MatchText(text=TEST_PATH),
|
||||
)
|
||||
]
|
||||
),
|
||||
with_vectors=True,
|
||||
limit=5,
|
||||
)
|
||||
|
||||
if not points:
|
||||
fail(
|
||||
f"No points found for {TEST_PATH} in collection "
|
||||
f"'{COLLECTION_NAME}' — ingestion may have succeeded "
|
||||
"but the source_file path may differ from expected."
|
||||
)
|
||||
|
||||
point = points[0]
|
||||
point_id = point.id
|
||||
ok(f"Found point {point_id}")
|
||||
|
||||
# ── 5. Verify dense vector dimensions ────────────────────────────────────
|
||||
print("5. Verifying embedding dimensions...")
|
||||
if "dense" not in point.vector:
|
||||
fail("Point has no 'dense' vector — named vectors may not be configured")
|
||||
|
||||
dims = len(point.vector["dense"])
|
||||
if dims != 4096:
|
||||
fail(f"Expected 4096 dimensions, got {dims}")
|
||||
|
||||
ok(f"{dims} dimensions ✓")
|
||||
|
||||
# ── 6. Cleanup ───────────────────────────────────────────────────────────
|
||||
print("6. Cleaning up test point...")
|
||||
await client.delete(
|
||||
collection_name=COLLECTION_NAME,
|
||||
points_selector=[point_id],
|
||||
)
|
||||
ok(f"Deleted point {point_id}")
|
||||
|
||||
# Clean up local temp file
|
||||
os.remove(test_file)
|
||||
os.rmdir(tmpdir)
|
||||
|
||||
await client.close()
|
||||
await redis.close()
|
||||
|
||||
# ── 7. Summary ───────────────────────────────────────────────────────────
|
||||
print()
|
||||
print("═" * 60)
|
||||
ok("All checks passed — ingestion pipeline is operational")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
|
@ -0,0 +1,355 @@
|
|||
#!/usr/bin/env bash
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Memory OS — Setup Script
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Installs the complete Memory OS stack into your Hermes Agent.
|
||||
#
|
||||
# Usage:
|
||||
# curl -sSL https://raw.githubusercontent.com/ClaudioDrews/memory-os/main/setup.sh | bash
|
||||
#
|
||||
# Or, if you already cloned the repo:
|
||||
# bash setup.sh
|
||||
#
|
||||
# What this script does:
|
||||
# 1. Checks prerequisites (Docker, Python, Hermes)
|
||||
# 2. Clones the repo (if needed)
|
||||
# 3. Installs Python dependencies
|
||||
# 4. Creates SQLite databases (state.db, memory_store.db)
|
||||
# 5. Installs the Icarus plugin
|
||||
# 6. Creates wiki/vault directory structure
|
||||
# 7. Starts Redis + Qdrant + Worker (Docker Compose)
|
||||
# 8. Configures environment variables
|
||||
# 9. Applies rulebook modifications
|
||||
#
|
||||
# Idempotent — safe to run multiple times.
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
set -euo pipefail
|
||||
|
||||
# ── Colors ────────────────────────────────────────────────────────────────────
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[0;33m'
|
||||
BOLD='\033[1m'
|
||||
NC='\033[0m'
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
WARN=0
|
||||
|
||||
ok() { printf " ${GREEN}✅${NC} %s\n" "$1"; PASS=$((PASS + 1)); }
|
||||
fail() { printf " ${RED}❌${NC} %s\n" "$1"; FAIL=$((FAIL + 1)); }
|
||||
warn() { printf " ${YELLOW}⚠️${NC} %s\n" "$1"; WARN=$((WARN + 1)); }
|
||||
info() { printf " 📘 %s\n" "$1"; }
|
||||
|
||||
banner() {
|
||||
echo ""
|
||||
echo -e "${BOLD}── $1 ──${NC}"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# ── Detect script directory ──────────────────────────────────────────────────
|
||||
# When run via curl|bash, SCRIPT_DIR is the current directory.
|
||||
# When run from a cloned repo, it's the script's location.
|
||||
if [ -n "${BASH_SOURCE[0]:-}" ] && [ "${BASH_SOURCE[0]}" != "bash" ]; then
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
else
|
||||
SCRIPT_DIR="$(pwd)"
|
||||
fi
|
||||
|
||||
REPO_URL="https://github.com/ClaudioDrews/memory-os.git"
|
||||
REPO_DIR="${HOME}/memory-os"
|
||||
HERMES_HOME="${HOME}/.hermes"
|
||||
VAULT_PATH="${VAULT_PATH:-${HOME}/vault}"
|
||||
ENV_FILE="${HERMES_HOME}/.env"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 1: Bootstrap — clone repo if needed
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 1: Bootstrap"
|
||||
|
||||
if [ -d "${REPO_DIR}/.git" ]; then
|
||||
ok "Repo already exists at ${REPO_DIR}"
|
||||
cd "${REPO_DIR}"
|
||||
else
|
||||
info "Cloning Memory OS..."
|
||||
git clone "${REPO_URL}" "${REPO_DIR}" 2>&1 | tail -1
|
||||
cd "${REPO_DIR}"
|
||||
ok "Repo cloned to ${REPO_DIR}"
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 2: Pre-flight Checks
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 2: Pre-flight Checks"
|
||||
|
||||
# Docker
|
||||
if docker info >/dev/null 2>&1; then
|
||||
ok "Docker $(docker --version | awk '{print $3}' | tr -d ',')"
|
||||
else
|
||||
fail "Docker is not running — install and start Docker first"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Docker Compose
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
ok "Docker Compose $(docker compose version --short 2>/dev/null || echo 'ok')"
|
||||
else
|
||||
warn "Docker Compose plugin not detected — required to start the stack"
|
||||
fi
|
||||
|
||||
# Python
|
||||
PYTHON_VERSION=$(python3 --version 2>/dev/null | awk '{print $2}' || echo "none")
|
||||
if [ "$PYTHON_VERSION" != "none" ]; then
|
||||
ok "Python ${PYTHON_VERSION}"
|
||||
else
|
||||
fail "Python 3 not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Hermes
|
||||
if command -v hermes >/dev/null 2>&1 || [ -f "${HERMES_HOME}/hermes-agent/cli.py" ]; then
|
||||
ok "Hermes Agent detected at ${HERMES_HOME}"
|
||||
else
|
||||
warn "Hermes Agent CLI not found — some features will be limited"
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 3: Python Dependencies
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 3: Python Dependencies"
|
||||
|
||||
if [ -f "requirements.txt" ]; then
|
||||
# Install in current environment (local user or venv)
|
||||
if pip install --user -r requirements.txt --quiet 2>&1 | tail -3; then
|
||||
ok "Python dependencies installed"
|
||||
else
|
||||
warn "pip install failed — trying with --break-system-packages"
|
||||
pip install --break-system-packages -r requirements.txt --quiet 2>&1 || {
|
||||
fail "Could not install Python dependencies"
|
||||
exit 1
|
||||
}
|
||||
ok "Python dependencies installed (--break-system-packages)"
|
||||
fi
|
||||
else
|
||||
fail "requirements.txt not found at $(pwd)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 4: SQLite Databases
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 4: Database Setup"
|
||||
|
||||
if [ -f "setup/setup_db.py" ]; then
|
||||
python3 setup/setup_db.py 2>&1 && \
|
||||
ok "SQLite databases created (state.db, memory_store.db)" || \
|
||||
fail "setup_db.py failed"
|
||||
else
|
||||
fail "setup/setup_db.py not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 5: Icarus Plugin
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 5: Icarus Plugin"
|
||||
|
||||
ICARUS_DEST="${HERMES_HOME}/plugins/icarus"
|
||||
|
||||
if [ -d "icarus" ]; then
|
||||
mkdir -p "${HERMES_HOME}/plugins"
|
||||
cp -r icarus/ "${ICARUS_DEST}/"
|
||||
ok "Icarus plugin installed at ${ICARUS_DEST}"
|
||||
else
|
||||
fail "icarus/ directory not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 6: Wiki + Vault Structure (BEFORE Docker — prevents root ownership)
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 6: Wiki & Vault"
|
||||
|
||||
mkdir -p "${VAULT_PATH}/wiki/"{raw,concepts,entities,comparisons,_meta,_archive}
|
||||
mkdir -p "${VAULT_PATH}/fabric"
|
||||
ok "Directory structure created at ${VAULT_PATH}"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 7: Docker Stack
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 7: Docker Stack"
|
||||
|
||||
DOCKER_DIR="${REPO_DIR}/docker"
|
||||
|
||||
if [ ! -d "${DOCKER_DIR}" ]; then
|
||||
fail "docker/ directory not found at ${REPO_DIR}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
cd "${DOCKER_DIR}"
|
||||
|
||||
# Detect OpenRouter API key from Hermes .env
|
||||
OPENROUTER_KEY=""
|
||||
if [ -f "${ENV_FILE}" ]; then
|
||||
# Try OPENROUTER_DS_API_KEY first (Hermes default), then OPENROUTER_API_KEY
|
||||
OPENROUTER_KEY=$(grep -oP 'OPENROUTER_DS_API_KEY=\K.*' "${ENV_FILE}" 2>/dev/null | head -1 || true)
|
||||
if [ -z "${OPENROUTER_KEY}" ]; then
|
||||
OPENROUTER_KEY=$(grep -oP 'OPENROUTER_API_KEY=\K.*' "${ENV_FILE}" 2>/dev/null | head -1 || true)
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "${OPENROUTER_KEY}" ]; then
|
||||
echo ""
|
||||
echo -e " ${YELLOW}Could not find your OpenRouter key in Hermes .env.${NC}"
|
||||
echo " The worker needs it to generate embeddings."
|
||||
echo ""
|
||||
read -r -p " Paste your OpenRouter key (e.g. sk-or-v1-...): " OPENROUTER_KEY
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# Generate random Redis password
|
||||
REDIS_PW=$(openssl rand -hex 16)
|
||||
|
||||
# Create Docker Compose .env
|
||||
cat > .env << DOCKERENV
|
||||
OPENROUTER_API_KEY=${OPENROUTER_KEY}
|
||||
REDIS_PASSWORD=${REDIS_PW}
|
||||
QDRANT_API_KEY=
|
||||
EMBEDDING_DIMS=4096
|
||||
COLLECTION_NAME=knowledge_base
|
||||
LOG_LEVEL=INFO
|
||||
MEMORY_OS_WIKI_PATH=${VAULT_PATH}/wiki
|
||||
MEMORY_OS_HERMES_HOME=${HERMES_HOME}
|
||||
MEMORY_OS_FABRIC_DIR=${VAULT_PATH}/fabric
|
||||
DOCKERENV
|
||||
|
||||
ok "docker/.env created"
|
||||
|
||||
# Pull pre-built images first (Redis, Qdrant) — fast
|
||||
info "Downloading pre-built images (Redis, Qdrant)..."
|
||||
docker compose pull redis qdrant 2>&1 | tail -3
|
||||
ok "Base images downloaded"
|
||||
|
||||
# Build worker image — SLOW on first run (gcc + build-essential)
|
||||
info "Building worker image (may take 5-10 minutes on first run)..."
|
||||
info " (Future builds will use Docker cache)"
|
||||
if docker compose build worker 2>&1; then
|
||||
ok "Worker image built"
|
||||
else
|
||||
fail "Failed to build worker image"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Start everything
|
||||
info "Starting containers..."
|
||||
if docker compose up -d 2>&1; then
|
||||
ok "Docker stack started (redis, qdrant, worker)"
|
||||
else
|
||||
fail "docker compose up failed — check Docker"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Wait for healthy
|
||||
info "Waiting for services to become healthy..."
|
||||
sleep 3
|
||||
if docker compose ps --format json 2>/dev/null | grep -q '"Health":"healthy"'; then
|
||||
ok "All services healthy"
|
||||
else
|
||||
warn "Services may still be starting — check with: docker compose ps"
|
||||
fi
|
||||
|
||||
# Return to repo directory
|
||||
cd "${REPO_DIR}"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 8: Environment Variables
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 8: Hermes .env"
|
||||
|
||||
if [ ! -f "${ENV_FILE}" ]; then
|
||||
warn "${ENV_FILE} not found — creating a new one"
|
||||
touch "${ENV_FILE}"
|
||||
fi
|
||||
|
||||
add_env() {
|
||||
local key="$1"
|
||||
local value="$2"
|
||||
if grep -q "^${key}=" "${ENV_FILE}" 2>/dev/null; then
|
||||
# Already exists — don't overwrite
|
||||
return 0
|
||||
fi
|
||||
echo "${key}=${value}" >> "${ENV_FILE}"
|
||||
}
|
||||
|
||||
add_env "FABRIC_DIR" "${VAULT_PATH}/fabric"
|
||||
add_env "ICARUS_EXTRACTION_MAX_TOKENS" "4096"
|
||||
add_env "ICARUS_EXTRACTION_MODEL" "deepseek/deepseek-v4-flash"
|
||||
add_env "EMBEDDING_DIMS" "4096"
|
||||
add_env "HERMES_AGENT_NAME" "hermes"
|
||||
add_env "REDIS_PASSWORD" "${REDIS_PW}"
|
||||
add_env "OPENROUTER_DS_API_KEY" "${OPENROUTER_KEY}"
|
||||
|
||||
ok "Environment variables added to Hermes .env"
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 9: Rulebook Modifications
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 9: Rulebook"
|
||||
|
||||
RULEBOOK="${HERMES_HOME}/rulebook.md"
|
||||
|
||||
if [ -f "${RULEBOOK}" ]; then
|
||||
if grep -q "Memory OS amendment" "${RULEBOOK}" 2>/dev/null; then
|
||||
ok "Rulebook amendments already applied"
|
||||
else
|
||||
info "Applying Memory OS amendments to rulebook..."
|
||||
info "(See modifications/execution-agent-protocol.md for details)"
|
||||
warn "Amendments NOT applied automatically — edit the rulebook manually"
|
||||
warn "See: ${REPO_DIR}/modifications/execution-agent-protocol.md"
|
||||
fi
|
||||
else
|
||||
warn "${RULEBOOK} not found — skipping modifications"
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Phase 10: Gateway
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Phase 10: Gateway"
|
||||
|
||||
if command -v hermes >/dev/null 2>&1; then
|
||||
info "Restarting Hermes gateway..."
|
||||
if hermes gateway restart 2>&1; then
|
||||
ok "Gateway restarted"
|
||||
else
|
||||
warn "Gateway restart failed — restart manually with: hermes gateway restart"
|
||||
fi
|
||||
else
|
||||
warn "'hermes' command not available — restart the gateway manually"
|
||||
fi
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
# Summary
|
||||
# ──────────────────────────────────────────────────────────────────────────────
|
||||
banner "Summary"
|
||||
|
||||
echo " Passed: ${PASS}"
|
||||
echo " Failed: ${FAIL}"
|
||||
echo " Warnings: ${WARN}"
|
||||
echo ""
|
||||
|
||||
if [ "${FAIL}" -eq 0 ]; then
|
||||
echo -e " ${GREEN}${BOLD}✅ Memory OS installed successfully!${NC}"
|
||||
echo ""
|
||||
echo " To verify:"
|
||||
echo " • /plugins → should show 'icarus'"
|
||||
echo " • docker compose ps → 3 services (redis, qdrant, worker)"
|
||||
echo " • fabric_brief() → fabric entries (initially empty)"
|
||||
echo " • qdrant_search() → semantic search (requires populated wiki)"
|
||||
echo ""
|
||||
echo " Next step: add .md files to ${VAULT_PATH}/wiki/raw/"
|
||||
echo " and the ingestion pipeline will index them automatically."
|
||||
echo ""
|
||||
else
|
||||
echo -e " ${RED}${BOLD}❌ ${FAIL} error(s) found — review the output above.${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
|
@ -1,5 +1,7 @@
|
|||
# Setup Guide
|
||||
|
||||
> **Prefer automated install?** Run `curl -sSL https://raw.githubusercontent.com/ClaudioDrews/memory-os/main/setup.sh | bash` — one command, 10 phases, fully idempotent. This manual guide is kept for reference and troubleshooting.
|
||||
|
||||
> Step-by-step installation of the Memory OS stack. Assumes Hermes Agent is already installed and configured.
|
||||
|
||||
## Prerequisites
|
||||
|
|
|
|||
|
|
@ -0,0 +1,169 @@
|
|||
#!/usr/bin/env bash
|
||||
# Memory OS Smoke Test
|
||||
# Verifies the entire stack is functional without needing to know what to check.
|
||||
#
|
||||
# Usage:
|
||||
# ./setup/smoke_test.sh # All checks
|
||||
# ./setup/smoke_test.sh --quick # Skip ingestion test (faster)
|
||||
# ./setup/smoke_test.sh --help # Show help
|
||||
#
|
||||
# Environment:
|
||||
# REDIS_PASSWORD Redis password
|
||||
# QDRANT_API_KEY Qdrant API key (default: "")
|
||||
# REDIS_HOST Redis host (default: localhost)
|
||||
# REDIS_PORT Redis port (default: 6379)
|
||||
# QDRANT_HOST Qdrant host (default: localhost)
|
||||
# QDRANT_PORT Qdrant port (default: 6333)
|
||||
# COLLECTION_NAME Qdrant collection (default: knowledge_base)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
PASS=0
|
||||
FAIL=0
|
||||
QUICK_MODE=false
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--quick) QUICK_MODE=true ;;
|
||||
--help) echo "Usage: ./setup/smoke_test.sh [--quick]"; exit 0 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
RED=''
|
||||
GREEN=''
|
||||
NC=''
|
||||
if [ -t 1 ]; then
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
NC='\033[0m'
|
||||
fi
|
||||
|
||||
check() {
|
||||
local label="$1"
|
||||
local cmd="$2"
|
||||
printf " %-40s " "$label"
|
||||
if eval "$cmd" >/dev/null 2>&1; then
|
||||
printf "${GREEN}✅${NC}\n"
|
||||
PASS=$((PASS + 1))
|
||||
else
|
||||
printf "${RED}❌${NC}\n"
|
||||
FAIL=$((FAIL + 1))
|
||||
fi
|
||||
}
|
||||
|
||||
# ── Resolve env vars ─────────────────────────────────────────────────────────
|
||||
REDIS_HOST="${REDIS_HOST:-localhost}"
|
||||
REDIS_PORT="${REDIS_PORT:-6379}"
|
||||
REDIS_PASSWORD="${REDIS_PASSWORD:-}"
|
||||
QDRANT_HOST="${QDRANT_HOST:-localhost}"
|
||||
QDRANT_PORT="${QDRANT_PORT:-6333}"
|
||||
QDRANT_API_KEY="${QDRANT_API_KEY:-}"
|
||||
COLLECTION_NAME="${COLLECTION_NAME:-knowledge_base}"
|
||||
|
||||
echo "=== Memory OS Smoke Test ==="
|
||||
echo " Redis: ${REDIS_HOST}:${REDIS_PORT}"
|
||||
echo " Qdrant: ${QDRANT_HOST}:${QDRANT_PORT}"
|
||||
echo " Collection: ${COLLECTION_NAME}"
|
||||
echo ""
|
||||
|
||||
# ── 1. Infrastructure ────────────────────────────────────────────────────────
|
||||
echo "── Infrastructure ──"
|
||||
|
||||
check "Docker running" \
|
||||
"docker info"
|
||||
|
||||
check "Redis reachable" \
|
||||
"python3 -c \"
|
||||
import redis
|
||||
r = redis.Redis(host='${REDIS_HOST}', port=${REDIS_PORT},
|
||||
password='${REDIS_PASSWORD}' or None)
|
||||
assert r.ping()
|
||||
\""
|
||||
|
||||
# Note: Qdrant healthcheck uses grep on /proc/net/tcp (shell built-in) because
|
||||
# the qdrant/qdrant image does not include curl, wget, or python3.
|
||||
# Port 6333 = 0x18BD in hex.
|
||||
check "Qdrant health" \
|
||||
"python3 -c \"
|
||||
from qdrant_client import QdrantClient
|
||||
c = QdrantClient(host='${QDRANT_HOST}', port=${QDRANT_PORT},
|
||||
api_key='${QDRANT_API_KEY}' or None, https=False)
|
||||
collections = c.get_collections()
|
||||
assert len(collections.collections) >= 1
|
||||
\""
|
||||
|
||||
check "Qdrant collection '${COLLECTION_NAME}'" \
|
||||
"python3 -c \"
|
||||
from qdrant_client import QdrantClient
|
||||
c = QdrantClient(host='${QDRANT_HOST}', port=${QDRANT_PORT},
|
||||
api_key='${QDRANT_API_KEY}' or None, https=False)
|
||||
info = c.get_collection('${COLLECTION_NAME}')
|
||||
assert info.config.params.vectors is not None
|
||||
\""
|
||||
|
||||
# ── 2. Icarus plugin ─────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "── Icarus Plugin ──"
|
||||
|
||||
check "Icarus plugin installed" \
|
||||
"test -f ~/.hermes/plugins/icarus/__init__.py"
|
||||
|
||||
check "Icarus plugin loaded" \
|
||||
"hermes plugins list 2>/dev/null | grep -q icarus"
|
||||
|
||||
# ── 3. Embedding ─────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "── Embedding ──"
|
||||
|
||||
check "Embedding produces 4096d vectors" \
|
||||
"python3 << 'PYEOF'
|
||||
from qdrant_client import QdrantClient
|
||||
c = QdrantClient(host='${QDRANT_HOST}', port=${QDRANT_PORT},
|
||||
api_key='${QDRANT_API_KEY}' or None, https=False)
|
||||
points, _ = c.scroll('${COLLECTION_NAME}', limit=1, with_vectors=True)
|
||||
assert len(points) > 0, 'no points found in collection'
|
||||
assert len(points[0].vector['dense']) == 4096, \\
|
||||
f'expected 4096 dims, got {len(points[0].vector[\"dense\"])}'
|
||||
PYEOF"
|
||||
|
||||
# ── 4. Ingestion pipeline ────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "── Ingestion Pipeline ──"
|
||||
|
||||
if [ "$QUICK_MODE" = true ]; then
|
||||
echo " (skipped — --quick mode)"
|
||||
else
|
||||
check "End-to-end ingestion" \
|
||||
"python3 scripts/test_ingestion.py"
|
||||
fi
|
||||
|
||||
# ── 5. Cron jobs ─────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "── Cron Jobs ──"
|
||||
|
||||
check "Cron jobs active (≥3)" \
|
||||
"python3 -c \"
|
||||
import subprocess, json
|
||||
out = subprocess.run(['hermes', 'cron', 'list'],
|
||||
capture_output=True, text=True).stdout
|
||||
# Count lines with '[active]'
|
||||
count = out.count('[active]')
|
||||
assert count >= 3, f'expected >=3 active cron jobs, got {count}'
|
||||
\""
|
||||
|
||||
# ── Summary ──────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo "──────────────────────────────────────────"
|
||||
if [ "$QUICK_MODE" = true ]; then
|
||||
echo "Result (quick mode): $PASS passed, $FAIL failed, 1 skipped"
|
||||
else
|
||||
echo "Result: $PASS passed, $FAIL failed"
|
||||
fi
|
||||
|
||||
if [ "$FAIL" -eq 0 ]; then
|
||||
echo "✅ All checks passed — Memory OS is operational."
|
||||
exit 0
|
||||
else
|
||||
echo "❌ $FAIL check(s) failed — review output above."
|
||||
exit 1
|
||||
fi
|
||||
Loading…
Reference in New Issue