* feat(serve): turnkey secure remote MCP server (#1877) Add `mempalace serve`: a secure-by-default wrapper over the HTTP MCP transport so a team can stand up a shared central palace with one command. Server capabilities (mempalace/mcp_server.py): - Native TLS via --tls-cert/--tls-key (env MEMPALACE_MCP_TLS_CERT/_KEY): wraps the socket in a TLS 1.2+ context, validated before bind. Token is still required on a non-loopback bind (TLS != auth). - Read-only mode via --read-only (env MEMPALACE_MCP_READ_ONLY): the 24 mutating tools are hidden from tools/list and refused at dispatch (-32003), enforced before arg handling — not merely hidden. Turnkey command (mempalace/cli.py): - Auto-generates a strong bearer token for non-loopback binds, stored 0600 under ~/.mempalace/server/ and printed once; reused across restarts. Token rides in the child env, never argv, so it can't leak via ps. - Prints a ready-to-paste client config (scheme reflects TLS), then foreground-execs the real server so Docker/systemd own the lifecycle. Deployment (deploy/): - docker-compose.server.yml wires the server + Qdrant with a /healthz healthcheck and persistent volumes. - server.env.example documents the env surface. - mempalace-server.service is a hardened systemd unit template. Tests: TLS handshake (openssl-gated), read-only enforcement, token autogen/0600/reuse, token-not-in-argv, secure-by-default gates. Docs: remote-server guide now leads with `mempalace serve` plus Compose and systemd subsections. * test(serve): fix Windows — don't patch os.name; gate 0600 asserts to POSIX Patching os.name to 'posix' broke Path.home() on Windows (pathlib mixed POSIX home resolution with Windows drive parsing). Capture both exec branches (os.execve + subprocess.run) instead, and guard the POSIX permission-bit assertions behind os.name == 'posix' (Windows files report 0o666).
This commit is contained in:
Igor Lins e Silva
GitHub
parent
8ec284db5d
commit
afd0428823
|
|
@ -0,0 +1,71 @@
|
|||
# MemPalace remote team server — MCP over HTTP, backed by a central Qdrant.
|
||||
#
|
||||
# One command stands up a shared memory service a whole team's AI clients
|
||||
# connect to. Embeddings are still produced locally inside the mempalace
|
||||
# container; only your own Qdrant ever receives the vectors and text.
|
||||
#
|
||||
# 1. cp deploy/server.env.example deploy/.env && edit deploy/.env
|
||||
# (at minimum set MEMPALACE_MCP_HTTP_TOKEN to a long random secret)
|
||||
# 2. docker compose -f deploy/docker-compose.server.yml --env-file deploy/.env up -d
|
||||
# 3. connect a client (see the Remote / Team Server guide):
|
||||
# claude mcp add --transport http mempalace http://YOUR_HOST:8765/mcp \
|
||||
# --header "Authorization: Bearer $MEMPALACE_MCP_HTTP_TOKEN"
|
||||
#
|
||||
# SECURITY: this exposes plaintext HTTP on :8765. For anything beyond a trusted
|
||||
# private network, put a TLS-terminating reverse proxy in front (nginx/Caddy/
|
||||
# Traefik) and only expose the proxy. The bearer token is mandatory for the
|
||||
# network-exposed (0.0.0.0) bind.
|
||||
|
||||
services:
|
||||
qdrant:
|
||||
image: qdrant/qdrant:latest
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- qdrant-storage:/qdrant/storage
|
||||
# Not published to the host: only the mempalace service reaches it over the
|
||||
# internal compose network. Uncomment to inspect Qdrant directly.
|
||||
# ports:
|
||||
# - "6333:6333"
|
||||
|
||||
mempalace:
|
||||
image: ghcr.io/mempalace/mempalace:latest
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- qdrant
|
||||
command:
|
||||
- serve
|
||||
- --host
|
||||
- "0.0.0.0"
|
||||
- --port
|
||||
- "8765"
|
||||
- --backend
|
||||
- qdrant
|
||||
# Uncomment to expose recall without write access to most clients:
|
||||
# - --read-only
|
||||
environment:
|
||||
# Required for the network-exposed bind. Set in deploy/.env.
|
||||
MEMPALACE_MCP_HTTP_TOKEN: ${MEMPALACE_MCP_HTTP_TOKEN:?set MEMPALACE_MCP_HTTP_TOKEN in deploy/.env}
|
||||
MEMPALACE_QDRANT_URL: http://qdrant:6333
|
||||
MEMPALACE_QDRANT_API_KEY: ${MEMPALACE_QDRANT_API_KEY:-}
|
||||
# Set to cuda/dml/coreml on an accelerated host (see Dockerfile.gpu).
|
||||
MEMPALACE_EMBEDDING_DEVICE: ${MEMPALACE_EMBEDDING_DEVICE:-auto}
|
||||
ports:
|
||||
- "8765:8765"
|
||||
volumes:
|
||||
- mempalace-data:/data
|
||||
healthcheck:
|
||||
# The image has no curl; use Python (always present). /healthz needs no auth.
|
||||
# If you enable TLS on the server itself, switch this to https + ssl context.
|
||||
test:
|
||||
- CMD
|
||||
- python
|
||||
- -c
|
||||
- "import urllib.request,sys; sys.exit(0) if urllib.request.urlopen('http://127.0.0.1:8765/healthz').read().strip()==b'ok' else sys.exit(1)"
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 40s
|
||||
|
||||
volumes:
|
||||
qdrant-storage:
|
||||
mempalace-data:
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
# MemPalace remote MCP server — systemd unit template.
|
||||
#
|
||||
# Install:
|
||||
# sudo useradd --system --home /var/lib/mempalace --shell /usr/sbin/nologin mempalace
|
||||
# sudo install -d -o mempalace -g mempalace -m 750 /var/lib/mempalace /etc/mempalace
|
||||
# sudo cp deploy/server.env.example /etc/mempalace/server.env # then edit + chmod 600
|
||||
# sudo install -m 600 -o mempalace -g mempalace /etc/mempalace/server.env /etc/mempalace/server.env
|
||||
# # install mempalace into a venv on PATH, or adjust ExecStart to its absolute path
|
||||
# sudo cp deploy/mempalace-server.service /etc/systemd/system/
|
||||
# sudo systemctl daemon-reload && sudo systemctl enable --now mempalace-server
|
||||
#
|
||||
# This binds 0.0.0.0:8765 and requires MEMPALACE_MCP_HTTP_TOKEN (set in the
|
||||
# EnvironmentFile). Front it with a TLS-terminating reverse proxy, or set
|
||||
# MEMPALACE_MCP_TLS_CERT / _KEY in the EnvironmentFile for native TLS.
|
||||
|
||||
[Unit]
|
||||
Description=MemPalace remote MCP server
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=exec
|
||||
User=mempalace
|
||||
Group=mempalace
|
||||
EnvironmentFile=/etc/mempalace/server.env
|
||||
ExecStart=mempalace serve --host 0.0.0.0 --port 8765
|
||||
Restart=on-failure
|
||||
RestartSec=2
|
||||
|
||||
# --- Hardening ---------------------------------------------------------------
|
||||
NoNewPrivileges=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
PrivateTmp=true
|
||||
PrivateDevices=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectKernelModules=true
|
||||
ProtectControlGroups=true
|
||||
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
|
||||
RestrictNamespaces=true
|
||||
LockPersonality=true
|
||||
MemoryDenyWriteExecute=false
|
||||
# The palace and any local state live here; everything else is read-only.
|
||||
ReadWritePaths=/var/lib/mempalace
|
||||
StateDirectory=mempalace
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
# MemPalace remote server environment.
|
||||
# Copy to deploy/.env (compose) or /etc/mempalace/server.env (systemd) and edit.
|
||||
# Keep this file readable only by the service account: chmod 600.
|
||||
|
||||
# --- Required for a network-exposed (0.0.0.0) bind ------------------------------
|
||||
# Clients send: Authorization: Bearer <this value>. Generate a strong secret:
|
||||
# openssl rand -hex 32
|
||||
MEMPALACE_MCP_HTTP_TOKEN=
|
||||
|
||||
# --- Storage backend ------------------------------------------------------------
|
||||
# The team server should use a networked backend so the palace is shared.
|
||||
MEMPALACE_BACKEND=qdrant
|
||||
MEMPALACE_QDRANT_URL=http://qdrant:6333
|
||||
# MEMPALACE_QDRANT_API_KEY=
|
||||
|
||||
# --- Embedding ------------------------------------------------------------------
|
||||
# auto | cpu | cuda | dml | coreml. Use cuda on a GPU host (needs the GPU image).
|
||||
MEMPALACE_EMBEDDING_DEVICE=auto
|
||||
|
||||
# --- Optional: native TLS (otherwise terminate TLS at a reverse proxy) ----------
|
||||
# Point these at a PEM cert/key the service account can read. When set, serve
|
||||
# speaks HTTPS directly and clients connect to https://...
|
||||
# MEMPALACE_MCP_TLS_CERT=/etc/mempalace/tls/cert.pem
|
||||
# MEMPALACE_MCP_TLS_KEY=/etc/mempalace/tls/key.pem
|
||||
|
||||
# --- Palace location (systemd / bare-metal) -------------------------------------
|
||||
# In Docker the palace lives on the mempalace-data volume by default.
|
||||
# MEMPALACE_PALACE_PATH=/var/lib/mempalace/palace
|
||||
181
mempalace/cli.py
181
mempalace/cli.py
|
|
@ -1354,6 +1354,154 @@ def cmd_mcp(args):
|
|||
print(f" {base_server_cmd} --palace /path/to/palace")
|
||||
|
||||
|
||||
_SERVER_LOOPBACK_HOSTS = {"127.0.0.1", "localhost", "::1", "[::1]"}
|
||||
_SERVER_BIND_ALL_HOSTS = {"0.0.0.0", "::", "[::]"}
|
||||
|
||||
|
||||
def _server_is_loopback(host: str) -> bool:
|
||||
return (host or "").strip().lower() in _SERVER_LOOPBACK_HOSTS
|
||||
|
||||
|
||||
def _server_token_path(palace_path: str) -> Path:
|
||||
"""Per-palace location for the auto-generated server bearer token.
|
||||
|
||||
Distinct from the daemon's token dir; keyed by the canonical palace path so
|
||||
one server per palace reuses a stable token across restarts.
|
||||
"""
|
||||
import hashlib
|
||||
|
||||
canonical = os.path.abspath(os.path.realpath(os.path.expanduser(palace_path)))
|
||||
key = hashlib.sha256(os.path.normcase(canonical).encode("utf-8")).hexdigest()[:24]
|
||||
return Path.home() / ".mempalace" / "server" / key / "token"
|
||||
|
||||
|
||||
def _load_or_create_server_token(palace_path: str) -> tuple[str, bool]:
|
||||
"""Return (token, created). Reuse an existing 0600 token or mint a new one."""
|
||||
import secrets
|
||||
|
||||
token_path = _server_token_path(palace_path)
|
||||
if token_path.exists():
|
||||
existing = token_path.read_text(encoding="utf-8").strip()
|
||||
if existing:
|
||||
return existing, False
|
||||
token = secrets.token_urlsafe(32)
|
||||
token_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
os.chmod(str(token_path.parent), 0o700)
|
||||
except OSError:
|
||||
pass
|
||||
# O_CREAT with 0600 so the token is never briefly world-readable on disk.
|
||||
fd = os.open(str(token_path), os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
fh.write(token + "\n")
|
||||
return token, True
|
||||
|
||||
|
||||
def cmd_serve(args):
|
||||
"""Run a secure remote HTTP MCP server for a team to share one palace (#1877).
|
||||
|
||||
A turnkey wrapper over ``mempalace-mcp --transport http``: it resolves a
|
||||
bearer token (auto-generating a strong one for non-loopback binds), prints a
|
||||
ready-to-paste client config, then execs the real server in the foreground so
|
||||
Docker/systemd own the process lifecycle. The token is passed via the
|
||||
environment, never argv, so it can't leak through ``ps``.
|
||||
"""
|
||||
host = args.host
|
||||
port = int(args.port)
|
||||
loopback = _server_is_loopback(host)
|
||||
palace_path = (
|
||||
os.path.abspath(os.path.expanduser(args.palace))
|
||||
if args.palace
|
||||
else MempalaceConfig().palace_path
|
||||
)
|
||||
backend = _backend_arg(args)
|
||||
|
||||
tls_cert = os.path.expanduser(args.tls_cert) if args.tls_cert else None
|
||||
tls_key = os.path.expanduser(args.tls_key) if args.tls_key else None
|
||||
if bool(tls_cert) != bool(tls_key):
|
||||
print("mempalace: --tls-cert and --tls-key must be given together", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
for label, path in (("--tls-cert", tls_cert), ("--tls-key", tls_key)):
|
||||
if path and not os.path.isfile(path):
|
||||
print(f"mempalace: {label} file not found: {path}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
scheme = "https" if tls_cert else "http"
|
||||
|
||||
# Token resolution. Explicit flag > existing env > (non-loopback) auto-generated.
|
||||
token = (args.token or os.environ.get("MEMPALACE_MCP_HTTP_TOKEN", "")).strip()
|
||||
token_created = False
|
||||
if not token and not loopback and not args.allow_insecure:
|
||||
token, token_created = _load_or_create_server_token(palace_path)
|
||||
|
||||
# Build the child environment. Token rides in the env (never argv) so it
|
||||
# stays out of the process table.
|
||||
env = dict(os.environ)
|
||||
env["MEMPALACE_PALACE_PATH"] = palace_path
|
||||
if backend:
|
||||
env["MEMPALACE_BACKEND"] = str(backend).strip().lower()
|
||||
if token:
|
||||
env["MEMPALACE_MCP_HTTP_TOKEN"] = token
|
||||
if args.allow_insecure:
|
||||
env["MEMPALACE_MCP_HTTP_ALLOW_INSECURE_NO_TOKEN"] = "1"
|
||||
|
||||
child = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"mempalace.mcp_server",
|
||||
"--transport",
|
||||
"http",
|
||||
"--host",
|
||||
host,
|
||||
"--port",
|
||||
str(port),
|
||||
]
|
||||
if backend:
|
||||
child += ["--backend", str(backend).strip().lower()]
|
||||
child += ["--palace", palace_path]
|
||||
if tls_cert:
|
||||
child += ["--tls-cert", tls_cert, "--tls-key", tls_key]
|
||||
if args.read_only:
|
||||
child.append("--read-only")
|
||||
|
||||
# Client-facing address: 0.0.0.0/:: means "all interfaces" — clients dial a
|
||||
# real reachable host, so show a placeholder rather than the bind wildcard.
|
||||
client_host = "YOUR_SERVER_HOST" if host.strip().lower() in _SERVER_BIND_ALL_HOSTS else host
|
||||
url = f"{scheme}://{client_host}:{port}/mcp"
|
||||
|
||||
print("Starting MemPalace remote MCP server")
|
||||
print(f" palace : {palace_path}")
|
||||
print(f" backend : {(backend or 'default').strip().lower() if backend else 'default'}")
|
||||
print(f" bind : {host}:{port} ({'loopback' if loopback else 'network-exposed'})")
|
||||
print(f" tls : {'on' if tls_cert else 'off (plaintext — terminate TLS at a proxy)'}")
|
||||
print(f" read-only: {'yes' if args.read_only else 'no'}")
|
||||
if token_created:
|
||||
print("\n A new bearer token was generated and stored 0600 at:")
|
||||
print(f" {_server_token_path(palace_path)}")
|
||||
print(" Store it securely — clients need it to connect:")
|
||||
print(f" {token}")
|
||||
print("\nConnect a client:")
|
||||
if token:
|
||||
print(
|
||||
f" claude mcp add --transport http mempalace {url} "
|
||||
f'--header "Authorization: Bearer {token if token_created else "$MEMPALACE_MCP_HTTP_TOKEN"}"'
|
||||
)
|
||||
else:
|
||||
print(f" claude mcp add --transport http mempalace {url}")
|
||||
print(f" curl {scheme}://{client_host}:{port}/healthz # liveness (no auth)\n")
|
||||
sys.stdout.flush()
|
||||
|
||||
# Foreground: hand the process to the real server so signals (SIGTERM from
|
||||
# Docker/systemd) reach it directly. exec on POSIX; subprocess on Windows
|
||||
# (no exec semantics) propagating the exit code.
|
||||
if os.name == "posix":
|
||||
os.execve(sys.executable, child, env)
|
||||
else:
|
||||
import subprocess
|
||||
|
||||
completed = subprocess.run(child, env=env)
|
||||
sys.exit(completed.returncode)
|
||||
|
||||
|
||||
def cmd_compress(args):
|
||||
"""Compress drawers in a wing using AAAK Dialect."""
|
||||
from .dialect import Dialect
|
||||
|
|
@ -1965,6 +2113,38 @@ def main():
|
|||
help="Storage backend to include in the MCP startup command",
|
||||
)
|
||||
|
||||
# serve — turnkey remote HTTP MCP server (#1877)
|
||||
p_serve = sub.add_parser(
|
||||
"serve",
|
||||
help="Run a secure remote HTTP MCP server for a team to share one palace",
|
||||
)
|
||||
p_serve.add_argument(
|
||||
"--host", default="127.0.0.1", help="Bind address (use 0.0.0.0 for remote clients)"
|
||||
)
|
||||
p_serve.add_argument("--port", type=int, default=8765, help="Bind port (default: 8765)")
|
||||
p_serve.add_argument(
|
||||
"--backend", default=None, help="Storage backend (default: config/env/detected)"
|
||||
)
|
||||
p_serve.add_argument("--palace", default=None, help="Palace path (overrides config/env)")
|
||||
p_serve.add_argument(
|
||||
"--token",
|
||||
default=None,
|
||||
help="Bearer token clients must present. Default: reuse/auto-generate one for "
|
||||
"non-loopback binds (stored 0600 under ~/.mempalace/server/).",
|
||||
)
|
||||
p_serve.add_argument("--tls-cert", default=None, help="PEM certificate to enable TLS")
|
||||
p_serve.add_argument("--tls-key", default=None, help="PEM private key matching --tls-cert")
|
||||
p_serve.add_argument(
|
||||
"--read-only",
|
||||
action="store_true",
|
||||
help="Expose recall only: mutating tools are hidden and refused",
|
||||
)
|
||||
p_serve.add_argument(
|
||||
"--allow-insecure",
|
||||
action="store_true",
|
||||
help="Permit a non-loopback bind with no token (only behind a trusted proxy)",
|
||||
)
|
||||
|
||||
# status
|
||||
# migrate
|
||||
p_migrate = sub.add_parser(
|
||||
|
|
@ -2073,6 +2253,7 @@ def main():
|
|||
"sweep": cmd_sweep,
|
||||
"sync": cmd_sync,
|
||||
"mcp": cmd_mcp,
|
||||
"serve": cmd_serve,
|
||||
"compress": cmd_compress,
|
||||
"wake-up": cmd_wakeup,
|
||||
"repair": cmd_repair,
|
||||
|
|
|
|||
|
|
@ -275,6 +275,23 @@ def _parse_args():
|
|||
default=8765,
|
||||
help="HTTP port to bind when --transport=http (default: 8765)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tls-cert",
|
||||
metavar="PATH",
|
||||
help="PEM certificate to terminate TLS on the HTTP transport "
|
||||
"(requires --tls-key; env MEMPALACE_MCP_TLS_CERT)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tls-key",
|
||||
metavar="PATH",
|
||||
help="PEM private key matching --tls-cert (env MEMPALACE_MCP_TLS_KEY)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--read-only",
|
||||
action="store_true",
|
||||
help="Serve a read-only tool surface: the mutating tools are hidden from "
|
||||
"tools/list and refused at dispatch (env MEMPALACE_MCP_READ_ONLY)",
|
||||
)
|
||||
args, unknown = parser.parse_known_args()
|
||||
if unknown:
|
||||
logger.debug("Ignoring unknown args: %s", unknown)
|
||||
|
|
@ -295,6 +312,14 @@ if _args.backend:
|
|||
|
||||
_config = MempalaceConfig()
|
||||
|
||||
# Read-only server mode: when on, the mutating tools are hidden from tools/list
|
||||
# and refused at dispatch (-32003). Resolved once at startup from --read-only or
|
||||
# MEMPALACE_MCP_READ_ONLY. Computed inline (not via _truthy_env, defined below)
|
||||
# so it is available to the request path regardless of import order.
|
||||
_READ_ONLY = bool(getattr(_args, "read_only", False)) or os.environ.get(
|
||||
"MEMPALACE_MCP_READ_ONLY", ""
|
||||
).strip().lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
_kg_by_path: dict[str, KnowledgeGraph] = {}
|
||||
_kg_cache_lock = threading.Lock()
|
||||
_palace_flag_given: bool = bool(_args.palace)
|
||||
|
|
@ -4426,9 +4451,36 @@ def _internal_tool_error(req_id, tool_name: str, exc: BaseException = None) -> d
|
|||
}
|
||||
|
||||
|
||||
def _mcp_read_only_refusal(req_id, tool_name: str):
|
||||
"""Refuse mutating tools when the server runs in read-only mode (#1877).
|
||||
|
||||
Read-only is an operator-set server mode (``--read-only`` /
|
||||
``MEMPALACE_MCP_READ_ONLY``), distinct from the dynamic peer-writer lock:
|
||||
it is an unconditional gate so a shared team server can expose recall
|
||||
without write access. Enforced at dispatch, not merely hidden from
|
||||
tools/list, so a client that calls a mutating tool by name is still refused.
|
||||
"""
|
||||
if not _READ_ONLY or tool_name not in _MUTATING_TOOLS:
|
||||
return None
|
||||
|
||||
return {
|
||||
"jsonrpc": "2.0",
|
||||
"id": req_id,
|
||||
"error": {
|
||||
"code": -32003,
|
||||
"message": "Server is in read-only mode; this tool is disabled",
|
||||
"data": {"tool": tool_name},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _mcp_tool_preflight_refusal(req_id, tool_name: str):
|
||||
"""Run MCP request preflight gates outside handle_request complexity."""
|
||||
|
||||
read_only_error = _mcp_read_only_refusal(req_id, tool_name)
|
||||
if read_only_error is not None:
|
||||
return read_only_error
|
||||
|
||||
sqlite_integrity_error = _mcp_sqlite_integrity_refusal(req_id, tool_name)
|
||||
if sqlite_integrity_error is not None:
|
||||
return sqlite_integrity_error
|
||||
|
|
@ -4480,6 +4532,8 @@ def handle_request(request):
|
|||
# Notifications (no id) never get a response per JSON-RPC spec
|
||||
return None
|
||||
elif method == "tools/list":
|
||||
# In read-only mode, hide the mutating tools so clients don't advertise
|
||||
# write capabilities they can't use (dispatch also refuses them, #1877).
|
||||
return {
|
||||
"jsonrpc": "2.0",
|
||||
"id": req_id,
|
||||
|
|
@ -4487,6 +4541,7 @@ def handle_request(request):
|
|||
"tools": [
|
||||
{"name": n, "description": t["description"], "inputSchema": t["input_schema"]}
|
||||
for n, t in TOOLS.items()
|
||||
if not (_READ_ONLY and n in _MUTATING_TOOLS)
|
||||
]
|
||||
},
|
||||
}
|
||||
|
|
@ -4856,6 +4911,37 @@ _HTTP_LOOPBACK_HOSTS = ("127.0.0.1", "localhost", "::1", "[::1]")
|
|||
_HTTP_ALLOW_INSECURE_NO_TOKEN_ENV = "MEMPALACE_MCP_HTTP_ALLOW_INSECURE_NO_TOKEN"
|
||||
|
||||
|
||||
def _resolve_tls_paths() -> tuple:
|
||||
"""Resolve the TLS cert/key from --tls-cert/--tls-key or env, or (None, None).
|
||||
|
||||
Flags take precedence over ``MEMPALACE_MCP_TLS_CERT`` / ``MEMPALACE_MCP_TLS_KEY``.
|
||||
Both must be given together; one without the other is a configuration error
|
||||
(raised here, before any bind, so it fails loudly at startup).
|
||||
"""
|
||||
cert = (
|
||||
getattr(_args, "tls_cert", None) or os.environ.get("MEMPALACE_MCP_TLS_CERT", "")
|
||||
).strip()
|
||||
key = (getattr(_args, "tls_key", None) or os.environ.get("MEMPALACE_MCP_TLS_KEY", "")).strip()
|
||||
if bool(cert) != bool(key):
|
||||
raise ValueError("TLS requires both --tls-cert and --tls-key (or the matching env vars)")
|
||||
if not cert:
|
||||
return None, None
|
||||
for label, path in (("--tls-cert", cert), ("--tls-key", key)):
|
||||
if not os.path.isfile(path):
|
||||
raise ValueError(f"{label} file not found: {path!r}")
|
||||
return cert, key
|
||||
|
||||
|
||||
def _wrap_tls(sock, cert: str, key: str):
|
||||
"""Wrap a server socket in a TLS 1.2+ context. Raises on bad cert/key."""
|
||||
import ssl
|
||||
|
||||
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
|
||||
ctx.minimum_version = ssl.TLSVersion.TLSv1_2
|
||||
ctx.load_cert_chain(certfile=cert, keyfile=key)
|
||||
return ctx.wrap_socket(sock, server_side=True)
|
||||
|
||||
|
||||
def _http_is_loopback(host: str) -> bool:
|
||||
"""Whether ``host`` binds only to this machine."""
|
||||
return (host or "").strip().lower() in _HTTP_LOOPBACK_HOSTS
|
||||
|
|
@ -4921,6 +5007,11 @@ def _build_http_server(host: str, port: int):
|
|||
"when a trusted fronting layer provides access control."
|
||||
)
|
||||
|
||||
# Resolve TLS before bind so a bad cert/key fails loudly rather than at the
|
||||
# first request. TLS is transport encryption only — the bearer-token guard
|
||||
# above still applies on a non-loopback bind.
|
||||
tls_cert, tls_key = _resolve_tls_paths()
|
||||
|
||||
class _MCPHTTPServer(ThreadingHTTPServer):
|
||||
daemon_threads = True
|
||||
allow_reuse_address = True
|
||||
|
|
@ -5043,6 +5134,10 @@ def _build_http_server(host: str, port: int):
|
|||
httpd.enforce_host_pin = _http_is_loopback(host)
|
||||
httpd.allowed_hosts = _http_allowed_host_values(host, bound_port)
|
||||
httpd.auth_token = auth_token
|
||||
httpd.scheme = "http"
|
||||
if tls_cert:
|
||||
httpd.socket = _wrap_tls(httpd.socket, tls_cert, tls_key)
|
||||
httpd.scheme = "https"
|
||||
return httpd
|
||||
|
||||
|
||||
|
|
@ -5076,7 +5171,14 @@ def _serve_http(host: str, port: int) -> None:
|
|||
_HTTP_ALLOW_INSECURE_NO_TOKEN_ENV,
|
||||
)
|
||||
with httpd:
|
||||
logger.info("MemPalace MCP HTTP server listening on http://%s:%s/mcp", host, bound_port)
|
||||
logger.info(
|
||||
"MemPalace MCP HTTP server listening on %s://%s:%s/mcp%s%s",
|
||||
getattr(httpd, "scheme", "http"),
|
||||
host,
|
||||
bound_port,
|
||||
" (TLS)" if getattr(httpd, "scheme", "http") == "https" else "",
|
||||
" (read-only)" if _READ_ONLY else "",
|
||||
)
|
||||
try:
|
||||
httpd.serve_forever(poll_interval=0.5)
|
||||
except KeyboardInterrupt:
|
||||
|
|
|
|||
|
|
@ -197,6 +197,123 @@ def test_bearer_token_enforced_when_configured(monkeypatch):
|
|||
thread.join(timeout=5)
|
||||
|
||||
|
||||
def test_read_only_hides_and_refuses_mutating_tools(http_server, monkeypatch):
|
||||
"""Read-only mode (#1877): mutating tools are hidden from tools/list AND
|
||||
refused at dispatch with -32003, while read tools still work."""
|
||||
monkeypatch.setattr(mcp, "_READ_ONLY", True)
|
||||
port, _ = http_server
|
||||
|
||||
status, body = _post(port, "/mcp", {"jsonrpc": "2.0", "id": 1, "method": "tools/list"})
|
||||
assert status == 200
|
||||
names = {t["name"] for t in json.loads(body)["result"]["tools"]}
|
||||
assert "mempalace_search" in names # read tool stays
|
||||
assert "mempalace_add_drawer" not in names # mutating tool hidden
|
||||
assert names.isdisjoint(mcp._MUTATING_TOOLS)
|
||||
|
||||
status, body = _post(
|
||||
port,
|
||||
"/mcp",
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"id": 2,
|
||||
"method": "tools/call",
|
||||
"params": {"name": "mempalace_add_drawer", "arguments": {"content": "x"}},
|
||||
},
|
||||
)
|
||||
assert status == 200
|
||||
assert json.loads(body)["error"]["code"] == -32003
|
||||
|
||||
|
||||
def test_read_only_off_exposes_mutating_tools(http_server):
|
||||
"""Sanity: without read-only, mutating tools are present (guards the test above)."""
|
||||
port, _ = http_server
|
||||
status, body = _post(port, "/mcp", {"jsonrpc": "2.0", "id": 1, "method": "tools/list"})
|
||||
names = {t["name"] for t in json.loads(body)["result"]["tools"]}
|
||||
assert "mempalace_add_drawer" in names
|
||||
|
||||
|
||||
def _make_self_signed_cert(tmp_path):
|
||||
"""Write a throwaway self-signed cert/key via openssl; skip if unavailable."""
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
if shutil.which("openssl") is None:
|
||||
pytest.skip("openssl not available to generate a test certificate")
|
||||
cert = tmp_path / "cert.pem"
|
||||
key = tmp_path / "key.pem"
|
||||
subprocess.run(
|
||||
[
|
||||
"openssl",
|
||||
"req",
|
||||
"-x509",
|
||||
"-newkey",
|
||||
"rsa:2048",
|
||||
"-keyout",
|
||||
str(key),
|
||||
"-out",
|
||||
str(cert),
|
||||
"-days",
|
||||
"1",
|
||||
"-nodes",
|
||||
"-subj",
|
||||
"/CN=localhost",
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
return cert, key
|
||||
|
||||
|
||||
def test_tls_serves_https(tmp_path, monkeypatch):
|
||||
"""With --tls-cert/--tls-key (via env), the server speaks TLS: a plain HTTP
|
||||
client cannot read it, and an HTTPS client trusting the cert can."""
|
||||
import ssl
|
||||
|
||||
cert, key = _make_self_signed_cert(tmp_path)
|
||||
monkeypatch.setenv("MEMPALACE_MCP_TLS_CERT", str(cert))
|
||||
monkeypatch.setenv("MEMPALACE_MCP_TLS_KEY", str(key))
|
||||
|
||||
httpd = mcp._build_http_server("127.0.0.1", 0)
|
||||
assert getattr(httpd, "scheme", "http") == "https"
|
||||
port = httpd.server_address[1]
|
||||
thread = threading.Thread(
|
||||
target=httpd.serve_forever, kwargs={"poll_interval": 0.05}, daemon=True
|
||||
)
|
||||
thread.start()
|
||||
try:
|
||||
# Full verification on: trust the self-signed cert as the CA and dial
|
||||
# "localhost" (the cert CN, resolves to 127.0.0.1) so hostname checking
|
||||
# passes without being disabled.
|
||||
ctx = ssl.create_default_context(cafile=str(cert))
|
||||
conn = http.client.HTTPSConnection("localhost", port, context=ctx, timeout=5)
|
||||
try:
|
||||
conn.request("GET", "/healthz")
|
||||
resp = conn.getresponse()
|
||||
assert resp.status == 200
|
||||
assert resp.read() == b"ok\n"
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
# A plaintext HTTP client must NOT be able to talk to the TLS socket.
|
||||
with pytest.raises(Exception):
|
||||
plain = http.client.HTTPConnection("127.0.0.1", port, timeout=5)
|
||||
plain.request("GET", "/healthz")
|
||||
plain.getresponse()
|
||||
finally:
|
||||
httpd.shutdown()
|
||||
httpd.server_close()
|
||||
thread.join(timeout=5)
|
||||
|
||||
|
||||
def test_tls_requires_both_cert_and_key(tmp_path, monkeypatch):
|
||||
"""A cert without a key (or vice versa) is a startup error, not a silent skip."""
|
||||
cert, _key = _make_self_signed_cert(tmp_path)
|
||||
monkeypatch.setenv("MEMPALACE_MCP_TLS_CERT", str(cert))
|
||||
monkeypatch.delenv("MEMPALACE_MCP_TLS_KEY", raising=False)
|
||||
with pytest.raises(ValueError, match="both"):
|
||||
mcp._build_http_server("127.0.0.1", 0)
|
||||
|
||||
|
||||
def test_loopback_and_origin_helpers():
|
||||
assert mcp._http_is_loopback("127.0.0.1")
|
||||
assert mcp._http_is_loopback("localhost")
|
||||
|
|
|
|||
|
|
@ -0,0 +1,160 @@
|
|||
"""Tests for the turnkey `mempalace serve` command (#1877).
|
||||
|
||||
These exercise the wrapper's security-relevant behavior — token autogeneration
|
||||
and 0600 persistence, the secure-by-default non-loopback gate, and that the
|
||||
bearer token is passed via the environment (never argv, so it can't leak via
|
||||
``ps``) — without binding a real socket. ``cmd_serve`` ends by exec'ing the real
|
||||
server; we intercept ``os.execve`` to capture the child invocation instead.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import stat
|
||||
|
||||
import pytest
|
||||
|
||||
from mempalace import cli
|
||||
|
||||
|
||||
class _ExecCalled(Exception):
|
||||
"""Raised by the patched os.execve to stop cmd_serve at the exec boundary."""
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def isolated_home(tmp_path, monkeypatch):
|
||||
"""Point ~ at a temp dir so server token state never touches the real home."""
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
monkeypatch.setenv("USERPROFILE", str(tmp_path)) # Windows
|
||||
# Don't inherit a token from the ambient environment.
|
||||
monkeypatch.delenv("MEMPALACE_MCP_HTTP_TOKEN", raising=False)
|
||||
return tmp_path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def capture_exec(monkeypatch):
|
||||
"""Capture the child argv/env cmd_serve would launch, instead of running it.
|
||||
|
||||
cmd_serve takes the os.execve branch on POSIX and the subprocess.run branch
|
||||
on Windows. Patch both (rather than forcing os.name, which breaks
|
||||
Path.home() on Windows) so the test is platform-agnostic.
|
||||
"""
|
||||
import subprocess
|
||||
|
||||
captured = {}
|
||||
|
||||
def _capture(argv, env):
|
||||
captured["argv"] = argv
|
||||
captured["env"] = env
|
||||
raise _ExecCalled()
|
||||
|
||||
monkeypatch.setattr(cli.os, "execve", lambda path, argv, env: _capture(argv, env))
|
||||
monkeypatch.setattr(subprocess, "run", lambda argv, env=None, **kw: _capture(argv, env))
|
||||
return captured
|
||||
|
||||
|
||||
def _serve_args(tmp_path, **over):
|
||||
base = dict(
|
||||
host="127.0.0.1",
|
||||
port=8765,
|
||||
backend=None,
|
||||
global_backend=None,
|
||||
palace=str(tmp_path / "palace"),
|
||||
token=None,
|
||||
tls_cert=None,
|
||||
tls_key=None,
|
||||
read_only=False,
|
||||
allow_insecure=False,
|
||||
)
|
||||
base.update(over)
|
||||
return argparse.Namespace(**base)
|
||||
|
||||
|
||||
def test_token_helper_creates_0600_and_reuses(isolated_home):
|
||||
palace = str(isolated_home / "palace")
|
||||
token1, created1 = cli._load_or_create_server_token(palace)
|
||||
assert created1 is True
|
||||
assert token1
|
||||
|
||||
path = cli._server_token_path(palace)
|
||||
assert path.exists()
|
||||
if os.name == "posix":
|
||||
# POSIX permission bits aren't meaningful on Windows (files report 0o666).
|
||||
mode = stat.S_IMODE(path.stat().st_mode)
|
||||
assert mode == 0o600, oct(mode)
|
||||
dir_mode = stat.S_IMODE(path.parent.stat().st_mode)
|
||||
assert dir_mode == 0o700, oct(dir_mode)
|
||||
|
||||
token2, created2 = cli._load_or_create_server_token(palace)
|
||||
assert created2 is False
|
||||
assert token2 == token1 # stable across restarts
|
||||
|
||||
|
||||
def test_loopback_serve_needs_no_token(isolated_home, capture_exec):
|
||||
with pytest.raises(_ExecCalled):
|
||||
cli.cmd_serve(_serve_args(isolated_home, host="127.0.0.1"))
|
||||
env = capture_exec["env"]
|
||||
assert "MEMPALACE_MCP_HTTP_TOKEN" not in env
|
||||
assert "MEMPALACE_MCP_HTTP_ALLOW_INSECURE_NO_TOKEN" not in env
|
||||
# No token persisted for a loopback bind.
|
||||
assert not cli._server_token_path(str(isolated_home / "palace")).exists()
|
||||
|
||||
|
||||
def test_non_loopback_autogenerates_token_in_env_not_argv(isolated_home, capture_exec):
|
||||
with pytest.raises(_ExecCalled):
|
||||
cli.cmd_serve(_serve_args(isolated_home, host="0.0.0.0"))
|
||||
env = capture_exec["env"]
|
||||
argv = capture_exec["argv"]
|
||||
token = env.get("MEMPALACE_MCP_HTTP_TOKEN")
|
||||
assert token, "a token must be generated for a network-exposed bind"
|
||||
# Security: the token rides in the env, never on the command line.
|
||||
assert all(token not in part for part in argv)
|
||||
assert "--token" not in argv
|
||||
# And it was persisted for reuse on the next start (0600 on POSIX).
|
||||
path = cli._server_token_path(str(isolated_home / "palace"))
|
||||
assert path.exists()
|
||||
if os.name == "posix":
|
||||
assert stat.S_IMODE(path.stat().st_mode) == 0o600
|
||||
|
||||
|
||||
def test_allow_insecure_skips_token_and_sets_escape_hatch(isolated_home, capture_exec):
|
||||
with pytest.raises(_ExecCalled):
|
||||
cli.cmd_serve(_serve_args(isolated_home, host="0.0.0.0", allow_insecure=True))
|
||||
env = capture_exec["env"]
|
||||
assert env.get("MEMPALACE_MCP_HTTP_ALLOW_INSECURE_NO_TOKEN") == "1"
|
||||
assert "MEMPALACE_MCP_HTTP_TOKEN" not in env
|
||||
assert not cli._server_token_path(str(isolated_home / "palace")).exists()
|
||||
|
||||
|
||||
def test_read_only_flag_forwarded_to_child(isolated_home, capture_exec):
|
||||
with pytest.raises(_ExecCalled):
|
||||
cli.cmd_serve(_serve_args(isolated_home, read_only=True))
|
||||
assert "--read-only" in capture_exec["argv"]
|
||||
|
||||
|
||||
def test_explicit_token_is_used_and_not_in_argv(isolated_home, capture_exec):
|
||||
with pytest.raises(_ExecCalled):
|
||||
cli.cmd_serve(_serve_args(isolated_home, host="0.0.0.0", token="my-secret-token"))
|
||||
env = capture_exec["env"]
|
||||
argv = capture_exec["argv"]
|
||||
assert env["MEMPALACE_MCP_HTTP_TOKEN"] == "my-secret-token"
|
||||
assert all("my-secret-token" not in part for part in argv)
|
||||
# An explicitly-provided token is not persisted to the server token file.
|
||||
assert not cli._server_token_path(str(isolated_home / "palace")).exists()
|
||||
|
||||
|
||||
def test_tls_paths_forwarded_and_validated(isolated_home, capture_exec, tmp_path):
|
||||
cert = tmp_path / "cert.pem"
|
||||
key = tmp_path / "key.pem"
|
||||
cert.write_text("x")
|
||||
key.write_text("x")
|
||||
with pytest.raises(_ExecCalled):
|
||||
cli.cmd_serve(_serve_args(isolated_home, tls_cert=str(cert), tls_key=str(key)))
|
||||
argv = capture_exec["argv"]
|
||||
assert "--tls-cert" in argv and "--tls-key" in argv
|
||||
|
||||
|
||||
def test_tls_requires_both_cert_and_key(isolated_home, capture_exec, tmp_path):
|
||||
cert = tmp_path / "cert.pem"
|
||||
cert.write_text("x")
|
||||
with pytest.raises(SystemExit):
|
||||
cli.cmd_serve(_serve_args(isolated_home, tls_cert=str(cert), tls_key=None))
|
||||
|
|
@ -89,35 +89,46 @@ and needs no extra.
|
|||
|
||||
## 3. Serve MCP over HTTP
|
||||
|
||||
The MCP server speaks JSON-RPC over `POST /mcp` and exposes an unauthenticated
|
||||
`GET /healthz` liveness probe for orchestrators. Binding to a **non-loopback**
|
||||
host requires a bearer token — MemPalace refuses to start otherwise.
|
||||
One command — `mempalace serve` — runs the server with secure defaults. On a
|
||||
network-exposed (`0.0.0.0`) bind it **auto-generates a strong bearer token**
|
||||
(stored `0600` under `~/.mempalace/server/`, printed once), prints a
|
||||
ready-to-paste client config, and runs in the foreground so Docker/systemd own
|
||||
the lifecycle.
|
||||
|
||||
```bash
|
||||
export MEMPALACE_MCP_HTTP_TOKEN="$(openssl rand -hex 32)"
|
||||
|
||||
mempalace-mcp --transport http --host 0.0.0.0 --port 8765 --backend qdrant
|
||||
mempalace serve --host 0.0.0.0 --port 8765 --backend qdrant
|
||||
```
|
||||
|
||||
| Flag / variable | Default | Purpose |
|
||||
Output includes the token and the exact client command. Useful flags:
|
||||
|
||||
| Flag | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `--transport http` | `stdio` | Serve over HTTP instead of stdio |
|
||||
| `--host` | `127.0.0.1` | Bind address (`0.0.0.0` to accept remote clients) |
|
||||
| `--port` | `8765` | Listen port |
|
||||
| `MEMPALACE_MCP_HTTP_TOKEN` | _(none)_ | **Required** for non-loopback binds; clients send `Authorization: Bearer <token>` |
|
||||
| `--backend` | config/env | Storage backend (e.g. `qdrant`) |
|
||||
| `--tls-cert` / `--tls-key` | _(none)_ | PEM cert + key to terminate **TLS natively** (server speaks `https`) |
|
||||
| `--read-only` | off | Expose recall only — the mutating tools are hidden and refused |
|
||||
| `--token` | auto | Use a specific bearer token instead of the generated one |
|
||||
| `--allow-insecure` | off | Permit a non-loopback bind with no token (only behind a trusted proxy) |
|
||||
|
||||
The server protects against DNS-rebinding with a `Host` allowlist and an
|
||||
`Origin` loopback check, and serializes concurrent writes — so multiple
|
||||
teammates can write to the shared palace at once over HTTP.
|
||||
The token always travels via the environment, never the command line, so it
|
||||
can't leak through `ps`. Binding to a non-loopback host with no token and no
|
||||
`--allow-insecure` refuses to start. The server also guards against
|
||||
DNS-rebinding with a `Host` allowlist and an `Origin` loopback check, and
|
||||
serializes concurrent writes — so multiple teammates can write to the shared
|
||||
palace at once over HTTP.
|
||||
|
||||
::: danger Put TLS in front of it
|
||||
The HTTP server is plaintext. For anything beyond a trusted private network,
|
||||
run it behind a reverse proxy (nginx/Caddy/Traefik) terminating TLS, and keep
|
||||
the bearer token secret. Only set
|
||||
`MEMPALACE_MCP_HTTP_ALLOW_INSECURE_NO_TOKEN=1` when a trusted fronting layer
|
||||
already enforces access control — never on a directly-exposed port.
|
||||
::: tip TLS
|
||||
Pass `--tls-cert`/`--tls-key` to terminate TLS in the server itself
|
||||
(`https://…`). Otherwise the server is plaintext and you should front it with a
|
||||
TLS-terminating reverse proxy (nginx/Caddy/Traefik) — never expose plaintext
|
||||
`/mcp` beyond a trusted private network.
|
||||
:::
|
||||
|
||||
The underlying server is `mempalace-mcp --transport http` (the same flags exist
|
||||
there if you'd rather wire the token/TLS yourself); `mempalace serve` is the
|
||||
turnkey wrapper over it.
|
||||
|
||||
## 4. Connect a client
|
||||
|
||||
Point each teammate's MCP client at the server's `/mcp` endpoint with the
|
||||
|
|
@ -151,6 +162,29 @@ whole team.
|
|||
- **Backups** are now your storage backend's responsibility (Qdrant snapshots
|
||||
/ Postgres backups) rather than a single laptop's palace directory.
|
||||
|
||||
## One-command deployments
|
||||
|
||||
The repo ships ready-to-edit deployment files under
|
||||
[`deploy/`](https://github.com/MemPalace/mempalace/tree/main/deploy):
|
||||
|
||||
**Docker Compose (server + Qdrant):**
|
||||
|
||||
```bash
|
||||
cp deploy/server.env.example deploy/.env # set MEMPALACE_MCP_HTTP_TOKEN
|
||||
docker compose -f deploy/docker-compose.server.yml --env-file deploy/.env up -d
|
||||
```
|
||||
|
||||
This brings up a Qdrant container and a MemPalace server running
|
||||
`serve --host 0.0.0.0 --backend qdrant`, with a `/healthz` healthcheck and
|
||||
persistent volumes. Embeddings stay local to the MemPalace container.
|
||||
|
||||
**systemd:**
|
||||
|
||||
`deploy/mempalace-server.service` is a hardened unit template
|
||||
(`NoNewPrivileges`, `ProtectSystem=strict`, dedicated user) that runs
|
||||
`mempalace serve` with its config from `/etc/mempalace/server.env`. Install
|
||||
steps are in the file's header comment.
|
||||
|
||||
## See also
|
||||
|
||||
- [MCP Integration](/guide/mcp-integration) — the tools clients get once connected
|
||||
|
|
|
|||
Loading…
Reference in New Issue