Five fixes from the Copilot review of the merged daemon PR:
1. Privacy: the queue DB's SQLite WAL/SHM sidecars hold un-checkpointed
verbatim payloads but were created with the caller's umask. Set the
owner-only umask in run_server BEFORE DaemonRuntime builds the QueueStore
(not only once the HTTP server starts), and harden any existing sidecars in
QueueStore._init_db as defense-in-depth.
2. DoS guard: reject a negative Content-Length in the request reader.
rfile.read(-1) would block until the client disconnects and bypass the
MAX_BODY_BYTES cap.
3. Side effects: extract _wal_log (+ _ensure_wal, _WAL_FILE, _WAL_REDACT_KEYS)
into a new side-effect-free mempalace/wal.py. The CLI sync path and the
daemon service layer obtained _wal_log via `from .mcp_server import _wal_log`,
which runs mcp_server's import-time stdio protection (os.dup2(2, 1);
sys.stdout = sys.stderr) in a non-MCP process and misroutes operator output.
mcp_server/cli/service now import from mempalace.wal.
4. Correctness: run_mcp_tool treated any dict as success. Write tools that
return a bare {"error": ...} (e.g. tool_create_tunnel/tool_delete_tunnel
validation) were recorded as succeeded; now the "error" key infers failure.
5. Hook budget: get_client_if_running()/health() take an explicit timeout, and
the hook "is the daemon up?" precheck uses a short HOOK_PROBE_TIMEOUT (0.5s)
so a wedged daemon can't stall the hook for the default 5s.
Adds tests/test_wal.py (import isolation + redaction) and daemon tests for the
umask ordering, negative Content-Length, run_mcp_tool error inference, and the
short probe timeout.
_pid_alive used os.kill(pid, 0) as an existence check. On Windows signal 0
is signal.CTRL_C_EVENT, so Python routes it to GenerateConsoleCtrlEvent and
sends a console Ctrl-C to the target's process group rather than probing the
pid. DaemonClient polls a same-process endpoint during startup, so on a CI
runner with an attached console that Ctrl-C was delivered back to the
interpreter as a spurious KeyboardInterrupt — the Windows CI hang that
interrupted the suite at the first daemon HTTP-lifecycle test (socket.py
recv). Probe via the Win32 OpenProcess/WaitForSingleObject handle API instead,
which has no signalling side effects. This is also a real Windows production
bug, not just a test artifact.
Tests:
- Skip the two owner-only (0600) permission tests on Windows: os.chmod cannot
represent POSIX mode bits there (files report 0o666); the daemon relies on
user-profile ACLs on Windows.
- _start_server now captures and re-surfaces a run_server thread crash instead
of spinning for 30s and failing with a bare assert (diagnoses the macOS
startup flake).
- Add a regression test asserting _pid_alive is correct and emits no console
control event when hammered like the poll loop.
- Remove the temporary win32 exit-hang diagnostic fixture from conftest now
that the root cause is fixed.
DaemonClient.request now uses a no-proxy opener (build_opener(ProxyHandler({})))
instead of urllib.urlopen. The daemon is always on 127.0.0.1, so a request must
never go through an HTTP proxy — this is the correct production choice. It also
bypasses urllib's proxy discovery (macOS _scproxy via SystemConfiguration), which
runs on the first request to any host and is NOT bounded by the per-request
timeout: on a CI runner with no network it hangs for tens of seconds, which looked
exactly like the daemon never came up (test_daemon_http_lifecycle_executes_job
timed out at 30.18s). With the no-proxy opener the lifecycle runs in 0.78s and no
server thread is leaked — which also removes the timing skew that made the
sqlite_exact concurrent-connection test flake on macOS CI.
The leaked server thread was also the Windows exit-hang root cause: a slow/failed
client.shutdown() POST left serve_forever running, and the interpreter blocked on
the open listening socket at process exit. Tests now capture the httpd run_server
creates (by subclassing daemon.ThreadingHTTPServer) and force httpd.shutdown() +
server_close() from the test thread if the normal shutdown path leaves the thread
alive, asserting the thread died so a leak becomes a visible failure instead of
a silent exit hang.
CI was red on all three platforms for the daemon-mode draft PR. Root causes
and fixes:
- Linux 3.9 collection error: `_submit_daemon_job`'s `dedupe_key: str | None`
parameter annotation is evaluated at def time, and hooks_cli.py has no
`from __future__ import annotations` — `str | None` raises TypeError on 3.9.
Reverted to `dedupe_key: str = None` (the original, 3.9-safe). The other
`int | None` in the file is a function-local annotation, which is never
evaluated, so it was never the problem.
- macOS/Windows daemon lifecycle flakes: the 3 HTTP-lifecycle tests failed at
the 10s readiness deadline on contended CI runners (localhost bind is
sub-second locally but took ~5s when it passed on the macOS fleet, >10s when
it didn't), and because the server thread never shuts down on timeout,
run_server's `os.environ["MEMPALACE_PALACE_PATH"]` + `os.umask(0o077)`
mutations leaked into the rest of the suite — poisoning every later test that
reads MempalaceConfig().palace_path (the 60+ test_mcp_server cascade on macOS;
the at-exit socket hang → SIGINT on Windows). Bumped the readiness deadline
to 30s and added a module-scoped snapshot + autouse fixture in test_daemon.py
that force-restores the env + umask to the pre-suite baseline after every
daemon test, so a leaked server thread can't poison other test files.
Gemini review comments (fixed in code, no thread replies per convention):
- daemon.py `_connect()` was a bare `sqlite3.connect` whose `with`-block only
managed the transaction, not the connection — an unbounded FD leak in a
long-lived daemon running thousands of jobs (also the source of the Windows
"unclosed database" ResourceWarning noise). Converted to a closing
@contextlib.contextmanager.
- `QueueStore.finish()` gained `only_if_running`; `_safe_finish` passes it so a
late worker finish can't overwrite a shutdown-cancelled job back to
succeeded/failed — removes the reliance on process-exit timing.
- `DaemonClient.request` wraps the final `json.loads` in try/except
JSONDecodeError → DaemonError, so a non-JSON 2xx response surfaces as a
structured error instead of a bare JSONDecodeError.
- test_sync.py: removed the module-level `import mempalace.mcp_server` and moved
the stdout-rebinding side effect into an autouse fixture scoped to
TestServiceRunSyncReport, so the embedder/Chroma import chain is no longer
forced at collection time for the existing sync tests.
Coverage: added focused happy-path tests for service.run_sync early-returns,
run_mine backend application + invalid mode, execute_job kind dispatch,
run_diary_write arg forwarding, run_mcp_tool write-tool dispatch, and
print_job_result — lifts service.py from 57% to 85% so the new files
(service 85%, daemon 80%) don't drag the total below the 80% CI gate now that
the daemon tests complete and the gate is actually evaluated.
- New mempalace/daemon.py: long-lived localhost HTTP server (127.0.0.1) with a
SQLite WAL job queue, single worker thread, bearer-token auth, and owner-only
file perms (0600/0700) on queue DB, token, endpoint, and log.
- New mempalace/service.py: transport-neutral job execution surface shared by the
daemon, with per-job env isolation so one job's backend/palace switch cannot
leak into the next. mcp_tool is allowlisted to write-classified tools only.
- Crash recovery re-queues jobs left 'running' by a killed daemon; jobs that
already exhausted MAX_ATTEMPTS are dead-lettered to 'failed' instead of being
retried (non-idempotent diary_write would otherwise duplicate verbatim
content on every restart).
- Bounded retention prunes terminal jobs older than 7 days
(MEMPALACE_DAEMON_RETENTION_DAYS); queued/running jobs are never touched so a
crash mid-prune cannot drop in-flight work.
- CLI: --daemon/--background on mine/sync submit to the queue; new
`mempalace daemon {start,stop,status,jobs,wait}` subcommand. Strictly opt-in:
no flag, env, or config means no daemon and no behavior change.
- Hooks opt in via MEMPALACE_HOOKS_DAEMON or config hooks.daemon; when the daemon
is not already running, hooks fall back to the existing direct/spawn path so
the 500ms hook budget is preserved (hooks never auto-start the daemon).
- service.run_sync renders the same operator-facing report shape as the direct
CLI sync path (no_source, out_of_scope, by_source, Re-run/Removed hints) and
drops the old KeyError-prone 'deleted' read.
Igor Lins e Silva