Files
memoryweave/go
小唯andprof-cloud 19afe3bfe2 perf(distill): AC2 concurrency matched to local 2B slots (t_9b2c7943 AC2)
AC2: concurrency no longer hardcoded 3, now matched to local 2B slot count.
Probe endpoint /props total_slots at startup and before each flush;
effective concurrency = min(slots, distillConcurrencyMax=3).
Probe failure (404/non-JSON/conn refused) conservatively defaults to 1
(serialize rather than hammer unknown slot limit).

Criterion (from 小怡): if 429 ratio rises instead of falls, slots are
overrun, must dial down. Measured 2026-09-21: 106 llama-server is
single-slot (total_slots=1) -> matched concurrency=1.

Changes:
- Engine gains concurrency field (effective concurrency, default 1)
- NewEngine calls RefreshConcurrency() at startup
- flush() re-probes via RefreshConcurrency() before each round to
  track slot changes after deploy
- probeLLMSlots: GET /props read total_slots; fail/404/nonJSON -> 1
- log lines print actual effConcurrency (no longer constant 3)

Tests (engine_concurrency_test.go rewritten, all PASS):
- slots 5 > cap 3 -> concurrency=3 (cap protects 2B)
- slots 2 < cap 3 -> concurrency=2 (matches slots)
- slots 1 (106 single-slot status quo) -> concurrency=1 (serial)
- /props 404 -> conservative concurrency=1
- peak concurrency <= min(slots,cap) and >=1 (truly effective)

NOT deployed / NOT restarted zhiyid (waiting 09-23 window; the AC2
30min before/after comparison is a separate evidence step).

Co-Authored-By: prof-cloud <prof-cloud@hermes>
2026-09-21 22:32:48 +08:00
..