fix: consolidation 风暴 — distill cooldown 60s→15min, cluster_only 跳过 PageRank

- triggers.go: TriggerDistill cooldown time.Minute → 15 * time.Minute
- consolidation_pipe.go: runGraphMaintenance 仅 full 模式执行(cluster_only 高频快速聚类跳过重负载 PageRank)
- 修复 zhiyid CPU 80-90% 风暴、recall API 120s+ 超时

根因:t_distill 每 60s 触发全量 DBSCAN + PageRank(10356 nodes),数据量大时单次 30-60s CPU 堆积
This commit is contained in:
小唯 2026-08-10 01:21:57 +08:00
parent 98e541b985
commit 11162e2f60
3 changed files with 72 additions and 4 deletions

View File

@ -0,0 +1,66 @@
# 织忆 Consolidation 风暴修复 — 实施计划
> 2026-08-10 | 小唯 | 优先级P0recall API 不可用)
## 问题
zhiyid CPU 80-90%recall/memories API 超时HTTP 000 / 120s+journalctl 每 15-60s 一次 `[consolidation] Rust sidecar 完成` + `PageRank 更新: 10341 nodes`
## 根因(已源码定位)
1. **server.go:1113-1125**30s ticker 检查 7 个触发器
2. **triggers.go**`t_distill` cooldown = **60s**`t_merge` = 10min
3. **consolidation_pipe.go:46-48**`Run()` → `cluster_only` 模式
4. **rust/main.rs:374-404**cluster_only 每次 `lancedb.search(&zero_vec, 10000)` 加载**全部 10000 条向量** + DBSCAN 全量聚类 + PageRank
**风暴机制**t_distill 每 60s 触发 → 全量聚类(数据量大时单次 30-60s→ 还没跑完下一轮又触发 → CPU 堆积HTTP 排队超时。
## 修复方案(选 A最小改动立竿见影
### A. 调大 distill 触发器 cooldown60s → 15min
**文件**`go/internal/api/routes/triggers.go`
```go
// 修改 cooldownMap
TriggerDistill: time.Minute, // 改为 15 * time.Minute
```
**影响**
- 蒸馏批量处理从每 60s 一次 → 每 15min 一次(数据仍在队列,不会丢)
- cluster_only 全量聚类从每 60s → 每 15minCPU 风暴消除)
- distill 的"5min 无蒸馏 → 批量"逻辑仍可触发(是另一条路径)
### B. 可选增强cluster_only 无新写入跳过
**文件**`go/internal/api/routes/consolidation_pipe.go`
在 Run() 前检查 `ldb.Stats()["total_episodes"]` 与上次相比无增长 → 直接 return 空报告。防止 15min 间隔内仍频繁全量跑。
## 改动清单
| 文件 | 改动 |
|------|------|
| `go/internal/api/routes/triggers.go` | TriggerDistill cooldown 60s → 15min |
| B`go/internal/api/routes/consolidation_pipe.go` | cluster_only 无新写入跳过 |
## 编译部署
```bash
cd /tmp/memoryweave/go && go build -o zhiyid-new ./cmd/zhiyid
# 成功 → 复制
systemctl --user stop zhiyid
cp zhiyid-new /home/muc/bin/zhiyid-new
systemctl --user start zhiyid
# 验证
curl -s -H "X-API-Key: zhiyi-dev-key-2026" http://localhost:7821/api/v1/health
ps aux | grep zhiyid-new | grep -v grep # CPU 应 < 20%
```
## 验证标准
1. `systemctl --user status zhiyid` → active (running)
2. CPU 稳定 < 20%之前 80-90%
3. recall API 10s 内响应(之前 120s+ 超时)
4. 日志 consolidation 频率≥15min 一次(之前 15-60s
5. memory_write → 5-10s → recall 能命中新写入

View File

@ -122,9 +122,11 @@ func (cp *ConsolidationPipeline) RunWithMode(mode string) (*ConsolidationReport,
}
metrics.SyncFromDashboard(selfoptimize.Dash.Metrics())
// ─── 图谱后处理:修剪 + PageRank无论 Rust/Go 都执行)──
pruned := cp.runGraphMaintenance()
report.GraphPruned += pruned
// ─── 图谱后处理:修剪 + PageRank仅 full 模式执行cluster_only 是高频快速聚类,跳过重负载的 PageRank 防止 CPU 风暴)──
if mode == "full" {
pruned := cp.runGraphMaintenance()
report.GraphPruned += pruned
}
report.FinishedAt = time.Now()
report.Duration = report.FinishedAt.Sub(report.StartedAt).String()

View File

@ -26,7 +26,7 @@ const (
// 冷却时间映射
var cooldownMap = map[TriggerType]time.Duration{
TriggerDistill: time.Minute,
TriggerDistill: 15 * time.Minute,
TriggerMerge: 10 * time.Minute,
TriggerPrune: 24 * time.Hour,
TriggerDecay: 6 * time.Hour,