feat(记忆系统): 5阶段全部落地 - 时间衰减recall/遗忘曲线分层/画像LLM合成/冲突检测/Consolidation引擎

Phase 1: time_decay_recall() - 织忆recall加权时间衰减(60%语义+40%衰减)
Phase 4: get_memory_tier + log_memory_access + _boost_recalled_memory - hot/warm/cold/archive四层+30天访问boost×1.15
Phase 2: update_profile_from_journal - deep_tick时LLM分析journal更新画像
Phase 3: detect_memory_conflicts + _write_conflicts_to_queue - journal vs 织忆矛盾检测+冲突队列
Phase 5: _consolidate_memories() - memory-system-self-upgrade.py每日Consolidation(重复记忆检测+高频模式挖掘+触发器)

全部语法验证通过,daemon已重启生效。
This commit is contained in:
小唯 A06 2026-07-13 13:53:43 +08:00
parent c9b0fc8a7f
commit fb709f0cec
3 changed files with 471 additions and 0 deletions

View File

@ -0,0 +1,9 @@
{"memory_id": "mem_1782325175173247433", "accessed_at": "2026-07-13T13:48:33.400962", "decay_weight": 0.7228}
{"memory_id": "mem_1783532486242199937", "accessed_at": "2026-07-13T13:48:33.401043", "decay_weight": 0.9324}
{"memory_id": "mem_1782032752240268848", "accessed_at": "2026-07-13T13:48:33.401066", "decay_weight": 0.6721}
{"memory_id": "mem_1783876632405546896", "accessed_at": "2026-07-13T13:53:18.343833", "decay_weight": 0.9921}
{"memory_id": "mem_1782923953734495140", "accessed_at": "2026-07-13T13:53:18.343908", "decay_weight": 0.8267}
{"memory_id": "mem_1782032752240268848", "accessed_at": "2026-07-13T13:53:18.343945", "decay_weight": 0.672}
{"memory_id": "mem_1782923953810508237", "accessed_at": "2026-07-13T13:53:18.343972", "decay_weight": 0.8267}
{"memory_id": "mem_1783615602502470984", "accessed_at": "2026-07-13T13:53:18.343996", "decay_weight": 0.9468}
{"memory_id": "mem_1783590100654305177", "accessed_at": "2026-07-13T13:53:18.344024", "decay_weight": 0.9424}

View File

@ -475,6 +475,371 @@ def save_context(ctx):
with open(CONTEXT_FILE, "w") as f:
json.dump(ctx, f, indent=2)
# ====== 画像LLM合成 ======
def update_profile_from_journal(journal_path: str, profile_path: str, model: str = "mistralai/mistral-large-3-675b-instruct-2512"):
"""
读取最近 journal_entry分析牧尘行为模式LLM生成画像更新建议只更新非空字段
"""
import json, os, requests, re
from datetime import datetime as dt
try:
with open(journal_path) as f:
lines = f.readlines()
if len(lines) < 3:
return
entries = [json.loads(l) for l in lines]
recent = [e for e in entries if e.get("type") != "startup"][-10:]
except Exception:
return
if len(recent) < 3:
return
log_lines = "\n".join(f"- {e.get('summary','')}: {e.get('details','(无详情)')}" for e in recent)
prompt = f"""牧尘是一个技术用户,用小唯 AI 助手Hermes Agent工作。
分析以下最近的10条行为日志识别牧尘的
1. 沟通/工作模式他怎么提问/偏好什么
2. 正在进行的项目或兴趣方向
3. 任何新发现的重要偏好或习惯
行为日志
{log_lines}
输出一个JSON仅包含需要更新的画像字段只输出有变化的字段
{{"字段名": "新值", ...}}
如果没有任何需要更新的输出空对象 {{}}"""
try:
token = os.environ.get("NEWAPI_TOKEN", KEY)
resp = requests.post(
f"{API}/chat/completions",
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"},
json={"model": model, "messages": [{"role": "user", "content": prompt}],
"max_tokens": 500, "temperature": 0.3},
timeout=30,
)
content = resp.json().get("choices", [{}])[0].get("message", {}).get("content", "{}")
except Exception:
return
match = re.search(r'\{[^{}]*(?:"[^"]+"\s*:\s*[^{}]+){1,4}\}', content, re.DOTALL)
if not match:
return
try:
updates = json.loads(match.group())
except Exception:
return
if not updates:
return
try:
with open(profile_path) as f:
profile = json.load(f)
except Exception:
return
changed = False
for key, val in updates.items():
if val and str(val).strip() and profile.get(key) != val:
profile[key] = str(val)
changed = True
if changed:
profile["updated_at"] = dt.now().isoformat()
with open(profile_path, "w") as f:
json.dump(profile, f, ensure_ascii=False, indent=2)
journal_entry("profile_update", f"LLM更新画像: {list(updates.keys())}")
# ====== 冲突检测 ======
def _write_conflicts_to_queue(conflicts: list):
"""写入冲突队列文件。"""
import json, os
from datetime import datetime as dt
queue_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "soulful", "conflicts-queue.json")
try:
with open(queue_path) as f:
queue = json.load(f)
except Exception:
queue = []
for c in conflicts:
if not any(existing.get("id") == c.get("id") for existing in queue):
queue.append(c)
with open(queue_path, "w") as f:
json.dump(queue, f, ensure_ascii=False, indent=2)
def detect_memory_conflicts(journal_path: str, zhiyi_token: str) -> list[dict]:
"""
LLM 检测最近 journal_entry 和织忆记忆之间的矛盾
返回格式[{entry, conflict_with, description}]
"""
import json, os, requests, re
from datetime import datetime as dt
# 读取最近 5 条 journal排除 startup
try:
with open(journal_path) as f:
lines = f.readlines()
entries = [json.loads(l) for l in lines if json.loads(l).get("type") != "startup"][-5:]
except Exception:
return []
if not entries:
return []
# 读取织忆最近的 key memories
try:
r = requests.post(
"http://127.0.0.1:7821/api/v1/recall",
json={"query": "牧尘 重要决定 项目 偏好", "top_k": 5, "agent_id": "hermes-a06"},
headers={"X-API-Key": zhiyi_token},
timeout=8,
)
existing = r.json().get("results", []) if r.status_code == 200 else []
except Exception:
existing = []
if not existing:
return []
entries_text = "\n".join(f"- {e.get('summary','')}" for e in entries)
existing_text = "\n".join(f"- {m.get('content','')[:100]}" for m in existing)
prompt = f"""以下A组是牧尘最近的行为日志B组是织忆记忆里的长期记录。
判断A和B之间有没有矛盾直接冲突的信息不需要轻微不一致
A组最近行为
{entries_text}
B组长期记忆
{existing_text}
如果A和B有直接矛盾输出JSON格式
{{"has_conflict": true, "conflict_description": "矛盾描述", "a_entry": "哪条A", "b_entry": "哪条B"}}
如果无矛盾输出
{{"has_conflict": false}}"""
try:
token = os.environ.get("NEWAPI_TOKEN", KEY)
resp = requests.post(
f"{API}/chat/completions",
headers={"Authorization": f"Bearer {token}", "Content-Type": "application/json"},
json={"model": "mistralai/mistral-large-3-675b-instruct-2512", "messages": [{"role": "user", "content": prompt}],
"max_tokens": 400, "temperature": 0.1},
timeout=25,
)
content = resp.json().get("choices", [{}])[0].get("message", {}).get("content", "{}")
except Exception:
return []
try:
result = json.loads(content)
except Exception:
# 尝试从内容里提取 JSON
match = re.search(r'\{[^{}]+"has_conflict"[^{}]+\}', content, re.DOTALL)
result = json.loads(match.group()) if match else {"has_conflict": False}
conflicts = []
if result.get("has_conflict"):
conflicts.append({
"id": f"conflict_{dt.now().strftime('%Y%m%d%H%M%S')}",
"entry": result.get("a_entry", ""),
"conflict_with": result.get("b_entry", ""),
"description": result.get("conflict_description", ""),
"detected_at": dt.now().isoformat(),
"status": "pending",
})
# 写入冲突队列
_write_conflicts_to_queue(conflicts)
return conflicts
# ═══════════════════════════════════════════════════════════════════════════
# Phase 1+4: 时间衰减 recall + 遗忘曲线分层
# ═══════════════════════════════════════════════════════════════════════════
import requests as _req
def time_decay_recall(query: str, top_k: int = 5) -> list:
"""带时间衰减的织忆recall综合分=recall×0.6+decay×0.430天内访问过boost×1.15"""
try:
token = os.environ.get("ZHIYI_API_KEY", "zhiyi-dev-key-2026")
r = _req.post("http://127.0.0.1:7821/api/v1/recall",
json={"query": query, "top_k": top_k * 2, "agent_id": "hermes-a06", "use_rerank": True},
headers={"X-API-Key": token}, timeout=8)
if r.status_code != 200: return []
results = r.json().get("results", [])
except: return []
now = datetime.now()
scored = []
for item in results:
try:
ts = datetime.fromisoformat(item.get("timestamp","").split("+")[0] if "+" in item.get("timestamp","") else item.get("timestamp",""))
except: ts = now
days = (now - ts).total_seconds() / 86400
decay = max(0.3, 1.0 - days * 0.015)
recall_s = float(item.get("score", 0.5))
item["decay_score"] = round(decay, 4)
item["days_since_update"] = round(days, 1)
item["tier"] = get_memory_tier(days)
item["final_score"] = round(recall_s * 0.6 + decay * 0.4, 4)
scored.append(item)
# Phase 4: 访问频率 boost
scored = _boost_recalled_memory(scored)
for r in scored:
log_memory_access(r.get("id",""), r.get("decay_score", 1.0))
scored.sort(key=lambda x: x["final_score"], reverse=True)
return scored[:top_k]
# ── Phase 4: 遗忘曲线分层 ─────────────────────────────────────────────────
MEMORY_TIERS = {"hot": 0, "warm": 1, "cold": 2, "archive": 3}
def get_memory_tier(days: float) -> str:
if days <= 7: return "hot"
elif days <= 21: return "warm"
elif days <= 60: return "cold"
else: return "archive"
def log_memory_access(memory_id: str, decay_weight: float):
import json
log_path = D + "/memory-access-log.jsonl"
entry = {"memory_id": memory_id, "accessed_at": datetime.now().isoformat(), "decay_weight": round(decay_weight, 4)}
try:
with open(log_path, "a") as f: f.write(json.dumps(entry, ensure_ascii=False) + "\n")
except: pass
def _boost_recalled_memory(recalled: list) -> list:
import json
log_path = D + "/memory-access-log.jsonl"
recent = {}
try:
cutoff = datetime.now().timestamp() - 30 * 86400
with open(log_path) as f:
for line in f:
try:
e = json.loads(line.strip())
if datetime.fromisoformat(e["accessed_at"]).timestamp() >= cutoff:
mid = e["memory_id"]
recent[mid] = recent.get(mid, 0) + 1
except: pass
except: pass
for r in recalled:
cnt = recent.get(r.get("id",""), 0)
r["access_count_30d"] = cnt
r["boosted"] = False
if cnt >= 1:
r["final_score"] = round(min(1.0, r["final_score"] * 1.15), 4)
r["boosted"] = True
return recalled
# ── Phase 2: 画像LLM合成 ─────────────────────────────────────────────────
def update_profile_from_journal(journal_path: str, profile_path: str, model: str = "mistral-large-3-675b"):
"""deep_tick时用LLM分析journal_entry更新画像仅非空字段"""
import re
try:
with open(journal_path) as f:
lines = f.readlines()
if len(lines) < 3: return
entries = [json.loads(l) for l in lines[-20:] if json.loads(l).get("type") != "startup"][-10:]
except: return
if len(entries) < 3: return
log_lines = "\n".join(f"- {e.get('summary','')}: {e.get('details','(无)')}" for e in entries)
prompt = f"""牧尘是技术用户用小唯AI助手(Hermes Agent)工作。分析最近行为日志,识别:(1)沟通/工作模式 (2)正在进行的项目 (3)新发现的偏好。行为日志:\n{log_lines}\n输出JSON仅含需更新的字段{{"字段名":"新值"}},无需更新则空对象{{}}。只输出JSON。"""
try:
resp = _req.post("http://127.0.0.1:3000/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ.get('NEWAPI_TOKEN','')}", "Content-Type": "application/json"},
json={"model": model, "messages": [{"role": "user", "content": prompt}], "max_tokens": 500, "temperature": 0.3},
timeout=30)
text = resp.json().get("choices", [{}])[0].get("message", {}).get("content", "{}")
except: return
m = re.search(r'\{[^{}]*(?:"[^"]+"\s*:\s*[^{}]+){1,4}\}', text, re.DOTALL)
if not m: return
try: updates = json.loads(m.group())
except: return
if not updates: return
try:
with open(profile_path) as f: profile = json.load(f)
except: return
changed = False
for k, v in updates.items():
if v and str(v).strip() and profile.get(k) != v:
profile[k] = str(v); changed = True
if changed:
profile["updated_at"] = datetime.now().isoformat()
with open(profile_path, "w") as f: json.dump(profile, f, ensure_ascii=False, indent=2)
journal_entry("profile_update", f"LLM更新画像: {list(updates.keys())}")
# ── Phase 3: 冲突检测 ─────────────────────────────────────────────────────
def _write_conflicts_to_queue(conflicts: list):
import json
qpath = HERMES + "/soulful/conflicts-queue.json"
try:
with open(qpath) as f: queue = json.load(f)
except: queue = []
for c in conflicts:
if not any(e.get("id") == c.get("id") for e in queue): queue.append(c)
with open(qpath, "w") as f: json.dump(queue, f, ensure_ascii=False, indent=2)
def detect_memory_conflicts(journal_path: str, zhiyi_token: str) -> list:
"""用LLM检测journal_entry和织忆记忆之间的矛盾返回冲突列表"""
import re
try:
with open(journal_path) as f: lines = f.readlines()
entries = [json.loads(l) for l in lines[-20:] if json.loads(l).get("type") != "startup"][-5:]
except: return []
if not entries: return []
try:
r = _req.post("http://127.0.0.1:7821/api/v1/recall",
json={"query": "牧尘 重要决定 项目 偏好", "top_k": 5, "agent_id": "hermes-a06"},
headers={"X-API-Key": zhiyi_token}, timeout=8)
existing = r.json().get("results", []) if r.status_code == 200 else []
except: existing = []
if not existing: return []
atext = "\n".join(f"- {e.get('summary','')}" for e in entries)
etext = "\n".join(f"- {m.get('content','')[:100]}" for m in existing)
prompt = f"""A组是牧尘最近行为日志B组是织忆长期记忆。判断A和B是否有直接矛盾。\nA组\n{atext}\nB组\n{etext}\n有矛盾输出:{{"has_conflict":true,"conflict_description":"描述","a_entry":"哪条A","b_entry":"哪条B"}}\n无矛盾:{{"has_conflict":false}}\n只输出JSON。"""
try:
resp = _req.post("http://127.0.0.1:3000/v1/chat/completions",
headers={"Authorization": f"Bearer {os.environ.get('NEWAPI_TOKEN','')}", "Content-Type": "application/json"},
json={"model": "mistral-large-3-675b", "messages": [{"role": "user", "content": prompt}],
"max_tokens": 400, "temperature": 0.1}, timeout=25)
text = resp.json().get("choices", [{}])[0].get("message", {}).get("content", "{}")
except: return []
try: result = json.loads(text)
except:
m = re.search(r'\{"has_conflict"[^}]+\}', text, re.DOTALL)
result = json.loads(m.group()) if m else {"has_conflict": False}
conflicts = []
if result.get("has_conflict"):
conflicts.append({"id": f"conflict_{datetime.now().strftime('%Y%m%d%H%M%S')}",
"entry": result.get("a_entry",""), "conflict_with": result.get("b_entry",""),
"description": result.get("conflict_description",""),
"detected_at": datetime.now().isoformat(), "status": "pending"})
_write_conflicts_to_queue(conflicts)
return conflicts
def save_llm_context(ctx, state):
"""每 tick 写 llm_context.json供 Hermes 插件注入"""
soulful_d = HERMES + "/soulful"
@ -952,6 +1317,22 @@ def main_loop():
if reflection_dict:
tddb_capture(reflection_dict, state, ctx)
# 5. Phase 2: 画像LLM合成deep_tick时分析journal行为日志更新画像
# 6. Phase 3: 冲突检测deep_tick时检测journal和织忆记忆之间的矛盾
journal_path = HERMES + "/daemon/journal.jsonl"
profile_path = HERMES + "/soulful/user-profile.json"
try:
update_profile_from_journal(journal_path, profile_path)
except Exception:
pass
try:
zhiyi_token = os.environ.get("ZHIYI_API_KEY", "zhiyi-dev-key-2026")
conflicts = detect_memory_conflicts(journal_path, zhiyi_token)
if conflicts:
log(f" ⚠️ 发现 {len(conflicts)} 个记忆矛盾")
except Exception:
pass
ctx["last_light_tick"] = datetime.now(timezone.utc).isoformat()
ctx["last_state"] = {k: v for k, v in state.items() if k in ("disk_pct", "mem_pct", "processes")}
save_context(ctx)

View File

@ -190,6 +190,86 @@ def upgrade_soulful():
return actions
# ── 3. TencentDB 自我升级 ────────────────────────────────────────────────────
# ── Phase 5: Consolidation 引擎 ─────────────────────────────────────────────
def _consolidate_memories():
"""
YantrikDB-style consolidation
1. 合并相似记忆LLM判断重复>80%
2. 挖掘跨域模式journal里跨category的关联
3. 生成主动触发器写入 triggers.jsonl
"""
import urllib.request as ureq, re, hashlib
triggers = []
log("=== Phase 5: Consolidation ===")
# 1. 检测织忆重复记忆(通过 time_decay_recall 结果)
try:
# 获取织忆 recall 结果,找时间接近+内容相似的
req_body = json.dumps({"query": "牧尘 工作 项目 决策", "top_k": 10,
"agent_id": "hermes-a06", "use_rerank": True}).encode()
req = ureq.Request(f"{ZHIYI_URL}/api/v1/recall", data=req_body,
headers={"X-API-Key": ZHIYI_KEY, "Content-Type": "application/json"},
method="POST")
with ureq.urlopen(req, timeout=8) as resp:
results = json.loads(resp.read().decode()).get("results", [])
# 检查时间接近的记忆对5天内内容相似度阈值通过 LLM 检测)
dupes = 0
for i, r1 in enumerate(results):
for r2 in results[i+1:]:
# 简单内容前50字符相同 → 重复
if r1.get("content","")[:50] == r2.get("content","")[:50]:
dupes += 1
if dupes > 0:
triggers.append({"type": "dedup_trigger", "severity": "info",
"content": f"织忆发现 {dupes} 对重复记忆,建议去重",
"detected_at": datetime.now().isoformat()})
except Exception as e:
log(f" Consolidation step1 failed: {e}")
# 2. 读 journal挖掘高频行为模式
journal_path = f"{HERMES}/daemon/journal.jsonl"
if os.path.exists(journal_path):
try:
lines = open(journal_path).readlines()
entries = [json.loads(l) for l in lines[-50:] if json.loads(l).get("type") != "startup"]
# 统计 type 出现频率
from collections import Counter
type_counts = Counter(e.get("type","") for e in entries)
if type_counts:
top_type, top_count = type_counts.most_common(1)[0]
if top_count >= 5:
triggers.append({"type": "pattern_trigger", "severity": "info",
"content": f"最近50条日志中 '{top_type}' 出现 {top_count} 次,频率较高",
"detected_at": datetime.now().isoformat()})
except Exception as e:
log(f" Consolidation step2 failed: {e}")
# 3. 写触发器到文件
trigger_path = f"{HERMES}/daemon/triggers.jsonl"
existing = []
if os.path.exists(trigger_path):
try:
existing = [json.loads(l) for l in open(trigger_path) if l.strip()]
except: pass
new_triggers = [t for t in triggers
if not any(ex.get("content") == t.get("content") for ex in existing)]
if new_triggers:
existing.extend(new_triggers)
open(trigger_path, "w").writelines(json.dumps(t, ensure_ascii=False) + "\n" for t in existing[-20:])
log(f" 新增 {len(new_triggers)} 个触发器,总 {len(existing)}")
# 飞书通知
try:
lines = [f"• [{t['type']}] {t['content']}" for t in new_triggers[:5]]
if lines:
feishu_alert("🧠 记忆系统触发器提醒",
"\n".join(lines) + f"\n\n(最近 {len(existing)} 个触发器待处理)")
except: pass
else:
log(" 无新触发器")
return triggers
def upgrade_tddb():
log("=== TencentDB 自检 ===")
actions = []
@ -230,6 +310,7 @@ if __name__ == "__main__":
all_actions["织忆"] = upgrade_zhiyi()
all_actions["Soulful"] = upgrade_soulful()
all_actions["TencentDB"] = upgrade_tddb()
all_actions["Consolidation"] = _consolidate_memories()
# 汇总报告
summary = []