xiaoxue_admin
/
memoryweave
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

G7.1: skill persistence - BayesianSkillManager Redis + CRUD API 

- BetaSkill extended: PromptTemplate, LinkedMemoryIDs, LinkedEntities, CreatedAt
- BayesianSkillManager: EnableRedisPersistence(), loadFromRedis(), persistSkill()
- Skill CRUD API: POST/GET/DELETE /api/v1/skills/{name}
- Skill Manager: Register(), Get(), Delete(), Stats()
- redis.go: added HDel method
- triggers.go: removed duplicate Skill/SkillManager (now in skill_bayes.go)
- server.go: EnableRedisPersistence() at startup + new routes
This commit is contained in:
xiaowei 2026-06-01 23:44:36 +08:00
parent 8b01ab2d90
commit 4f2c0fe769
5 changed files with 514 additions and 88 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

154
G7-IMPLEMENTATION.md Normal file
View File

@ -0,0 +1,154 @@
# G7: 遗忘 + 技能系统 — 实施计划
> 核心理念：记忆是原料，Hermes skill 是执行体，Bayesian 是裁判，三者构成自举循环。
## 现状
| 组件 | 状态 | 说明 |
|------|------|------|
| Forgetter | ✅ 完成 | E4.3 刚修好 |
| BayesianSkillManager | ✅ 存在 | Beta-Bernoulli 模型，纯内存 |
| SkillManager | ✅ 存在 | List/Trial API，纯内存 |
| Skill 持久化 | ❌ | 重启丢失 |
| Skill 生成 | ❌ | 无自动从记忆生成机制 |
| Skill 执行 | ❌ | 无 execute API |
## 目标架构
```
[高质量记忆] --distill/高频recall--> [Skill候选]
|
[LLM结晶：生成prompt模板]
|
[Skill执行成功] <--execute-- [Hermes Skill 执行]
| |
| [trial反馈] |
v |
[Bayesian更新ETA] ────────────> [ETA>0.8: active]
|
[active skill → 关联记忆 degree+5]
|
[ETA<0.5: retired → decay_rate×1.2]
```
## 阶段划分
### G7.1 — 技能持久化（基础）
**扩展 BetaSkill**（`skill_bayes.go`）：
- 新增字段：`PromptTemplate string`、`LinkedMemoryIDs []string`、`LinkedEntities []string`、`CreatedAt time.Time`
- 新增 Redis HSET：`zhiyi:skills` → `{name: json(BetaSkill)}`
**扩展 BayesianSkillManager**（`skill_bayes.go`）：
- `EnableRedisPersistence()` — 启动时从 Redis 加载，重启不丢
- `persistSkill(skill)` — 每次 RecordTrial 后同步写 Redis
- `LoadFromRedis()` — 启动时加载所有 skill
**新增 API**（`skill_crud.go`）：
- `POST /api/v1/skills` — 手动注册新 skill（含 prompt_template）
- `GET /api/v1/skills/{name}` — 获取单个 skill 详情
- `DELETE /api/v1/skills/{name}` — 删除 skill
**文件变更**：
- `go/internal/api/routes/skill_bayes.go` — 扩展结构体 + 持久化
- `go/internal/api/routes/skill_crud.go` — 新建，CRUD API
### G7.2 — 技能生成（LLM 结晶）
**新增 `skill_crystallize.go`**：
- `CrystallizeFromMemory(mem *MemoryRecord) (*BetaSkill, error)` — 将高质量记忆转为 skill
- 调用 LLM 生成 prompt 模板（从 content 提取参数占位符）
- 从 content 提取关键实体作为 LinkedEntities
- `SuggestSkillCandidates(limit int) []*MemoryRecord` — 找出适合结晶的记忆
- 条件：`recall_count >= 5` AND `quality_score >= 0.7` AND `tier != "core"` AND 未关联任何 skill
**Consolidation 集成**（`consolidation_pipe.go`）：
- 每次 consolidation 后，对候选记忆调用 `CrystallizeFromMemory`
**新增 API**：
- `POST /api/v1/skills/crystallize` — 手动触发结晶
- `GET /api/v1/skills/candidates` — 查看当前候选列表
**LLM Prompt（生成 prompt 模板）**：
```
给定记忆内容，生成一个可执行的 prompt 模板：
1. 识别记忆中的可变参数，用 {param} 格式标注
2. 生成一段可直接执行的指令文本
3. 提取 3-5 个关键实体作为关联实体
记忆内容：{content}
```
### G7.3 — 技能执行 + 反馈闭环
**执行 API**（`skill_execute.go`）：
- `POST /api/v1/skills/{name}/execute`
- Body: `{"params": {"key": "value"}, "context": "optional context override"}`
- 加载 linked memories 作为 context
- 调用 Hermes agent（`delegate_task`）执行 prompt
- 返回执行结果
**与 BayesianSkills 联动**：
- 执行成功 → `BayesianSkills.RecordTrial(name, true)`
- 执行失败 → `BayesianSkills.RecordTrial(name, false)`
- 若 `ETA < 0.5`：对 linked memories 的 decay_rate ×1.2（加速遗忘）
**与遗忘联动**：
- `admin.go` Forget 函数扩展：
- 扫描记忆时，检查是否有 active skill 关联
- 若有关联且 skill.ETA > 0.8：degree +5
- 若有关联且 skill.ETA < 0.5：decay_rate ×1.2
**新增 API**：
- `POST /api/v1/skills/{name}/execute` — 执行 skill
## 技术细节
### Redis Schema
```
zhiyi:skills → HASH {skill_name: JSON(BetaSkill)}
zhiyi:skill_meta → HASH {skill_name: JSON(SkillMeta)} # LinkedMemoryIDs, LinkedEntities
```
### BetaSkill 扩展结构
```go
type BetaSkill struct {
Name string `json:"name"`
Alpha float64 `json:"alpha"`
Beta float64 `json:"beta"`
Trials int `json:"trials"`
Successes int `json:"successes"`
ETA float64 `json:"eta"`
Status string `json:"status"` // active / probation / retired
LastUpdated time.Time `json:"last_updated"`
// G7 新增
PromptTemplate string `json:"prompt_template,omitempty"`
LinkedMemoryIDs []string `json:"linked_memory_ids,omitempty"`
LinkedEntities []string `json:"linked_entities,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
```
## 验证计划
1. **G7.1 验证**：注册 skill → 重启 Go API → skill 仍在 Redis
2. **G7.2 验证**：POST `/api/v1/skills/crystallize` → skill 生成，prompt_template 非空
3. **G7.3 验证**：
- `POST /api/v1/skills/{name}/execute` → 返回执行结果
- trial 反馈 → Bayesian ETA 更新
- ETA > 0.8 → linked memory 保护性增强
## 文件清单
| 文件 | 操作 | 说明 |
|------|------|------|
| `go/internal/api/routes/skill_bayes.go` | 修改 | 扩展结构体 + Redis 持久化 |
| `go/internal/api/routes/skill_crud.go` | 新建 | CRUD API |
| `go/internal/api/routes/skill_execute.go` | 新建 | 执行 API + 反馈闭环 |
| `go/internal/api/routes/skill_crystallize.go` | 新建 | LLM 结晶逻辑 |
| `go/internal/api/server.go` | 修改 | 注册新路由 |
## 实施顺序
1. G7.1（持久化）→ 2. G7.2（生成）→ 3. G7.3（执行+联动）

358
go/internal/api/routes/skill_bayes.go
View File

@ -1,33 +1,99 @@
// 织忆 MemoryWeave — Skill 贝叶斯后验更新（Beta-Bernoulli）
// 织忆 MemoryWeave — Skill 贝叶斯后验更新（Beta-Bernoulli）+ Redis 持久化
package routes
import (
"encoding/json"
"fmt"
"math"
"net/http"
"sync"
"time"
"github.com/xiaoxue/memoryweave/internal/storage"
)
// BetaSkill 贝叶斯 Skill 评分
// skillRedisKey Redis 持久化 key
const skillRedisKey = "zhiyi:skills"
// BetaSkill 贝叶斯 Skill 评分（G7 扩展版）
type BetaSkill struct {
Name string `json:"name"`
Alpha float64 `json:"alpha"` // α = successes + 1
Beta float64 `json:"beta"` // β = failures + 1
Trials int `json:"trials"`
Successes int `json:"successes"`
ETA float64 `json:"eta"` // α/(α+β) 贝叶斯均值
Status string `json:"status"` // active / probation / retired
LastUpdated time.Time `json:"last_updated"`
Name string `json:"name"`
Alpha float64 `json:"alpha"` // α = successes + 1
Beta float64 `json:"beta"` // β = failures + 1
Trials int `json:"trials"`
Successes int `json:"successes"`
ETA float64 `json:"eta"` // α/(α+β) 贝叶斯均值
Status string `json:"status"` // active / probation / retired
LastUpdated time.Time `json:"last_updated"`
// G7 扩展字段
PromptTemplate string `json:"prompt_template,omitempty"` // 可执行 prompt（含 {param} 占位符）
LinkedMemoryIDs []string `json:"linked_memory_ids,omitempty"` // 关联的记忆 ID
LinkedEntities []string `json:"linked_entities,omitempty"` // 关联的图谱实体
CreatedAt time.Time `json:"created_at"`
}
type BayesianSkillManager struct {
mu sync.RWMutex
skills map[string]*BetaSkill
mu sync.RWMutex
skills map[string]*BetaSkill
// Redis 持久化
redisClient *storage.RedisClient
persisted bool // 是否已从 Redis 加载
}
var BayesianSkills = &BayesianSkillManager{
skills: make(map[string]*BetaSkill),
}
// EnableRedisPersistence 启动 Skill 持久化（从 Redis 加载 + 每次变更同步）
func (bsm *BayesianSkillManager) EnableRedisPersistence() {
rc := storage.GetRedisClient()
if rc == nil {
fmt.Println("[skill] Redis not available, skills in-memory only")
return
}
bsm.redisClient = rc
if err := bsm.loadFromRedis(); err != nil {
fmt.Printf("[skill] Redis load failed: %v\n", err)
return
}
bsm.persisted = true
fmt.Printf("[skill] Redis persistence enabled, loaded %d skills\n", len(bsm.skills))
}
// loadFromRedis 从 Redis 加载所有 skill
func (bsm *BayesianSkillManager) loadFromRedis() error {
if bsm.redisClient == nil {
return nil
}
data, err := bsm.redisClient.HGetAll(skillRedisKey)
if err != nil || len(data) == 0 {
return err
}
bsm.mu.Lock()
defer bsm.mu.Unlock()
for name, jsonStr := range data {
var skill BetaSkill
if err := json.Unmarshal([]byte(jsonStr), &skill); err == nil {
bsm.skills[name] = &skill
}
}
return nil
}
// persistSkill 持久化单个 skill 到 Redis
func (bsm *BayesianSkillManager) persistSkill(skill *BetaSkill) {
if bsm.redisClient == nil {
return
}
data, err := json.Marshal(skill)
if err != nil {
return
}
_ = bsm.redisClient.HSet(skillRedisKey, skill.Name, string(data))
_ = bsm.redisClient.Expire(skillRedisKey, 90*24*time.Hour)
}
// RecordTrial 记录一次 trial 结果，更新 α/β 后验
func (bsm *BayesianSkillManager) RecordTrial(name string, success bool) *BetaSkill {
bsm.mu.Lock()
@ -36,9 +102,10 @@ func (bsm *BayesianSkillManager) RecordTrial(name string, success bool) *BetaSki
skill, exists := bsm.skills[name]
if !exists {
skill = &BetaSkill{
Name: name,
Alpha: 1.0, // prior: Beta(1,1) = uniform
Beta: 1.0,
Name: name,
Alpha: 1.0, // prior: Beta(1,1) = uniform
Beta: 1.0,
CreatedAt: time.Now(),
}
bsm.skills[name] = skill
}
@ -65,6 +132,46 @@ func (bsm *BayesianSkillManager) RecordTrial(name string, success bool) *BetaSki
skill.Status = "retired"
}
// 持久化
bsm.persistSkill(skill)
return skill
}
// Register 注册一个新 skill（含 prompt template）
func (bsm *BayesianSkillManager) Register(name, promptTemplate string, linkedEntities []string) *BetaSkill {
bsm.mu.Lock()
defer bsm.mu.Unlock()
skill, exists := bsm.skills[name]
if !exists {
skill = &BetaSkill{
Name: name,
Alpha: 1.0,
Beta: 1.0,
CreatedAt: time.Now(),
PromptTemplate: promptTemplate,
LinkedEntities: linkedEntities,
}
bsm.skills[name] = skill
} else {
skill.PromptTemplate = promptTemplate
if len(linkedEntities) > 0 {
skill.LinkedEntities = linkedEntities
}
skill.LastUpdated = time.Now()
}
skill.ETA = math.Round(skill.Alpha/(skill.Alpha+skill.Beta)*100) / 100
if skill.ETA == 0 {
skill.ETA = 0.5
}
// 注册即 probation 状态（需要 trial 来验证）
if skill.Status == "" {
skill.Status = "probation"
}
bsm.persistSkill(skill)
return skill
}
@ -88,6 +195,13 @@ func (bsm *BayesianSkillManager) List() []*BetaSkill {
return list
}
// Get 获取单个 skill
func (bsm *BayesianSkillManager) Get(name string) *BetaSkill {
bsm.mu.RLock()
defer bsm.mu.RUnlock()
return bsm.skills[name]
}
// GetActive 获取所有 active skill
func (bsm *BayesianSkillManager) GetActive() []*BetaSkill {
bsm.mu.RLock()
@ -100,3 +214,217 @@ func (bsm *BayesianSkillManager) GetActive() []*BetaSkill {
}
return active
}
// Delete 删除 skill
func (bsm *BayesianSkillManager) Delete(name string) error {
bsm.mu.Lock()
defer bsm.mu.Unlock()
if _, exists := bsm.skills[name]; !exists {
return fmt.Errorf("skill not found: %s", name)
}
delete(bsm.skills, name)
if bsm.redisClient != nil {
_ = bsm.redisClient.HDel(skillRedisKey, name)
}
return nil
}
// SetLinkedMemory 设置 skill 关联的记忆 ID
func (bsm *BayesianSkillManager) SetLinkedMemory(name string, memIDs []string) error {
bsm.mu.Lock()
defer bsm.mu.Unlock()
skill, exists := bsm.skills[name]
if !exists {
return fmt.Errorf("skill not found: %s", name)
}
skill.LinkedMemoryIDs = memIDs
skill.LastUpdated = time.Now()
bsm.persistSkill(skill)
return nil
}
// GetLinkedMemoryDegree 获取记忆关联的所有 skill 中最高 ETA 度
// 用于 admin.go 的遗忘决策：关联 skill 的 degree 保护
func (bsm *BayesianSkillManager) GetLinkedMemoryDegree(memID string) int {
bsm.mu.RLock()
defer bsm.mu.RUnlock()
for _, skill := range bsm.skills {
for _, id := range skill.LinkedMemoryIDs {
if id == memID {
// active skill 提供 +5 degree 保护
if skill.Status == "active" {
return 5
}
}
}
}
return 0
}
// GetSkillForMemory 获取记忆关联的 skill（用于 decay 加速）
func (bsm *BayesianSkillManager) GetSkillForMemory(memID string) *BetaSkill {
bsm.mu.RLock()
defer bsm.mu.RUnlock()
for _, skill := range bsm.skills {
for _, id := range skill.LinkedMemoryIDs {
if id == memID {
return skill
}
}
}
return nil
}
// UpdateLinkedEntities 更新 skill 的关联实体
func (bsm *BayesianSkillManager) UpdateLinkedEntities(name string, entities []string) error {
bsm.mu.Lock()
defer bsm.mu.Unlock()
skill, exists := bsm.skills[name]
if !exists {
return fmt.Errorf("skill not found: %s", name)
}
skill.LinkedEntities = entities
skill.LastUpdated = time.Now()
bsm.persistSkill(skill)
return nil
}
// Stats 返回 skill 统计（供 dashboard 使用）
func (bsm *BayesianSkillManager) Stats() map[string]interface{} {
bsm.mu.RLock()
defer bsm.mu.RUnlock()
active, probation, retired := 0, 0, 0
for _, s := range bsm.skills {
switch s.Status {
case "active":
active++
case "probation":
probation++
case "retired":
retired++
}
}
return map[string]interface{}{
"total": len(bsm.skills),
"active": active,
"probation": probation,
"retired": retired,
"persisted": bsm.persisted,
"redis_connected": bsm.redisClient != nil,
}
}
// Exists 检查 skill 是否存在
func (bsm *BayesianSkillManager) Exists(name string) bool {
bsm.mu.RLock()
defer bsm.mu.RUnlock()
_, exists := bsm.skills[name]
return exists
}
// ─── SkillManager（HTTP 路由层，与 BayesianSkills 双写）─────────────────────────
type SkillManager struct {
mu sync.RWMutex
skills map[string]*Skill
}
type Skill struct {
Name string `json:"name"`
Description string `json:"description"`
Trials int `json:"trials"`
ETA float64 `json:"eta"`
CreatedAt time.Time `json:"created_at"`
}
var Skills = &SkillManager{
skills: make(map[string]*Skill),
}
// GET /api/v1/skills
func (sm *SkillManager) List(w http.ResponseWriter, r *http.Request) {
list := BayesianSkills.List()
respond(w, 200, map[string]interface{}{"skills": list, "count": len(list)})
}
// POST /api/v1/skills/{name}/trial
func (sm *SkillManager) Trial(w http.ResponseWriter, r *http.Request) {
name := r.PathValue("name")
if name == "" {
respondError(w, 400, "name required")
return
}
var req struct {
Success bool `json:"success"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
respondError(w, 400, "invalid body")
return
}
skill := BayesianSkills.RecordTrial(name, req.Success)
respond(w, 200, skill)
}
// POST /api/v1/skills — 注册新 skill（G7 新增）
func (sm *SkillManager) Register(w http.ResponseWriter, r *http.Request) {
var req struct {
Name string `json:"name"`
PromptTemplate string `json:"prompt_template"`
LinkedEntities []string `json:"linked_entities"`
Description string `json:"description"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
respondError(w, 400, "invalid body: "+err.Error())
return
}
if req.Name == "" {
respondError(w, 400, "name required")
return
}
skill := BayesianSkills.Register(req.Name, req.PromptTemplate, req.LinkedEntities)
respond(w, 200, skill)
}
// GET /api/v1/skills/{name}
func (sm *SkillManager) Get(w http.ResponseWriter, r *http.Request) {
name := r.PathValue("name")
skill := BayesianSkills.Get(name)
if skill == nil {
respondError(w, 404, "skill not found: "+name)
return
}
respond(w, 200, skill)
}
// DELETE /api/v1/skills/{name}
func (sm *SkillManager) Delete(w http.ResponseWriter, r *http.Request) {
name := r.PathValue("name")
if err := BayesianSkills.Delete(name); err != nil {
respondError(w, 404, err.Error())
return
}
respond(w, 200, map[string]string{"status": "deleted", "name": name})
}
// GET /api/v1/skills/stats
func (sm *SkillManager) Stats(w http.ResponseWriter, r *http.Request) {
respond(w, 200, BayesianSkills.Stats())
}
// RecordTrial 程序化记录一次技能试验（无需 HTTP）
func (sm *SkillManager) RecordTrial(name string, success bool) {
BayesianSkills.RecordTrial(name, success)
}
// GetSkillETA 获取 skill 的 ETA（供遗忘决策使用）
func (sm *SkillManager) GetSkillETA(name string) float64 {
skill := BayesianSkills.Get(name)
if skill == nil {
return 0.5
}
return skill.ETA
}

72
go/internal/api/routes/triggers.go
View File

@ -190,76 +190,4 @@ func (tm *TriggerManager) RecordFail(id string) {
}
}
// ─── Skill 结晶 ──────────────────────────────────────────
type Skill struct {
Name string `json:"name"`
Description string `json:"description"`
Trials int `json:"trials"`
ETA float64 `json:"eta"` // 有效性 η = successes / trials
CreatedAt time.Time `json:"created_at"`
}
type SkillManager struct {
mu sync.RWMutex
skills map[string]*Skill
}
var Skills = &SkillManager{
skills: make(map[string]*Skill),
}
// GET /api/v1/skills
func (sm *SkillManager) List(w http.ResponseWriter, r *http.Request) {
list := BayesianSkills.List()
respond(w, 200, map[string]interface{}{"skills": list, "count": len(list)})
}
// POST /api/v1/skills/{name}/trial
func (sm *SkillManager) Trial(w http.ResponseWriter, r *http.Request) {
name := r.PathValue("name")
if name == "" {
respondError(w, 400, "name required")
return
}
var req struct {
Success bool `json:"success"`
}
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
respondError(w, 400, "invalid body")
return
}
sm.mu.Lock()
defer sm.mu.Unlock()
skill := BayesianSkills.RecordTrial(name, req.Success)
respond(w, 200, skill)
}
// RecordTrial 程序化记录一次技能试验（无需 HTTP）
func (sm *SkillManager) RecordTrial(name string, success bool) {
sm.mu.Lock()
defer sm.mu.Unlock()
sm.recordTrialLocked(name, success)
}
func (sm *SkillManager) recordTrialLocked(name string, success bool) *Skill {
skill, exists := sm.skills[name]
if !exists {
skill = &Skill{
Name: name,
Description: "自动发现的工作模式",
CreatedAt: time.Now(),
}
sm.skills[name] = skill
}
skill.Trials++
if success {
skill.ETA = float64(skill.Trials-1) / float64(skill.Trials)
} else {
skill.ETA = float64(skill.Trials-1) / float64(skill.Trials)
}
return skill
}

9
go/internal/api/server.go
View File

@ -224,6 +224,9 @@ func NewServer() http.Handler {
// ─── VProp 持久化到 Redis（restart 不丢）───
selfoptimize.VProp.EnableVPropRedisPersistence()
// ─── Skill 持久化到 Redis（G7: restart 不丢 skills）───
routes.BayesianSkills.EnableRedisPersistence()
// ─── V 值传播器（竞争性架构核心）─────────────
vPropagator := selfoptimize.VProp
@ -662,8 +665,12 @@ func NewServer() http.Handler {
}
})
// Skills
// Skills（G7）
mux.HandleFunc("/api/v1/skills", routes.Skills.List)
mux.HandleFunc("POST /api/v1/skills", routes.Skills.Register) // 注册新 skill
mux.HandleFunc("/api/v1/skills/stats", routes.Skills.Stats) // skill 统计
mux.HandleFunc("/api/v1/skills/{name}", routes.Skills.Get) // 获取单个
mux.HandleFunc("DELETE /api/v1/skills/{name}", routes.Skills.Delete) // 删除
mux.HandleFunc("/api/v1/skills/bayes", func(w http.ResponseWriter, r *http.Request) {
list := routes.BayesianSkills.List()
respondJSON(w, 200, list)

9
go/internal/storage/redis.go
View File

@ -189,6 +189,15 @@ func (rc *RedisClient) HSet(key, field, value string) error {
return err
}
// HDel 从 Hash 中删除一个或多个 field
func (rc *RedisClient) HDel(key, field string) error {
if rc.conn == nil {
return errors.New("redis: not connected")
}
_, err := rc.conn.Do("HDEL", key, field)
return err
}
func (rc *RedisClient) HGetAll(key string) (map[string]string, error) {
if rc.conn == nil {
return nil, errors.New("redis: not connected")