28 KiB
Phase 3-4 Integration Guide
Context Engineering System Integration
Date: 2025-12-04 Status: ✅ Phase 3 Complete | Phase 4 Integration Guide
Overview
This guide demonstrates how to integrate the Phase 3-4 Context Engineering System with agentUniverse agents. The system provides:
- Knowledge-Context Synchronization (Phase 3)
- Benchmarking Framework (Phase 3)
- Agent Integration Patterns (Phase 4)
- Cross-Agent Context Sharing (Phase 4)
- Production Monitoring (Phase 4)
Phase 3: New Components
1. Knowledge-Context Synchronization
Location: agentuniverse/agent/context/sync/knowledge_context_synchronizer.py
Purpose: Bidirectional synchronization between Knowledge and Context systems with conflict resolution.
Key Features
-
4 Conflict Resolution Strategies:
NEWEST_WINS: Most recent knowledge takes precedenceCRITICAL_PRESERVED: CRITICAL priority segments always keptMERGE: Combine both versionsVERSION_BOTH: Keep both as versioned segments
-
Version Tracking: SHA256 hash-based change detection
-
Selective Invalidation: Mark outdated segments without deletion
-
Efficient Updates: Only sync when content changes
Basic Usage
from agentuniverse.agent.context.sync.knowledge_context_synchronizer import (
KnowledgeContextSynchronizer,
ConflictResolutionStrategy
)
from agentuniverse.agent.context.context_manager_manager import ContextManagerManager
# Get context manager instance
context_manager = ContextManagerManager().get_instance_obj("default_context_manager")
# Create synchronizer
synchronizer = KnowledgeContextSynchronizer(
context_manager=context_manager,
conflict_strategy=ConflictResolutionStrategy.NEWEST_WINS,
enable_versioning=True
)
# Sync knowledge to context
result = synchronizer.sync_knowledge_to_context(
knowledge_id="user_manual_v2",
documents=["Updated user manual content...", "New API documentation..."],
session_id="session_123",
source="knowledge_base",
priority=ContextPriority.HIGH
)
print(f"Added {result.segments_added} segments")
print(f"Invalidated {result.segments_invalidated} old segments")
Update Existing Knowledge
# Update when knowledge changes
result = synchronizer.update_knowledge_context(
knowledge_id="user_manual_v2",
session_id="session_123",
new_documents=["Updated content with corrections..."],
conflict_strategy=ConflictResolutionStrategy.CRITICAL_PRESERVED
)
print(f"Resolved {result.conflicts_resolved} conflicts")
print(f"Updated {result.segments_updated} segments")
Version Tracking
# Get version information
version = synchronizer.get_knowledge_version("user_manual_v2")
if version:
print(f"Version ID: {version.version_id}")
print(f"Source: {version.source}")
print(f"Hash: {version.hash}")
print(f"Timestamp: {version.timestamp}")
# List all versions
all_versions = synchronizer.list_knowledge_versions()
for knowledge_id, version in all_versions.items():
print(f"{knowledge_id}: {version.version_id}")
2. Benchmarking Framework
Location: agentuniverse/agent/context/benchmark/benchmark_suite.py
Purpose: Comprehensive evaluation against industry standards (Cursor, Claude).
Benchmark Targets
| Metric | Target | Description |
|---|---|---|
| Multi-turn coherence | >0.85 | Context preservation across turns |
| Compression ratio | 60-80% | Token reduction efficiency |
| Information loss | <10% | Semantic preservation |
| Retrieval precision | >0.90 | Search accuracy |
| Retrieval latency | <100ms | Search speed |
| Memory usage | <500MB | For 10K turns |
Basic Usage
from agentuniverse.agent.context.benchmark.benchmark_suite import (
ContextBenchmarkSuite
)
from agentuniverse.agent.context.context_manager_manager import ContextManagerManager
# Get context manager
context_manager = ContextManagerManager().get_instance_obj("default_context_manager")
# Create benchmark suite
suite = ContextBenchmarkSuite(context_manager)
# Run full benchmark (100 turns by default)
result = suite.run_full_suite(num_turns=100)
# Check results
print(f"Overall Score: {result.metrics.get_score():.1f}/100")
print(f"Passes Targets: {result.metrics.passes_targets()}")
print(f"Multi-turn Coherence: {result.metrics.multi_turn_coherence:.3f}")
print(f"Compression Ratio: {result.metrics.compression_ratio:.1%}")
print(f"Information Loss: {result.metrics.information_loss:.1%}")
print(f"Retrieval Precision: {result.metrics.retrieval_precision:.3f}")
print(f"Average Latency: {result.metrics.average_latency_ms:.1f}ms")
print(f"Memory Usage: {result.metrics.memory_usage_mb:.1f}MB")
Detailed Metrics
# Access detailed metrics
metrics = result.metrics
# Coherence metrics
print(f"Multi-turn coherence: {metrics.multi_turn_coherence:.3f}")
print(f"Context consistency: {metrics.context_consistency:.3f}")
# Compression metrics
print(f"Compression ratio: {metrics.compression_ratio:.1%}")
print(f"Information loss: {metrics.information_loss:.1%}")
print(f"Context preservation: {metrics.context_preservation:.1%}")
# Retrieval metrics
print(f"Precision: {metrics.retrieval_precision:.3f}")
print(f"Recall: {metrics.retrieval_recall:.3f}")
print(f"F1 Score: {metrics.retrieval_f1:.3f}")
# Performance metrics
print(f"Average latency: {metrics.average_latency_ms:.1f}ms")
print(f"P95 latency: {metrics.p95_latency_ms:.1f}ms")
print(f"P99 latency: {metrics.p99_latency_ms:.1f}ms")
print(f"Throughput: {metrics.throughput_ops_per_sec:.1f} ops/sec")
# Resource metrics
print(f"Memory usage: {metrics.memory_usage_mb:.1f}MB")
print(f"Tokens per turn (avg): {metrics.tokens_per_turn_avg:.1f}")
Phase 4: Agent Integration Patterns
Pattern 1: Context-Aware Agent (Basic)
Use Case: Single agent with context management
Configuration (agent_context_aware.yaml):
name: 'context_aware_agent'
description: 'Agent with advanced context management'
profile:
llm_model:
name: 'gpt-4'
max_tokens: 2000
# Enable context manager
context_manager_name: 'default_context_manager'
task_type: 'dialogue' # code_generation, data_analysis, dialogue
system_prompt: |
You are an intelligent assistant with advanced context awareness.
You can maintain coherent conversations across multiple turns.
memory:
name: 'demo_memory'
action:
knowledge:
- 'demo_knowledge'
plan:
planner:
name: 'react_planner'
Implementation:
from agentuniverse.agent.agent import Agent
from agentuniverse.agent.agent_manager import AgentManager
from agentuniverse.agent.context.context_manager_manager import ContextManagerManager
class ContextAwareAgent(Agent):
"""Agent with integrated context management."""
def __init__(self):
super().__init__()
self._context_manager = None
def initialize_by_component_configer(self, component_configer):
"""Initialize with context manager."""
super().initialize_by_component_configer(component_configer)
# Get context manager name from profile
context_manager_name = self.agent_model.profile.get('context_manager_name')
if context_manager_name:
self._context_manager = ContextManagerManager().get_instance_obj(
context_manager_name
)
return self
def pre_parse_input(self, input_object):
"""Enhanced input parsing with context extraction."""
agent_input = super().pre_parse_input(input_object)
# If context manager is enabled
if self._context_manager:
session_id = input_object.get_data('session_id')
if session_id:
# Create or get context window
task_type = self.agent_model.profile.get('task_type', 'dialogue')
window = self._context_manager.create_context_window(
session_id=session_id,
agent_id=self.agent_model.info.get('name'),
task_type=task_type
)
# Add user input to context
user_input = input_object.get_data('input')
if user_input:
self._context_manager.add_context(
session_id,
user_input,
ContextType.CONVERSATION,
ContextPriority.HIGH,
metadata={'role': 'user'}
)
# Add system prompt to context (CRITICAL priority)
system_prompt = self.agent_model.profile.get('system_prompt')
if system_prompt:
self._context_manager.add_context(
session_id,
system_prompt,
ContextType.SYSTEM,
ContextPriority.CRITICAL
)
# Get context for building agent input
agent_input['context_window'] = window
return agent_input
def parse_result(self, agent_result):
"""Enhanced result parsing with context storage."""
result = super().parse_result(agent_result)
# Store agent output in context
if self._context_manager:
session_id = agent_result.get('session_id')
output = result.get('output')
if session_id and output:
self._context_manager.add_context(
session_id,
output,
ContextType.CONVERSATION,
ContextPriority.HIGH,
metadata={'role': 'assistant'}
)
return result
Pattern 2: Knowledge-Integrated Agent
Use Case: Agent with automatic knowledge-context synchronization
Implementation:
from agentuniverse.agent.context.sync.knowledge_context_synchronizer import (
KnowledgeContextSynchronizer
)
class KnowledgeIntegratedAgent(ContextAwareAgent):
"""Agent with knowledge-context synchronization."""
def __init__(self):
super().__init__()
self._synchronizer = None
def initialize_by_component_configer(self, component_configer):
"""Initialize with synchronizer."""
super().initialize_by_component_configer(component_configer)
# Create synchronizer if context manager exists
if self._context_manager:
self._synchronizer = KnowledgeContextSynchronizer(
context_manager=self._context_manager,
enable_versioning=True
)
return self
def execute(self, input_object, agent_input):
"""Execute with knowledge synchronization."""
session_id = input_object.get_data('session_id')
# Sync knowledge before execution
if self._synchronizer and session_id:
knowledge_configs = self.agent_model.action.get('knowledge', [])
for knowledge_config in knowledge_configs:
knowledge_name = knowledge_config.get('name') if isinstance(knowledge_config, dict) else knowledge_config
# Get knowledge documents (simplified)
from agentuniverse.agent.action.knowledge.knowledge_manager import KnowledgeManager
knowledge = KnowledgeManager().get_instance_obj(knowledge_name)
if knowledge:
# Query relevant documents
query = input_object.get_data('input', '')
documents = knowledge.query_knowledge(query, top_k=5)
# Sync to context
if documents:
self._synchronizer.sync_knowledge_to_context(
knowledge_id=knowledge_name,
documents=[doc.text for doc in documents],
session_id=session_id,
priority=ContextPriority.HIGH
)
# Execute normally
return super().execute(input_object, agent_input)
Pattern 3: Cross-Agent Context Sharing
Use Case: Multiple agents sharing context in a multi-agent system
Implementation:
class AgentContextCoordinator:
"""Coordinates context sharing across multiple agents."""
def __init__(self, context_manager_name: str = "default_context_manager"):
self.context_manager = ContextManagerManager().get_instance_obj(
context_manager_name
)
self._shared_sessions = {} # agent_group_id -> session_id
def create_shared_session(self, agent_group_id: str) -> str:
"""Create a shared context session for a group of agents."""
session_id = f"shared_{agent_group_id}_{datetime.now().timestamp()}"
window = self.context_manager.create_context_window(
session_id=session_id,
task_type="dialogue"
)
self._shared_sessions[agent_group_id] = session_id
return session_id
def share_context(
self,
agent_group_id: str,
content: str,
source_agent: str,
context_type: ContextType = ContextType.CONVERSATION,
priority: ContextPriority = ContextPriority.MEDIUM
):
"""Share context from one agent to the group."""
session_id = self._shared_sessions.get(agent_group_id)
if not session_id:
session_id = self.create_shared_session(agent_group_id)
self.context_manager.add_context(
session_id,
content,
context_type,
priority,
metadata={
'source_agent': source_agent,
'shared': True,
'timestamp': datetime.now().isoformat()
}
)
def get_shared_context(
self,
agent_group_id: str,
context_type: Optional[ContextType] = None
) -> List[ContextSegment]:
"""Get shared context for the agent group."""
session_id = self._shared_sessions.get(agent_group_id)
if not session_id:
return []
return self.context_manager.get_context(
session_id,
context_type=context_type
)
# Usage
coordinator = AgentContextCoordinator()
# Create shared session for multiple agents
session_id = coordinator.create_shared_session("research_team")
# Agent 1 shares findings
coordinator.share_context(
agent_group_id="research_team",
content="Found relevant research paper on topic X",
source_agent="research_agent",
priority=ContextPriority.HIGH
)
# Agent 2 shares analysis
coordinator.share_context(
agent_group_id="research_team",
content="Analysis shows correlation with previous findings",
source_agent="analysis_agent",
priority=ContextPriority.HIGH
)
# Agent 3 retrieves all shared context
shared_context = coordinator.get_shared_context("research_team")
for segment in shared_context:
print(f"From {segment.metadata.custom['source_agent']}: {segment.content}")
Pattern 4: Production Monitoring
Implementation:
from dataclasses import dataclass
from typing import Dict, List
import time
@dataclass
class ContextMetrics:
"""Production metrics for context system."""
# Usage metrics
total_sessions: int = 0
active_sessions: int = 0
total_segments: int = 0
average_segments_per_session: float = 0.0
# Performance metrics
average_add_latency_ms: float = 0.0
average_search_latency_ms: float = 0.0
average_compression_latency_ms: float = 0.0
# Quality metrics
average_compression_ratio: float = 0.0
average_retrieval_precision: float = 0.0
# Resource metrics
total_memory_mb: float = 0.0
average_tokens_per_session: float = 0.0
class ContextMonitor:
"""Monitor context system performance in production."""
def __init__(self, context_manager_name: str = "default_context_manager"):
self.context_manager = ContextManagerManager().get_instance_obj(
context_manager_name
)
self._metrics_history: List[ContextMetrics] = []
self._operation_latencies: Dict[str, List[float]] = {
'add': [],
'search': [],
'compression': []
}
def track_operation(self, operation_type: str, latency_ms: float):
"""Track operation latency."""
if operation_type in self._operation_latencies:
self._operation_latencies[operation_type].append(latency_ms)
def collect_metrics(self) -> ContextMetrics:
"""Collect current metrics."""
import sys
metrics = ContextMetrics()
# Count active windows
if hasattr(self.context_manager, '_windows'):
metrics.total_sessions = len(self.context_manager._windows)
metrics.active_sessions = len([
w for w in self.context_manager._windows.values()
if w.total_tokens > 0
])
# Calculate average latencies
if self._operation_latencies['add']:
metrics.average_add_latency_ms = sum(self._operation_latencies['add']) / len(self._operation_latencies['add'])
if self._operation_latencies['search']:
metrics.average_search_latency_ms = sum(self._operation_latencies['search']) / len(self._operation_latencies['search'])
if self._operation_latencies['compression']:
metrics.average_compression_latency_ms = sum(self._operation_latencies['compression']) / len(self._operation_latencies['compression'])
# Memory usage
metrics.total_memory_mb = sys.getsizeof(self.context_manager) / (1024 * 1024)
self._metrics_history.append(metrics)
return metrics
def get_health_status(self) -> str:
"""Get system health status."""
metrics = self.collect_metrics()
# Check against targets
issues = []
if metrics.average_add_latency_ms > 100:
issues.append("High add latency")
if metrics.average_search_latency_ms > 100:
issues.append("High search latency")
if metrics.total_memory_mb > 500:
issues.append("High memory usage")
if not issues:
return "HEALTHY"
elif len(issues) <= 1:
return f"WARNING: {issues[0]}"
else:
return f"CRITICAL: {', '.join(issues)}"
def export_metrics(self) -> Dict[str, Any]:
"""Export metrics for monitoring system."""
metrics = self.collect_metrics()
return {
'timestamp': datetime.now().isoformat(),
'health_status': self.get_health_status(),
'metrics': {
'sessions': {
'total': metrics.total_sessions,
'active': metrics.active_sessions,
},
'performance': {
'add_latency_ms': metrics.average_add_latency_ms,
'search_latency_ms': metrics.average_search_latency_ms,
'compression_latency_ms': metrics.average_compression_latency_ms,
},
'resources': {
'memory_mb': metrics.total_memory_mb,
'average_tokens_per_session': metrics.average_tokens_per_session,
}
}
}
# Usage
monitor = ContextMonitor()
# Track operations
start = time.time()
context_manager.add_context(session_id, content, context_type, priority)
monitor.track_operation('add', (time.time() - start) * 1000)
# Collect metrics
metrics = monitor.collect_metrics()
print(f"Average add latency: {metrics.average_add_latency_ms:.1f}ms")
print(f"Memory usage: {metrics.total_memory_mb:.1f}MB")
print(f"Health: {monitor.get_health_status()}")
# Export for monitoring system
metrics_export = monitor.export_metrics()
# Send to Prometheus, DataDog, etc.
Complete Example: Context-Aware Chat Agent
# agent_context_chat.yaml
"""
name: 'context_chat_agent'
description: 'Context-aware chat agent with benchmarking'
profile:
llm_model:
name: 'gpt-4'
context_manager_name: 'default_context_manager'
task_type: 'dialogue'
system_prompt: |
You are a helpful assistant with excellent memory.
memory:
name: 'chat_memory'
plan:
planner:
name: 'react_planner'
"""
# Implementation
from agentuniverse.agent.default.default_agent import DefaultAgent
from agentuniverse.agent.context.benchmark.benchmark_suite import ContextBenchmarkSuite
class ContextChatAgent(DefaultAgent):
"""Production-ready context-aware chat agent."""
def __init__(self):
super().__init__()
self._context_manager = None
self._monitor = None
self._benchmark_suite = None
def initialize_by_component_configer(self, component_configer):
super().initialize_by_component_configer(component_configer)
# Initialize context manager
context_manager_name = self.agent_model.profile.get('context_manager_name')
if context_manager_name:
self._context_manager = ContextManagerManager().get_instance_obj(
context_manager_name
)
# Initialize monitor
self._monitor = ContextMonitor(context_manager_name)
# Initialize benchmark suite
self._benchmark_suite = ContextBenchmarkSuite(self._context_manager)
return self
def run(self, **kwargs):
"""Run with monitoring."""
import time
# Track operation
start = time.time()
result = super().run(**kwargs)
latency = (time.time() - start) * 1000
if self._monitor:
self._monitor.track_operation('run', latency)
return result
def run_benchmark(self, num_turns: int = 100):
"""Run benchmark evaluation."""
if not self._benchmark_suite:
raise ValueError("Benchmark suite not initialized")
result = self._benchmark_suite.run_full_suite(num_turns=num_turns)
return result
def get_health_metrics(self):
"""Get current health metrics."""
if not self._monitor:
return {}
return self._monitor.export_metrics()
# Usage
agent = AgentManager().get_instance_obj('context_chat_agent')
# Normal operation
output = agent.run(
input="What did we discuss yesterday?",
session_id="user_123"
)
# Get metrics
metrics = agent.get_health_metrics()
print(f"Health: {metrics['health_status']}")
# Run benchmark
benchmark_result = agent.run_benchmark(num_turns=50)
print(f"Score: {benchmark_result.metrics.get_score():.1f}/100")
print(f"Passes: {benchmark_result.metrics.passes_targets()}")
Testing Phase 3-4 Features
Test 1: Knowledge Synchronization
def test_knowledge_sync():
"""Test knowledge-context synchronization."""
from agentuniverse.agent.context.sync.knowledge_context_synchronizer import (
KnowledgeContextSynchronizer,
ConflictResolutionStrategy
)
context_manager = ContextManagerManager().get_instance_obj("default_context_manager")
synchronizer = KnowledgeContextSynchronizer(context_manager)
# Test 1: Initial sync
result = synchronizer.sync_knowledge_to_context(
knowledge_id="test_doc",
documents=["Content version 1"],
session_id="test_session",
priority=ContextPriority.HIGH
)
assert result.segments_added == 1
assert result.segments_invalidated == 0
# Test 2: Update with no changes
result = synchronizer.sync_knowledge_to_context(
knowledge_id="test_doc",
documents=["Content version 1"], # Same content
session_id="test_session"
)
assert result.segments_added == 0 # No changes
# Test 3: Update with changes
result = synchronizer.sync_knowledge_to_context(
knowledge_id="test_doc",
documents=["Content version 2"], # Updated
session_id="test_session",
invalidate_old=True
)
assert result.segments_added == 1
assert result.segments_invalidated == 1
# Test 4: Version tracking
version = synchronizer.get_knowledge_version("test_doc")
assert version is not None
assert version.version_id.startswith("test_doc_")
print("✅ All knowledge sync tests passed")
test_knowledge_sync()
Test 2: Benchmarking
def test_benchmarking():
"""Test benchmark suite."""
from agentuniverse.agent.context.benchmark.benchmark_suite import (
ContextBenchmarkSuite
)
context_manager = ContextManagerManager().get_instance_obj("default_context_manager")
suite = ContextBenchmarkSuite(context_manager)
# Run full suite
result = suite.run_full_suite(num_turns=20) # Small for testing
# Check result structure
assert result.test_name == "full_suite"
assert result.metrics is not None
assert result.timestamp is not None
# Check metrics
metrics = result.metrics
assert 0 <= metrics.multi_turn_coherence <= 1
assert 0 <= metrics.compression_ratio <= 1
assert 0 <= metrics.information_loss <= 1
assert 0 <= metrics.retrieval_precision <= 1
assert metrics.average_latency_ms >= 0
assert metrics.memory_usage_mb >= 0
# Check scoring
score = metrics.get_score()
assert 0 <= score <= 100
print(f"✅ Benchmark test passed")
print(f" Score: {score:.1f}/100")
print(f" Coherence: {metrics.multi_turn_coherence:.3f}")
print(f" Compression: {metrics.compression_ratio:.1%}")
test_benchmarking()
Production Deployment Checklist
Configuration
- Set up context manager YAML with appropriate storage tiers
- Configure task-specific budgets for your use case
- Enable compression with appropriate strategy
- Set up monitoring and metrics collection
Integration
- Integrate context manager with agents
- Set up knowledge synchronization if using knowledge base
- Implement cross-agent sharing if using multi-agent system
- Add monitoring to track performance
Testing
- Run benchmark suite to validate performance
- Test knowledge synchronization with real data
- Verify compression quality and ratio
- Load test with production-scale data
Monitoring
- Set up health checks and alerts
- Monitor latency metrics (add, search, compression)
- Track memory usage and resource consumption
- Set up benchmarking schedule (weekly/monthly)
Performance Targets (vs Cursor/Claude)
| Metric | Target | Production Minimum |
|---|---|---|
| Multi-turn coherence | >0.85 | >0.80 |
| Compression ratio | 60-80% | 50-80% |
| Information loss | <10% | <15% |
| Retrieval precision | >0.90 | >0.85 |
| Add latency | <50ms | <100ms |
| Search latency | <100ms | <200ms |
| Memory usage | <500MB | <750MB |
| Overall score | >85/100 | >75/100 |
Troubleshooting
Issue: Low coherence score
Symptoms: multi_turn_coherence < 0.8
Solutions:
- Increase compression quality threshold
- Use CRITICAL priority for important context
- Enable VERSION_BOTH conflict resolution
- Increase context window size
Issue: High latency
Symptoms: latency > 100ms
Solutions:
- Enable caching in storage tier
- Use truncate compression for speed
- Reduce top_k in searches
- Optimize token counting
Issue: Memory exhaustion
Symptoms: memory_usage > 500MB
Solutions:
- Enable warm/cold storage tiers
- Reduce TTL for ephemeral segments
- Increase compression aggressiveness
- Implement proactive pruning
Next Steps
- Implement Integration: Choose appropriate pattern for your use case
- Run Benchmarks: Validate performance against targets
- Deploy to Staging: Test with production-like load
- Monitor Metrics: Track performance and quality metrics
- Optimize: Tune configuration based on metrics
- Scale: Deploy to production with monitoring
Support
For issues or questions:
- Review Phase 2 test results:
PHASE2_最终测试报告.md - Check implementation details in source files
- Run benchmark suite for diagnostics
- Monitor production metrics for insights
Phase 3-4 Status: ✅ Complete with integration patterns ready for production use.