# Phase 3-4 Integration Guide ## Context Engineering System Integration **Date**: 2025-12-04 **Status**: ✅ Phase 3 Complete | Phase 4 Integration Guide --- ## Overview This guide demonstrates how to integrate the Phase 3-4 Context Engineering System with agentUniverse agents. The system provides: 1. **Knowledge-Context Synchronization** (Phase 3) 2. **Benchmarking Framework** (Phase 3) 3. **Agent Integration Patterns** (Phase 4) 4. **Cross-Agent Context Sharing** (Phase 4) 5. **Production Monitoring** (Phase 4) --- ## Phase 3: New Components ### 1. Knowledge-Context Synchronization **Location**: `agentuniverse/agent/context/sync/knowledge_context_synchronizer.py` **Purpose**: Bidirectional synchronization between Knowledge and Context systems with conflict resolution. #### Key Features - **4 Conflict Resolution Strategies**: - `NEWEST_WINS`: Most recent knowledge takes precedence - `CRITICAL_PRESERVED`: CRITICAL priority segments always kept - `MERGE`: Combine both versions - `VERSION_BOTH`: Keep both as versioned segments - **Version Tracking**: SHA256 hash-based change detection - **Selective Invalidation**: Mark outdated segments without deletion - **Efficient Updates**: Only sync when content changes #### Basic Usage ```python from agentuniverse.agent.context.sync.knowledge_context_synchronizer import ( KnowledgeContextSynchronizer, ConflictResolutionStrategy ) from agentuniverse.agent.context.context_manager_manager import ContextManagerManager # Get context manager instance context_manager = ContextManagerManager().get_instance_obj("default_context_manager") # Create synchronizer synchronizer = KnowledgeContextSynchronizer( context_manager=context_manager, conflict_strategy=ConflictResolutionStrategy.NEWEST_WINS, enable_versioning=True ) # Sync knowledge to context result = synchronizer.sync_knowledge_to_context( knowledge_id="user_manual_v2", documents=["Updated user manual content...", "New API documentation..."], session_id="session_123", source="knowledge_base", priority=ContextPriority.HIGH ) print(f"Added {result.segments_added} segments") print(f"Invalidated {result.segments_invalidated} old segments") ``` #### Update Existing Knowledge ```python # Update when knowledge changes result = synchronizer.update_knowledge_context( knowledge_id="user_manual_v2", session_id="session_123", new_documents=["Updated content with corrections..."], conflict_strategy=ConflictResolutionStrategy.CRITICAL_PRESERVED ) print(f"Resolved {result.conflicts_resolved} conflicts") print(f"Updated {result.segments_updated} segments") ``` #### Version Tracking ```python # Get version information version = synchronizer.get_knowledge_version("user_manual_v2") if version: print(f"Version ID: {version.version_id}") print(f"Source: {version.source}") print(f"Hash: {version.hash}") print(f"Timestamp: {version.timestamp}") # List all versions all_versions = synchronizer.list_knowledge_versions() for knowledge_id, version in all_versions.items(): print(f"{knowledge_id}: {version.version_id}") ``` ### 2. Benchmarking Framework **Location**: `agentuniverse/agent/context/benchmark/benchmark_suite.py` **Purpose**: Comprehensive evaluation against industry standards (Cursor, Claude). #### Benchmark Targets | Metric | Target | Description | |--------|--------|-------------| | Multi-turn coherence | >0.85 | Context preservation across turns | | Compression ratio | 60-80% | Token reduction efficiency | | Information loss | <10% | Semantic preservation | | Retrieval precision | >0.90 | Search accuracy | | Retrieval latency | <100ms | Search speed | | Memory usage | <500MB | For 10K turns | #### Basic Usage ```python from agentuniverse.agent.context.benchmark.benchmark_suite import ( ContextBenchmarkSuite ) from agentuniverse.agent.context.context_manager_manager import ContextManagerManager # Get context manager context_manager = ContextManagerManager().get_instance_obj("default_context_manager") # Create benchmark suite suite = ContextBenchmarkSuite(context_manager) # Run full benchmark (100 turns by default) result = suite.run_full_suite(num_turns=100) # Check results print(f"Overall Score: {result.metrics.get_score():.1f}/100") print(f"Passes Targets: {result.metrics.passes_targets()}") print(f"Multi-turn Coherence: {result.metrics.multi_turn_coherence:.3f}") print(f"Compression Ratio: {result.metrics.compression_ratio:.1%}") print(f"Information Loss: {result.metrics.information_loss:.1%}") print(f"Retrieval Precision: {result.metrics.retrieval_precision:.3f}") print(f"Average Latency: {result.metrics.average_latency_ms:.1f}ms") print(f"Memory Usage: {result.metrics.memory_usage_mb:.1f}MB") ``` #### Detailed Metrics ```python # Access detailed metrics metrics = result.metrics # Coherence metrics print(f"Multi-turn coherence: {metrics.multi_turn_coherence:.3f}") print(f"Context consistency: {metrics.context_consistency:.3f}") # Compression metrics print(f"Compression ratio: {metrics.compression_ratio:.1%}") print(f"Information loss: {metrics.information_loss:.1%}") print(f"Context preservation: {metrics.context_preservation:.1%}") # Retrieval metrics print(f"Precision: {metrics.retrieval_precision:.3f}") print(f"Recall: {metrics.retrieval_recall:.3f}") print(f"F1 Score: {metrics.retrieval_f1:.3f}") # Performance metrics print(f"Average latency: {metrics.average_latency_ms:.1f}ms") print(f"P95 latency: {metrics.p95_latency_ms:.1f}ms") print(f"P99 latency: {metrics.p99_latency_ms:.1f}ms") print(f"Throughput: {metrics.throughput_ops_per_sec:.1f} ops/sec") # Resource metrics print(f"Memory usage: {metrics.memory_usage_mb:.1f}MB") print(f"Tokens per turn (avg): {metrics.tokens_per_turn_avg:.1f}") ``` --- ## Phase 4: Agent Integration Patterns ### Pattern 1: Context-Aware Agent (Basic) **Use Case**: Single agent with context management **Configuration** (`agent_context_aware.yaml`): ```yaml name: 'context_aware_agent' description: 'Agent with advanced context management' profile: llm_model: name: 'gpt-4' max_tokens: 2000 # Enable context manager context_manager_name: 'default_context_manager' task_type: 'dialogue' # code_generation, data_analysis, dialogue system_prompt: | You are an intelligent assistant with advanced context awareness. You can maintain coherent conversations across multiple turns. memory: name: 'demo_memory' action: knowledge: - 'demo_knowledge' plan: planner: name: 'react_planner' ``` **Implementation**: ```python from agentuniverse.agent.agent import Agent from agentuniverse.agent.agent_manager import AgentManager from agentuniverse.agent.context.context_manager_manager import ContextManagerManager class ContextAwareAgent(Agent): """Agent with integrated context management.""" def __init__(self): super().__init__() self._context_manager = None def initialize_by_component_configer(self, component_configer): """Initialize with context manager.""" super().initialize_by_component_configer(component_configer) # Get context manager name from profile context_manager_name = self.agent_model.profile.get('context_manager_name') if context_manager_name: self._context_manager = ContextManagerManager().get_instance_obj( context_manager_name ) return self def pre_parse_input(self, input_object): """Enhanced input parsing with context extraction.""" agent_input = super().pre_parse_input(input_object) # If context manager is enabled if self._context_manager: session_id = input_object.get_data('session_id') if session_id: # Create or get context window task_type = self.agent_model.profile.get('task_type', 'dialogue') window = self._context_manager.create_context_window( session_id=session_id, agent_id=self.agent_model.info.get('name'), task_type=task_type ) # Add user input to context user_input = input_object.get_data('input') if user_input: self._context_manager.add_context( session_id, user_input, ContextType.CONVERSATION, ContextPriority.HIGH, metadata={'role': 'user'} ) # Add system prompt to context (CRITICAL priority) system_prompt = self.agent_model.profile.get('system_prompt') if system_prompt: self._context_manager.add_context( session_id, system_prompt, ContextType.SYSTEM, ContextPriority.CRITICAL ) # Get context for building agent input agent_input['context_window'] = window return agent_input def parse_result(self, agent_result): """Enhanced result parsing with context storage.""" result = super().parse_result(agent_result) # Store agent output in context if self._context_manager: session_id = agent_result.get('session_id') output = result.get('output') if session_id and output: self._context_manager.add_context( session_id, output, ContextType.CONVERSATION, ContextPriority.HIGH, metadata={'role': 'assistant'} ) return result ``` ### Pattern 2: Knowledge-Integrated Agent **Use Case**: Agent with automatic knowledge-context synchronization **Implementation**: ```python from agentuniverse.agent.context.sync.knowledge_context_synchronizer import ( KnowledgeContextSynchronizer ) class KnowledgeIntegratedAgent(ContextAwareAgent): """Agent with knowledge-context synchronization.""" def __init__(self): super().__init__() self._synchronizer = None def initialize_by_component_configer(self, component_configer): """Initialize with synchronizer.""" super().initialize_by_component_configer(component_configer) # Create synchronizer if context manager exists if self._context_manager: self._synchronizer = KnowledgeContextSynchronizer( context_manager=self._context_manager, enable_versioning=True ) return self def execute(self, input_object, agent_input): """Execute with knowledge synchronization.""" session_id = input_object.get_data('session_id') # Sync knowledge before execution if self._synchronizer and session_id: knowledge_configs = self.agent_model.action.get('knowledge', []) for knowledge_config in knowledge_configs: knowledge_name = knowledge_config.get('name') if isinstance(knowledge_config, dict) else knowledge_config # Get knowledge documents (simplified) from agentuniverse.agent.action.knowledge.knowledge_manager import KnowledgeManager knowledge = KnowledgeManager().get_instance_obj(knowledge_name) if knowledge: # Query relevant documents query = input_object.get_data('input', '') documents = knowledge.query_knowledge(query, top_k=5) # Sync to context if documents: self._synchronizer.sync_knowledge_to_context( knowledge_id=knowledge_name, documents=[doc.text for doc in documents], session_id=session_id, priority=ContextPriority.HIGH ) # Execute normally return super().execute(input_object, agent_input) ``` ### Pattern 3: Cross-Agent Context Sharing **Use Case**: Multiple agents sharing context in a multi-agent system **Implementation**: ```python class AgentContextCoordinator: """Coordinates context sharing across multiple agents.""" def __init__(self, context_manager_name: str = "default_context_manager"): self.context_manager = ContextManagerManager().get_instance_obj( context_manager_name ) self._shared_sessions = {} # agent_group_id -> session_id def create_shared_session(self, agent_group_id: str) -> str: """Create a shared context session for a group of agents.""" session_id = f"shared_{agent_group_id}_{datetime.now().timestamp()}" window = self.context_manager.create_context_window( session_id=session_id, task_type="dialogue" ) self._shared_sessions[agent_group_id] = session_id return session_id def share_context( self, agent_group_id: str, content: str, source_agent: str, context_type: ContextType = ContextType.CONVERSATION, priority: ContextPriority = ContextPriority.MEDIUM ): """Share context from one agent to the group.""" session_id = self._shared_sessions.get(agent_group_id) if not session_id: session_id = self.create_shared_session(agent_group_id) self.context_manager.add_context( session_id, content, context_type, priority, metadata={ 'source_agent': source_agent, 'shared': True, 'timestamp': datetime.now().isoformat() } ) def get_shared_context( self, agent_group_id: str, context_type: Optional[ContextType] = None ) -> List[ContextSegment]: """Get shared context for the agent group.""" session_id = self._shared_sessions.get(agent_group_id) if not session_id: return [] return self.context_manager.get_context( session_id, context_type=context_type ) # Usage coordinator = AgentContextCoordinator() # Create shared session for multiple agents session_id = coordinator.create_shared_session("research_team") # Agent 1 shares findings coordinator.share_context( agent_group_id="research_team", content="Found relevant research paper on topic X", source_agent="research_agent", priority=ContextPriority.HIGH ) # Agent 2 shares analysis coordinator.share_context( agent_group_id="research_team", content="Analysis shows correlation with previous findings", source_agent="analysis_agent", priority=ContextPriority.HIGH ) # Agent 3 retrieves all shared context shared_context = coordinator.get_shared_context("research_team") for segment in shared_context: print(f"From {segment.metadata.custom['source_agent']}: {segment.content}") ``` ### Pattern 4: Production Monitoring **Implementation**: ```python from dataclasses import dataclass from typing import Dict, List import time @dataclass class ContextMetrics: """Production metrics for context system.""" # Usage metrics total_sessions: int = 0 active_sessions: int = 0 total_segments: int = 0 average_segments_per_session: float = 0.0 # Performance metrics average_add_latency_ms: float = 0.0 average_search_latency_ms: float = 0.0 average_compression_latency_ms: float = 0.0 # Quality metrics average_compression_ratio: float = 0.0 average_retrieval_precision: float = 0.0 # Resource metrics total_memory_mb: float = 0.0 average_tokens_per_session: float = 0.0 class ContextMonitor: """Monitor context system performance in production.""" def __init__(self, context_manager_name: str = "default_context_manager"): self.context_manager = ContextManagerManager().get_instance_obj( context_manager_name ) self._metrics_history: List[ContextMetrics] = [] self._operation_latencies: Dict[str, List[float]] = { 'add': [], 'search': [], 'compression': [] } def track_operation(self, operation_type: str, latency_ms: float): """Track operation latency.""" if operation_type in self._operation_latencies: self._operation_latencies[operation_type].append(latency_ms) def collect_metrics(self) -> ContextMetrics: """Collect current metrics.""" import sys metrics = ContextMetrics() # Count active windows if hasattr(self.context_manager, '_windows'): metrics.total_sessions = len(self.context_manager._windows) metrics.active_sessions = len([ w for w in self.context_manager._windows.values() if w.total_tokens > 0 ]) # Calculate average latencies if self._operation_latencies['add']: metrics.average_add_latency_ms = sum(self._operation_latencies['add']) / len(self._operation_latencies['add']) if self._operation_latencies['search']: metrics.average_search_latency_ms = sum(self._operation_latencies['search']) / len(self._operation_latencies['search']) if self._operation_latencies['compression']: metrics.average_compression_latency_ms = sum(self._operation_latencies['compression']) / len(self._operation_latencies['compression']) # Memory usage metrics.total_memory_mb = sys.getsizeof(self.context_manager) / (1024 * 1024) self._metrics_history.append(metrics) return metrics def get_health_status(self) -> str: """Get system health status.""" metrics = self.collect_metrics() # Check against targets issues = [] if metrics.average_add_latency_ms > 100: issues.append("High add latency") if metrics.average_search_latency_ms > 100: issues.append("High search latency") if metrics.total_memory_mb > 500: issues.append("High memory usage") if not issues: return "HEALTHY" elif len(issues) <= 1: return f"WARNING: {issues[0]}" else: return f"CRITICAL: {', '.join(issues)}" def export_metrics(self) -> Dict[str, Any]: """Export metrics for monitoring system.""" metrics = self.collect_metrics() return { 'timestamp': datetime.now().isoformat(), 'health_status': self.get_health_status(), 'metrics': { 'sessions': { 'total': metrics.total_sessions, 'active': metrics.active_sessions, }, 'performance': { 'add_latency_ms': metrics.average_add_latency_ms, 'search_latency_ms': metrics.average_search_latency_ms, 'compression_latency_ms': metrics.average_compression_latency_ms, }, 'resources': { 'memory_mb': metrics.total_memory_mb, 'average_tokens_per_session': metrics.average_tokens_per_session, } } } # Usage monitor = ContextMonitor() # Track operations start = time.time() context_manager.add_context(session_id, content, context_type, priority) monitor.track_operation('add', (time.time() - start) * 1000) # Collect metrics metrics = monitor.collect_metrics() print(f"Average add latency: {metrics.average_add_latency_ms:.1f}ms") print(f"Memory usage: {metrics.total_memory_mb:.1f}MB") print(f"Health: {monitor.get_health_status()}") # Export for monitoring system metrics_export = monitor.export_metrics() # Send to Prometheus, DataDog, etc. ``` --- ## Complete Example: Context-Aware Chat Agent ```python # agent_context_chat.yaml """ name: 'context_chat_agent' description: 'Context-aware chat agent with benchmarking' profile: llm_model: name: 'gpt-4' context_manager_name: 'default_context_manager' task_type: 'dialogue' system_prompt: | You are a helpful assistant with excellent memory. memory: name: 'chat_memory' plan: planner: name: 'react_planner' """ # Implementation from agentuniverse.agent.default.default_agent import DefaultAgent from agentuniverse.agent.context.benchmark.benchmark_suite import ContextBenchmarkSuite class ContextChatAgent(DefaultAgent): """Production-ready context-aware chat agent.""" def __init__(self): super().__init__() self._context_manager = None self._monitor = None self._benchmark_suite = None def initialize_by_component_configer(self, component_configer): super().initialize_by_component_configer(component_configer) # Initialize context manager context_manager_name = self.agent_model.profile.get('context_manager_name') if context_manager_name: self._context_manager = ContextManagerManager().get_instance_obj( context_manager_name ) # Initialize monitor self._monitor = ContextMonitor(context_manager_name) # Initialize benchmark suite self._benchmark_suite = ContextBenchmarkSuite(self._context_manager) return self def run(self, **kwargs): """Run with monitoring.""" import time # Track operation start = time.time() result = super().run(**kwargs) latency = (time.time() - start) * 1000 if self._monitor: self._monitor.track_operation('run', latency) return result def run_benchmark(self, num_turns: int = 100): """Run benchmark evaluation.""" if not self._benchmark_suite: raise ValueError("Benchmark suite not initialized") result = self._benchmark_suite.run_full_suite(num_turns=num_turns) return result def get_health_metrics(self): """Get current health metrics.""" if not self._monitor: return {} return self._monitor.export_metrics() # Usage agent = AgentManager().get_instance_obj('context_chat_agent') # Normal operation output = agent.run( input="What did we discuss yesterday?", session_id="user_123" ) # Get metrics metrics = agent.get_health_metrics() print(f"Health: {metrics['health_status']}") # Run benchmark benchmark_result = agent.run_benchmark(num_turns=50) print(f"Score: {benchmark_result.metrics.get_score():.1f}/100") print(f"Passes: {benchmark_result.metrics.passes_targets()}") ``` --- ## Testing Phase 3-4 Features ### Test 1: Knowledge Synchronization ```python def test_knowledge_sync(): """Test knowledge-context synchronization.""" from agentuniverse.agent.context.sync.knowledge_context_synchronizer import ( KnowledgeContextSynchronizer, ConflictResolutionStrategy ) context_manager = ContextManagerManager().get_instance_obj("default_context_manager") synchronizer = KnowledgeContextSynchronizer(context_manager) # Test 1: Initial sync result = synchronizer.sync_knowledge_to_context( knowledge_id="test_doc", documents=["Content version 1"], session_id="test_session", priority=ContextPriority.HIGH ) assert result.segments_added == 1 assert result.segments_invalidated == 0 # Test 2: Update with no changes result = synchronizer.sync_knowledge_to_context( knowledge_id="test_doc", documents=["Content version 1"], # Same content session_id="test_session" ) assert result.segments_added == 0 # No changes # Test 3: Update with changes result = synchronizer.sync_knowledge_to_context( knowledge_id="test_doc", documents=["Content version 2"], # Updated session_id="test_session", invalidate_old=True ) assert result.segments_added == 1 assert result.segments_invalidated == 1 # Test 4: Version tracking version = synchronizer.get_knowledge_version("test_doc") assert version is not None assert version.version_id.startswith("test_doc_") print("✅ All knowledge sync tests passed") test_knowledge_sync() ``` ### Test 2: Benchmarking ```python def test_benchmarking(): """Test benchmark suite.""" from agentuniverse.agent.context.benchmark.benchmark_suite import ( ContextBenchmarkSuite ) context_manager = ContextManagerManager().get_instance_obj("default_context_manager") suite = ContextBenchmarkSuite(context_manager) # Run full suite result = suite.run_full_suite(num_turns=20) # Small for testing # Check result structure assert result.test_name == "full_suite" assert result.metrics is not None assert result.timestamp is not None # Check metrics metrics = result.metrics assert 0 <= metrics.multi_turn_coherence <= 1 assert 0 <= metrics.compression_ratio <= 1 assert 0 <= metrics.information_loss <= 1 assert 0 <= metrics.retrieval_precision <= 1 assert metrics.average_latency_ms >= 0 assert metrics.memory_usage_mb >= 0 # Check scoring score = metrics.get_score() assert 0 <= score <= 100 print(f"✅ Benchmark test passed") print(f" Score: {score:.1f}/100") print(f" Coherence: {metrics.multi_turn_coherence:.3f}") print(f" Compression: {metrics.compression_ratio:.1%}") test_benchmarking() ``` --- ## Production Deployment Checklist ### Configuration - [ ] Set up context manager YAML with appropriate storage tiers - [ ] Configure task-specific budgets for your use case - [ ] Enable compression with appropriate strategy - [ ] Set up monitoring and metrics collection ### Integration - [ ] Integrate context manager with agents - [ ] Set up knowledge synchronization if using knowledge base - [ ] Implement cross-agent sharing if using multi-agent system - [ ] Add monitoring to track performance ### Testing - [ ] Run benchmark suite to validate performance - [ ] Test knowledge synchronization with real data - [ ] Verify compression quality and ratio - [ ] Load test with production-scale data ### Monitoring - [ ] Set up health checks and alerts - [ ] Monitor latency metrics (add, search, compression) - [ ] Track memory usage and resource consumption - [ ] Set up benchmarking schedule (weekly/monthly) --- ## Performance Targets (vs Cursor/Claude) | Metric | Target | Production Minimum | |--------|--------|-------------------| | Multi-turn coherence | >0.85 | >0.80 | | Compression ratio | 60-80% | 50-80% | | Information loss | <10% | <15% | | Retrieval precision | >0.90 | >0.85 | | Add latency | <50ms | <100ms | | Search latency | <100ms | <200ms | | Memory usage | <500MB | <750MB | | Overall score | >85/100 | >75/100 | --- ## Troubleshooting ### Issue: Low coherence score **Symptoms**: `multi_turn_coherence < 0.8` **Solutions**: - Increase compression quality threshold - Use CRITICAL priority for important context - Enable VERSION_BOTH conflict resolution - Increase context window size ### Issue: High latency **Symptoms**: `latency > 100ms` **Solutions**: - Enable caching in storage tier - Use truncate compression for speed - Reduce top_k in searches - Optimize token counting ### Issue: Memory exhaustion **Symptoms**: `memory_usage > 500MB` **Solutions**: - Enable warm/cold storage tiers - Reduce TTL for ephemeral segments - Increase compression aggressiveness - Implement proactive pruning --- ## Next Steps 1. **Implement Integration**: Choose appropriate pattern for your use case 2. **Run Benchmarks**: Validate performance against targets 3. **Deploy to Staging**: Test with production-like load 4. **Monitor Metrics**: Track performance and quality metrics 5. **Optimize**: Tune configuration based on metrics 6. **Scale**: Deploy to production with monitoring --- ## Support For issues or questions: - Review Phase 2 test results: `PHASE2_最终测试报告.md` - Check implementation details in source files - Run benchmark suite for diagnostics - Monitor production metrics for insights **Phase 3-4 Status**: ✅ Complete with integration patterns ready for production use.