同步完整源码 - 2026-05-25
This commit is contained in:
@@ -0,0 +1 @@
|
||||
uv.lock binary
|
||||
@@ -0,0 +1,20 @@
|
||||
# Code of Conduct
|
||||
|
||||
Be kind and be mindful. We're here to accelerate together.
|
||||
|
||||
All participants in our project's community are expected to follow this code of conduct, which covers our behavior as contributors, maintainers, and community members.
|
||||
|
||||
## Our Pledge
|
||||
|
||||
|
||||
We pledge to make participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to creating a positive environment include:
|
||||
|
||||
* Using welcoming and inclusive language
|
||||
* Being respectful of differing viewpoints and experiences
|
||||
* Gracefully accepting constructive criticism
|
||||
* Focusing on what is best for the community
|
||||
* Showing empathy towards other community members
|
||||
@@ -0,0 +1,24 @@
|
||||
## Open Source and Proprietary Components
|
||||
|
||||
The AgentOps SDK is open source, but the backend and UI are not. We are source available and also support on-prem/VPC hosting upon request.
|
||||
|
||||
That said, we love when community members contribute to our open source SDK-- we even mail over swag packs to developers who help out :)
|
||||
|
||||
## Contribution Guidelines
|
||||
|
||||
We use issues primarily to communicate our thoughts and plans to the community. However, not every issue is suitable for the open-source distribution, as some features may depend on non-open-source parts of our stack.
|
||||
|
||||
When looking for issues to work on look for following tags:
|
||||
- `good first issue`
|
||||
- `help wanted`
|
||||
|
||||
If you have feature requests or bug reports, please open an issue.
|
||||
|
||||
## Getting Started
|
||||
|
||||
The easiest way to contribute is to start a conversation with us. We highly recommend:
|
||||
|
||||
1. Joining our Discord community
|
||||
2. Sending us a message through our contact form at [agentops.ai/contact](https://agentops.ai/contact)
|
||||
|
||||
We look forward to collaborating with you and appreciate your interest in improving AgentOps!
|
||||
@@ -0,0 +1,53 @@
|
||||
name: 🐛 Bug Report
|
||||
description: Noticed something off with Agentops? Let us know!
|
||||
title: "[Bug]: "
|
||||
labels: ["bug"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for taking the time to fill out this bug report! Your feedback helps us improve AgentOps.
|
||||
- type: input
|
||||
id: contact
|
||||
attributes:
|
||||
label: Contact Details
|
||||
description: How can we get in touch with you if we need more info?
|
||||
placeholder: ex. email@example.com
|
||||
validations:
|
||||
required: false
|
||||
- type: input
|
||||
id: package-version
|
||||
attributes:
|
||||
label: 📦 Package Version
|
||||
description: What version of Agentops are you using?
|
||||
placeholder: ex. 0.3.12
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: framework-version
|
||||
attributes:
|
||||
label: 🎞️ Framework Version
|
||||
description: Which frameworks are you using? (if applicable)
|
||||
placeholder: ex. Autogen 0.3.0, CrewAI 0.61.0, Langchain 0.2.1
|
||||
validations:
|
||||
required: false
|
||||
- type: textarea
|
||||
id: bug-description
|
||||
attributes:
|
||||
label: 🔎 Describe the Bug
|
||||
description: Give a clear and concise description of the bug and how to reproduce it. Include any error messages and screenshots.
|
||||
placeholder: What went wrong?
|
||||
validations:
|
||||
required: true
|
||||
- type: checkboxes
|
||||
id: contribution
|
||||
attributes:
|
||||
label: 🤝 Contribution
|
||||
description: Would you be willing to contribute to this update?
|
||||
options:
|
||||
- label: Yes, I'd be happy to submit a pull request with these changes.
|
||||
- label: I need some guidance on how to contribute.
|
||||
- label: I'd prefer the AgentOps team to handle this update.
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1,53 @@
|
||||
name: 📚 Documentation Update
|
||||
description: Suggest an improvement or addition to our documentation
|
||||
title: "[Docs]: "
|
||||
labels: ["documentation"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for helping us improve AgentOps documentation! Your input is valuable in making our docs more helpful and comprehensive.
|
||||
- type: textarea
|
||||
id: current-state
|
||||
attributes:
|
||||
label: 📘 Current State of Documentation
|
||||
description: What information is missing or could be improved?
|
||||
placeholder: Describe the current state of the documentation you'd like to update.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: suggested-improvement
|
||||
attributes:
|
||||
label: 📖 Suggested Improvement
|
||||
description: Describe your idea for the new/improved documentation.
|
||||
placeholder: What changes or additions would you like to see?
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: affected-pages
|
||||
attributes:
|
||||
label: 🔗 Affected Documentation Pages
|
||||
description: If applicable, provide links to the documentation pages that need updating.
|
||||
placeholder: https://docs.agentops.ai/page-to-update
|
||||
validations:
|
||||
required: false
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: 🔍 Additional Context
|
||||
description: Any extra information to help us understand the change?
|
||||
placeholder: Provide any additional details or context for the documentation update.
|
||||
validations:
|
||||
required: false
|
||||
- type: checkboxes
|
||||
id: contribution
|
||||
attributes:
|
||||
label: 🤝 Contribution
|
||||
description: Would you be willing to contribute to this documentation update?
|
||||
options:
|
||||
- label: Yes, I'd be happy to submit a pull request with these changes.
|
||||
- label: I need some guidance on how to contribute.
|
||||
- label: I'd prefer the Agentops team to handle this update.
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1,37 @@
|
||||
name: 🚀 Feature Request
|
||||
description: Got an idea to improve AgentOps? We're all ears!
|
||||
title: "[Feature]: "
|
||||
labels: ["enhancement"]
|
||||
assignees: []
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for taking the time to share your feature idea! Your input helps us improve AgentOps.
|
||||
- type: textarea
|
||||
id: feature-description-and-solution
|
||||
attributes:
|
||||
label: 💡 Feature Description and Proposed Solution
|
||||
description: Briefly describe your feature suggestion and how you envision it working. Include any mockups/screenshots if applicable.
|
||||
placeholder: What's your brilliant idea?
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: problem-related
|
||||
attributes:
|
||||
label: 🤔 Related Problem
|
||||
description: Is your feature request related to a problem you're facing?
|
||||
placeholder: Describe the problem, if applicable. The more we understand your struggle, the better we can address it.
|
||||
validations:
|
||||
required: false
|
||||
- type: checkboxes
|
||||
id: contribution
|
||||
attributes:
|
||||
label: 🤝 Contribution
|
||||
description: Would you be willing to contribute to this update?
|
||||
options:
|
||||
- label: Yes, I'd be happy to submit a pull request with these changes.
|
||||
- label: I need some guidance on how to contribute.
|
||||
- label: I'd prefer the AgentOps team to handle this update.
|
||||
validations:
|
||||
required: true
|
||||
@@ -0,0 +1,8 @@
|
||||
## 📥 Pull Request
|
||||
|
||||
**📘 Description**
|
||||
_Briefly describe the changes you've made._
|
||||
|
||||
**🧪 Testing**
|
||||
_Describe the tests you performed to validate your changes._
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
# Configuration for probot-stale - https://github.com/probot/stale
|
||||
|
||||
# Number of days of inactivity before an issue becomes stale
|
||||
daysUntilStale: 60
|
||||
@@ -0,0 +1,132 @@
|
||||
name: Add Notebook Examples to Docs
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- 'examples/**'
|
||||
- 'docs/v2/examples/**'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
add-notebook-examples-to-docs:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v3
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.x'
|
||||
|
||||
- name: Install dependencies
|
||||
run: pip install jupyter nbconvert
|
||||
|
||||
- name: Detect and process notebooks
|
||||
id: process
|
||||
run: |
|
||||
# Determine comparison range
|
||||
case "${{ github.event_name }}" in
|
||||
"push")
|
||||
BASE_SHA="${{ github.event.before }}"
|
||||
[[ "$BASE_SHA" == "0000000000000000000000000000000000000000" ]] && BASE_SHA="$(git hash-object -t tree /dev/null)"
|
||||
HEAD_SHA="${{ github.event.after }}"
|
||||
;;
|
||||
"pull_request")
|
||||
BASE_SHA="${{ github.event.pull_request.base.sha }}"
|
||||
HEAD_SHA="${{ github.event.pull_request.head.sha }}"
|
||||
;;
|
||||
"workflow_dispatch")
|
||||
git fetch origin "${{ github.event.repository.default_branch }}" --depth=50
|
||||
BASE_SHA="origin/${{ github.event.repository.default_branch }}"
|
||||
HEAD_SHA="HEAD"
|
||||
;;
|
||||
*)
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
# Get changed files and filter in one pass
|
||||
CHANGED_FILES=$(git diff --name-only "$BASE_SHA" "$HEAD_SHA")
|
||||
|
||||
# Use arrays for better performance
|
||||
declare -A notebooks_set
|
||||
|
||||
# Process notebooks directly from changed files
|
||||
while IFS= read -r file; do
|
||||
if [[ "$file" =~ ^examples/.*\.ipynb$ ]]; then
|
||||
notebooks_set["$file"]=1
|
||||
elif [[ "$file" =~ ^docs/v2/examples/.*\.mdx$ ]] && [[ -f "$file" ]]; then
|
||||
# Extract source notebook from MDX file
|
||||
source_notebook=$(grep -oP '(?<=SOURCE_FILE: )[^ ]+' "$file" 2>/dev/null || true)
|
||||
if [[ -n "$source_notebook" && -f "$source_notebook" ]]; then
|
||||
notebooks_set["$source_notebook"]=1
|
||||
fi
|
||||
fi
|
||||
done <<< "$CHANGED_FILES"
|
||||
|
||||
# Exit if no notebooks found
|
||||
[[ ${#notebooks_set[@]} -eq 0 ]] && exit 0
|
||||
|
||||
# Process each notebook and track results
|
||||
failed_count=0
|
||||
processed_notebooks=""
|
||||
for notebook in "${!notebooks_set[@]}"; do
|
||||
if python examples/generate_documentation.py "$notebook"; then
|
||||
processed_notebooks="$processed_notebooks$notebook"$'\n'
|
||||
else
|
||||
((failed_count++))
|
||||
fi
|
||||
done
|
||||
|
||||
[[ $failed_count -gt 0 ]] && exit 1
|
||||
|
||||
# Get modified files for PR description
|
||||
modified_files=$(git status --porcelain | grep -E '^\s*[AM]' | awk '{print $2}' | sort)
|
||||
|
||||
# Save outputs for PR
|
||||
echo "processed_notebooks<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$processed_notebooks" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
echo "modified_files<<EOF" >> $GITHUB_OUTPUT
|
||||
echo "$modified_files" >> $GITHUB_OUTPUT
|
||||
echo "EOF" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Create Pull Request
|
||||
uses: peter-evans/create-pull-request@v7
|
||||
with:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
committer: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
||||
author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
|
||||
commit-message: "docs: update examples from notebooks"
|
||||
title: "docs: update examples from notebooks"
|
||||
body: |
|
||||
Updates documentation examples from their source notebooks.
|
||||
|
||||
**Triggered by:** @${{ github.actor }}
|
||||
**Event:** ${{ github.event_name }}
|
||||
${{ steps.process.outputs.processed_notebooks && format('
|
||||
|
||||
## Processed Notebooks
|
||||
```
|
||||
{0}
|
||||
```', steps.process.outputs.processed_notebooks) || '' }}
|
||||
${{ steps.process.outputs.modified_files && format('
|
||||
|
||||
## Modified Files
|
||||
```
|
||||
{0}
|
||||
```', steps.process.outputs.modified_files) || '' }}
|
||||
branch: update-examples-docs
|
||||
delete-branch: true
|
||||
assignees: the-praxs,bboynton97,areibman
|
||||
reviewers: the-praxs,bboynton97,areibman
|
||||
@@ -0,0 +1,42 @@
|
||||
name: Compile llms.txt
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [ main ]
|
||||
paths:
|
||||
- 'docs/**'
|
||||
- 'README.md'
|
||||
- 'CONTRIBUTING.md'
|
||||
- 'examples/*/README.md'
|
||||
- 'agentops/*/README.md'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
compile-llms-txt:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: '3.11'
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
pip install llms-txt
|
||||
|
||||
- name: Compile llms.txt
|
||||
run: |
|
||||
cd docs
|
||||
python compile_llms_txt.py
|
||||
|
||||
- name: Commit and push if changed
|
||||
run: |
|
||||
git config --local user.email "action@github.com"
|
||||
git config --local user.name "GitHub Action"
|
||||
git add llms.txt
|
||||
git diff --staged --quiet || git commit -m "Auto-update llms.txt from documentation changes"
|
||||
git push
|
||||
@@ -0,0 +1,28 @@
|
||||
name: Delete Bot Comments
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
delete-bot-comment:
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ github.event.comment.user.login == 'entelligence-ai-pr-reviews' }}
|
||||
|
||||
steps:
|
||||
- name: Delete bot comment
|
||||
uses: actions/github-script@v6
|
||||
with:
|
||||
github-token: ${{ secrets.GITHUB_TOKEN }}
|
||||
script: |
|
||||
const commentId = context.payload.comment.id;
|
||||
|
||||
console.log(`Deleting comment ID: ${commentId} from bot: entelligence-ai-pr-reviews`);
|
||||
|
||||
await github.rest.issues.deleteComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
comment_id: commentId
|
||||
});
|
||||
|
||||
console.log('Comment deleted successfully');
|
||||
@@ -0,0 +1,53 @@
|
||||
name: Build and Push MCP Docker Image
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
tags:
|
||||
- 'v*'
|
||||
- 'mcp-v*'
|
||||
paths:
|
||||
- 'mcp/**'
|
||||
- '.github/workflows/docker-mcp-publish.yml'
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Log in to DockerHub
|
||||
uses: docker/login-action@v3
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Extract metadata
|
||||
id: meta
|
||||
uses: docker/metadata-action@v5
|
||||
with:
|
||||
images: |
|
||||
${{ secrets.DOCKERHUB_USERNAME }}/agentops-mcp
|
||||
tags: |
|
||||
type=ref,event=branch
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
|
||||
- name: Build and push Docker image
|
||||
uses: docker/build-push-action@v5
|
||||
with:
|
||||
context: ./mcp
|
||||
file: ./mcp/Dockerfile
|
||||
push: true
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
@@ -0,0 +1,191 @@
|
||||
name: Examples Integration Test
|
||||
|
||||
# This workflow runs all example scripts to ensure they work correctly
|
||||
# and that LLM spans are properly tracked in AgentOps using the
|
||||
# integrated validation functionality.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [ main, develop ]
|
||||
paths:
|
||||
- 'examples/**/*.py'
|
||||
- 'agentops/**'
|
||||
- '.github/workflows/examples-integration-test.yml'
|
||||
types: [opened, reopened, ready_for_review]
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
PYTHON_VERSION: '3.11'
|
||||
|
||||
jobs:
|
||||
test-examples:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
example:
|
||||
# OpenAI examples
|
||||
- { path: 'examples/openai/openai_example_sync.py', name: 'OpenAI Sync' }
|
||||
- { path: 'examples/openai/openai_example_async.py', name: 'OpenAI Async' }
|
||||
- { path: 'examples/openai/multi_tool_orchestration.py', name: 'OpenAI Multi-Tool' }
|
||||
- { path: 'examples/openai/web_search.py', name: 'OpenAI Web Search' }
|
||||
- { path: 'examples/openai/o3_responses_example.py', name: 'OpenAI o3 Responses' }
|
||||
|
||||
# Anthropic examples
|
||||
- { path: 'examples/anthropic/anthropic-example-sync.py', name: 'Anthropic Sync' }
|
||||
- { path: 'examples/anthropic/anthropic-example-async.py', name: 'Anthropic Async' }
|
||||
- { path: 'examples/anthropic/agentops-anthropic-understanding-tools.py', name: 'Anthropic Tools' }
|
||||
|
||||
# LangChain examples
|
||||
- { path: 'examples/langchain/langchain_examples.py', name: 'LangChain' }
|
||||
|
||||
# LiteLLM examples
|
||||
- { path: 'examples/litellm/litellm_example.py', name: 'LiteLLM' }
|
||||
|
||||
# Google Generative AI examples
|
||||
- { path: 'examples/google_genai/gemini_example.py', name: 'Google Gemini' }
|
||||
|
||||
# xAI examples
|
||||
- { path: 'examples/xai/grok_examples.py', name: 'xAI Grok' }
|
||||
- { path: 'examples/xai/grok_vision_examples.py', name: 'xAI Grok Vision' }
|
||||
|
||||
# CrewAI examples
|
||||
- { path: 'examples/crewai/job_posting.py', name: 'CrewAI Job Posting' }
|
||||
- { path: 'examples/crewai/markdown_validator.py', name: 'CrewAI Markdown' }
|
||||
|
||||
# AutoGen examples
|
||||
- { path: 'examples/autogen/AgentChat.py', name: 'AutoGen Agent Chat' }
|
||||
- { path: 'examples/autogen/MathAgent.py', name: 'AutoGen Math Agent' }
|
||||
|
||||
# AG2 examples
|
||||
- { path: 'examples/ag2/async_human_input.py', name: 'AG2 Async Human Input' }
|
||||
- { path: 'examples/ag2/tools_wikipedia_search.py', name: 'AG2 Wikipedia Search' }
|
||||
|
||||
# Agno examples
|
||||
- { path: 'examples/agno/agno_async_operations.py', name: 'Agno Async Operations' }
|
||||
- { path: 'examples/agno/agno_basic_agents.py', name: 'Agno Basic Agents' }
|
||||
- { path: 'examples/agno/agno_research_team.py', name: 'Agno Research Team' }
|
||||
- { path: 'examples/agno/agno_tool_integrations.py', name: 'Agno Tool Integrations' }
|
||||
- { path: 'examples/agno/agno_workflow_setup.py', name: 'Agno Workflow Setup' }
|
||||
|
||||
# Google ADK examples
|
||||
- { path: 'examples/google_adk/human_approval.py', name: 'Google ADK Human Approval' }
|
||||
- { path: 'examples/google_adk/sse_function_call_example.py', name: 'Google ADK SSE Function Call' }
|
||||
|
||||
# LlamaIndex examples
|
||||
# - { path: 'examples/llamaindex/llamaindex_example.py', name: 'LlamaIndex' }
|
||||
|
||||
# Mem0 examples
|
||||
- { path: 'examples/mem0/mem0_memoryclient_example.py', name: 'Mem0 Memory Client' }
|
||||
|
||||
# Watsonx examples
|
||||
- { path: 'examples/watsonx/watsonx-streaming.py', name: 'Watsonx Streaming' }
|
||||
- { path: 'examples/watsonx/watsonx-text-chat.py', name: 'Watsonx Text Chat' }
|
||||
- { path: 'examples/watsonx/watsonx-tokeniation-model.py', name: 'Watsonx Tokenization' }
|
||||
|
||||
# LangGraph examples
|
||||
- { path: 'examples/langgraph/langgraph_example.py', name: 'LangGraph' }
|
||||
|
||||
# Smolagents examples
|
||||
- { path: 'examples/smolagents/multi_smolagents_system.py', name: 'Smolagents Multi System' }
|
||||
- { path: 'examples/smolagents/text_to_sql.py', name: 'Smolagents Text to SQL' }
|
||||
|
||||
# OpenAI Agents examples
|
||||
- { path: 'examples/openai_agents/agent_guardrails.py', name: 'OpenAI Agents Guardrails' }
|
||||
- { path: 'examples/openai_agents/agent_patterns.py', name: 'OpenAI Agents Patterns' }
|
||||
- { path: 'examples/openai_agents/agents_tools.py', name: 'OpenAI Agents Tools' }
|
||||
- { path: 'examples/openai_agents/customer_service_agent.py', name: 'OpenAI Agents Customer Service' }
|
||||
|
||||
# DSPy examples
|
||||
- { path: 'examples/dspy/dspy_calculator.py', name: 'DSPy ReAct Agent' }
|
||||
|
||||
# Haystack examples
|
||||
- { path: 'examples/haystack/haystack_example.py', name: 'Haystack OpenAI' }
|
||||
- { path: 'examples/haystack/azure_haystack_example.py', name: 'Haystack Azure Chat' }
|
||||
# Add more examples as needed
|
||||
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v4
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
|
||||
- name: Install AgentOps
|
||||
run: |
|
||||
pip install -e .
|
||||
|
||||
- name: Install example dependencies
|
||||
run: |
|
||||
# Install common dependencies
|
||||
pip install python-dotenv requests
|
||||
|
||||
# Install from requirements.txt in the example's directory
|
||||
example_dir=$(dirname "${{ matrix.example.path }}")
|
||||
if [ -f "$example_dir/requirements.txt" ]; then
|
||||
echo "Installing dependencies from $example_dir/requirements.txt"
|
||||
pip install -r "$example_dir/requirements.txt"
|
||||
else
|
||||
echo "No requirements.txt found in $example_dir"
|
||||
fi
|
||||
|
||||
- name: Run example - ${{ matrix.example.name }}
|
||||
env:
|
||||
AGENTOPS_API_KEY: ${{ secrets.AGENTOPS_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
|
||||
WATSONX_API_KEY: ${{ secrets.WATSONX_API_KEY }}
|
||||
WATSONX_PROJECT_ID: ${{ secrets.WATSONX_PROJECT_ID }}
|
||||
WATSONX_URL: ${{ secrets.WATSONX_URL }}
|
||||
MEM0_API_KEY: ${{ secrets.MEM0_API_KEY }}
|
||||
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
|
||||
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
|
||||
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
|
||||
FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
|
||||
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
|
||||
AI21_API_KEY: ${{ secrets.AI21_API_KEY }}
|
||||
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
|
||||
EXA_API_KEY: ${{ secrets.EXA_API_KEY }}
|
||||
LLAMA_API_KEY: ${{ secrets.LLAMA_API_KEY }}
|
||||
PERPLEXITY_API_KEY: ${{ secrets.PERPLEXITY_API_KEY }}
|
||||
REPLICATE_API_TOKEN: ${{ secrets.REPLICATE_API_TOKEN }}
|
||||
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
|
||||
PYTHONPATH: ${{ github.workspace }}
|
||||
run: |
|
||||
echo "Running ${{ matrix.example.name }}..."
|
||||
python "${{ matrix.example.path }}" || exit 1
|
||||
|
||||
- name: Check for errors
|
||||
if: failure()
|
||||
run: |
|
||||
echo "Example ${{ matrix.example.name }} failed!"
|
||||
echo "Path: ${{ matrix.example.path }}"
|
||||
|
||||
# Show last 50 lines of any log files
|
||||
if [ -f agentops.log ]; then
|
||||
echo "=== AgentOps Log ==="
|
||||
tail -n 50 agentops.log
|
||||
fi
|
||||
|
||||
summary:
|
||||
needs: test-examples
|
||||
runs-on: ubuntu-latest
|
||||
if: always()
|
||||
|
||||
steps:
|
||||
- name: Summary
|
||||
run: |
|
||||
echo "## Examples Integration Test Summary" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
if [ "${{ needs.test-examples.result }}" == "success" ]; then
|
||||
echo "✅ All examples passed!" >> $GITHUB_STEP_SUMMARY
|
||||
else
|
||||
echo "❌ Some examples failed. Check the logs above." >> $GITHUB_STEP_SUMMARY
|
||||
fi
|
||||
@@ -0,0 +1,81 @@
|
||||
# This workflow will upload a Python Package using Twine when a release is created
|
||||
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries
|
||||
|
||||
# To run this workflow manually using GitHub CLI:
|
||||
# Test build only:
|
||||
# gh workflow run "Upload Python Package" --ref main --field test_mode=build-only
|
||||
#
|
||||
# Test with dry run (no actual publish):
|
||||
# gh workflow run "Upload Python Package" --ref main --field test_mode=dry-run
|
||||
#
|
||||
# Test publish to TestPyPI (requires TEST_PYPI_API_TOKEN):
|
||||
# gh workflow run "Upload Python Package" --ref main --field test_mode=testpypi
|
||||
#
|
||||
# To check workflow status:
|
||||
# gh run list --workflow "Upload Python Package"
|
||||
# gh run view <run-id>
|
||||
|
||||
name: Upload Python Package
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
test_mode:
|
||||
description: 'Test mode (dry-run, testpypi, build-only)'
|
||||
required: true
|
||||
default: 'dry-run'
|
||||
type: choice
|
||||
options:
|
||||
- dry-run
|
||||
- testpypi
|
||||
- build-only
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write # Required for trusted publishing
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
environment:
|
||||
name: ${{ github.event_name == 'release' && 'pypi' || 'test' }}
|
||||
url: ${{ github.event_name == 'release' && 'https://pypi.org/p/agentops' || 'https://test.pypi.org/p/agentops' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup UV
|
||||
uses: astral-sh/setup-uv@v5
|
||||
with:
|
||||
python-version: "3.11" # Specify exact version
|
||||
cache-dependency-glob: |
|
||||
**/pyproject.toml
|
||||
**/requirements*.txt
|
||||
|
||||
- name: Build and publish
|
||||
env:
|
||||
PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
|
||||
TEST_PYPI_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
|
||||
run: |
|
||||
uv pip install build
|
||||
uv build
|
||||
|
||||
# Extra safety check
|
||||
if [[ "${{ github.event_name }}" == "workflow_dispatch" && "${{ github.event.inputs.test_mode }}" == "testpypi" ]]; then
|
||||
echo "⚠️ Publishing to TestPyPI..."
|
||||
sleep 5 # Give time to cancel if needed
|
||||
fi
|
||||
|
||||
if [[ "${{ github.event.inputs.test_mode }}" == "dry-run" ]]; then
|
||||
echo "🔍 Performing dry run..."
|
||||
uv publish --check-url https://pypi.org/pypi
|
||||
elif [[ "${{ github.event.inputs.test_mode }}" == "testpypi" ]]; then
|
||||
uv publish --publish-url https://test.pypi.org/legacy/ --token $TEST_PYPI_TOKEN
|
||||
elif [[ "${{ github.event.inputs.test_mode }}" == "build-only" ]]; then
|
||||
echo "✅ Build completed successfully. Skipping publish."
|
||||
elif [[ "${{ github.event_name }}" == "release" ]]; then
|
||||
echo "⚠️ Publishing to PyPI in 10 seconds... (Ctrl+C to cancel)"
|
||||
sleep 10
|
||||
uv publish --token $PYPI_TOKEN
|
||||
fi
|
||||
@@ -0,0 +1,109 @@
|
||||
# :: Use nektos/act to run this locally
|
||||
# :: Example:
|
||||
# :: `act push -j unit-tests --matrix python-version:3.10 --container-architecture linux/amd64`
|
||||
#
|
||||
# This workflow runs two separate test suites:
|
||||
# 1. Unit Tests (python-tests job):
|
||||
# - Runs across Python 3.9 to 3.13
|
||||
# - Located in tests/unit directory
|
||||
# - Coverage report uploaded to Codecov for Python 3.11 only
|
||||
#
|
||||
# 2. Integration Tests (integration-tests job):
|
||||
# - Runs only on Python 3.13
|
||||
# - Located in tests/integration directory
|
||||
# - Longer timeout (15 min vs 10 min for unit tests)
|
||||
# - Separate cache for dependencies
|
||||
|
||||
name: Python Tests
|
||||
on:
|
||||
workflow_dispatch: {}
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- 'agentops/**/*.py'
|
||||
- 'agentops/**/*.ipynb'
|
||||
- 'tests/**/*.py'
|
||||
- 'tests/**/*.ipynb'
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- 'pyproject.toml'
|
||||
- 'agentops/**/*.py'
|
||||
- 'agentops/**/*.ipynb'
|
||||
- 'tests/**/*.py'
|
||||
- 'tests/**/*.ipynb'
|
||||
|
||||
jobs:
|
||||
unit-tests:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
AGENTOPS_API_KEY: ${{ secrets.AGENTOPS_API_KEY }}
|
||||
PYTHONUNBUFFERED: "1"
|
||||
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
|
||||
fail-fast: false
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup UV
|
||||
uses: astral-sh/setup-uv@v5
|
||||
continue-on-error: true
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
enable-cache: true
|
||||
cache-suffix: uv-${{ matrix.python-version }}
|
||||
cache-dependency-glob: "**/pyproject.toml"
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
uv sync --group test --group dev
|
||||
|
||||
- name: Run unit tests with coverage
|
||||
timeout-minutes: 5
|
||||
run: |
|
||||
uv run -m pytest tests/unit -v --cov=agentops --cov-report=xml
|
||||
|
||||
# Only upload coverage report for python3.11
|
||||
- name: Upload coverage to Codecov
|
||||
if: ${{matrix.python-version == '3.11'}}
|
||||
uses: codecov/codecov-action@v5
|
||||
with:
|
||||
token: ${{ secrets.CODECOV_TOKEN }}
|
||||
files: ./coverage.xml
|
||||
flags: unittests
|
||||
name: codecov-umbrella
|
||||
fail_ci_if_error: true # Should we?
|
||||
|
||||
integration-tests:
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
AGENTOPS_API_KEY: ${{ secrets.AGENTOPS_API_KEY }}
|
||||
PYTHONUNBUFFERED: "1"
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup UV
|
||||
uses: astral-sh/setup-uv@v5
|
||||
continue-on-error: true
|
||||
with:
|
||||
python-version: "3.12"
|
||||
enable-cache: true
|
||||
cache-suffix: uv-3.12-integration
|
||||
cache-dependency-glob: "**/pyproject.toml"
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
uv sync --group test --group dev
|
||||
|
||||
- name: Run integration tests
|
||||
timeout-minutes: 5
|
||||
run: |
|
||||
uv run pytest tests/integration
|
||||
@@ -0,0 +1,44 @@
|
||||
name: 'Check for Stale PRs'
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 0 * * *'
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
stale-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/stale@v9
|
||||
with:
|
||||
# Only process pull requests (not issues)
|
||||
days-before-stale: 14
|
||||
days-before-close: 7
|
||||
|
||||
# Only target PRs
|
||||
days-before-issue-stale: -1
|
||||
days-before-issue-close: -1
|
||||
|
||||
# Custom messages
|
||||
stale-pr-message: |
|
||||
This pull request has been automatically marked as stale because it has not had any activity in the last 14 days.
|
||||
|
||||
If no updates are made within 7 days, this PR will be automatically closed.
|
||||
|
||||
close-pr-message: |
|
||||
This pull request has been automatically closed because it has been stale for 7 days with no activity.
|
||||
|
||||
Feel free to reopen this PR if you'd like to continue working on it.
|
||||
|
||||
# Labels
|
||||
stale-pr-label: 'stale'
|
||||
|
||||
# Remove stale label on PR update
|
||||
remove-stale-when-updated: true
|
||||
|
||||
# Don't remove any existing labels
|
||||
labels-to-remove-when-stale: ''
|
||||
labels-to-remove-when-unstale: ''
|
||||
@@ -0,0 +1,51 @@
|
||||
name: Static analysis
|
||||
|
||||
# on PR and push to main
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
paths:
|
||||
- '**/*.py'
|
||||
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
# Limit concurrency by workflow/branch combination.
|
||||
#
|
||||
# For pull request builds, pushing additional changes to the
|
||||
# branch will cancel prior in-progress and pending builds.
|
||||
#
|
||||
# For builds triggered on a branch push, additional changes
|
||||
# will wait for prior builds to complete before starting.
|
||||
#
|
||||
# https://docs.github.com/en/actions/using-jobs/using-concurrency
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
|
||||
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||
|
||||
jobs:
|
||||
pre-commit-checks:
|
||||
name: Pre-commit checks
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- uses: astral-sh/setup-uv@v4
|
||||
with:
|
||||
enable-cache: true
|
||||
cache-dependency-glob: "**/pyproject.toml"
|
||||
python-version: "3.12.2"
|
||||
|
||||
- name: Install packages
|
||||
run: |
|
||||
uv sync --group dev
|
||||
|
||||
- name: Run pre-commit
|
||||
run: |
|
||||
uvx pre-commit run --show-diff-on-failure --color=always --all-files
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
!app/dashboard/lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
share/python-wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
|
||||
# PyInstaller
|
||||
# Usually these files are written by a python script from a template
|
||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
||||
*.manifest
|
||||
*.spec
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
.coverage
|
||||
.coverage.*
|
||||
.cache
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
*.cover
|
||||
*.py,cover
|
||||
.hypothesis/
|
||||
.pytest_cache/
|
||||
cover/
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Django stuff:
|
||||
*.log
|
||||
local_settings.py
|
||||
db.sqlite3
|
||||
db.sqlite3-journal
|
||||
|
||||
# Flask stuff:
|
||||
instance/
|
||||
.webassets-cache
|
||||
|
||||
# Scrapy stuff:
|
||||
.scrapy
|
||||
|
||||
# Sphinx documentation
|
||||
docs/_build/
|
||||
|
||||
# PyBuilder
|
||||
.pybuilder/
|
||||
target/
|
||||
|
||||
# Jupyter Notebook
|
||||
.ipynb_checkpoints
|
||||
|
||||
# IPython
|
||||
profile_default/
|
||||
ipython_config.py
|
||||
|
||||
# pyenv
|
||||
# For a library or package, you might want to ignore these files since the code is
|
||||
# intended to run in multiple environments; otherwise, check them in:
|
||||
.python-version
|
||||
|
||||
# pipenv
|
||||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
||||
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
||||
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
||||
# install all needed dependencies.
|
||||
#Pipfile.lock
|
||||
|
||||
# poetry
|
||||
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
|
||||
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
||||
# commonly ignored for libraries.
|
||||
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
|
||||
#poetry.lock
|
||||
|
||||
# pdm
|
||||
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
|
||||
#pdm.lock
|
||||
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
|
||||
# in version control.
|
||||
# https://pdm.fming.dev/#use-with-ide
|
||||
.pdm.toml
|
||||
|
||||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
|
||||
__pypackages__/
|
||||
|
||||
# Celery stuff
|
||||
celerybeat-schedule
|
||||
celerybeat.pid
|
||||
|
||||
# SageMath parsed files
|
||||
*.sage.py
|
||||
|
||||
# Environments
|
||||
.env
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# Spyder project settings
|
||||
.spyderproject
|
||||
.spyproject
|
||||
|
||||
# Rope project settings
|
||||
.ropeproject
|
||||
|
||||
# mkdocs documentation
|
||||
/site
|
||||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
.dmypy.json
|
||||
dmypy.json
|
||||
|
||||
# Pyre type checker
|
||||
.pyre/
|
||||
|
||||
# pytype static type analyzer
|
||||
.pytype/
|
||||
|
||||
# Cython debug symbols
|
||||
cython_debug/
|
||||
|
||||
# PyCharm
|
||||
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
|
||||
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
|
||||
# and can be added to the global gitignore or merged into this file. For a more nuclear
|
||||
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
||||
.idea/
|
||||
|
||||
# VSCode
|
||||
.vscode/
|
||||
|
||||
# Cursor
|
||||
.cursorrules
|
||||
|
||||
# Benchmarks
|
||||
.benchmarks/
|
||||
|
||||
# MacOS
|
||||
.DS_Store
|
||||
|
||||
# Database
|
||||
.db
|
||||
|
||||
# Time travel
|
||||
agentops_time_travel.json
|
||||
.agentops_time_travel.yaml
|
||||
|
||||
# Node
|
||||
node_modules
|
||||
@@ -0,0 +1,9 @@
|
||||
repos:
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: "v0.12.9"
|
||||
hooks:
|
||||
- id: ruff
|
||||
args: [--fix, --exit-non-zero-on-fix]
|
||||
exclude: ^app/
|
||||
- id: ruff-format
|
||||
exclude: ^app/
|
||||
+390
@@ -0,0 +1,390 @@
|
||||
# Contributing to AgentOps
|
||||
|
||||
Thanks for checking out AgentOps. We're building tools to help developers like you make AI agents that actually work reliably. If you've ever tried to build an agent system, you know the pain - they're a nightmare to debug, impossible to monitor, and when something goes wrong... good luck figuring out why.
|
||||
|
||||
We created AgentOps to solve these headaches, and we'd love your help making it even better. Our SDK hooks into all the major Python frameworks (AG2, CrewAI, LangChain) and LLM providers (OpenAI, Anthropic, Cohere, etc.) to give you visibility into what your agents are actually doing.
|
||||
|
||||
## How You Can Help
|
||||
|
||||
There are tons of ways to contribute, and we genuinely appreciate all of them:
|
||||
|
||||
1. **Add More Providers**: Help us support new LLM providers. Each one helps more developers monitor their agents.
|
||||
2. **Improve Framework Support**: Using a framework we don't support yet? Help us add it!
|
||||
3. **Make Docs Better**: Found our docs confusing? Help us fix them! Clear documentation makes everyone's life easier.
|
||||
4. **Share Your Experience**: Using AgentOps? Let us know what's working and what isn't. Your feedback shapes our roadmap.
|
||||
|
||||
Even if you're not ready to contribute code, we'd love to hear your thoughts. Drop into our Discord, open an issue, or start a discussion. We're building this for developers like you, so your input matters.
|
||||
|
||||
## Table of Contents
|
||||
- [Getting Started](#getting-started)
|
||||
- [Development Environment](#development-environment)
|
||||
- [Testing](#testing)
|
||||
- [Adding LLM Providers](#adding-llm-providers)
|
||||
- [Code Style](#code-style)
|
||||
- [Pull Request Process](#pull-request-process)
|
||||
- [Documentation](#documentation)
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. **Fork and Clone**:
|
||||
First, fork the repository by clicking the 'Fork' button in the top right of the [AgentOps repository](https://github.com/AgentOps-AI/agentops). This creates your own copy of the repository where you can make changes.
|
||||
|
||||
Then clone your fork:
|
||||
```bash
|
||||
git clone https://github.com/YOUR_USERNAME/agentops.git
|
||||
cd agentops
|
||||
```
|
||||
|
||||
Add the upstream repository to stay in sync:
|
||||
```bash
|
||||
git remote add upstream https://github.com/AgentOps-AI/agentops.git
|
||||
git fetch upstream
|
||||
```
|
||||
|
||||
Before starting work on a new feature:
|
||||
```bash
|
||||
git checkout main
|
||||
git pull upstream main
|
||||
git checkout -b feature/your-feature-name
|
||||
```
|
||||
|
||||
2. **Install Dependencies**:
|
||||
```bash
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
3. **Set Up Pre-commit Hooks**:
|
||||
```bash
|
||||
pre-commit install
|
||||
```
|
||||
|
||||
## Development Environment
|
||||
|
||||
1. **Environment Variables**:
|
||||
Create a `.env` file:
|
||||
```
|
||||
AGENTOPS_API_KEY=your_api_key
|
||||
OPENAI_API_KEY=your_openai_key # For testing
|
||||
ANTHROPIC_API_KEY=your_anthropic_key # For testing
|
||||
# Other keys...
|
||||
```
|
||||
|
||||
2. **Virtual Environment**:
|
||||
We recommend using `poetry` or `venv`:
|
||||
```bash
|
||||
python -m venv venv
|
||||
source venv/bin/activate # Unix
|
||||
.\venv\Scripts\activate # Windows
|
||||
```
|
||||
|
||||
3. **Pre-commit Setup**:
|
||||
We use pre-commit hooks to automatically format and lint code. Set them up with:
|
||||
```bash
|
||||
pip install pre-commit
|
||||
pre-commit install
|
||||
```
|
||||
|
||||
That's it! The hooks will run automatically when you commit. To manually check all files:
|
||||
```bash
|
||||
pre-commit run --all-files
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
We use a comprehensive testing stack to ensure code quality and reliability. Our testing framework includes pytest and several specialized testing tools.
|
||||
|
||||
### Testing Dependencies
|
||||
|
||||
Install all testing dependencies:
|
||||
```bash
|
||||
pip install -e ".[dev]"
|
||||
```
|
||||
|
||||
We use the following testing packages:
|
||||
- `pytest==7.4.0`: Core testing framework
|
||||
- `pytest-depends`: Manage test dependencies
|
||||
- `pytest-asyncio`: Test async code
|
||||
- `pytest-vcr`: Record and replay HTTP interactions
|
||||
- `pytest-mock`: Mocking functionality
|
||||
- `pyfakefs`: Mock filesystem operations
|
||||
- `requests_mock==1.11.0`: Mock HTTP requests
|
||||
|
||||
### Using Tox
|
||||
|
||||
We use tox to automate and standardize testing. Tox:
|
||||
- Creates isolated virtual environments for testing
|
||||
- Tests against multiple Python versions (3.7-3.12)
|
||||
- Runs all test suites consistently
|
||||
- Ensures dependencies are correctly specified
|
||||
- Verifies the package installs correctly
|
||||
|
||||
Run tox:
|
||||
```bash
|
||||
tox
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Create fresh virtual environments
|
||||
2. Install dependencies
|
||||
3. Run pytest with our test suite
|
||||
4. Generate coverage reports
|
||||
|
||||
### Running Tests
|
||||
|
||||
1. **Run All Tests**:
|
||||
```bash
|
||||
tox
|
||||
```
|
||||
|
||||
2. **Run Specific Test File**:
|
||||
```bash
|
||||
pytest tests/llms/test_anthropic.py -v
|
||||
```
|
||||
|
||||
3. **Run with Coverage**:
|
||||
```bash
|
||||
coverage run -m pytest
|
||||
coverage report
|
||||
```
|
||||
|
||||
### Writing Tests
|
||||
|
||||
1. **Test Structure**:
|
||||
```python
|
||||
import pytest
|
||||
from pytest_mock import MockerFixture
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
@pytest.mark.asyncio # For async tests
|
||||
async def test_async_function():
|
||||
# Test implementation
|
||||
|
||||
@pytest.mark.depends(on=['test_prerequisite']) # Declare test dependencies
|
||||
def test_dependent_function():
|
||||
# Test implementation
|
||||
```
|
||||
|
||||
2. **Recording HTTP Interactions**:
|
||||
```python
|
||||
@pytest.mark.vcr() # Records HTTP interactions
|
||||
def test_api_call():
|
||||
response = client.make_request()
|
||||
assert response.status_code == 200
|
||||
```
|
||||
|
||||
3. **Mocking Filesystem**:
|
||||
```python
|
||||
def test_file_operations(fs): # fs fixture provided by pyfakefs
|
||||
fs.create_file('/fake/file.txt', contents='test')
|
||||
assert os.path.exists('/fake/file.txt')
|
||||
```
|
||||
|
||||
4. **Mocking HTTP Requests**:
|
||||
```python
|
||||
def test_http_client(requests_mock):
|
||||
requests_mock.get('http://api.example.com', json={'key': 'value'})
|
||||
response = make_request()
|
||||
assert response.json()['key'] == 'value'
|
||||
```
|
||||
|
||||
### Testing Best Practices
|
||||
|
||||
1. **Test Categories**:
|
||||
- Unit tests: Test individual components
|
||||
- Integration tests: Test component interactions
|
||||
- End-to-end tests: Test complete workflows
|
||||
- Performance tests: Test response times and resource usage
|
||||
|
||||
2. **Fixtures**:
|
||||
Create reusable test fixtures in `conftest.py`:
|
||||
```python
|
||||
@pytest.fixture
|
||||
def mock_llm_client():
|
||||
client = Mock()
|
||||
client.chat.completions.create.return_value = Mock()
|
||||
return client
|
||||
```
|
||||
|
||||
3. **Test Data**:
|
||||
- Store test data in `tests/data/`
|
||||
- Use meaningful test data names
|
||||
- Document data format and purpose
|
||||
|
||||
4. **VCR Cassettes**:
|
||||
- Store in `tests/cassettes/`
|
||||
- Sanitize sensitive information
|
||||
- Update cassettes when API changes
|
||||
|
||||
### CI Testing Strategy
|
||||
|
||||
We use Jupyter notebooks as integration tests for LLM providers. This approach:
|
||||
- Tests real-world usage patterns
|
||||
- Verifies end-to-end functionality
|
||||
- Ensures examples stay up-to-date
|
||||
- Tests against actual LLM APIs
|
||||
|
||||
1. **Notebook Tests**:
|
||||
- Located in `examples/` directory
|
||||
- Each LLM provider has example notebooks
|
||||
- CI runs notebooks on PR merges to main
|
||||
- Tests run against multiple Python versions
|
||||
|
||||
2. **Test Workflow**:
|
||||
The `test-notebooks.yml` workflow:
|
||||
```yaml
|
||||
name: Test Notebooks
|
||||
on:
|
||||
pull_request:
|
||||
paths:
|
||||
- "agentops/**"
|
||||
- "examples/**"
|
||||
- "tests/**"
|
||||
```
|
||||
- Runs on PR merges and manual triggers
|
||||
- Sets up environment with provider API keys
|
||||
- Installs AgentOps from main branch
|
||||
- Executes each notebook
|
||||
- Excludes specific notebooks that require manual testing
|
||||
|
||||
3. **Provider Coverage**:
|
||||
Each provider should have notebooks demonstrating:
|
||||
- Basic completion calls
|
||||
- Streaming responses
|
||||
- Async operations (if supported)
|
||||
- Error handling
|
||||
- Tool usage (if applicable)
|
||||
|
||||
4. **Adding Provider Tests**:
|
||||
- Create notebook in `examples/provider_name/`
|
||||
- Include all provider functionality
|
||||
- Add necessary secrets to GitHub Actions
|
||||
- Update `exclude_notebooks` in workflow if manual testing needed
|
||||
|
||||
## Adding LLM Providers
|
||||
|
||||
The `agentops/llms/` directory contains provider implementations. Each provider must:
|
||||
|
||||
1. **Inherit from BaseProvider**:
|
||||
```python
|
||||
@singleton
|
||||
class NewProvider(BaseProvider):
|
||||
def __init__(self, client):
|
||||
super().__init__(client)
|
||||
self._provider_name = "ProviderName"
|
||||
```
|
||||
|
||||
2. **Implement Required Methods**:
|
||||
- `handle_response()`: Process LLM responses
|
||||
- `override()`: Patch the provider's methods
|
||||
- `undo_override()`: Restore original methods
|
||||
|
||||
3. **Handle Events**:
|
||||
Track:
|
||||
- Prompts and completions
|
||||
- Token usage
|
||||
- Timestamps
|
||||
- Errors
|
||||
- Tool usage (if applicable)
|
||||
|
||||
4. **Example Implementation Structure**:
|
||||
```python
|
||||
def handle_response(self, response, kwargs, init_timestamp, session=None):
|
||||
llm_event = LLMEvent(init_timestamp=init_timestamp, params=kwargs)
|
||||
try:
|
||||
# Process response
|
||||
llm_event.returns = response.model_dump()
|
||||
llm_event.prompt = kwargs["messages"]
|
||||
# ... additional processing
|
||||
self._safe_record(session, llm_event)
|
||||
except Exception as e:
|
||||
self._safe_record(session, ErrorEvent(trigger_event=llm_event, exception=e))
|
||||
```
|
||||
|
||||
## Code Style
|
||||
|
||||
1. **Formatting**:
|
||||
- Use Black for Python code formatting
|
||||
- Maximum line length: 88 characters
|
||||
- Use type hints
|
||||
|
||||
2. **Documentation**:
|
||||
- Docstrings for all public methods
|
||||
- Clear inline comments
|
||||
- Update relevant documentation
|
||||
|
||||
3. **Error Handling**:
|
||||
- Use specific exception types
|
||||
- Log errors with meaningful messages
|
||||
- Include context in error messages
|
||||
|
||||
## Pull Request Process
|
||||
|
||||
1. **Branch Naming**:
|
||||
- `feature/description`
|
||||
- `fix/description`
|
||||
- `docs/description`
|
||||
|
||||
2. **Commit Messages**:
|
||||
- Clear and descriptive
|
||||
- Reference issues when applicable
|
||||
|
||||
3. **PR Requirements**:
|
||||
- Pass all tests
|
||||
- Maintain or improve code coverage
|
||||
- Include relevant documentation
|
||||
- Update CHANGELOG.md if applicable
|
||||
|
||||
4. **Review Process**:
|
||||
- At least one approval required
|
||||
- Address all review comments
|
||||
- Maintain PR scope
|
||||
|
||||
## Documentation
|
||||
|
||||
1. **Types of Documentation**:
|
||||
- API reference
|
||||
- Integration guides
|
||||
- Examples
|
||||
- Troubleshooting guides
|
||||
|
||||
2. **Documentation Location**:
|
||||
- Code documentation in docstrings
|
||||
- User guides in `docs/`
|
||||
- Examples in `examples/`
|
||||
|
||||
3. **Documentation Style**:
|
||||
- Clear and concise
|
||||
- Include code examples
|
||||
- Explain the why, not just the what
|
||||
|
||||
## Getting Help & Community
|
||||
|
||||
We encourage active community participation and are here to help!
|
||||
|
||||
### Preferred Communication Channels
|
||||
|
||||
1. **GitHub Issues & Discussions**:
|
||||
- Open an [issue](https://github.com/AgentOps-AI/agentops/issues) for:
|
||||
- Bug reports
|
||||
- Feature requests
|
||||
- Documentation improvements
|
||||
- Start a [discussion](https://github.com/AgentOps-AI/agentops/discussions) for:
|
||||
- Questions about usage
|
||||
- Ideas for new features
|
||||
- Community showcase
|
||||
- General feedback
|
||||
|
||||
2. **Discord Community**:
|
||||
- Join our [Discord server](https://discord.gg/FagdcwwXRR) for:
|
||||
- Real-time help
|
||||
- Community discussions
|
||||
- Feature announcements
|
||||
- Sharing your projects
|
||||
|
||||
3. **Contact Form**:
|
||||
- For private inquiries, use our [contact form](https://agentops.ai/contact)
|
||||
- Please note that public channels are preferred for technical discussions
|
||||
|
||||
## License
|
||||
|
||||
By contributing to AgentOps, you agree that your contributions will be licensed under the MIT License.
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2023 AgentOps-AI
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,862 @@
|
||||
<div align="center">
|
||||
<a href="https://agentops.ai?ref=gh">
|
||||
<img src="docs/images/external/logo/github-banner.png" alt="Logo">
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<em>Observability and DevTool platform for AI Agents</em>
|
||||
</div>
|
||||
|
||||
<br />
|
||||
|
||||
<div align="center">
|
||||
<a href="https://pepy.tech/project/agentops">
|
||||
<img src="https://static.pepy.tech/badge/agentops/month" alt="Downloads">
|
||||
</a>
|
||||
<a href="https://github.com/agentops-ai/agentops/issues">
|
||||
<img src="https://img.shields.io/github/commit-activity/m/agentops-ai/agentops" alt="git commit activity">
|
||||
</a>
|
||||
<img src="https://img.shields.io/pypi/v/agentops?&color=3670A0" alt="PyPI - Version">
|
||||
<a href="https://opensource.org/licenses/MIT">
|
||||
<img src="https://img.shields.io/badge/License-MIT-yellow.svg?&color=3670A0" alt="License: MIT">
|
||||
</a>
|
||||
<a href="https://smithery.ai/server/@AgentOps-AI/agentops-mcp">
|
||||
<img src="https://smithery.ai/badge/@AgentOps-AI/agentops-mcp"/>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://twitter.com/agentopsai/">
|
||||
<img src="https://img.shields.io/twitter/follow/agentopsai?style=social" alt="Twitter" style="height: 20px;">
|
||||
</a>
|
||||
<a href="https://discord.gg/FagdcwwXRR">
|
||||
<img src="https://img.shields.io/badge/discord-7289da.svg?style=flat-square&logo=discord" alt="Discord" style="height: 20px;">
|
||||
</a>
|
||||
<a href="https://app.agentops.ai/?ref=gh">
|
||||
<img src="https://img.shields.io/badge/Dashboard-blue.svg?style=flat-square" alt="Dashboard" style="height: 20px;">
|
||||
</a>
|
||||
<a href="https://docs.agentops.ai/introduction">
|
||||
<img src="https://img.shields.io/badge/Documentation-orange.svg?style=flat-square" alt="Documentation" style="height: 20px;">
|
||||
</a>
|
||||
<a href="https://entelligence.ai/AgentOps-AI&agentops">
|
||||
<img src="https://img.shields.io/badge/Chat%20with%20Docs-green.svg?style=flat-square" alt="Chat with Docs" style="height: 20px;">
|
||||
</a>
|
||||
</p>
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/dfb4fa8d-d8c4-4965-9ff6-5b8514c1c22f" width="650" autoplay loop muted></video>
|
||||
</div>
|
||||
|
||||
<br/>
|
||||
|
||||
AgentOps helps developers build, evaluate, and monitor AI agents. From prototype to production.
|
||||
|
||||
## Open Source
|
||||
|
||||
The AgentOps app is open source under the MIT license. Explore the code in our [app directory](https://github.com/AgentOps-AI/agentops/tree/main/app).
|
||||
|
||||
## Key Integrations 🔌
|
||||
|
||||
<div align="center" style="background-color: white; padding: 20px; border-radius: 10px; margin: 0 auto; max-width: 800px;">
|
||||
<div style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 30px; margin-bottom: 20px;">
|
||||
<a href="https://docs.agentops.ai/v2/integrations/openai_agents_python"><img src="docs/images/external/openai/agents-sdk.svg" height="45" alt="OpenAI Agents SDK"></a>
|
||||
<a href="https://docs.agentops.ai/v1/integrations/crewai"><img src="docs/v1/img/docs-icons/crew-banner.png" height="45" alt="CrewAI"></a>
|
||||
<a href="https://docs.ag2.ai/docs/ecosystem/agentops"><img src="docs/images/external/ag2/ag2-logo.svg" height="45" alt="AG2 (AutoGen)"></a>
|
||||
<a href="https://docs.agentops.ai/v1/integrations/microsoft"><img src="docs/images/external/microsoft/microsoft_logo.svg" height="45" alt="Microsoft"></a>
|
||||
</div>
|
||||
|
||||
<div style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 30px; margin-bottom: 20px;">
|
||||
<a href="https://docs.agentops.ai/v1/integrations/langchain"><img src="docs/images/external/langchain/langchain-logo.svg" height="45" alt="LangChain"></a>
|
||||
<a href="https://docs.agentops.ai/v1/integrations/camel"><img src="docs/images/external/camel/camel.png" height="45" alt="Camel AI"></a>
|
||||
<a href="https://docs.llamaindex.ai/en/stable/module_guides/observability/?h=agentops#agentops"><img src="docs/images/external/ollama/ollama-icon.png" height="45" alt="LlamaIndex"></a>
|
||||
<a href="https://docs.agentops.ai/v1/integrations/cohere"><img src="docs/images/external/cohere/cohere-logo.svg" height="45" alt="Cohere"></a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
| | |
|
||||
| ------------------------------------- | ------------------------------------------------------------- |
|
||||
| 📊 **Replay Analytics and Debugging** | Step-by-step agent execution graphs |
|
||||
| 💸 **LLM Cost Management** | Track spend with LLM foundation model providers |
|
||||
| 🤝 **Framework Integrations** | Native Integrations with CrewAI, AG2 (AutoGen), Agno, LangGraph, & more |
|
||||
| ⚒️ **Self-Host** | Want to run AgentOps on your own cloud? You're covered |
|
||||
|
||||
## Quick Start ⌨️
|
||||
|
||||
```bash
|
||||
pip install agentops
|
||||
```
|
||||
|
||||
|
||||
#### Session replays in 2 lines of code
|
||||
|
||||
Initialize the AgentOps client and automatically get analytics on all your LLM calls.
|
||||
|
||||
[Get an API key](https://app.agentops.ai/settings/projects)
|
||||
|
||||
```python
|
||||
import agentops
|
||||
|
||||
# Beginning of your program (i.e. main.py, __init__.py)
|
||||
agentops.init( < INSERT YOUR API KEY HERE >)
|
||||
|
||||
...
|
||||
|
||||
# End of program
|
||||
agentops.end_session('Success')
|
||||
```
|
||||
|
||||
All your sessions can be viewed on the [AgentOps dashboard](https://app.agentops.ai?ref=gh)
|
||||
<br/>
|
||||
|
||||
## Self-Hosting
|
||||
|
||||
Looking to run the full AgentOps app (Dashboard + API backend) on your machine? Follow the setup guide in `app/README.md`:
|
||||
|
||||
- [Run the App and Backend (Dashboard + API)](app/README.md)
|
||||
|
||||
|
||||
<details>
|
||||
<summary>Agent Debugging</summary>
|
||||
<a href="https://app.agentops.ai?ref=gh">
|
||||
<img src="docs/images/external/app_screenshots/session-drilldown-metadata.png" style="width: 90%;" alt="Agent Metadata"/>
|
||||
</a>
|
||||
<a href="https://app.agentops.ai?ref=gh">
|
||||
<img src="docs/images/external/app_screenshots/chat-viewer.png" style="width: 90%;" alt="Chat Viewer"/>
|
||||
</a>
|
||||
<a href="https://app.agentops.ai?ref=gh">
|
||||
<img src="docs/images/external/app_screenshots/session-drilldown-graphs.png" style="width: 90%;" alt="Event Graphs"/>
|
||||
</a>
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Session Replays</summary>
|
||||
<a href="https://app.agentops.ai?ref=gh">
|
||||
<img src="docs/images/external/app_screenshots/session-replay.png" style="width: 90%;" alt="Session Replays"/>
|
||||
</a>
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Summary Analytics</summary>
|
||||
<a href="https://app.agentops.ai?ref=gh">
|
||||
<img src="docs/images/external/app_screenshots/overview.png" style="width: 90%;" alt="Summary Analytics"/>
|
||||
</a>
|
||||
<a href="https://app.agentops.ai?ref=gh">
|
||||
<img src="docs/images/external/app_screenshots/overview-charts.png" style="width: 90%;" alt="Summary Analytics Charts"/>
|
||||
</a>
|
||||
</details>
|
||||
|
||||
|
||||
### First class Developer Experience
|
||||
Add powerful observability to your agents, tools, and functions with as little code as possible: one line at a time.
|
||||
<br/>
|
||||
Refer to our [documentation](http://docs.agentops.ai)
|
||||
|
||||
```python
|
||||
# Create a session span (root for all other spans)
|
||||
from agentops.sdk.decorators import session
|
||||
|
||||
@session
|
||||
def my_workflow():
|
||||
# Your session code here
|
||||
return result
|
||||
```
|
||||
|
||||
```python
|
||||
# Create an agent span for tracking agent operations
|
||||
from agentops.sdk.decorators import agent
|
||||
|
||||
@agent
|
||||
class MyAgent:
|
||||
def __init__(self, name):
|
||||
self.name = name
|
||||
|
||||
# Agent methods here
|
||||
```
|
||||
|
||||
```python
|
||||
# Create operation/task spans for tracking specific operations
|
||||
from agentops.sdk.decorators import operation, task
|
||||
|
||||
@operation # or @task
|
||||
def process_data(data):
|
||||
# Process the data
|
||||
return result
|
||||
```
|
||||
|
||||
```python
|
||||
# Create workflow spans for tracking multi-operation workflows
|
||||
from agentops.sdk.decorators import workflow
|
||||
|
||||
@workflow
|
||||
def my_workflow(data):
|
||||
# Workflow implementation
|
||||
return result
|
||||
```
|
||||
|
||||
```python
|
||||
# Nest decorators for proper span hierarchy
|
||||
from agentops.sdk.decorators import session, agent, operation
|
||||
|
||||
@agent
|
||||
class MyAgent:
|
||||
@operation
|
||||
def nested_operation(self, message):
|
||||
return f"Processed: {message}"
|
||||
|
||||
@operation
|
||||
def main_operation(self):
|
||||
result = self.nested_operation("test message")
|
||||
return result
|
||||
|
||||
@session
|
||||
def my_session():
|
||||
agent = MyAgent()
|
||||
return agent.main_operation()
|
||||
```
|
||||
|
||||
All decorators support:
|
||||
- Input/Output Recording
|
||||
- Exception Handling
|
||||
- Async/await functions
|
||||
- Generator functions
|
||||
- Custom attributes and names
|
||||
|
||||
## Integrations 🦾
|
||||
|
||||
### OpenAI Agents SDK 🖇️
|
||||
|
||||
Build multi-agent systems with tools, handoffs, and guardrails. AgentOps natively integrates with the OpenAI Agents SDKs for both Python and TypeScript.
|
||||
|
||||
#### Python
|
||||
|
||||
```bash
|
||||
pip install openai-agents
|
||||
```
|
||||
|
||||
- [Python integration guide](https://docs.agentops.ai/v2/integrations/openai_agents_python)
|
||||
- [OpenAI Agents Python documentation](https://openai.github.io/openai-agents-python/)
|
||||
|
||||
#### TypeScript
|
||||
|
||||
```bash
|
||||
npm install agentops @openai/agents
|
||||
```
|
||||
|
||||
- [TypeScript integration guide](https://docs.agentops.ai/v2/integrations/openai_agents_js)
|
||||
- [OpenAI Agents JS documentation](https://openai.github.io/openai-agents-js)
|
||||
|
||||
### CrewAI 🛶
|
||||
|
||||
Build Crew agents with observability in just 2 lines of code. Simply set an `AGENTOPS_API_KEY` in your environment, and your crews will get automatic monitoring on the AgentOps dashboard.
|
||||
|
||||
```bash
|
||||
pip install 'crewai[agentops]'
|
||||
```
|
||||
|
||||
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/crewai)
|
||||
- [Official CrewAI documentation](https://docs.crewai.com/how-to/AgentOps-Observability)
|
||||
|
||||
### AG2 🤖
|
||||
With only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an `AGENTOPS_API_KEY` in your environment and call `agentops.init()`
|
||||
|
||||
- [AG2 Observability Example](https://github.com/ag2ai/ag2/blob/main/notebook/agentchat_agentops.ipynb)
|
||||
- [AG2 - AgentOps Documentation](https://docs.ag2.ai/latest/docs/ecosystem/agentops/)
|
||||
|
||||
### Camel AI 🐪
|
||||
|
||||
Track and analyze CAMEL agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.
|
||||
|
||||
- [Camel AI](https://www.camel-ai.org/) - Advanced agent communication framework
|
||||
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/camel)
|
||||
- [Official Camel AI documentation](https://docs.camel-ai.org/cookbooks/agents_tracking.html)
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```bash
|
||||
pip install "camel-ai[all]==0.2.11"
|
||||
pip install agentops
|
||||
```
|
||||
|
||||
```python
|
||||
import os
|
||||
import agentops
|
||||
from camel.agents import ChatAgent
|
||||
from camel.messages import BaseMessage
|
||||
from camel.models import ModelFactory
|
||||
from camel.types import ModelPlatformType, ModelType
|
||||
|
||||
# Initialize AgentOps
|
||||
agentops.init(os.getenv("AGENTOPS_API_KEY"), tags=["CAMEL Example"])
|
||||
|
||||
# Import toolkits after AgentOps init for tracking
|
||||
from camel.toolkits import SearchToolkit
|
||||
|
||||
# Set up the agent with search tools
|
||||
sys_msg = BaseMessage.make_assistant_message(
|
||||
role_name='Tools calling operator',
|
||||
content='You are a helpful assistant'
|
||||
)
|
||||
|
||||
# Configure tools and model
|
||||
tools = [*SearchToolkit().get_tools()]
|
||||
model = ModelFactory.create(
|
||||
model_platform=ModelPlatformType.OPENAI,
|
||||
model_type=ModelType.GPT_4O_MINI,
|
||||
)
|
||||
|
||||
# Create and run the agent
|
||||
camel_agent = ChatAgent(
|
||||
system_message=sys_msg,
|
||||
model=model,
|
||||
tools=tools,
|
||||
)
|
||||
|
||||
response = camel_agent.step("What is AgentOps?")
|
||||
print(response)
|
||||
|
||||
agentops.end_session("Success")
|
||||
```
|
||||
|
||||
Check out our [Camel integration guide](https://docs.agentops.ai/v1/integrations/camel) for more examples including multi-agent scenarios.
|
||||
</details>
|
||||
|
||||
### Langchain 🦜🔗
|
||||
|
||||
AgentOps works seamlessly with applications built using Langchain. To use the handler, install Langchain as an optional dependency:
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```shell
|
||||
pip install agentops[langchain]
|
||||
```
|
||||
|
||||
To use the handler, import and set
|
||||
|
||||
```python
|
||||
import os
|
||||
from langchain.chat_models import ChatOpenAI
|
||||
from langchain.agents import initialize_agent, AgentType
|
||||
from agentops.integration.callbacks.langchain import LangchainCallbackHandler
|
||||
|
||||
AGENTOPS_API_KEY = os.environ['AGENTOPS_API_KEY']
|
||||
handler = LangchainCallbackHandler(api_key=AGENTOPS_API_KEY, tags=['Langchain Example'])
|
||||
|
||||
llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY,
|
||||
callbacks=[handler],
|
||||
model='gpt-3.5-turbo')
|
||||
|
||||
agent = initialize_agent(tools,
|
||||
llm,
|
||||
agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
|
||||
verbose=True,
|
||||
callbacks=[handler], # You must pass in a callback handler to record your agent
|
||||
handle_parsing_errors=True)
|
||||
```
|
||||
|
||||
Check out the [Langchain Examples Notebook](./examples/langchain/langchain_examples.ipynb) for more details including Async handlers.
|
||||
|
||||
</details>
|
||||
|
||||
### Cohere ⌨️
|
||||
|
||||
First class support for Cohere(>=5.4.0). This is a living integration, should you need any added functionality please message us on Discord!
|
||||
|
||||
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/cohere)
|
||||
- [Official Cohere documentation](https://docs.cohere.com/reference/about)
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```bash
|
||||
pip install cohere
|
||||
```
|
||||
|
||||
```python python
|
||||
import cohere
|
||||
import agentops
|
||||
|
||||
# Beginning of program's code (i.e. main.py, __init__.py)
|
||||
agentops.init(<INSERT YOUR API KEY HERE>)
|
||||
co = cohere.Client()
|
||||
|
||||
chat = co.chat(
|
||||
message="Is it pronounced ceaux-hear or co-hehray?"
|
||||
)
|
||||
|
||||
print(chat)
|
||||
|
||||
agentops.end_session('Success')
|
||||
```
|
||||
|
||||
```python python
|
||||
import cohere
|
||||
import agentops
|
||||
|
||||
# Beginning of program's code (i.e. main.py, __init__.py)
|
||||
agentops.init(<INSERT YOUR API KEY HERE>)
|
||||
|
||||
co = cohere.Client()
|
||||
|
||||
stream = co.chat_stream(
|
||||
message="Write me a haiku about the synergies between Cohere and AgentOps"
|
||||
)
|
||||
|
||||
for event in stream:
|
||||
if event.event_type == "text-generation":
|
||||
print(event.text, end='')
|
||||
|
||||
agentops.end_session('Success')
|
||||
```
|
||||
</details>
|
||||
|
||||
|
||||
### Anthropic ﹨
|
||||
|
||||
Track agents built with the Anthropic Python SDK (>=0.32.0).
|
||||
|
||||
- [AgentOps integration guide](https://docs.agentops.ai/v1/integrations/anthropic)
|
||||
- [Official Anthropic documentation](https://docs.anthropic.com/en/docs/welcome)
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```bash
|
||||
pip install anthropic
|
||||
```
|
||||
|
||||
```python python
|
||||
import anthropic
|
||||
import agentops
|
||||
|
||||
# Beginning of program's code (i.e. main.py, __init__.py)
|
||||
agentops.init(<INSERT YOUR API KEY HERE>)
|
||||
|
||||
client = anthropic.Anthropic(
|
||||
# This is the default and can be omitted
|
||||
api_key=os.environ.get("ANTHROPIC_API_KEY"),
|
||||
)
|
||||
|
||||
message = client.messages.create(
|
||||
max_tokens=1024,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Tell me a cool fact about AgentOps",
|
||||
}
|
||||
],
|
||||
model="claude-3-opus-20240229",
|
||||
)
|
||||
print(message.content)
|
||||
|
||||
agentops.end_session('Success')
|
||||
```
|
||||
|
||||
Streaming
|
||||
```python python
|
||||
import anthropic
|
||||
import agentops
|
||||
|
||||
# Beginning of program's code (i.e. main.py, __init__.py)
|
||||
agentops.init(<INSERT YOUR API KEY HERE>)
|
||||
|
||||
client = anthropic.Anthropic(
|
||||
# This is the default and can be omitted
|
||||
api_key=os.environ.get("ANTHROPIC_API_KEY"),
|
||||
)
|
||||
|
||||
stream = client.messages.create(
|
||||
max_tokens=1024,
|
||||
model="claude-3-opus-20240229",
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Tell me something cool about streaming agents",
|
||||
}
|
||||
],
|
||||
stream=True,
|
||||
)
|
||||
|
||||
response = ""
|
||||
for event in stream:
|
||||
if event.type == "content_block_delta":
|
||||
response += event.delta.text
|
||||
elif event.type == "message_stop":
|
||||
print("\n")
|
||||
print(response)
|
||||
print("\n")
|
||||
```
|
||||
|
||||
Async
|
||||
|
||||
```python python
|
||||
import asyncio
|
||||
from anthropic import AsyncAnthropic
|
||||
|
||||
client = AsyncAnthropic(
|
||||
# This is the default and can be omitted
|
||||
api_key=os.environ.get("ANTHROPIC_API_KEY"),
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
message = await client.messages.create(
|
||||
max_tokens=1024,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Tell me something interesting about async agents",
|
||||
}
|
||||
],
|
||||
model="claude-3-opus-20240229",
|
||||
)
|
||||
print(message.content)
|
||||
|
||||
|
||||
await main()
|
||||
```
|
||||
</details>
|
||||
|
||||
### Mistral 〽️
|
||||
|
||||
Track agents built with the Mistral Python SDK (>=0.32.0).
|
||||
|
||||
- [AgentOps integration example](./examples/mistral//mistral_example.ipynb)
|
||||
- [Official Mistral documentation](https://docs.mistral.ai)
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```bash
|
||||
pip install mistralai
|
||||
```
|
||||
|
||||
Sync
|
||||
|
||||
```python python
|
||||
from mistralai import Mistral
|
||||
import agentops
|
||||
|
||||
# Beginning of program's code (i.e. main.py, __init__.py)
|
||||
agentops.init(<INSERT YOUR API KEY HERE>)
|
||||
|
||||
client = Mistral(
|
||||
# This is the default and can be omitted
|
||||
api_key=os.environ.get("MISTRAL_API_KEY"),
|
||||
)
|
||||
|
||||
message = client.chat.complete(
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Tell me a cool fact about AgentOps",
|
||||
}
|
||||
],
|
||||
model="open-mistral-nemo",
|
||||
)
|
||||
print(message.choices[0].message.content)
|
||||
|
||||
agentops.end_session('Success')
|
||||
```
|
||||
|
||||
Streaming
|
||||
|
||||
```python python
|
||||
from mistralai import Mistral
|
||||
import agentops
|
||||
|
||||
# Beginning of program's code (i.e. main.py, __init__.py)
|
||||
agentops.init(<INSERT YOUR API KEY HERE>)
|
||||
|
||||
client = Mistral(
|
||||
# This is the default and can be omitted
|
||||
api_key=os.environ.get("MISTRAL_API_KEY"),
|
||||
)
|
||||
|
||||
message = client.chat.stream(
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Tell me something cool about streaming agents",
|
||||
}
|
||||
],
|
||||
model="open-mistral-nemo",
|
||||
)
|
||||
|
||||
response = ""
|
||||
for event in message:
|
||||
if event.data.choices[0].finish_reason == "stop":
|
||||
print("\n")
|
||||
print(response)
|
||||
print("\n")
|
||||
else:
|
||||
response += event.text
|
||||
|
||||
agentops.end_session('Success')
|
||||
```
|
||||
|
||||
Async
|
||||
|
||||
```python python
|
||||
import asyncio
|
||||
from mistralai import Mistral
|
||||
|
||||
client = Mistral(
|
||||
# This is the default and can be omitted
|
||||
api_key=os.environ.get("MISTRAL_API_KEY"),
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
message = await client.chat.complete_async(
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Tell me something interesting about async agents",
|
||||
}
|
||||
],
|
||||
model="open-mistral-nemo",
|
||||
)
|
||||
print(message.choices[0].message.content)
|
||||
|
||||
|
||||
await main()
|
||||
```
|
||||
|
||||
Async Streaming
|
||||
|
||||
```python python
|
||||
import asyncio
|
||||
from mistralai import Mistral
|
||||
|
||||
client = Mistral(
|
||||
# This is the default and can be omitted
|
||||
api_key=os.environ.get("MISTRAL_API_KEY"),
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
message = await client.chat.stream_async(
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Tell me something interesting about async streaming agents",
|
||||
}
|
||||
],
|
||||
model="open-mistral-nemo",
|
||||
)
|
||||
|
||||
response = ""
|
||||
async for event in message:
|
||||
if event.data.choices[0].finish_reason == "stop":
|
||||
print("\n")
|
||||
print(response)
|
||||
print("\n")
|
||||
else:
|
||||
response += event.text
|
||||
|
||||
|
||||
await main()
|
||||
```
|
||||
</details>
|
||||
|
||||
|
||||
|
||||
### CamelAI ﹨
|
||||
|
||||
Track agents built with the CamelAI Python SDK (>=0.32.0).
|
||||
|
||||
- [CamelAI integration guide](https://docs.camel-ai.org/cookbooks/agents_tracking.html#)
|
||||
- [Official CamelAI documentation](https://docs.camel-ai.org/index.html)
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```bash
|
||||
pip install camel-ai[all]
|
||||
pip install agentops
|
||||
```
|
||||
|
||||
```python python
|
||||
#Import Dependencies
|
||||
import agentops
|
||||
import os
|
||||
from getpass import getpass
|
||||
from dotenv import load_dotenv
|
||||
|
||||
#Set Keys
|
||||
load_dotenv()
|
||||
openai_api_key = os.getenv("OPENAI_API_KEY") or "<your openai key here>"
|
||||
agentops_api_key = os.getenv("AGENTOPS_API_KEY") or "<your agentops key here>"
|
||||
|
||||
|
||||
|
||||
```
|
||||
</details>
|
||||
|
||||
[You can find usage examples here!](examples/camelai_examples/README.md).
|
||||
|
||||
|
||||
|
||||
### LiteLLM 🚅
|
||||
|
||||
AgentOps provides support for LiteLLM(>=1.3.1), allowing you to call 100+ LLMs using the same Input/Output Format.
|
||||
|
||||
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/litellm)
|
||||
- [Official LiteLLM documentation](https://docs.litellm.ai/docs/providers)
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```bash
|
||||
pip install litellm
|
||||
```
|
||||
|
||||
```python python
|
||||
# Do not use LiteLLM like this
|
||||
# from litellm import completion
|
||||
# ...
|
||||
# response = completion(model="claude-3", messages=messages)
|
||||
|
||||
# Use LiteLLM like this
|
||||
import litellm
|
||||
...
|
||||
response = litellm.completion(model="claude-3", messages=messages)
|
||||
# or
|
||||
response = await litellm.acompletion(model="claude-3", messages=messages)
|
||||
```
|
||||
</details>
|
||||
|
||||
### LlamaIndex 🦙
|
||||
|
||||
|
||||
AgentOps works seamlessly with applications built using LlamaIndex, a framework for building context-augmented generative AI applications with LLMs.
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```shell
|
||||
pip install llama-index-instrumentation-agentops
|
||||
```
|
||||
|
||||
To use the handler, import and set
|
||||
|
||||
```python
|
||||
from llama_index.core import set_global_handler
|
||||
|
||||
# NOTE: Feel free to set your AgentOps environment variables (e.g., 'AGENTOPS_API_KEY')
|
||||
# as outlined in the AgentOps documentation, or pass the equivalent keyword arguments
|
||||
# anticipated by AgentOps' AOClient as **eval_params in set_global_handler.
|
||||
|
||||
set_global_handler("agentops")
|
||||
```
|
||||
|
||||
Check out the [LlamaIndex docs](https://docs.llamaindex.ai/en/stable/module_guides/observability/?h=agentops#agentops) for more details.
|
||||
|
||||
</details>
|
||||
|
||||
### Llama Stack 🦙🥞
|
||||
|
||||
AgentOps provides support for Llama Stack Python Client(>=0.0.53), allowing you to monitor your Agentic applications.
|
||||
|
||||
- [AgentOps integration example 1](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-fdddf65549f3714f8f007ce7dfd1cde720329fe54155d54389dd50fbd81813cb)
|
||||
- [AgentOps integration example 2](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-6688ff4fb7ab1ce7b1cc9b8362ca27264a3060c16737fb1d850305787a6e3699)
|
||||
- [Official Llama Stack Python Client](https://github.com/meta-llama/llama-stack-client-python)
|
||||
|
||||
### SwarmZero AI 🐝
|
||||
|
||||
Track and analyze SwarmZero agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.
|
||||
|
||||
- [SwarmZero](https://swarmzero.ai) - Advanced multi-agent framework
|
||||
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/swarmzero)
|
||||
- [SwarmZero AI integration example](https://docs.swarmzero.ai/examples/ai-agents/build-and-monitor-a-web-search-agent)
|
||||
- [SwarmZero AI - AgentOps documentation](https://docs.swarmzero.ai/sdk/observability/agentops)
|
||||
- [Official SwarmZero Python SDK](https://github.com/swarmzero/swarmzero)
|
||||
|
||||
<details>
|
||||
<summary>Installation</summary>
|
||||
|
||||
```bash
|
||||
pip install swarmzero
|
||||
pip install agentops
|
||||
```
|
||||
|
||||
```python
|
||||
from dotenv import load_dotenv
|
||||
load_dotenv()
|
||||
|
||||
import agentops
|
||||
agentops.init(<INSERT YOUR API KEY HERE>)
|
||||
|
||||
from swarmzero import Agent, Swarm
|
||||
# ...
|
||||
```
|
||||
</details>
|
||||
|
||||
## Evaluations Roadmap 🧭
|
||||
|
||||
| Platform | Dashboard | Evals |
|
||||
| ---------------------------------------------------------------------------- | ------------------------------------------ | -------------------------------------- |
|
||||
| ✅ Python SDK | ✅ Multi-session and Cross-session metrics | ✅ Custom eval metrics |
|
||||
| 🚧 Evaluation builder API | ✅ Custom event tag tracking | 🔜 Agent scorecards |
|
||||
| 🚧 [Javascript/Typescript SDK (Alpha)](https://github.com/AgentOps-AI/agentops-node) | ✅ Session replays | 🔜 Evaluation playground + leaderboard |
|
||||
|
||||
## Debugging Roadmap 🧭
|
||||
|
||||
| Performance testing | Environments | LLM Testing | Reasoning and execution testing |
|
||||
| ----------------------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------- |
|
||||
| ✅ Event latency analysis | 🔜 Non-stationary environment testing | 🔜 LLM non-deterministic function detection | 🚧 Infinite loops and recursive thought detection |
|
||||
| ✅ Agent workflow execution pricing | 🔜 Multi-modal environments | 🚧 Token limit overflow flags | 🔜 Faulty reasoning detection |
|
||||
| 🚧 Success validators (external) | 🔜 Execution containers | 🔜 Context limit overflow flags | 🔜 Generative code validators |
|
||||
| 🔜 Agent controllers/skill tests | ✅ Honeypot and prompt injection detection ([PromptArmor](https://promptarmor.com)) | ✅ API bill tracking | 🔜 Error breakpoint analysis |
|
||||
| 🔜 Information context constraint testing | 🔜 Anti-agent roadblocks (i.e. Captchas) | 🔜 CI/CD integration checks | |
|
||||
| 🔜 Regression testing | ✅ Multi-agent framework visualization | | |
|
||||
|
||||
### Why AgentOps? 🤔
|
||||
|
||||
Without the right tools, AI agents are slow, expensive, and unreliable. Our mission is to bring your agent from prototype to production. Here's why AgentOps stands out:
|
||||
|
||||
- **Comprehensive Observability**: Track your AI agents' performance, user interactions, and API usage.
|
||||
- **Real-Time Monitoring**: Get instant insights with session replays, metrics, and live monitoring tools.
|
||||
- **Cost Control**: Monitor and manage your spend on LLM and API calls.
|
||||
- **Failure Detection**: Quickly identify and respond to agent failures and multi-agent interaction issues.
|
||||
- **Tool Usage Statistics**: Understand how your agents utilize external tools with detailed analytics.
|
||||
- **Session-Wide Metrics**: Gain a holistic view of your agents' sessions with comprehensive statistics.
|
||||
|
||||
AgentOps is designed to make agent observability, testing, and monitoring easy.
|
||||
|
||||
|
||||
## Star History
|
||||
|
||||
Check out our growth in the community:
|
||||
|
||||
<img src="https://api.star-history.com/svg?repos=AgentOps-AI/agentops&type=Date" style="max-width: 500px" width="50%" alt="Logo">
|
||||
|
||||
## Popular projects using AgentOps
|
||||
|
||||
|
||||
| Repository | Stars |
|
||||
| :-------- | -----: |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/2707039?s=40&v=4" width="20" height="20" alt=""> [geekan](https://github.com/geekan) / [MetaGPT](https://github.com/geekan/MetaGPT) | 42787 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/130722866?s=40&v=4" width="20" height="20" alt=""> [run-llama](https://github.com/run-llama) / [llama_index](https://github.com/run-llama/llama_index) | 34446 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/170677839?s=40&v=4" width="20" height="20" alt=""> [crewAIInc](https://github.com/crewAIInc) / [crewAI](https://github.com/crewAIInc/crewAI) | 18287 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/134388954?s=40&v=4" width="20" height="20" alt=""> [camel-ai](https://github.com/camel-ai) / [camel](https://github.com/camel-ai/camel) | 5166 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/152537519?s=40&v=4" width="20" height="20" alt=""> [superagent-ai](https://github.com/superagent-ai) / [superagent](https://github.com/superagent-ai/superagent) | 5050 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/30197649?s=40&v=4" width="20" height="20" alt=""> [iyaja](https://github.com/iyaja) / [llama-fs](https://github.com/iyaja/llama-fs) | 4713 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/188122941?s=40&v=4" width="20" height="20" alt=""> [ag2ai](https://github.com/ag2ai) / [ag2](https://github.com/ag2ai/ag2) | 4240 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/162546372?s=40&v=4" width="20" height="20" alt=""> [BasedHardware](https://github.com/BasedHardware) / [Omi](https://github.com/BasedHardware/Omi) | 2723 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/454862?s=40&v=4" width="20" height="20" alt=""> [MervinPraison](https://github.com/MervinPraison) / [PraisonAI](https://github.com/MervinPraison/PraisonAI) | 2007 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/140554352?s=40&v=4" width="20" height="20" alt=""> [AgentOps-AI](https://github.com/AgentOps-AI) / [Jaiqu](https://github.com/AgentOps-AI/Jaiqu) | 272 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/173542722?s=48&v=4" width="20" height="20" alt=""> [swarmzero](https://github.com/swarmzero) / [swarmzero](https://github.com/swarmzero/swarmzero) | 195 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/3074263?s=40&v=4" width="20" height="20" alt=""> [strnad](https://github.com/strnad) / [CrewAI-Studio](https://github.com/strnad/CrewAI-Studio) | 134 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/18406448?s=40&v=4" width="20" height="20" alt=""> [alejandro-ao](https://github.com/alejandro-ao) / [exa-crewai](https://github.com/alejandro-ao/exa-crewai) | 55 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/64493665?s=40&v=4" width="20" height="20" alt=""> [tonykipkemboi](https://github.com/tonykipkemboi) / [youtube_yapper_trapper](https://github.com/tonykipkemboi/youtube_yapper_trapper) | 47 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/17598928?s=40&v=4" width="20" height="20" alt=""> [sethcoast](https://github.com/sethcoast) / [cover-letter-builder](https://github.com/sethcoast/cover-letter-builder) | 27 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/109994880?s=40&v=4" width="20" height="20" alt=""> [bhancockio](https://github.com/bhancockio) / [chatgpt4o-analysis](https://github.com/bhancockio/chatgpt4o-analysis) | 19 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/14105911?s=40&v=4" width="20" height="20" alt=""> [breakstring](https://github.com/breakstring) / [Agentic_Story_Book_Workflow](https://github.com/breakstring/Agentic_Story_Book_Workflow) | 14 |
|
||||
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/124134656?s=40&v=4" width="20" height="20" alt=""> [MULTI-ON](https://github.com/MULTI-ON) / [multion-python](https://github.com/MULTI-ON/multion-python) | 13 |
|
||||
|
||||
|
||||
_Generated using [github-dependents-info](https://github.com/nvuillam/github-dependents-info), by [Nicolas Vuillamy](https://github.com/nvuillam)_
|
||||
@@ -0,0 +1,5 @@
|
||||
#### Disclaimer on Error Reporting
|
||||
|
||||
_By implementing this SDK, you acknowledge and agree that the SDK may automatically send error reports to our servers. These reports are strictly limited to information relevant for identifying and resolving issues, such as error messages, stack traces (including file paths that may include your system username), and the SDK version. Our primary aim in collecting this data is to enhance the developer experience by identifying, diagnosing, and fixing issues more efficiently. No personal information or sensitive data will be transmitted as part of these reports without your explicit consent._
|
||||
|
||||
_This collection is in compliance with the California Consumer Privacy Act (CCPA). We commit to not selling your Personal Information and ensuring its confidentiality and security. California residents have the right to know about, access, and delete their Personal Information collected by us. For details on how to exercise these rights, please refer to our [Privacy Policy](https://docs.google.com/document/d/1SjWwa8nhHFLXa0ytSHv5F7osfNDclA6GU9-RvTevGDs/edit?usp=sharing). Our policy also includes information on how to opt-out of the sale of Personal Information, though we do not engage in this practice._
|
||||
@@ -0,0 +1,488 @@
|
||||
# For backwards compatibility
|
||||
from agentops.legacy import (
|
||||
start_session,
|
||||
end_session,
|
||||
track_agent,
|
||||
track_tool,
|
||||
end_all_sessions,
|
||||
Session,
|
||||
ToolEvent,
|
||||
ErrorEvent,
|
||||
ActionEvent,
|
||||
LLMEvent,
|
||||
) # type: ignore
|
||||
|
||||
# Import all required modules at the top
|
||||
from opentelemetry.trace import get_current_span
|
||||
from agentops.semconv import (
|
||||
AgentAttributes,
|
||||
ToolAttributes,
|
||||
WorkflowAttributes,
|
||||
CoreAttributes,
|
||||
SpanKind,
|
||||
SpanAttributes,
|
||||
)
|
||||
import json
|
||||
from typing import List, Optional, Union, Dict, Any
|
||||
from agentops.client import Client
|
||||
from agentops.sdk.core import TraceContext, tracer
|
||||
from agentops.sdk.decorators import trace, session, agent, task, workflow, operation, tool, guardrail, track_endpoint
|
||||
from agentops.enums import TraceState, SUCCESS, ERROR, UNSET
|
||||
from opentelemetry.trace.status import StatusCode
|
||||
|
||||
from agentops.logging.config import logger
|
||||
from agentops.helpers.deprecation import deprecated, warn_deprecated_param
|
||||
import threading
|
||||
|
||||
# Import validation functions
|
||||
from agentops.validation import validate_trace_spans, print_validation_summary, ValidationError
|
||||
|
||||
# Thread-safe client management
|
||||
_client_lock = threading.Lock()
|
||||
_client = None
|
||||
|
||||
|
||||
def get_client() -> Client:
|
||||
"""Get the singleton client instance in a thread-safe manner"""
|
||||
global _client
|
||||
|
||||
# Double-checked locking pattern for thread safety
|
||||
if _client is None:
|
||||
with _client_lock:
|
||||
if _client is None:
|
||||
_client = Client()
|
||||
|
||||
return _client
|
||||
|
||||
|
||||
@deprecated("Automatically tracked in v4.")
|
||||
def record(event):
|
||||
"""
|
||||
Legacy function to record an event. This is kept for backward compatibility.
|
||||
|
||||
In the current version, this simply sets the end_timestamp on the event.
|
||||
|
||||
Args:
|
||||
event: The event to record
|
||||
"""
|
||||
from agentops.helpers.time import get_ISO_time
|
||||
|
||||
# TODO: Manual timestamp assignment is a temporary fix; should use proper event lifecycle
|
||||
if event and hasattr(event, "end_timestamp"):
|
||||
event.end_timestamp = get_ISO_time()
|
||||
|
||||
return event
|
||||
|
||||
|
||||
def init(
|
||||
api_key: Optional[str] = None,
|
||||
endpoint: Optional[str] = None,
|
||||
app_url: Optional[str] = None,
|
||||
max_wait_time: Optional[int] = None,
|
||||
max_queue_size: Optional[int] = None,
|
||||
tags: Optional[List[str]] = None,
|
||||
default_tags: Optional[List[str]] = None,
|
||||
trace_name: Optional[str] = None,
|
||||
instrument_llm_calls: Optional[bool] = None,
|
||||
auto_start_session: Optional[bool] = None,
|
||||
auto_init: Optional[bool] = None,
|
||||
skip_auto_end_session: Optional[bool] = None,
|
||||
env_data_opt_out: Optional[bool] = None,
|
||||
log_level: Optional[Union[str, int]] = None,
|
||||
fail_safe: Optional[bool] = None,
|
||||
log_session_replay_url: Optional[bool] = None,
|
||||
exporter_endpoint: Optional[str] = None,
|
||||
**kwargs,
|
||||
):
|
||||
"""
|
||||
Initializes the AgentOps SDK.
|
||||
|
||||
Args:
|
||||
api_key (str, optional): API Key for AgentOps services. If none is provided, key will
|
||||
be read from the AGENTOPS_API_KEY environment variable.
|
||||
endpoint (str, optional): The endpoint for the AgentOps service. If none is provided, key will
|
||||
be read from the AGENTOPS_API_ENDPOINT environment variable. Defaults to 'https://api.agentops.ai'.
|
||||
app_url (str, optional): The dashboard URL for the AgentOps app. If none is provided, key will
|
||||
be read from the AGENTOPS_APP_URL environment variable. Defaults to 'https://app.agentops.ai'.
|
||||
max_wait_time (int, optional): The maximum time to wait in milliseconds before flushing the queue.
|
||||
Defaults to 5,000 (5 seconds)
|
||||
max_queue_size (int, optional): The maximum size of the event queue. Defaults to 512.
|
||||
tags (List[str], optional): [Deprecated] Use `default_tags` instead.
|
||||
default_tags (List[str], optional): Default tags for the sessions that can be used for grouping or sorting later (e.g. ["GPT-4"]).
|
||||
trace_name (str, optional): Name for the default trace/session. If none is provided, defaults to "default".
|
||||
instrument_llm_calls (bool): Whether to instrument LLM calls and emit LLMEvents.
|
||||
auto_start_session (bool): Whether to start a session automatically when the client is created.
|
||||
auto_init (bool): Whether to automatically initialize the client on import. Defaults to True.
|
||||
skip_auto_end_session (optional, bool): Don't automatically end session based on your framework's decision-making
|
||||
(i.e. Crew determining when tasks are complete and ending the session)
|
||||
env_data_opt_out (bool): Whether to opt out of collecting environment data.
|
||||
log_level (str, int): The log level to use for the client. Defaults to 'CRITICAL'.
|
||||
fail_safe (bool): Whether to suppress errors and continue execution when possible.
|
||||
log_session_replay_url (bool): Whether to log session replay URLs to the console. Defaults to True.
|
||||
exporter_endpoint (str, optional): Endpoint for the exporter. If none is provided, key will
|
||||
be read from the AGENTOPS_EXPORTER_ENDPOINT environment variable.
|
||||
**kwargs: Additional configuration parameters to be passed to the client.
|
||||
"""
|
||||
global _client
|
||||
|
||||
# Check for deprecated parameters and emit warnings
|
||||
if tags is not None:
|
||||
warn_deprecated_param("tags", "default_tags")
|
||||
|
||||
# Merge tags and default_tags if both are provided
|
||||
merged_tags = None
|
||||
if tags and default_tags:
|
||||
merged_tags = list(set(tags + default_tags))
|
||||
elif tags:
|
||||
merged_tags = tags
|
||||
elif default_tags:
|
||||
merged_tags = default_tags
|
||||
|
||||
# Check if in a Jupyter Notebook (manual start/end_trace())
|
||||
try:
|
||||
get_ipython().__class__.__name__ == "ZMQInteractiveShell" # type: ignore
|
||||
auto_start_session = False
|
||||
except NameError:
|
||||
pass
|
||||
|
||||
# Prepare initialization arguments
|
||||
init_kwargs = {
|
||||
"api_key": api_key,
|
||||
"endpoint": endpoint,
|
||||
"app_url": app_url,
|
||||
"max_wait_time": max_wait_time,
|
||||
"max_queue_size": max_queue_size,
|
||||
"default_tags": merged_tags,
|
||||
"trace_name": trace_name,
|
||||
"instrument_llm_calls": instrument_llm_calls,
|
||||
"auto_start_session": auto_start_session,
|
||||
"auto_init": auto_init,
|
||||
"skip_auto_end_session": skip_auto_end_session,
|
||||
"env_data_opt_out": env_data_opt_out,
|
||||
"log_level": log_level,
|
||||
"fail_safe": fail_safe,
|
||||
"log_session_replay_url": log_session_replay_url,
|
||||
"exporter_endpoint": exporter_endpoint,
|
||||
**kwargs,
|
||||
}
|
||||
|
||||
# Get the current client instance (creates new one if needed)
|
||||
client = get_client()
|
||||
|
||||
# Initialize the client directly
|
||||
return client.init(**init_kwargs)
|
||||
|
||||
|
||||
def configure(**kwargs):
|
||||
"""Update client configuration
|
||||
|
||||
Args:
|
||||
**kwargs: Configuration parameters. Supported parameters include:
|
||||
- api_key: API Key for AgentOps services
|
||||
- endpoint: The endpoint for the AgentOps service
|
||||
- app_url: The dashboard URL for the AgentOps app
|
||||
- max_wait_time: Maximum time to wait in milliseconds before flushing the queue
|
||||
- max_queue_size: Maximum size of the event queue
|
||||
- default_tags: Default tags for the sessions
|
||||
- instrument_llm_calls: Whether to instrument LLM calls
|
||||
- auto_start_session: Whether to start a session automatically
|
||||
- skip_auto_end_session: Don't automatically end session
|
||||
- env_data_opt_out: Whether to opt out of collecting environment data
|
||||
- log_level: The log level to use for the client
|
||||
- fail_safe: Whether to suppress errors and continue execution
|
||||
- exporter: Custom span exporter for OpenTelemetry trace data
|
||||
- processor: Custom span processor for OpenTelemetry trace data
|
||||
- exporter_endpoint: Endpoint for the exporter
|
||||
"""
|
||||
global _client
|
||||
|
||||
# List of valid parameters that can be passed to configure
|
||||
valid_params = {
|
||||
"api_key",
|
||||
"endpoint",
|
||||
"app_url",
|
||||
"max_wait_time",
|
||||
"max_queue_size",
|
||||
"default_tags",
|
||||
"instrument_llm_calls",
|
||||
"auto_start_session",
|
||||
"skip_auto_end_session",
|
||||
"env_data_opt_out",
|
||||
"log_level",
|
||||
"fail_safe",
|
||||
"exporter",
|
||||
"processor",
|
||||
"exporter_endpoint",
|
||||
}
|
||||
|
||||
# Check for invalid parameters
|
||||
invalid_params = set(kwargs.keys()) - valid_params
|
||||
if invalid_params:
|
||||
logger.warning(f"Invalid configuration parameters: {invalid_params}")
|
||||
|
||||
client = get_client()
|
||||
client.configure(**kwargs)
|
||||
|
||||
|
||||
def start_trace(
|
||||
trace_name: str = "session", tags: Optional[Union[Dict[str, Any], List[str]]] = None
|
||||
) -> Optional[TraceContext]:
|
||||
"""
|
||||
Starts a new trace (root span) and returns its context.
|
||||
This allows for multiple concurrent, user-managed traces.
|
||||
|
||||
Args:
|
||||
trace_name: Name for the trace (e.g., "session", "my_custom_task").
|
||||
tags: Optional tags to attach to the trace span (list of strings or dict).
|
||||
|
||||
Returns:
|
||||
A TraceContext object containing the span and context token, or None if SDK not initialized.
|
||||
"""
|
||||
if not tracer.initialized:
|
||||
# Optionally, attempt to initialize the client if not already, or log a more severe warning.
|
||||
# For now, align with legacy start_session that would try to init.
|
||||
# However, explicit init is preferred before starting traces.
|
||||
logger.warning("AgentOps SDK not initialized. Attempting to initialize with defaults before starting trace.")
|
||||
try:
|
||||
init() # Attempt to initialize with environment variables / defaults
|
||||
if not tracer.initialized:
|
||||
logger.error("SDK initialization failed. Cannot start trace.")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.error(f"SDK auto-initialization failed during start_trace: {e}. Cannot start trace.")
|
||||
return None
|
||||
|
||||
return tracer.start_trace(trace_name=trace_name, tags=tags)
|
||||
|
||||
|
||||
def end_trace(
|
||||
trace_context: Optional[TraceContext] = None, end_state: Union[TraceState, StatusCode, str] = TraceState.SUCCESS
|
||||
) -> None:
|
||||
"""
|
||||
Ends a trace (its root span) and finalizes it.
|
||||
If no trace_context is provided, ends all active session spans.
|
||||
|
||||
Args:
|
||||
trace_context: The TraceContext object returned by start_trace. If None, ends all active traces.
|
||||
end_state: The final state of the trace (e.g., "Success", "Indeterminate", "Error").
|
||||
"""
|
||||
if not tracer.initialized:
|
||||
logger.warning("AgentOps SDK not initialized. Cannot end trace.")
|
||||
return
|
||||
tracer.end_trace(trace_context=trace_context, end_state=end_state)
|
||||
|
||||
|
||||
def update_trace_metadata(metadata: Dict[str, Any], prefix: str = "trace.metadata") -> bool:
|
||||
"""
|
||||
Update metadata on the current running trace.
|
||||
|
||||
Args:
|
||||
metadata: Dictionary of key-value pairs to set as trace metadata.
|
||||
Values must be strings, numbers, booleans, or lists of these types.
|
||||
Lists are converted to JSON string representation.
|
||||
Keys can be either custom keys or semantic convention aliases.
|
||||
prefix: Prefix for metadata attributes (default: "trace.metadata").
|
||||
Ignored for semantic convention attributes.
|
||||
|
||||
Returns:
|
||||
bool: True if metadata was successfully updated, False otherwise.
|
||||
|
||||
"""
|
||||
if not tracer.initialized:
|
||||
logger.warning("AgentOps SDK not initialized. Cannot update trace metadata.")
|
||||
return False
|
||||
|
||||
# Build semantic convention mappings dynamically
|
||||
def build_semconv_mappings():
|
||||
"""Build mappings from user-friendly keys to semantic convention attributes."""
|
||||
mappings = {}
|
||||
|
||||
# Helper function to extract attribute name from semantic convention
|
||||
def extract_key_from_attr(attr_value: str) -> str:
|
||||
parts = attr_value.split(".")
|
||||
if len(parts) >= 2:
|
||||
# Handle special cases
|
||||
if parts[0] == "error":
|
||||
# error.type -> error_type
|
||||
return "_".join(parts)
|
||||
else:
|
||||
# Default: entity.attribute -> entity_attribute
|
||||
return "_".join(parts)
|
||||
return attr_value
|
||||
|
||||
# Process each semantic convention class
|
||||
for cls in [AgentAttributes, ToolAttributes, WorkflowAttributes, CoreAttributes, SpanAttributes]:
|
||||
for attr_name, attr_value in cls.__dict__.items():
|
||||
if not attr_name.startswith("_") and isinstance(attr_value, str):
|
||||
# Skip gen_ai attributes
|
||||
if attr_value.startswith("gen_ai."):
|
||||
continue
|
||||
|
||||
# Generate user-friendly key
|
||||
user_key = extract_key_from_attr(attr_value)
|
||||
mappings[user_key] = attr_value
|
||||
|
||||
# Add some additional convenience mappings
|
||||
if attr_value == CoreAttributes.TAGS:
|
||||
mappings["tags"] = attr_value
|
||||
|
||||
return mappings
|
||||
|
||||
# Build mappings if using semantic conventions
|
||||
SEMCONV_MAPPINGS = build_semconv_mappings()
|
||||
|
||||
# Collect all valid semantic convention attributes
|
||||
VALID_SEMCONV_ATTRS = set()
|
||||
for cls in [AgentAttributes, ToolAttributes, WorkflowAttributes, CoreAttributes, SpanAttributes]:
|
||||
for key, value in cls.__dict__.items():
|
||||
if not key.startswith("_") and isinstance(value, str):
|
||||
# Include all attributes except gen_ai ones
|
||||
if not value.startswith("gen_ai."):
|
||||
VALID_SEMCONV_ATTRS.add(value)
|
||||
|
||||
# Find the current trace span
|
||||
span = None
|
||||
|
||||
# Get the current span from OpenTelemetry context
|
||||
current_span = get_current_span()
|
||||
|
||||
# Check if the current span is valid and recording
|
||||
if current_span and hasattr(current_span, "is_recording") and current_span.is_recording():
|
||||
# Check if this is a trace/session span or a child span
|
||||
span_name = getattr(current_span, "name", "")
|
||||
|
||||
# If it's a session/trace span, use it directly
|
||||
if span_name.endswith(f".{SpanKind.SESSION}"):
|
||||
span = current_span
|
||||
else:
|
||||
# It's a child span, try to find the root trace span
|
||||
# Get all active traces
|
||||
active_traces = tracer.get_active_traces()
|
||||
if active_traces:
|
||||
# Find the trace that contains the current span
|
||||
current_trace_id = current_span.get_span_context().trace_id
|
||||
|
||||
for trace_id_str, trace_ctx in active_traces.items():
|
||||
try:
|
||||
# Convert hex string back to int for comparison
|
||||
trace_id = int(trace_id_str, 16)
|
||||
if trace_id == current_trace_id:
|
||||
span = trace_ctx.span
|
||||
break
|
||||
except (ValueError, AttributeError):
|
||||
continue
|
||||
|
||||
# If we couldn't find the parent trace, use the current span
|
||||
if not span:
|
||||
span = current_span
|
||||
else:
|
||||
# No active traces, use the current span
|
||||
span = current_span
|
||||
|
||||
# If no current span or it's not recording, check active traces
|
||||
if not span:
|
||||
active_traces = tracer.get_active_traces()
|
||||
if active_traces:
|
||||
# Get the most recently created trace (last in the dict)
|
||||
trace_context = list(active_traces.values())[-1]
|
||||
span = trace_context.span
|
||||
logger.debug("Using most recent active trace for metadata update")
|
||||
else:
|
||||
logger.warning("No active trace found. Cannot update metadata.")
|
||||
return False
|
||||
|
||||
# Ensure the span is recording before updating
|
||||
if not span or (hasattr(span, "is_recording") and not span.is_recording()):
|
||||
logger.warning("Span is not recording. Cannot update metadata.")
|
||||
return False
|
||||
|
||||
# Update the span attributes with the metadata
|
||||
try:
|
||||
updated_count = 0
|
||||
for key, value in metadata.items():
|
||||
# Validate the value type
|
||||
if value is None:
|
||||
continue
|
||||
|
||||
# Convert lists to JSON string representation for OpenTelemetry compatibility
|
||||
if isinstance(value, list):
|
||||
# Ensure all list items are valid types
|
||||
if all(isinstance(item, (str, int, float, bool)) for item in value):
|
||||
value = json.dumps(value)
|
||||
else:
|
||||
logger.warning(f"Skipping metadata key '{key}': list contains invalid types")
|
||||
continue
|
||||
elif not isinstance(value, (str, int, float, bool)):
|
||||
logger.warning(f"Skipping metadata key '{key}': value type {type(value)} not supported")
|
||||
continue
|
||||
|
||||
# Determine the attribute key
|
||||
attribute_key = key
|
||||
|
||||
# Check if key is already a valid semantic convention attribute
|
||||
if key in VALID_SEMCONV_ATTRS:
|
||||
# Key is already a valid semantic convention, use as-is
|
||||
attribute_key = key
|
||||
elif key in SEMCONV_MAPPINGS:
|
||||
# It's a user-friendly key, map it to semantic convention
|
||||
attribute_key = SEMCONV_MAPPINGS[key]
|
||||
logger.debug(f"Mapped '{key}' to semantic convention '{attribute_key}'")
|
||||
else:
|
||||
# Not a semantic convention, use with prefix
|
||||
attribute_key = f"{prefix}.{key}"
|
||||
|
||||
# Set the attribute
|
||||
span.set_attribute(attribute_key, value)
|
||||
updated_count += 1
|
||||
|
||||
if updated_count > 0:
|
||||
logger.debug(f"Successfully updated {updated_count} metadata attributes on trace")
|
||||
return True
|
||||
else:
|
||||
logger.warning("No valid metadata attributes were updated")
|
||||
return False
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error updating trace metadata: {e}")
|
||||
return False
|
||||
|
||||
|
||||
__all__ = [
|
||||
# Legacy exports
|
||||
"start_session",
|
||||
"end_session",
|
||||
"track_agent",
|
||||
"track_tool",
|
||||
"end_all_sessions",
|
||||
"Session",
|
||||
"ToolEvent",
|
||||
"ErrorEvent",
|
||||
"ActionEvent",
|
||||
"LLMEvent",
|
||||
# Modern exports
|
||||
"init",
|
||||
"start_trace",
|
||||
"end_trace",
|
||||
"update_trace_metadata",
|
||||
"Client",
|
||||
"get_client",
|
||||
# Decorators
|
||||
"trace",
|
||||
"session",
|
||||
"agent",
|
||||
"task",
|
||||
"workflow",
|
||||
"operation",
|
||||
"tool",
|
||||
"guardrail",
|
||||
"track_endpoint",
|
||||
# Enums
|
||||
"TraceState",
|
||||
"SUCCESS",
|
||||
"ERROR",
|
||||
"UNSET",
|
||||
# Validation
|
||||
"validate_trace_spans",
|
||||
"print_validation_summary",
|
||||
"ValidationError",
|
||||
]
|
||||
@@ -0,0 +1,5 @@
|
||||
from agentops.client.client import Client
|
||||
from agentops.client.api import ApiClient
|
||||
|
||||
|
||||
__all__ = ["Client", "ApiClient"]
|
||||
@@ -0,0 +1,71 @@
|
||||
"""
|
||||
API client for the AgentOps API.
|
||||
|
||||
This module provides the client for the AgentOps API.
|
||||
"""
|
||||
|
||||
from typing import Dict, Type, TypeVar, cast
|
||||
|
||||
from agentops.client.api.base import BaseApiClient
|
||||
from agentops.client.api.types import AuthTokenResponse
|
||||
from agentops.client.api.versions.v3 import V3Client
|
||||
from agentops.client.api.versions.v4 import V4Client
|
||||
|
||||
# Define a type variable for client classes
|
||||
T = TypeVar("T", bound=BaseApiClient)
|
||||
|
||||
__all__ = ["ApiClient", "BaseApiClient", "AuthTokenResponse"]
|
||||
|
||||
|
||||
class ApiClient:
|
||||
"""
|
||||
Master API client that contains all version-specific clients.
|
||||
|
||||
This client provides a unified interface for accessing different API versions.
|
||||
It lazily initializes version-specific clients when they are first accessed.
|
||||
"""
|
||||
|
||||
def __init__(self, endpoint: str = "https://api.agentops.ai"):
|
||||
"""
|
||||
Initialize the master API client.
|
||||
|
||||
Args:
|
||||
endpoint: The base URL for the API
|
||||
"""
|
||||
self.endpoint = endpoint
|
||||
self._clients: Dict[str, BaseApiClient] = {}
|
||||
|
||||
@property
|
||||
def v3(self) -> V3Client:
|
||||
"""
|
||||
Get the V3 API client.
|
||||
|
||||
Returns:
|
||||
The V3 API client
|
||||
"""
|
||||
return self._get_client("v3", V3Client)
|
||||
|
||||
@property
|
||||
def v4(self) -> V4Client:
|
||||
"""
|
||||
Get the V4 API client.
|
||||
|
||||
Returns:
|
||||
The V4 API client
|
||||
"""
|
||||
return self._get_client("v4", V4Client)
|
||||
|
||||
def _get_client(self, version: str, client_class: Type[T]) -> T:
|
||||
"""
|
||||
Get or create a version-specific client.
|
||||
|
||||
Args:
|
||||
version: The API version
|
||||
client_class: The client class to instantiate
|
||||
|
||||
Returns:
|
||||
The version-specific client
|
||||
"""
|
||||
if version not in self._clients:
|
||||
self._clients[version] = client_class(self.endpoint)
|
||||
return cast(T, self._clients[version])
|
||||
@@ -0,0 +1,160 @@
|
||||
"""
|
||||
Base API client classes for making HTTP requests.
|
||||
|
||||
This module provides the foundation for all API clients in the AgentOps SDK.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, Optional, Protocol
|
||||
|
||||
import requests
|
||||
|
||||
from agentops.client.http.http_client import HttpClient
|
||||
from agentops.helpers.version import get_agentops_version
|
||||
|
||||
|
||||
class TokenFetcher(Protocol):
|
||||
"""Protocol for token fetching functions"""
|
||||
|
||||
def __call__(self, api_key: str) -> str: ...
|
||||
|
||||
|
||||
class BaseApiClient:
|
||||
"""
|
||||
Base class for API communication with async HTTP methods.
|
||||
|
||||
This class provides the core HTTP functionality without authentication.
|
||||
All HTTP methods are asynchronous.
|
||||
"""
|
||||
|
||||
def __init__(self, endpoint: str):
|
||||
"""
|
||||
Initialize the base API client.
|
||||
|
||||
Args:
|
||||
endpoint: The base URL for the API
|
||||
"""
|
||||
self.endpoint = endpoint
|
||||
self.http_client = HttpClient()
|
||||
self.last_response: Optional[requests.Response] = None
|
||||
|
||||
def prepare_headers(self, custom_headers: Optional[Dict[str, str]] = None) -> Dict[str, str]:
|
||||
"""
|
||||
Prepare headers for API requests.
|
||||
|
||||
Args:
|
||||
custom_headers: Additional headers to include
|
||||
|
||||
Returns:
|
||||
Headers dictionary with standard headers and any custom headers
|
||||
"""
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Connection": "keep-alive",
|
||||
"Keep-Alive": "timeout=10, max=1000",
|
||||
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
|
||||
}
|
||||
|
||||
if custom_headers:
|
||||
headers.update(custom_headers)
|
||||
|
||||
return headers
|
||||
|
||||
def _get_full_url(self, path: str) -> str:
|
||||
"""
|
||||
Get the full URL for a path.
|
||||
|
||||
Args:
|
||||
path: The API endpoint path
|
||||
|
||||
Returns:
|
||||
The full URL
|
||||
"""
|
||||
return f"{self.endpoint}{path}"
|
||||
|
||||
async def async_request(
|
||||
self,
|
||||
method: str,
|
||||
path: str,
|
||||
data: Optional[Dict[str, Any]] = None,
|
||||
headers: Optional[Dict[str, str]] = None,
|
||||
timeout: int = 30,
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Make a generic async HTTP request
|
||||
|
||||
Args:
|
||||
method: HTTP method (e.g., 'get', 'post', 'put', 'delete')
|
||||
path: API endpoint path
|
||||
data: Request payload (for POST, PUT methods)
|
||||
headers: Request headers
|
||||
timeout: Request timeout in seconds
|
||||
|
||||
Returns:
|
||||
JSON response as dictionary, or None if request failed
|
||||
|
||||
Raises:
|
||||
Exception: If the request fails
|
||||
"""
|
||||
url = self._get_full_url(path)
|
||||
|
||||
try:
|
||||
response_data = await self.http_client.async_request(
|
||||
method=method, url=url, data=data, headers=headers, timeout=timeout
|
||||
)
|
||||
return response_data
|
||||
except Exception as e:
|
||||
raise Exception(f"{method.upper()} request failed: {str(e)}") from e
|
||||
|
||||
async def post(self, path: str, data: Dict[str, Any], headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Make async POST request
|
||||
|
||||
Args:
|
||||
path: API endpoint path
|
||||
data: Request payload
|
||||
headers: Request headers
|
||||
|
||||
Returns:
|
||||
JSON response as dictionary, or None if request failed
|
||||
"""
|
||||
return await self.async_request("post", path, data=data, headers=headers)
|
||||
|
||||
async def get(self, path: str, headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Make async GET request
|
||||
|
||||
Args:
|
||||
path: API endpoint path
|
||||
headers: Request headers
|
||||
|
||||
Returns:
|
||||
JSON response as dictionary, or None if request failed
|
||||
"""
|
||||
return await self.async_request("get", path, headers=headers)
|
||||
|
||||
async def put(self, path: str, data: Dict[str, Any], headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Make async PUT request
|
||||
|
||||
Args:
|
||||
path: API endpoint path
|
||||
data: Request payload
|
||||
headers: Request headers
|
||||
|
||||
Returns:
|
||||
JSON response as dictionary, or None if request failed
|
||||
"""
|
||||
return await self.async_request("put", path, data=data, headers=headers)
|
||||
|
||||
async def delete(self, path: str, headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Make async DELETE request
|
||||
|
||||
Args:
|
||||
path: API endpoint path
|
||||
headers: Request headers
|
||||
|
||||
Returns:
|
||||
JSON response as dictionary, or None if request failed
|
||||
"""
|
||||
return await self.async_request("delete", path, headers=headers)
|
||||
@@ -0,0 +1,21 @@
|
||||
"""
|
||||
Common types used across API client modules.
|
||||
|
||||
This module contains type definitions used by multiple API client modules.
|
||||
"""
|
||||
|
||||
from typing import TypedDict
|
||||
|
||||
|
||||
class AuthTokenResponse(TypedDict):
|
||||
"""Response from the auth/token endpoint"""
|
||||
|
||||
token: str
|
||||
project_id: str
|
||||
|
||||
|
||||
class UploadedObjectResponse(TypedDict):
|
||||
"""Response from the v4/objects/upload endpoint"""
|
||||
|
||||
url: str
|
||||
size: int
|
||||
@@ -0,0 +1,10 @@
|
||||
"""
|
||||
API client versions package.
|
||||
|
||||
This package contains client implementations for different API versions.
|
||||
"""
|
||||
|
||||
from agentops.client.api.versions.v3 import V3Client
|
||||
from agentops.client.api.versions.v4 import V4Client
|
||||
|
||||
__all__ = ["V3Client", "V4Client"]
|
||||
@@ -0,0 +1,69 @@
|
||||
"""
|
||||
V3 API client for the AgentOps API.
|
||||
|
||||
This module provides the client for the V3 version of the AgentOps API.
|
||||
"""
|
||||
|
||||
from agentops.client.api.base import BaseApiClient
|
||||
from agentops.client.api.types import AuthTokenResponse
|
||||
from agentops.client.http.http_client import HttpClient
|
||||
from agentops.logging import logger
|
||||
from termcolor import colored
|
||||
|
||||
|
||||
class V3Client(BaseApiClient):
|
||||
"""Client for the AgentOps V3 API"""
|
||||
|
||||
def __init__(self, endpoint: str):
|
||||
"""
|
||||
Initialize the V3 API client.
|
||||
|
||||
Args:
|
||||
endpoint: The base URL for the API
|
||||
"""
|
||||
# Set up with V3-specific auth endpoint
|
||||
super().__init__(endpoint)
|
||||
|
||||
async def fetch_auth_token(self, api_key: str) -> AuthTokenResponse:
|
||||
"""
|
||||
Asynchronously fetch authentication token.
|
||||
|
||||
Args:
|
||||
api_key: The API key to authenticate with
|
||||
|
||||
Returns:
|
||||
AuthTokenResponse containing token and project information, or None if failed
|
||||
"""
|
||||
try:
|
||||
path = "/v3/auth/token"
|
||||
data = {"api_key": api_key}
|
||||
headers = self.prepare_headers()
|
||||
|
||||
# Build full URL
|
||||
url = self._get_full_url(path)
|
||||
|
||||
# Make async request
|
||||
response_data = await HttpClient.async_request(
|
||||
method="POST", url=url, data=data, headers=headers, timeout=30
|
||||
)
|
||||
|
||||
token = response_data.get("token")
|
||||
if not token:
|
||||
logger.warning("Authentication failed: Perhaps an invalid API key?")
|
||||
return None
|
||||
|
||||
# Check project premium status
|
||||
if response_data.get("project_prem_status") != "pro":
|
||||
logger.info(
|
||||
colored(
|
||||
"\x1b[34mYou're on the agentops free plan 🤔\x1b[0m",
|
||||
"blue",
|
||||
)
|
||||
)
|
||||
|
||||
return response_data
|
||||
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
# Add V3-specific API methods here
|
||||
@@ -0,0 +1,145 @@
|
||||
"""
|
||||
V4 API client for the AgentOps API.
|
||||
|
||||
This module provides the client for the V4 version of the AgentOps API.
|
||||
"""
|
||||
|
||||
from typing import Optional, Union, Dict, Any
|
||||
|
||||
import requests
|
||||
from agentops.client.api.base import BaseApiClient
|
||||
from agentops.client.http.http_client import HttpClient
|
||||
from agentops.exceptions import ApiServerException
|
||||
from agentops.helpers.version import get_agentops_version
|
||||
|
||||
|
||||
class V4Client(BaseApiClient):
|
||||
"""Client for the AgentOps V4 API"""
|
||||
|
||||
def __init__(self, endpoint: str):
|
||||
"""Initialize the V4 API client."""
|
||||
super().__init__(endpoint)
|
||||
self.auth_token: Optional[str] = None
|
||||
|
||||
def set_auth_token(self, token: str):
|
||||
"""
|
||||
Set the authentication token for API requests.
|
||||
|
||||
Args:
|
||||
token: The authentication token to set
|
||||
"""
|
||||
self.auth_token = token
|
||||
|
||||
def prepare_headers(self, custom_headers: Optional[Dict[str, str]] = None) -> Dict[str, str]:
|
||||
"""
|
||||
Prepare headers for API requests.
|
||||
|
||||
Args:
|
||||
custom_headers: Additional headers to include
|
||||
Returns:
|
||||
Headers dictionary with standard headers and any custom headers
|
||||
"""
|
||||
headers = {
|
||||
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
|
||||
}
|
||||
|
||||
# Only add Authorization header if we have a token
|
||||
if self.auth_token:
|
||||
headers["Authorization"] = f"Bearer {self.auth_token}"
|
||||
|
||||
if custom_headers:
|
||||
headers.update(custom_headers)
|
||||
return headers
|
||||
|
||||
def post(self, path: str, body: Union[str, bytes], headers: Optional[Dict[str, str]] = None) -> requests.Response:
|
||||
"""
|
||||
Make a POST request to the V4 API.
|
||||
|
||||
Args:
|
||||
path: The API path to POST to
|
||||
body: The request body (string or bytes)
|
||||
headers: Optional headers to include
|
||||
|
||||
Returns:
|
||||
The response object
|
||||
"""
|
||||
url = self._get_full_url(path)
|
||||
request_headers = headers or self.prepare_headers()
|
||||
|
||||
return HttpClient.get_session().post(url, json={"body": body}, headers=request_headers, timeout=30)
|
||||
|
||||
def upload_object(self, body: Union[str, bytes]) -> Dict[str, Any]:
|
||||
"""
|
||||
Upload an object to the V4 API.
|
||||
|
||||
Args:
|
||||
body: The object body to upload
|
||||
|
||||
Returns:
|
||||
Dictionary containing upload response data
|
||||
|
||||
Raises:
|
||||
ApiServerException: If the upload fails
|
||||
"""
|
||||
try:
|
||||
# Convert bytes to string for consistency with test expectations
|
||||
if isinstance(body, bytes):
|
||||
body = body.decode("utf-8")
|
||||
|
||||
response = self.post("/v4/objects/upload/", body, self.prepare_headers())
|
||||
|
||||
if response.status_code != 200:
|
||||
error_msg = f"Upload failed: {response.status_code}"
|
||||
try:
|
||||
error_data = response.json()
|
||||
if "error" in error_data:
|
||||
error_msg = error_data["error"]
|
||||
except:
|
||||
pass
|
||||
raise ApiServerException(error_msg)
|
||||
|
||||
try:
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
raise ApiServerException(f"Failed to process upload response: {str(e)}")
|
||||
except requests.exceptions.RequestException as e:
|
||||
raise ApiServerException(f"Failed to upload object: {e}")
|
||||
|
||||
def upload_logfile(self, body: Union[str, bytes], trace_id: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Upload a logfile to the V4 API.
|
||||
|
||||
Args:
|
||||
body: The logfile content to upload
|
||||
trace_id: The trace ID associated with the logfile
|
||||
|
||||
Returns:
|
||||
Dictionary containing upload response data
|
||||
|
||||
Raises:
|
||||
ApiServerException: If the upload fails
|
||||
"""
|
||||
try:
|
||||
# Convert bytes to string for consistency with test expectations
|
||||
if isinstance(body, bytes):
|
||||
body = body.decode("utf-8")
|
||||
|
||||
headers = {**self.prepare_headers(), "Trace-Id": str(trace_id)}
|
||||
response = self.post("/v4/logs/upload/", body, headers)
|
||||
|
||||
if response.status_code != 200:
|
||||
error_msg = f"Upload failed: {response.status_code}"
|
||||
try:
|
||||
error_data = response.json()
|
||||
if "error" in error_data:
|
||||
error_msg = error_data["error"]
|
||||
except:
|
||||
pass
|
||||
raise ApiServerException(error_msg)
|
||||
|
||||
try:
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
raise ApiServerException(f"Failed to process upload response: {str(e)}")
|
||||
except requests.exceptions.RequestException as e:
|
||||
raise ApiServerException(f"Failed to upload logfile: {e}")
|
||||
@@ -0,0 +1,280 @@
|
||||
import atexit
|
||||
import asyncio
|
||||
import threading
|
||||
from typing import Optional, Any
|
||||
|
||||
from agentops.client.api import ApiClient
|
||||
from agentops.config import Config
|
||||
from agentops.instrumentation import instrument_all
|
||||
from agentops.logging import logger
|
||||
from agentops.logging.config import configure_logging, intercept_opentelemetry_logging
|
||||
from agentops.sdk.core import TraceContext, tracer
|
||||
from agentops.legacy import Session
|
||||
|
||||
# Global variables to hold the client's auto-started trace and its legacy session wrapper
|
||||
_client_init_trace_context: Optional[TraceContext] = None
|
||||
_client_legacy_session_for_init_trace: Optional[Session] = None
|
||||
|
||||
# Single atexit handler registered flag
|
||||
_atexit_registered = False
|
||||
|
||||
|
||||
def _end_init_trace_atexit():
|
||||
"""Global atexit handler to end the client's auto-initialized trace during shutdown."""
|
||||
global _client_init_trace_context, _client_legacy_session_for_init_trace
|
||||
if _client_init_trace_context is not None:
|
||||
logger.debug("Auto-ending client's init trace during shutdown.")
|
||||
try:
|
||||
# Use global tracer to end the trace directly
|
||||
if tracer.initialized and _client_init_trace_context.span.is_recording():
|
||||
tracer.end_trace(_client_init_trace_context, end_state="Shutdown")
|
||||
except Exception as e:
|
||||
logger.warning(f"Error ending client's init trace during shutdown: {e}")
|
||||
finally:
|
||||
_client_init_trace_context = None
|
||||
_client_legacy_session_for_init_trace = None # Clear its legacy wrapper too
|
||||
|
||||
|
||||
class Client:
|
||||
"""Singleton client for AgentOps service"""
|
||||
|
||||
config: Config
|
||||
_initialized: bool
|
||||
_init_trace_context: Optional[TraceContext] = None # Stores the context of the auto-started trace
|
||||
_legacy_session_for_init_trace: Optional[Session] = (
|
||||
None # Stores the legacy Session wrapper for the auto-started trace
|
||||
)
|
||||
|
||||
__instance = None # Class variable for singleton pattern
|
||||
|
||||
api: ApiClient
|
||||
_auth_token: Optional[str] = None
|
||||
_project_id: Optional[str] = None
|
||||
_auth_lock = threading.Lock()
|
||||
_auth_task: Optional[asyncio.Task] = None
|
||||
|
||||
def __new__(cls, *args: Any, **kwargs: Any) -> "Client":
|
||||
if cls.__instance is None:
|
||||
cls.__instance = super(Client, cls).__new__(cls)
|
||||
# Initialize instance variables that should only be set once per instance
|
||||
cls.__instance._init_trace_context = None
|
||||
cls.__instance._legacy_session_for_init_trace = None
|
||||
cls.__instance._auth_token = None
|
||||
cls.__instance._project_id = None
|
||||
cls.__instance._auth_lock = threading.Lock()
|
||||
cls.__instance._auth_task = None
|
||||
return cls.__instance
|
||||
|
||||
def __init__(self):
|
||||
# Initialization of attributes like config, _initialized should happen here if they are instance-specific
|
||||
# and not shared via __new__ for a true singleton that can be re-configured.
|
||||
# However, the current pattern re-initializes config in init().
|
||||
if (
|
||||
not hasattr(self, "_initialized") or not self._initialized
|
||||
): # Ensure init logic runs only once per actual initialization intent
|
||||
self.config = Config() # Initialize config here for the instance
|
||||
self._initialized = False
|
||||
# self._init_trace_context = None # Already done in __new__
|
||||
# self._legacy_session_for_init_trace = None # Already done in __new__
|
||||
|
||||
def get_current_jwt(self) -> Optional[str]:
|
||||
"""Get the current JWT token."""
|
||||
with self._auth_lock:
|
||||
return self._auth_token
|
||||
|
||||
def _set_auth_data(self, token: str, project_id: str):
|
||||
"""Set authentication data thread-safely."""
|
||||
with self._auth_lock:
|
||||
self._auth_token = token
|
||||
self._project_id = project_id
|
||||
|
||||
# Update the HTTP client's project ID
|
||||
from agentops.client.http.http_client import HttpClient
|
||||
|
||||
HttpClient.set_project_id(project_id)
|
||||
|
||||
async def _fetch_auth_async(self, api_key: str) -> Optional[dict]:
|
||||
"""Asynchronously fetch authentication token."""
|
||||
try:
|
||||
response = await self.api.v3.fetch_auth_token(api_key)
|
||||
if response:
|
||||
self._set_auth_data(response["token"], response["project_id"])
|
||||
|
||||
# Update V4 client with token
|
||||
self.api.v4.set_auth_token(response["token"])
|
||||
|
||||
# Update tracer config with real project ID
|
||||
tracing_config = {"project_id": response["project_id"]}
|
||||
tracer.update_config(tracing_config)
|
||||
|
||||
logger.debug("Successfully fetched authentication token asynchronously")
|
||||
return response
|
||||
else:
|
||||
logger.debug("Authentication failed - will continue without authentication")
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def _start_auth_task(self, api_key: str):
|
||||
"""Start the async authentication task."""
|
||||
if self._auth_task and not self._auth_task.done():
|
||||
return # Task already running
|
||||
|
||||
try:
|
||||
loop = asyncio.get_event_loop()
|
||||
if loop.is_running():
|
||||
# Use existing event loop
|
||||
self._auth_task = loop.create_task(self._fetch_auth_async(api_key))
|
||||
else:
|
||||
# Create new event loop in background thread
|
||||
def run_async_auth():
|
||||
asyncio.run(self._fetch_auth_async(api_key))
|
||||
|
||||
import threading
|
||||
|
||||
auth_thread = threading.Thread(target=run_async_auth, daemon=True)
|
||||
auth_thread.start()
|
||||
except RuntimeError:
|
||||
# Create new event loop in background thread
|
||||
def run_async_auth():
|
||||
asyncio.run(self._fetch_auth_async(api_key))
|
||||
|
||||
import threading
|
||||
|
||||
auth_thread = threading.Thread(target=run_async_auth, daemon=True)
|
||||
auth_thread.start()
|
||||
|
||||
def init(self, **kwargs: Any) -> None: # Return type updated to None
|
||||
# Recreate the Config object to parse environment variables at the time of initialization
|
||||
# This allows re-init with new env vars if needed, though true singletons usually init once.
|
||||
self.config = Config()
|
||||
self.configure(**kwargs)
|
||||
|
||||
# Only treat as re-initialization if a different non-None API key is explicitly provided
|
||||
provided_api_key = kwargs.get("api_key")
|
||||
if self.initialized and provided_api_key is not None and provided_api_key != self.config.api_key:
|
||||
logger.warning("AgentOps Client being re-initialized with a different API key. This is unusual.")
|
||||
# Reset initialization status to allow re-init with new key/config
|
||||
self._initialized = False
|
||||
if self._init_trace_context and self._init_trace_context.span.is_recording():
|
||||
logger.warning("Ending previously auto-started trace due to re-initialization.")
|
||||
tracer.end_trace(self._init_trace_context, "Reinitialized")
|
||||
self._init_trace_context = None
|
||||
self._legacy_session_for_init_trace = None
|
||||
|
||||
if self.initialized:
|
||||
logger.debug("AgentOps Client already initialized.")
|
||||
# If auto_start_session was true, return the existing legacy session wrapper
|
||||
if self.config.auto_start_session:
|
||||
return self._legacy_session_for_init_trace
|
||||
return None # If not auto-starting, and already initialized, return None
|
||||
|
||||
if not self.config.api_key:
|
||||
logger.warning(
|
||||
"No API key provided. AgentOps will initialize but authentication will fail. "
|
||||
"Set AGENTOPS_API_KEY environment variable or pass api_key parameter."
|
||||
)
|
||||
# Continue without API key - spans will be created but exports will fail gracefully
|
||||
|
||||
configure_logging(self.config)
|
||||
intercept_opentelemetry_logging()
|
||||
|
||||
self.api = ApiClient(self.config.endpoint)
|
||||
|
||||
# Initialize tracer with JWT provider for dynamic updates
|
||||
tracing_config = self.config.dict()
|
||||
tracing_config["project_id"] = "temporary" # Will be updated when auth completes
|
||||
|
||||
# Create JWT provider function for dynamic updates
|
||||
def jwt_provider():
|
||||
return self.get_current_jwt()
|
||||
|
||||
# Initialize tracer with JWT provider
|
||||
tracer.initialize_from_config(tracing_config, jwt_provider=jwt_provider)
|
||||
|
||||
if self.config.instrument_llm_calls:
|
||||
instrument_all()
|
||||
|
||||
# Start authentication task only if we have an API key
|
||||
if self.config.api_key:
|
||||
self._start_auth_task(self.config.api_key)
|
||||
else:
|
||||
logger.debug("No API key available - skipping authentication task")
|
||||
|
||||
global _atexit_registered
|
||||
if not _atexit_registered:
|
||||
atexit.register(_end_init_trace_atexit) # Register new atexit handler
|
||||
_atexit_registered = True
|
||||
|
||||
# Auto-start trace if configured
|
||||
if self.config.auto_start_session:
|
||||
if self._init_trace_context is None or not self._init_trace_context.span.is_recording():
|
||||
logger.debug("Auto-starting init trace.")
|
||||
trace_name = self.config.trace_name or "default"
|
||||
self._init_trace_context = tracer.start_trace(
|
||||
trace_name=trace_name,
|
||||
tags=list(self.config.default_tags) if self.config.default_tags else None,
|
||||
is_init_trace=True,
|
||||
)
|
||||
if self._init_trace_context:
|
||||
self._legacy_session_for_init_trace = Session(self._init_trace_context)
|
||||
|
||||
# For backward compatibility, also update the global references in legacy and client modules
|
||||
# These globals are what old code might have been using via agentops.legacy.get_session() or similar indirect access.
|
||||
global _client_init_trace_context, _client_legacy_session_for_init_trace
|
||||
_client_init_trace_context = self._init_trace_context
|
||||
_client_legacy_session_for_init_trace = self._legacy_session_for_init_trace
|
||||
|
||||
# Update legacy module's _current_session and _current_trace_context
|
||||
# This is tricky; direct access to another module's globals is not ideal.
|
||||
# Prefer explicit calls if possible, but for maximum BC:
|
||||
try:
|
||||
import agentops.legacy
|
||||
|
||||
agentops.legacy._current_session = self._legacy_session_for_init_trace
|
||||
agentops.legacy._current_trace_context = self._init_trace_context
|
||||
except ImportError:
|
||||
pass # Should not happen
|
||||
|
||||
else:
|
||||
logger.error("Failed to start the auto-init trace.")
|
||||
# Even if auto-start fails, core services up to the tracer might be initialized.
|
||||
# Set self.initialized to True if tracer is up, but return None.
|
||||
self._initialized = tracer.initialized
|
||||
return None # Failed to start trace
|
||||
|
||||
self._initialized = True # Successfully initialized and auto-trace started (if configured)
|
||||
# For backward compatibility, return the legacy session wrapper when auto_start_session=True
|
||||
return self._legacy_session_for_init_trace
|
||||
else:
|
||||
logger.debug("Auto-start session is disabled. No init trace started by client.")
|
||||
self._initialized = True # Successfully initialized, just no auto-trace
|
||||
return None # No auto-session, so return None
|
||||
|
||||
def configure(self, **kwargs: Any) -> None:
|
||||
"""Update client configuration"""
|
||||
self.config.configure(**kwargs)
|
||||
|
||||
@property
|
||||
def initialized(self) -> bool:
|
||||
return self._initialized
|
||||
|
||||
@initialized.setter
|
||||
def initialized(self, value: bool) -> None:
|
||||
if self._initialized and self._initialized != value:
|
||||
# Allow re-setting to False if we are intentionally re-initializing
|
||||
# This logic is now partly in init() to handle re-init cases
|
||||
pass
|
||||
self._initialized = value
|
||||
|
||||
# ------------------------------------------------------------
|
||||
# Remove the old __instance = None at the end of the class definition if it's a repeat
|
||||
# __instance = None # This was a class variable, should be defined once
|
||||
|
||||
# Make _init_trace_context and _legacy_session_for_init_trace accessible
|
||||
# to the atexit handler if it becomes a static/class method or needs access
|
||||
# For now, the atexit handler is global and uses global vars copied from these.
|
||||
|
||||
# Deprecate and remove the old global _active_session from this module.
|
||||
# Consumers should use agentops.start_trace() or rely on the auto-init trace.
|
||||
# For a transition, the auto-init trace's legacy wrapper is set to legacy module's globals.
|
||||
@@ -0,0 +1,87 @@
|
||||
# AgentOps HTTP Client Architecture
|
||||
|
||||
This directory contains the HTTP client architecture for the AgentOps SDK. The architecture follows a clean separation of concerns design principle.
|
||||
|
||||
## Components
|
||||
|
||||
### HttpClient
|
||||
|
||||
The `HttpClient` class provides low-level HTTP functionality:
|
||||
- Connection pooling
|
||||
- Retry logic
|
||||
- Basic HTTP methods (GET, POST, PUT, DELETE)
|
||||
|
||||
### AuthManager
|
||||
|
||||
The `AuthManager` class handles authentication concerns:
|
||||
- Token acquisition and storage
|
||||
- Token refresh logic
|
||||
- Authentication header preparation
|
||||
- Thread-safe token operations
|
||||
|
||||
### HTTP Adapters
|
||||
|
||||
#### BaseHTTPAdapter
|
||||
- Enhanced connection pooling and retry logic
|
||||
- Used by the `HttpClient` for basic HTTP operations
|
||||
|
||||
#### AuthenticatedHttpAdapter
|
||||
- Extends `BaseHTTPAdapter` with authentication capabilities
|
||||
- Automatically adds authentication headers to requests
|
||||
- Handles token refresh when authentication fails
|
||||
- Can be mounted to any requests.Session
|
||||
|
||||
## Design Principles
|
||||
|
||||
1. **Separation of Concerns**
|
||||
- HTTP concerns are isolated from authentication concerns
|
||||
- Each component has a single responsibility
|
||||
|
||||
2. **Composition over Inheritance**
|
||||
- Components use composition rather than inheritance
|
||||
- `ApiClient` composes `HttpClient` and `AuthManager`
|
||||
|
||||
3. **Clear Interfaces**
|
||||
- Each component has a well-defined interface
|
||||
- Implementation details are hidden
|
||||
|
||||
4. **Dependency Flow**
|
||||
- Dependencies flow in one direction
|
||||
- Lower-level components (HTTP, Auth) don't depend on higher-level components
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic API Client Usage
|
||||
|
||||
The HTTP client architecture is used by the `ApiClient` class, which provides a high-level interface for making API calls. Specific API versions (like `V3Client`) extend the `ApiClient` to provide version-specific functionality.
|
||||
|
||||
```python
|
||||
# Example usage
|
||||
from agentops.client.v3_client import V3Client
|
||||
|
||||
client = V3Client(endpoint="https://api.agentops.ai")
|
||||
response = client.authenticated_request(
|
||||
method="get",
|
||||
path="/v3/some/endpoint",
|
||||
api_key="your-api-key"
|
||||
)
|
||||
```
|
||||
|
||||
### Using with External Libraries
|
||||
|
||||
The architecture also supports integration with external libraries that need authenticated HTTP sessions:
|
||||
|
||||
```python
|
||||
# Example with OpenTelemetry exporter
|
||||
from agentops.client.v3_client import V3Client
|
||||
from agentops.client.exporters import AuthenticatedOTLPExporter
|
||||
|
||||
client = V3Client(endpoint="https://api.agentops.ai")
|
||||
session = client.create_authenticated_session(api_key="your-api-key")
|
||||
|
||||
exporter = AuthenticatedOTLPExporter(
|
||||
endpoint="https://api.agentops.ai/v3/traces",
|
||||
api_client=client,
|
||||
api_key="your-api-key"
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,116 @@
|
||||
from typing import Optional
|
||||
|
||||
from requests.adapters import HTTPAdapter
|
||||
from urllib3.util import Retry
|
||||
|
||||
# from agentops.client.auth_manager import AuthManager
|
||||
|
||||
|
||||
class BaseHTTPAdapter(HTTPAdapter):
|
||||
"""Base HTTP adapter with enhanced connection pooling and retry logic"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
pool_connections: int = 15,
|
||||
pool_maxsize: int = 256,
|
||||
max_retries: Optional[Retry] = None,
|
||||
):
|
||||
"""
|
||||
Initialize the base HTTP adapter.
|
||||
|
||||
Args:
|
||||
pool_connections: Number of connection pools to cache
|
||||
pool_maxsize: Maximum number of connections to save in the pool
|
||||
max_retries: Retry configuration for failed requests
|
||||
"""
|
||||
if max_retries is None:
|
||||
max_retries = Retry(
|
||||
total=3,
|
||||
backoff_factor=0.1,
|
||||
status_forcelist=[500, 502, 503, 504],
|
||||
)
|
||||
|
||||
super().__init__(pool_connections=pool_connections, pool_maxsize=pool_maxsize, max_retries=max_retries)
|
||||
|
||||
|
||||
# class AuthenticatedHttpAdapter(BaseHTTPAdapter):
|
||||
# """HTTP adapter with automatic JWT authentication and refresh"""
|
||||
#
|
||||
# def __init__(
|
||||
# self,
|
||||
# auth_manager: AuthManager,
|
||||
# api_key: str,
|
||||
# token_fetcher: Callable[[str], Union[str, AuthTokenResponse]],
|
||||
# pool_connections: int = 15,
|
||||
# pool_maxsize: int = 256,
|
||||
# max_retries: Optional[Retry] = None,
|
||||
# ):
|
||||
# """
|
||||
# Initialize the authenticated HTTP adapter.
|
||||
#
|
||||
# Args:
|
||||
# auth_manager: The authentication manager to use
|
||||
# api_key: The API key to authenticate with
|
||||
# token_fetcher: Function to fetch a new token if needed
|
||||
# pool_connections: Number of connection pools to cache
|
||||
# pool_maxsize: Maximum number of connections to save in the pool
|
||||
# max_retries: Retry configuration for failed requests
|
||||
# """
|
||||
# self.auth_manager = auth_manager
|
||||
# self.api_key = api_key
|
||||
# self.token_fetcher = token_fetcher
|
||||
#
|
||||
# super().__init__(
|
||||
# pool_connections=pool_connections,
|
||||
# pool_maxsize=pool_maxsize,
|
||||
# max_retries=max_retries
|
||||
# )
|
||||
#
|
||||
# def add_headers(self, request, **kwargs):
|
||||
# """Add authentication headers to the request"""
|
||||
# # Get fresh auth headers from the auth manager
|
||||
# self.auth_manager.maybe_fetch(self.api_key, self.token_fetcher)
|
||||
# auth_headers = self.auth_manager.prepare_auth_headers(self.api_key)
|
||||
#
|
||||
# # Update request headers
|
||||
# for key, value in auth_headers.items():
|
||||
# request.headers[key] = value
|
||||
#
|
||||
# return request
|
||||
#
|
||||
# def send(self, request, **kwargs):
|
||||
# """Send the request with authentication retry logic"""
|
||||
# # Ensure allow_redirects is set to False
|
||||
# kwargs["allow_redirects"] = False
|
||||
#
|
||||
# # Add auth headers to initial request
|
||||
# request = self.add_headers(request, **kwargs)
|
||||
#
|
||||
# # Make the initial request
|
||||
# response = super().send(request, **kwargs)
|
||||
#
|
||||
# # If we get a 401/403, check if it's due to token expiration
|
||||
# if self.auth_manager.is_token_expired_response(response):
|
||||
# logger.debug("Token expired, attempting to refresh")
|
||||
# try:
|
||||
# # Force token refresh
|
||||
# self.auth_manager.clear_token()
|
||||
# self.auth_manager.maybe_fetch(self.api_key, self.token_fetcher)
|
||||
#
|
||||
# # Update request with new token
|
||||
# request = self.add_headers(request, **kwargs)
|
||||
#
|
||||
# # Retry the request
|
||||
# logger.debug("Retrying request with new token")
|
||||
# response = super().send(request, **kwargs)
|
||||
# except AgentOpsApiJwtExpiredException as e:
|
||||
# # Authentication failed
|
||||
# logger.warning(f"Failed to refresh authentication token: {e}")
|
||||
# except ApiServerException as e:
|
||||
# # Server error during token refresh
|
||||
# logger.error(f"Server error during token refresh: {e}")
|
||||
# except Exception as e:
|
||||
# # Unexpected error during token refresh
|
||||
# logger.error(f"Unexpected error during token refresh: {e}")
|
||||
#
|
||||
# return response
|
||||
@@ -0,0 +1,169 @@
|
||||
from typing import Dict, Optional
|
||||
import threading
|
||||
|
||||
import requests
|
||||
|
||||
from agentops.client.http.http_adapter import BaseHTTPAdapter
|
||||
from agentops.logging import logger
|
||||
from agentops.helpers.version import get_agentops_version
|
||||
|
||||
# Import aiohttp for async requests
|
||||
try:
|
||||
import aiohttp
|
||||
|
||||
AIOHTTP_AVAILABLE = True
|
||||
except ImportError:
|
||||
AIOHTTP_AVAILABLE = False
|
||||
# Don't log warning here, only when actually trying to use async functionality
|
||||
|
||||
|
||||
class HttpClient:
|
||||
"""HTTP client with async-first design and optional sync fallback for log uploads"""
|
||||
|
||||
_session: Optional[requests.Session] = None
|
||||
_async_session: Optional[aiohttp.ClientSession] = None
|
||||
_project_id: Optional[str] = None
|
||||
_session_lock = threading.Lock()
|
||||
|
||||
@classmethod
|
||||
def get_project_id(cls) -> Optional[str]:
|
||||
"""Get the stored project ID"""
|
||||
return cls._project_id
|
||||
|
||||
@classmethod
|
||||
def set_project_id(cls, project_id: str) -> None:
|
||||
"""Set the project ID"""
|
||||
cls._project_id = project_id
|
||||
|
||||
@classmethod
|
||||
def get_session(cls) -> requests.Session:
|
||||
"""
|
||||
Get or create the global session with optimized connection pooling.
|
||||
|
||||
Note: This method is deprecated. Use async_request() instead.
|
||||
Only kept for log upload module compatibility.
|
||||
"""
|
||||
if cls._session is None:
|
||||
with cls._session_lock:
|
||||
if cls._session is None: # Double-check locking
|
||||
cls._session = requests.Session()
|
||||
|
||||
# Configure connection pooling
|
||||
adapter = BaseHTTPAdapter()
|
||||
|
||||
# Mount adapter for both HTTP and HTTPS
|
||||
cls._session.mount("http://", adapter)
|
||||
cls._session.mount("https://", adapter)
|
||||
|
||||
# Set default headers
|
||||
cls._session.headers.update(
|
||||
{
|
||||
"Connection": "keep-alive",
|
||||
"Keep-Alive": "timeout=10, max=1000",
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
|
||||
}
|
||||
)
|
||||
logger.debug(f"Agentops version: agentops-python/{get_agentops_version() or 'unknown'}")
|
||||
return cls._session
|
||||
|
||||
@classmethod
|
||||
async def get_async_session(cls) -> Optional[aiohttp.ClientSession]:
|
||||
"""Get or create the global async session with optimized connection pooling"""
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
logger.warning("aiohttp not available, cannot create async session")
|
||||
return None
|
||||
|
||||
# Always create a new session if the current one is None or closed
|
||||
if cls._async_session is None or cls._async_session.closed:
|
||||
# Close the old session if it exists but is closed
|
||||
if cls._async_session is not None and cls._async_session.closed:
|
||||
cls._async_session = None
|
||||
|
||||
# Create connector with connection pooling
|
||||
connector = aiohttp.TCPConnector(
|
||||
limit=100, # Total connection pool size
|
||||
limit_per_host=30, # Per-host connection limit
|
||||
ttl_dns_cache=300, # DNS cache TTL
|
||||
use_dns_cache=True,
|
||||
enable_cleanup_closed=True,
|
||||
)
|
||||
|
||||
# Create session with default headers
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
|
||||
}
|
||||
|
||||
cls._async_session = aiohttp.ClientSession(
|
||||
connector=connector, headers=headers, timeout=aiohttp.ClientTimeout(total=30)
|
||||
)
|
||||
|
||||
return cls._async_session
|
||||
|
||||
@classmethod
|
||||
async def close_async_session(cls):
|
||||
"""Close the async session"""
|
||||
if cls._async_session and not cls._async_session.closed:
|
||||
await cls._async_session.close()
|
||||
cls._async_session = None
|
||||
|
||||
@classmethod
|
||||
async def async_request(
|
||||
cls,
|
||||
method: str,
|
||||
url: str,
|
||||
data: Optional[Dict] = None,
|
||||
headers: Optional[Dict] = None,
|
||||
timeout: int = 30,
|
||||
) -> Optional[Dict]:
|
||||
"""
|
||||
Make an async HTTP request and return JSON response
|
||||
|
||||
Args:
|
||||
method: HTTP method (e.g., 'get', 'post', 'put', 'delete')
|
||||
url: Full URL for the request
|
||||
data: Request payload (for POST, PUT methods)
|
||||
headers: Request headers
|
||||
timeout: Request timeout in seconds
|
||||
|
||||
Returns:
|
||||
JSON response as dictionary, or None if request failed
|
||||
"""
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
logger.warning("aiohttp not available, cannot make async request")
|
||||
return None
|
||||
|
||||
try:
|
||||
session = await cls.get_async_session()
|
||||
if not session:
|
||||
return None
|
||||
|
||||
logger.debug(f"Making async {method} request to {url}")
|
||||
|
||||
# Prepare request parameters
|
||||
kwargs = {"timeout": aiohttp.ClientTimeout(total=timeout), "headers": headers or {}}
|
||||
|
||||
if data and method.lower() in ["post", "put", "patch"]:
|
||||
kwargs["json"] = data
|
||||
|
||||
# Make the request
|
||||
async with session.request(method.upper(), url, **kwargs) as response:
|
||||
logger.debug(f"Async request response status: {response.status}")
|
||||
|
||||
# Check if response is successful
|
||||
if response.status >= 400:
|
||||
return None
|
||||
|
||||
# Parse JSON response
|
||||
try:
|
||||
response_data = await response.json()
|
||||
logger.debug(
|
||||
f"Async request successful, response keys: {list(response_data.keys()) if response_data else 'None'}"
|
||||
)
|
||||
return response_data
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
except Exception:
|
||||
return None
|
||||
@@ -0,0 +1,274 @@
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from typing import List, Optional, Set, TypedDict, Union
|
||||
from uuid import UUID
|
||||
|
||||
from opentelemetry.sdk.trace import SpanProcessor
|
||||
from opentelemetry.sdk.trace.export import SpanExporter
|
||||
|
||||
from agentops.helpers.env import get_env_bool, get_env_int, get_env_list
|
||||
from agentops.helpers.serialization import AgentOpsJSONEncoder
|
||||
|
||||
|
||||
class ConfigDict(TypedDict):
|
||||
api_key: Optional[str]
|
||||
endpoint: Optional[str]
|
||||
app_url: Optional[str]
|
||||
max_wait_time: Optional[int]
|
||||
export_flush_interval: Optional[int]
|
||||
max_queue_size: Optional[int]
|
||||
default_tags: Optional[List[str]]
|
||||
trace_name: Optional[str]
|
||||
instrument_llm_calls: Optional[bool]
|
||||
auto_start_session: Optional[bool]
|
||||
auto_init: Optional[bool]
|
||||
skip_auto_end_session: Optional[bool]
|
||||
env_data_opt_out: Optional[bool]
|
||||
log_level: Optional[Union[str, int]]
|
||||
fail_safe: Optional[bool]
|
||||
prefetch_jwt_token: Optional[bool]
|
||||
log_session_replay_url: Optional[bool]
|
||||
|
||||
|
||||
@dataclass
|
||||
class Config:
|
||||
api_key: Optional[str] = field(
|
||||
default_factory=lambda: os.getenv("AGENTOPS_API_KEY"),
|
||||
metadata={"description": "API key for authentication with AgentOps services"},
|
||||
)
|
||||
|
||||
endpoint: str = field(
|
||||
default_factory=lambda: os.getenv("AGENTOPS_API_ENDPOINT", "https://api.agentops.ai"),
|
||||
metadata={"description": "Base URL for the AgentOps API"},
|
||||
)
|
||||
|
||||
app_url: str = field(
|
||||
default_factory=lambda: os.getenv("AGENTOPS_APP_URL", "https://app.agentops.ai"),
|
||||
metadata={"description": "Dashboard URL for the AgentOps application"},
|
||||
)
|
||||
|
||||
max_wait_time: int = field(
|
||||
default_factory=lambda: get_env_int("AGENTOPS_MAX_WAIT_TIME", 5000),
|
||||
metadata={"description": "Maximum time in milliseconds to wait for API responses"},
|
||||
)
|
||||
|
||||
export_flush_interval: int = field(
|
||||
default_factory=lambda: get_env_int("AGENTOPS_EXPORT_FLUSH_INTERVAL", 1000),
|
||||
metadata={"description": "Time interval in milliseconds between automatic exports of telemetry data"},
|
||||
)
|
||||
|
||||
max_queue_size: int = field(
|
||||
default_factory=lambda: get_env_int("AGENTOPS_MAX_QUEUE_SIZE", 512),
|
||||
metadata={"description": "Maximum number of events to queue before forcing a flush"},
|
||||
)
|
||||
|
||||
default_tags: Set[str] = field(
|
||||
default_factory=lambda: get_env_list("AGENTOPS_DEFAULT_TAGS"),
|
||||
metadata={"description": "Default tags to apply to all sessions"},
|
||||
)
|
||||
|
||||
trace_name: Optional[str] = field(
|
||||
default_factory=lambda: os.getenv("AGENTOPS_TRACE_NAME"),
|
||||
metadata={"description": "Default name for the trace/session"},
|
||||
)
|
||||
|
||||
instrument_llm_calls: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_INSTRUMENT_LLM_CALLS", True),
|
||||
metadata={"description": "Whether to automatically instrument and track LLM API calls"},
|
||||
)
|
||||
|
||||
auto_start_session: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_AUTO_START_SESSION", True),
|
||||
metadata={"description": "Whether to automatically start a session when initializing"},
|
||||
)
|
||||
|
||||
auto_init: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_AUTO_INIT", True),
|
||||
metadata={"description": "Whether to automatically initialize the client on import"},
|
||||
)
|
||||
|
||||
skip_auto_end_session: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_SKIP_AUTO_END_SESSION", False),
|
||||
metadata={"description": "Whether to skip automatically ending sessions on program exit"},
|
||||
)
|
||||
|
||||
env_data_opt_out: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_ENV_DATA_OPT_OUT", False),
|
||||
metadata={"description": "Whether to opt out of collecting environment data"},
|
||||
)
|
||||
|
||||
log_level: Union[str, int] = field(
|
||||
default_factory=lambda: os.getenv("AGENTOPS_LOG_LEVEL", "INFO"),
|
||||
metadata={"description": "Logging level for AgentOps logs"},
|
||||
)
|
||||
|
||||
fail_safe: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_FAIL_SAFE", False),
|
||||
metadata={"description": "Whether to suppress errors and continue execution when possible"},
|
||||
)
|
||||
|
||||
prefetch_jwt_token: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_PREFETCH_JWT_TOKEN", True),
|
||||
metadata={"description": "Whether to prefetch JWT token during initialization"},
|
||||
)
|
||||
|
||||
log_session_replay_url: bool = field(
|
||||
default_factory=lambda: get_env_bool("AGENTOPS_LOG_SESSION_REPLAY_URL", True),
|
||||
metadata={"description": "Whether to log session replay URLs to the console"},
|
||||
)
|
||||
|
||||
exporter_endpoint: Optional[str] = field(
|
||||
default_factory=lambda: os.getenv("AGENTOPS_EXPORTER_ENDPOINT", "https://otlp.agentops.ai/v1/traces"),
|
||||
metadata={
|
||||
"description": "Endpoint for the span exporter. When not provided, the default AgentOps endpoint will be used."
|
||||
},
|
||||
)
|
||||
|
||||
exporter: Optional[SpanExporter] = field(
|
||||
default_factory=lambda: None, metadata={"description": "Custom span exporter for OpenTelemetry trace data"}
|
||||
)
|
||||
|
||||
processor: Optional[SpanProcessor] = field(
|
||||
default_factory=lambda: None, metadata={"description": "Custom span processor for OpenTelemetry trace data"}
|
||||
)
|
||||
|
||||
def configure(
|
||||
self,
|
||||
api_key: Optional[str] = None,
|
||||
endpoint: Optional[str] = None,
|
||||
app_url: Optional[str] = None,
|
||||
max_wait_time: Optional[int] = None,
|
||||
export_flush_interval: Optional[int] = None,
|
||||
max_queue_size: Optional[int] = None,
|
||||
default_tags: Optional[List[str]] = None,
|
||||
trace_name: Optional[str] = None,
|
||||
instrument_llm_calls: Optional[bool] = None,
|
||||
auto_start_session: Optional[bool] = None,
|
||||
auto_init: Optional[bool] = None,
|
||||
skip_auto_end_session: Optional[bool] = None,
|
||||
env_data_opt_out: Optional[bool] = None,
|
||||
log_level: Optional[Union[str, int]] = None,
|
||||
fail_safe: Optional[bool] = None,
|
||||
prefetch_jwt_token: Optional[bool] = None,
|
||||
log_session_replay_url: Optional[bool] = None,
|
||||
exporter: Optional[SpanExporter] = None,
|
||||
processor: Optional[SpanProcessor] = None,
|
||||
exporter_endpoint: Optional[str] = None,
|
||||
):
|
||||
"""Configure settings from kwargs, validating where necessary"""
|
||||
if api_key is not None:
|
||||
self.api_key = api_key
|
||||
if not TESTING: # Allow setting dummy keys in tests
|
||||
try:
|
||||
UUID(api_key)
|
||||
except ValueError:
|
||||
# Log warning but don't throw exception - let async auth handle it
|
||||
from agentops.logging import logger
|
||||
|
||||
logger.warning(
|
||||
f"API key format appears invalid: {api_key[:8]}... "
|
||||
f"Authentication may fail. Find your API key at {self.endpoint}/settings/projects"
|
||||
)
|
||||
# Continue with the invalid key - async auth will handle the failure gracefully
|
||||
|
||||
if endpoint is not None:
|
||||
self.endpoint = endpoint
|
||||
|
||||
if app_url is not None:
|
||||
self.app_url = app_url
|
||||
|
||||
if max_wait_time is not None:
|
||||
self.max_wait_time = max_wait_time
|
||||
|
||||
if export_flush_interval is not None:
|
||||
self.export_flush_interval = export_flush_interval
|
||||
|
||||
if max_queue_size is not None:
|
||||
self.max_queue_size = max_queue_size
|
||||
|
||||
if default_tags is not None:
|
||||
self.default_tags = set(default_tags)
|
||||
|
||||
if trace_name is not None:
|
||||
self.trace_name = trace_name
|
||||
|
||||
if instrument_llm_calls is not None:
|
||||
self.instrument_llm_calls = instrument_llm_calls
|
||||
|
||||
if auto_start_session is not None:
|
||||
self.auto_start_session = auto_start_session
|
||||
|
||||
if auto_init is not None:
|
||||
self.auto_init = auto_init
|
||||
|
||||
if skip_auto_end_session is not None:
|
||||
self.skip_auto_end_session = skip_auto_end_session
|
||||
|
||||
if env_data_opt_out is not None:
|
||||
self.env_data_opt_out = env_data_opt_out
|
||||
|
||||
if log_level is not None:
|
||||
if isinstance(log_level, str):
|
||||
log_level_str = log_level.upper()
|
||||
if hasattr(logging, log_level_str):
|
||||
self.log_level = getattr(logging, log_level_str)
|
||||
else:
|
||||
self.log_level = logging.INFO
|
||||
else:
|
||||
self.log_level = log_level
|
||||
|
||||
if fail_safe is not None:
|
||||
self.fail_safe = fail_safe
|
||||
|
||||
if prefetch_jwt_token is not None:
|
||||
self.prefetch_jwt_token = prefetch_jwt_token
|
||||
|
||||
if log_session_replay_url is not None:
|
||||
self.log_session_replay_url = log_session_replay_url
|
||||
|
||||
if exporter is not None:
|
||||
self.exporter = exporter
|
||||
|
||||
if processor is not None:
|
||||
self.processor = processor
|
||||
|
||||
if exporter_endpoint is not None:
|
||||
self.exporter_endpoint = exporter_endpoint
|
||||
# else:
|
||||
# self.exporter_endpoint = self.endpoint
|
||||
|
||||
def dict(self):
|
||||
"""Return a dictionary representation of the config"""
|
||||
return {
|
||||
"api_key": self.api_key,
|
||||
"endpoint": self.endpoint,
|
||||
"app_url": self.app_url,
|
||||
"max_wait_time": self.max_wait_time,
|
||||
"export_flush_interval": self.export_flush_interval,
|
||||
"max_queue_size": self.max_queue_size,
|
||||
"default_tags": self.default_tags,
|
||||
"trace_name": self.trace_name,
|
||||
"instrument_llm_calls": self.instrument_llm_calls,
|
||||
"auto_start_session": self.auto_start_session,
|
||||
"auto_init": self.auto_init,
|
||||
"skip_auto_end_session": self.skip_auto_end_session,
|
||||
"env_data_opt_out": self.env_data_opt_out,
|
||||
"log_level": self.log_level,
|
||||
"fail_safe": self.fail_safe,
|
||||
"prefetch_jwt_token": self.prefetch_jwt_token,
|
||||
"log_session_replay_url": self.log_session_replay_url,
|
||||
"exporter": self.exporter,
|
||||
"processor": self.processor,
|
||||
"exporter_endpoint": self.exporter_endpoint,
|
||||
}
|
||||
|
||||
def json(self):
|
||||
"""Return a JSON representation of the config"""
|
||||
return json.dumps(self.dict(), cls=AgentOpsJSONEncoder)
|
||||
|
||||
|
||||
# checks if pytest is imported
|
||||
TESTING = "pytest" in sys.modules
|
||||
@@ -0,0 +1,36 @@
|
||||
"""
|
||||
AgentOps enums for user-friendly API.
|
||||
|
||||
This module provides simple enums that users can import from agentops
|
||||
without needing to know about OpenTelemetry internals.
|
||||
"""
|
||||
|
||||
from enum import Enum
|
||||
from opentelemetry.trace.status import StatusCode
|
||||
|
||||
|
||||
class TraceState(Enum):
|
||||
"""
|
||||
Enum for trace end states.
|
||||
|
||||
This provides a user-friendly interface that maps to OpenTelemetry StatusCode internally.
|
||||
Users can simply use agentops.TraceState.SUCCESS instead of importing OpenTelemetry.
|
||||
"""
|
||||
|
||||
SUCCESS = StatusCode.OK
|
||||
ERROR = StatusCode.ERROR
|
||||
UNSET = StatusCode.UNSET
|
||||
|
||||
def __str__(self) -> str:
|
||||
"""Return the name for string representation."""
|
||||
return self.name
|
||||
|
||||
def to_status_code(self) -> StatusCode:
|
||||
"""Convert to OpenTelemetry StatusCode."""
|
||||
return self.value
|
||||
|
||||
|
||||
# For backward compatibility, also provide these as module-level constants
|
||||
SUCCESS = TraceState.SUCCESS
|
||||
ERROR = TraceState.ERROR
|
||||
UNSET = TraceState.UNSET
|
||||
@@ -0,0 +1,38 @@
|
||||
class MultiSessionException(Exception):
|
||||
def __init__(self, message):
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class NoSessionException(Exception):
|
||||
def __init__(self, message="No session found"):
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class NoApiKeyException(Exception):
|
||||
def __init__(
|
||||
self,
|
||||
message="Could not initialize AgentOps client - API Key is missing."
|
||||
+ "\n\t Find your API key at https://app.agentops.ai/settings/projects",
|
||||
):
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class InvalidApiKeyException(Exception):
|
||||
def __init__(self, api_key, endpoint):
|
||||
message = f"API Key is invalid: {{{api_key}}}.\n\t Find your API key at {endpoint}/settings/projects"
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class ApiServerException(Exception):
|
||||
def __init__(self, message):
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class AgentOpsClientNotInitializedException(RuntimeError):
|
||||
def __init__(self, message="AgentOps client must be initialized before using this feature"):
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
class AgentOpsApiJwtExpiredException(Exception):
|
||||
def __init__(self, message="JWT token has expired"):
|
||||
super().__init__(message)
|
||||
@@ -0,0 +1,44 @@
|
||||
from agentops.helpers.time import get_ISO_time
|
||||
from agentops.helpers.serialization import (
|
||||
AgentOpsJSONEncoder,
|
||||
serialize_uuid,
|
||||
safe_serialize,
|
||||
is_jsonable,
|
||||
filter_unjsonable,
|
||||
)
|
||||
from agentops.helpers.system import (
|
||||
get_host_env,
|
||||
get_sdk_details,
|
||||
get_os_details,
|
||||
get_cpu_details,
|
||||
get_ram_details,
|
||||
get_disk_details,
|
||||
get_installed_packages,
|
||||
get_current_directory,
|
||||
get_virtual_env,
|
||||
)
|
||||
from agentops.helpers.version import get_agentops_version, check_agentops_update
|
||||
from agentops.helpers.env import get_env_bool, get_env_int, get_env_list
|
||||
|
||||
__all__ = [
|
||||
"get_ISO_time",
|
||||
"AgentOpsJSONEncoder",
|
||||
"serialize_uuid",
|
||||
"safe_serialize",
|
||||
"is_jsonable",
|
||||
"filter_unjsonable",
|
||||
"get_host_env",
|
||||
"get_sdk_details",
|
||||
"get_os_details",
|
||||
"get_cpu_details",
|
||||
"get_ram_details",
|
||||
"get_disk_details",
|
||||
"get_installed_packages",
|
||||
"get_current_directory",
|
||||
"get_virtual_env",
|
||||
"get_agentops_version",
|
||||
"check_agentops_update",
|
||||
"get_env_bool",
|
||||
"get_env_int",
|
||||
"get_env_list",
|
||||
]
|
||||
@@ -0,0 +1,54 @@
|
||||
"""
|
||||
Helpers for interacting with the AgentOps dashboard.
|
||||
"""
|
||||
|
||||
from typing import Union, Optional
|
||||
from termcolor import colored
|
||||
from opentelemetry.sdk.trace import Span, ReadableSpan
|
||||
from agentops.logging import logger
|
||||
|
||||
|
||||
def get_trace_url(span: Union[Span, ReadableSpan]) -> str:
|
||||
"""
|
||||
Generate a trace URL for a direct link to the session on the AgentOps dashboard.
|
||||
|
||||
Args:
|
||||
span: The span to generate the URL for.
|
||||
|
||||
Returns:
|
||||
The session URL.
|
||||
"""
|
||||
trace_id: Union[int, str] = span.context.trace_id
|
||||
|
||||
# Convert trace_id to hex string if it's not already
|
||||
# We don't add dashes to this to format it as a UUID since the dashboard doesn't either
|
||||
if isinstance(trace_id, int):
|
||||
trace_id = format(trace_id, "032x")
|
||||
|
||||
# Get the app_url from the config - import here to avoid circular imports
|
||||
from agentops import get_client
|
||||
|
||||
app_url = get_client().config.app_url
|
||||
|
||||
return f"{app_url}/sessions?trace_id={trace_id}"
|
||||
|
||||
|
||||
def log_trace_url(span: Union[Span, ReadableSpan], title: Optional[str] = None) -> None:
|
||||
"""
|
||||
Log the trace URL for the AgentOps dashboard.
|
||||
|
||||
Args:
|
||||
span: The span to log the URL for.
|
||||
title: Optional title for the trace.
|
||||
"""
|
||||
from agentops import get_client
|
||||
|
||||
try:
|
||||
client = get_client()
|
||||
if not client.config.log_session_replay_url:
|
||||
return
|
||||
except Exception:
|
||||
return
|
||||
|
||||
session_url = get_trace_url(span)
|
||||
logger.info(colored(f"\x1b[34mSession Replay for {title} trace: {session_url}\x1b[0m", "blue"))
|
||||
@@ -0,0 +1,50 @@
|
||||
"""
|
||||
Deprecation utilities for AgentOps SDK.
|
||||
"""
|
||||
|
||||
import functools
|
||||
from typing import Set, Callable, Any
|
||||
from agentops.logging import logger
|
||||
|
||||
# Track which deprecation warnings have been shown to avoid spam
|
||||
_shown_warnings: Set[str] = set()
|
||||
|
||||
|
||||
def deprecated(message: str):
|
||||
"""
|
||||
Decorator to mark functions as deprecated.
|
||||
|
||||
Args:
|
||||
message: Deprecation message to show
|
||||
"""
|
||||
|
||||
def decorator(func: Callable) -> Callable:
|
||||
@functools.wraps(func)
|
||||
def wrapper(*args: Any, **kwargs: Any) -> Any:
|
||||
warning_key = f"{func.__module__}.{func.__name__}"
|
||||
if warning_key not in _shown_warnings:
|
||||
logger.warning(f"{func.__name__}() is deprecated and will be removed in v4 in the future. {message}")
|
||||
_shown_warnings.add(warning_key)
|
||||
return func(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
return decorator
|
||||
|
||||
|
||||
def warn_deprecated_param(param_name: str, replacement: str = None):
|
||||
"""
|
||||
Warn about deprecated parameter usage.
|
||||
|
||||
Args:
|
||||
param_name: Name of the deprecated parameter
|
||||
replacement: Suggested replacement parameter
|
||||
"""
|
||||
warning_key = f"param.{param_name}"
|
||||
if warning_key not in _shown_warnings:
|
||||
if replacement:
|
||||
message = f"Parameter '{param_name}' is deprecated and will be removed in v4 in the future. Use '{replacement}' instead."
|
||||
else:
|
||||
message = f"Parameter '{param_name}' is deprecated and will be removed in v4 in the future."
|
||||
logger.warning(message)
|
||||
_shown_warnings.add(warning_key)
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Environment variable helper functions"""
|
||||
|
||||
import os
|
||||
from typing import List, Optional, Set
|
||||
|
||||
|
||||
def get_env_bool(key: str, default: bool) -> bool:
|
||||
"""Get boolean from environment variable
|
||||
|
||||
Args:
|
||||
key: Environment variable name
|
||||
default: Default value if not set
|
||||
|
||||
Returns:
|
||||
bool: Parsed boolean value
|
||||
"""
|
||||
val = os.getenv(key)
|
||||
if val is None:
|
||||
return default
|
||||
return val.lower() in ("true", "1", "t", "yes")
|
||||
|
||||
|
||||
def get_env_int(key: str, default: int) -> int:
|
||||
"""Get integer from environment variable
|
||||
|
||||
Args:
|
||||
key: Environment variable name
|
||||
default: Default value if not set
|
||||
|
||||
Returns:
|
||||
int: Parsed integer value
|
||||
"""
|
||||
try:
|
||||
return int(os.getenv(key, default))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def get_env_list(key: str, default: Optional[List[str]] = None) -> Set[str]:
|
||||
"""Get comma-separated list from environment variable
|
||||
|
||||
Args:
|
||||
key: Environment variable name
|
||||
default: Default list if not set
|
||||
|
||||
Returns:
|
||||
Set[str]: Set of parsed values
|
||||
"""
|
||||
val = os.getenv(key)
|
||||
if val is None:
|
||||
return set(default or [])
|
||||
return set(val.split(","))
|
||||
@@ -0,0 +1,137 @@
|
||||
"""Serialization helpers for AgentOps"""
|
||||
|
||||
import json
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from enum import Enum
|
||||
from typing import Any
|
||||
from uuid import UUID
|
||||
|
||||
from agentops.logging import logger
|
||||
|
||||
|
||||
def is_jsonable(x):
|
||||
try:
|
||||
json.dumps(x)
|
||||
return True
|
||||
except (TypeError, OverflowError):
|
||||
return False
|
||||
|
||||
|
||||
def filter_unjsonable(d: dict) -> dict:
|
||||
def filter_dict(obj):
|
||||
if isinstance(obj, dict):
|
||||
return {
|
||||
k: (
|
||||
filter_dict(v)
|
||||
if isinstance(v, (dict, list)) or is_jsonable(v)
|
||||
else str(v)
|
||||
if isinstance(v, UUID)
|
||||
else ""
|
||||
)
|
||||
for k, v in obj.items()
|
||||
}
|
||||
elif isinstance(obj, list):
|
||||
return [
|
||||
(
|
||||
filter_dict(x)
|
||||
if isinstance(x, (dict, list)) or is_jsonable(x)
|
||||
else str(x)
|
||||
if isinstance(x, UUID)
|
||||
else ""
|
||||
)
|
||||
for x in obj
|
||||
]
|
||||
else:
|
||||
return obj if is_jsonable(obj) or isinstance(obj, UUID) else ""
|
||||
|
||||
return filter_dict(d)
|
||||
|
||||
|
||||
class AgentOpsJSONEncoder(json.JSONEncoder):
|
||||
"""Custom JSON encoder for AgentOps types"""
|
||||
|
||||
def default(self, obj: Any) -> Any:
|
||||
if isinstance(obj, UUID):
|
||||
return str(obj)
|
||||
if isinstance(obj, datetime):
|
||||
return obj.isoformat()
|
||||
if isinstance(obj, Decimal):
|
||||
return str(obj)
|
||||
if isinstance(obj, set):
|
||||
return list(obj)
|
||||
if hasattr(obj, "to_json"):
|
||||
return obj.to_json()
|
||||
if isinstance(obj, Enum):
|
||||
return obj.value
|
||||
return str(obj)
|
||||
|
||||
|
||||
def serialize_uuid(obj: UUID) -> str:
|
||||
"""Serialize UUID to string"""
|
||||
return str(obj)
|
||||
|
||||
|
||||
def model_to_dict(obj: Any) -> dict:
|
||||
"""Convert a model object to a dictionary safely.
|
||||
|
||||
Handles various model types including:
|
||||
- Pydantic models (model_dump/dict methods)
|
||||
- Dictionary-like objects
|
||||
- API response objects with parse method
|
||||
- Objects with __dict__ attribute
|
||||
|
||||
Args:
|
||||
obj: The model object to convert to dictionary
|
||||
|
||||
Returns:
|
||||
Dictionary representation of the object, or empty dict if conversion fails
|
||||
"""
|
||||
if obj is None:
|
||||
return {}
|
||||
if isinstance(obj, dict):
|
||||
return obj
|
||||
if hasattr(obj, "model_dump"): # Pydantic v2
|
||||
return obj.model_dump()
|
||||
elif hasattr(obj, "dict"): # Pydantic v1
|
||||
return obj.dict()
|
||||
# TODO this is causing recursion on nested objects.
|
||||
# elif hasattr(obj, "parse"): # Raw API response
|
||||
# return model_to_dict(obj.parse())
|
||||
else:
|
||||
# Try to use __dict__ as fallback
|
||||
try:
|
||||
return obj.__dict__
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def safe_serialize(obj: Any) -> Any:
|
||||
"""Safely serialize an object to JSON-compatible format
|
||||
|
||||
This function handles complex objects by:
|
||||
1. Returning strings untouched (even if they contain JSON)
|
||||
2. Converting models to dictionaries
|
||||
3. Using custom JSON encoder to handle special types
|
||||
4. Falling back to string representation only when necessary
|
||||
|
||||
Args:
|
||||
obj: The object to serialize
|
||||
|
||||
Returns:
|
||||
If obj is a string, returns the original string untouched.
|
||||
Otherwise, returns a JSON string representation of the object.
|
||||
"""
|
||||
# Return strings untouched
|
||||
if isinstance(obj, str):
|
||||
return obj
|
||||
|
||||
# Convert any model objects to dictionaries
|
||||
if hasattr(obj, "model_dump") or hasattr(obj, "dict") or hasattr(obj, "parse"):
|
||||
obj = model_to_dict(obj)
|
||||
|
||||
try:
|
||||
return json.dumps(obj, cls=AgentOpsJSONEncoder)
|
||||
except (TypeError, ValueError) as e:
|
||||
logger.warning(f"Failed to serialize object: {e}")
|
||||
return str(obj)
|
||||
@@ -0,0 +1,178 @@
|
||||
import importlib.metadata
|
||||
import os
|
||||
import platform
|
||||
import socket
|
||||
import sys
|
||||
|
||||
import psutil # type: ignore
|
||||
|
||||
from agentops.logging import logger
|
||||
from agentops.helpers.version import get_agentops_version
|
||||
|
||||
|
||||
def get_imported_libraries():
|
||||
"""
|
||||
Get the top-level imported libraries in the current script.
|
||||
|
||||
Returns:
|
||||
list: List of imported libraries
|
||||
"""
|
||||
user_libs = []
|
||||
|
||||
builtin_modules = {
|
||||
"builtins",
|
||||
"sys",
|
||||
"os",
|
||||
"_thread",
|
||||
"abc",
|
||||
"io",
|
||||
"re",
|
||||
"types",
|
||||
"collections",
|
||||
"enum",
|
||||
"math",
|
||||
"datetime",
|
||||
"time",
|
||||
"warnings",
|
||||
}
|
||||
|
||||
try:
|
||||
main_module = sys.modules.get("__main__")
|
||||
if main_module and hasattr(main_module, "__dict__"):
|
||||
for name, obj in main_module.__dict__.items():
|
||||
if isinstance(obj, type(sys)) and hasattr(obj, "__name__"):
|
||||
mod_name = obj.__name__.split(".")[0]
|
||||
if mod_name and not mod_name.startswith("_") and mod_name not in builtin_modules:
|
||||
user_libs.append(mod_name)
|
||||
except Exception as e:
|
||||
logger.debug(f"Error getting imports: {e}")
|
||||
|
||||
return user_libs
|
||||
|
||||
|
||||
def get_sdk_details():
|
||||
try:
|
||||
return {
|
||||
"AgentOps SDK Version": get_agentops_version(),
|
||||
"Python Version": platform.python_version(),
|
||||
"System Packages": get_sys_packages(),
|
||||
}
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def get_sys_packages():
|
||||
sys_packages = {}
|
||||
for module in sys.modules:
|
||||
try:
|
||||
version = importlib.metadata.version(module)
|
||||
sys_packages[module] = version
|
||||
except importlib.metadata.PackageNotFoundError:
|
||||
# Skip built-in modules and those without package metadata
|
||||
continue
|
||||
|
||||
return sys_packages
|
||||
|
||||
|
||||
def get_installed_packages():
|
||||
try:
|
||||
return {
|
||||
# TODO: add to opt out
|
||||
"Installed Packages": {
|
||||
dist.metadata.get("Name"): dist.metadata.get("Version") for dist in importlib.metadata.distributions()
|
||||
}
|
||||
}
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def get_current_directory():
|
||||
try:
|
||||
return {"Project Working Directory": os.getcwd()}
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def get_virtual_env():
|
||||
try:
|
||||
return {"Virtual Environment": os.environ.get("VIRTUAL_ENV", None)}
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def get_os_details():
|
||||
try:
|
||||
return {
|
||||
"Hostname": socket.gethostname(),
|
||||
"OS": platform.system(),
|
||||
"OS Version": platform.version(),
|
||||
"OS Release": platform.release(),
|
||||
}
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def get_cpu_details():
|
||||
try:
|
||||
return {
|
||||
"Physical cores": psutil.cpu_count(logical=False),
|
||||
"Total cores": psutil.cpu_count(logical=True),
|
||||
# "Max Frequency": f"{psutil.cpu_freq().max:.2f}Mhz", # Fails right now
|
||||
"CPU Usage": f"{psutil.cpu_percent()}%",
|
||||
}
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def get_ram_details():
|
||||
try:
|
||||
ram_info = psutil.virtual_memory()
|
||||
return {
|
||||
"Total": f"{ram_info.total / (1024**3):.2f} GB",
|
||||
"Available": f"{ram_info.available / (1024**3):.2f} GB",
|
||||
"Used": f"{ram_info.used / (1024**3):.2f} GB",
|
||||
"Percentage": f"{ram_info.percent}%",
|
||||
}
|
||||
except:
|
||||
return {}
|
||||
|
||||
|
||||
def get_disk_details():
|
||||
partitions = psutil.disk_partitions()
|
||||
disk_info = {}
|
||||
for partition in partitions:
|
||||
try:
|
||||
usage = psutil.disk_usage(partition.mountpoint)
|
||||
disk_info[partition.device] = {
|
||||
"Mountpoint": partition.mountpoint,
|
||||
"Total": f"{usage.total / (1024**3):.2f} GB",
|
||||
"Used": f"{usage.used / (1024**3):.2f} GB",
|
||||
"Free": f"{usage.free / (1024**3):.2f} GB",
|
||||
"Percentage": f"{usage.percent}%",
|
||||
}
|
||||
except OSError as inaccessible:
|
||||
# Skip inaccessible partitions, such as removable drives with no media
|
||||
logger.debug("Mountpoint %s inaccessible: %s", partition.mountpoint, inaccessible)
|
||||
|
||||
return disk_info
|
||||
|
||||
|
||||
def get_host_env(opt_out: bool = False):
|
||||
if opt_out:
|
||||
return {
|
||||
"SDK": get_sdk_details(),
|
||||
"OS": get_os_details(),
|
||||
"Project Working Directory": get_current_directory(),
|
||||
"Virtual Environment": get_virtual_env(),
|
||||
}
|
||||
else:
|
||||
return {
|
||||
"SDK": get_sdk_details(),
|
||||
"OS": get_os_details(),
|
||||
"CPU": get_cpu_details(),
|
||||
"RAM": get_ram_details(),
|
||||
"Disk": get_disk_details(),
|
||||
"Installed Packages": get_installed_packages(),
|
||||
"Project Working Directory": get_current_directory(),
|
||||
"Virtual Environment": get_virtual_env(),
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def get_ISO_time():
|
||||
"""
|
||||
Get the current UTC time in ISO 8601 format with milliseconds precision in UTC timezone.
|
||||
|
||||
Returns:
|
||||
str: The current UTC time as a string in ISO 8601 format.
|
||||
"""
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
@@ -0,0 +1,36 @@
|
||||
from importlib.metadata import PackageNotFoundError, version
|
||||
|
||||
import requests
|
||||
|
||||
from agentops.logging import logger
|
||||
|
||||
|
||||
def get_agentops_version():
|
||||
try:
|
||||
pkg_version = version("agentops")
|
||||
return pkg_version
|
||||
except Exception as e:
|
||||
logger.warning("Error reading package version: %s", e)
|
||||
return None
|
||||
|
||||
|
||||
def check_agentops_update():
|
||||
try:
|
||||
response = requests.get("https://pypi.org/pypi/agentops/json")
|
||||
|
||||
if response.status_code == 200:
|
||||
json_data = response.json()
|
||||
latest_version = json_data["info"]["version"]
|
||||
|
||||
try:
|
||||
current_version = version("agentops")
|
||||
except PackageNotFoundError:
|
||||
return None
|
||||
|
||||
if not latest_version == current_version:
|
||||
logger.warning(
|
||||
" WARNING: agentops is out of date. Please update with the command: 'pip install --upgrade agentops'"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to check for updates: {e}")
|
||||
return None
|
||||
@@ -0,0 +1,133 @@
|
||||
# OpenTelemetry Implementation Notes
|
||||
|
||||
This document outlines best practices and implementation details for OpenTelemetry in AgentOps instrumentations.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Context Propagation
|
||||
|
||||
OpenTelemetry relies on proper context propagation to maintain parent-child relationships between spans. This is essential for:
|
||||
|
||||
- Creating accurate trace waterfalls in visualizations
|
||||
- Ensuring all spans from the same logical operation share a trace ID
|
||||
- Allowing proper querying and filtering of related operations
|
||||
|
||||
### Core Patterns
|
||||
|
||||
When implementing instrumentations that need to maintain context across different execution contexts:
|
||||
|
||||
1. **Store span contexts in dictionaries:**
|
||||
```python
|
||||
# Use weakref dictionaries to avoid memory leaks
|
||||
self._span_contexts = weakref.WeakKeyDictionary()
|
||||
self._trace_root_contexts = weakref.WeakKeyDictionary()
|
||||
```
|
||||
|
||||
2. **Create spans with explicit parent contexts:**
|
||||
```python
|
||||
parent_context = self._get_parent_context(trace_obj)
|
||||
with trace.start_as_current_span(
|
||||
name=span_name,
|
||||
context=parent_context,
|
||||
kind=trace.SpanKind.CLIENT,
|
||||
attributes=attributes,
|
||||
) as span:
|
||||
# Span operations here
|
||||
# Store the span's context for future reference
|
||||
context = trace.set_span_in_context(span)
|
||||
self._span_contexts[span_obj] = context
|
||||
```
|
||||
|
||||
3. **Implement helper methods to retrieve appropriate parent contexts:**
|
||||
```python
|
||||
def _get_parent_context(self, trace_obj):
|
||||
# Try to get the trace's root context if it exists
|
||||
if trace_obj in self._trace_root_contexts:
|
||||
return self._trace_root_contexts[trace_obj]
|
||||
|
||||
# Otherwise, use the current context
|
||||
return context_api.context.get_current()
|
||||
```
|
||||
|
||||
4. **Debug trace continuity:**
|
||||
```python
|
||||
current_span = trace.get_current_span()
|
||||
span_context = current_span.get_span_context()
|
||||
trace_id = format_trace_id(span_context.trace_id)
|
||||
logging.debug(f"Current span trace ID: {trace_id}")
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Naming conflicts:** Avoid using `trace` as a parameter name when you're also importing the OpenTelemetry `trace` module
|
||||
```python
|
||||
# Bad
|
||||
def on_trace_start(self, trace):
|
||||
# This will cause conflicts with the imported trace module
|
||||
|
||||
# Good
|
||||
def on_trace_start(self, trace_obj):
|
||||
# No conflicts with OpenTelemetry's trace module
|
||||
```
|
||||
|
||||
2. **Missing parent contexts:** Always explicitly provide parent contexts when available, don't rely on current context alone
|
||||
|
||||
3. **Memory leaks:** Use `weakref.WeakKeyDictionary()` for storing spans to allow garbage collection
|
||||
|
||||
4. **Lost context:** When calling async or callback functions, be sure to preserve and pass the context
|
||||
|
||||
## Testing Context Propagation
|
||||
|
||||
To verify proper context propagation:
|
||||
|
||||
1. Enable debug logging for trace IDs
|
||||
2. Run a simple end-to-end test that generates multiple spans
|
||||
3. Verify all spans share the same trace ID
|
||||
4. Check that parent-child relationships are correctly established
|
||||
|
||||
```python
|
||||
# Example debug logging
|
||||
logging.debug(f"Span {span.name} has trace ID: {format_trace_id(span.get_span_context().trace_id)}")
|
||||
```
|
||||
|
||||
## Timestamp Handling in OpenTelemetry
|
||||
|
||||
When working with OpenTelemetry spans and timestamps:
|
||||
|
||||
1. **Automatic Timestamp Tracking:** OpenTelemetry automatically tracks timestamps for spans. When a span is created with `tracer.start_span()` or `tracer.start_as_current_span()`, the start time is captured automatically. When `span.end()` is called, the end time is recorded.
|
||||
|
||||
2. **No Manual Timestamp Setting Required:** The standard instrumentation pattern does not require manually setting timestamp attributes on spans. Instead, OpenTelemetry handles this internally through the SpanProcessor and Exporter classes.
|
||||
|
||||
3. **Timestamp Representation:** In the OpenTelemetry data model, timestamps are stored as nanoseconds since the Unix epoch (January 1, 1970).
|
||||
|
||||
4. **Serialization Responsibility:** The serialization of timestamps from OTel spans to output formats like JSON is handled by the Exporter components. If timestamps aren't appearing correctly in output APIs, the issue is likely in the API exporter, not in the span creation code.
|
||||
|
||||
5. **Debugging Timestamps:** To debug timestamp issues, verify that spans are properly starting and ending, rather than manually setting timestamp attributes:
|
||||
|
||||
```python
|
||||
# Good pattern - timestamps handled by OpenTelemetry automatically
|
||||
with tracer.start_as_current_span("my_operation") as span:
|
||||
# Do work
|
||||
pass # span.end() is called automatically
|
||||
```
|
||||
|
||||
Note: If timestamps are missing in API output (e.g., empty "start_time" fields), focus on fixes in the exporter and serialization layer, not by manually tracking timestamps in instrumentation code.
|
||||
|
||||
## Attributes in OpenTelemetry
|
||||
|
||||
When working with span attributes in OpenTelemetry:
|
||||
|
||||
1. **Root Attributes Node:** The root `attributes` object in the API output JSON should always be empty. This is by design. All attribute data should be stored in the `span_attributes` object.
|
||||
|
||||
2. **Span Attributes:** The `span_attributes` object is where all user-defined and semantic attribute data should be stored. This allows for a structured, hierarchical representation of attributes.
|
||||
|
||||
3. **Structure Difference:** While the root `attributes` appears as an empty object in the API output, this is normal and expected. Do not attempt to populate this object directly or duplicate data from `span_attributes` into it.
|
||||
|
||||
4. **Setting Attributes:** Always set span attributes using the semantic conventions defined in the `agentops/semconv` module:
|
||||
|
||||
```python
|
||||
from agentops.semconv import agent
|
||||
|
||||
# Good pattern - using semantic conventions
|
||||
span.set_attribute(agent.AGENT_NAME, "My Agent")
|
||||
```
|
||||
@@ -0,0 +1,167 @@
|
||||
# AgentOps Instrumentation
|
||||
|
||||
This package provides OpenTelemetry instrumentation for various LLM providers and related services.
|
||||
|
||||
## Available Instrumentors
|
||||
|
||||
- **OpenAI** (`v0.27.0+` and `v1.0.0+`)
|
||||
- **Anthropic** (`v0.7.0+`)
|
||||
- **Google GenAI** (`v0.1.0+`)
|
||||
- **IBM WatsonX AI** (`v0.1.0+`)
|
||||
- **CrewAI** (`v0.56.0+`)
|
||||
- **AG2/AutoGen** (`v0.3.2+`)
|
||||
- **Google ADK** (`v0.1.0+`)
|
||||
- **Agno** (`v0.0.1+`)
|
||||
- **Mem0** (`v0.1.0+`)
|
||||
- **smolagents** (`v0.1.0+`)
|
||||
|
||||
## Common Module Usage
|
||||
|
||||
The `agentops.instrumentation.common` module provides shared utilities for creating instrumentations:
|
||||
|
||||
### Base Instrumentor
|
||||
|
||||
Use `CommonInstrumentor` for creating new instrumentations:
|
||||
|
||||
```python
|
||||
from agentops.instrumentation.common import CommonInstrumentor, InstrumentorConfig, WrapConfig
|
||||
|
||||
class MyInstrumentor(CommonInstrumentor):
|
||||
def __init__(self):
|
||||
config = InstrumentorConfig(
|
||||
library_name="my-library",
|
||||
library_version="1.0.0",
|
||||
wrapped_methods=[
|
||||
WrapConfig(
|
||||
trace_name="my.method",
|
||||
package="my_library.module",
|
||||
class_name="MyClass",
|
||||
method_name="my_method",
|
||||
handler=my_attribute_handler
|
||||
)
|
||||
],
|
||||
dependencies=["my-library >= 1.0.0"]
|
||||
)
|
||||
super().__init__(config)
|
||||
```
|
||||
|
||||
### Attribute Handlers
|
||||
|
||||
Create attribute handlers to extract data from method calls:
|
||||
|
||||
```python
|
||||
from agentops.instrumentation.common import AttributeMap
|
||||
|
||||
def my_attribute_handler(args=None, kwargs=None, return_value=None) -> AttributeMap:
|
||||
attributes = {}
|
||||
|
||||
if kwargs and "model" in kwargs:
|
||||
attributes["llm.request.model"] = kwargs["model"]
|
||||
|
||||
if return_value and hasattr(return_value, "usage"):
|
||||
attributes["llm.usage.total_tokens"] = return_value.usage.total_tokens
|
||||
|
||||
return attributes
|
||||
```
|
||||
|
||||
### Span Management
|
||||
|
||||
Use the span management utilities for consistent span creation:
|
||||
|
||||
```python
|
||||
from agentops.instrumentation.common import create_span, SpanAttributeManager
|
||||
|
||||
# Create an attribute manager
|
||||
attr_manager = SpanAttributeManager(service_name="my-service")
|
||||
|
||||
# Use the create_span context manager
|
||||
with create_span(
|
||||
tracer,
|
||||
"my.operation",
|
||||
attributes={"my.attribute": "value"},
|
||||
attribute_manager=attr_manager
|
||||
) as span:
|
||||
# Your operation code here
|
||||
pass
|
||||
```
|
||||
|
||||
### Token Counting
|
||||
|
||||
Use the token counting utilities for consistent token usage extraction:
|
||||
|
||||
```python
|
||||
from agentops.instrumentation.common import TokenUsageExtractor, set_token_usage_attributes
|
||||
|
||||
# Extract token usage from a response
|
||||
usage = TokenUsageExtractor.extract_from_response(response)
|
||||
|
||||
# Set token usage attributes on a span
|
||||
set_token_usage_attributes(span, response)
|
||||
```
|
||||
|
||||
### Streaming Support
|
||||
|
||||
Use streaming utilities for handling streaming responses:
|
||||
|
||||
```python
|
||||
from agentops.instrumentation.common import create_stream_wrapper_factory, StreamingResponseHandler
|
||||
|
||||
# Create a stream wrapper factory
|
||||
wrapper = create_stream_wrapper_factory(
|
||||
tracer,
|
||||
"my.stream",
|
||||
extract_chunk_content=StreamingResponseHandler.extract_generic_chunk_content,
|
||||
initial_attributes={"stream.type": "text"}
|
||||
)
|
||||
|
||||
# Apply to streaming methods
|
||||
wrap_function_wrapper("my_module", "stream_method", wrapper)
|
||||
```
|
||||
|
||||
### Metrics
|
||||
|
||||
Use standard metrics for consistency across instrumentations:
|
||||
|
||||
```python
|
||||
from agentops.instrumentation.common import StandardMetrics, MetricsRecorder
|
||||
|
||||
# Create standard metrics
|
||||
metrics = StandardMetrics.create_standard_metrics(meter)
|
||||
|
||||
# Use the metrics recorder
|
||||
recorder = MetricsRecorder(metrics)
|
||||
recorder.record_token_usage(prompt_tokens=100, completion_tokens=50)
|
||||
recorder.record_duration(1.5)
|
||||
```
|
||||
|
||||
## Creating a New Instrumentor
|
||||
|
||||
1. Create a new directory under `agentops/instrumentation/` for your provider
|
||||
2. Create an `__init__.py` file with version information
|
||||
3. Create an `instrumentor.py` file extending `CommonInstrumentor`
|
||||
4. Create attribute handlers in an `attributes/` subdirectory
|
||||
5. Add your instrumentor to the main `__init__.py` configuration
|
||||
|
||||
Example structure:
|
||||
```
|
||||
agentops/instrumentation/
|
||||
├── my_provider/
|
||||
│ ├── __init__.py
|
||||
│ ├── instrumentor.py
|
||||
│ └── attributes/
|
||||
│ ├── __init__.py
|
||||
│ └── handlers.py
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Use Common Utilities**: Leverage the common module for consistency
|
||||
2. **Follow Semantic Conventions**: Use attributes from `agentops.semconv`
|
||||
3. **Handle Errors Gracefully**: Wrap operations in try-except blocks
|
||||
4. **Support Async**: Provide both sync and async method wrapping
|
||||
5. **Document Attributes**: Comment on what attributes are captured
|
||||
6. **Test Thoroughly**: Write unit tests for your instrumentor
|
||||
|
||||
## Examples
|
||||
|
||||
See the `examples/` directory for usage examples of each instrumentor.
|
||||
@@ -0,0 +1,610 @@
|
||||
"""
|
||||
AgentOps Instrumentation Module
|
||||
|
||||
This module provides automatic instrumentation for various LLM providers and agentic libraries.
|
||||
It works by monitoring Python imports and automatically instrumenting packages as they are imported.
|
||||
|
||||
Key Features:
|
||||
- Automatic detection and instrumentation of LLM providers (OpenAI, Anthropic, etc.)
|
||||
- Support for agentic libraries (CrewAI, AutoGen, etc.)
|
||||
- Version-aware instrumentation (only activates for supported versions)
|
||||
- Smart handling of provider vs agentic library conflicts
|
||||
- Non-intrusive monitoring using Python's import system
|
||||
"""
|
||||
|
||||
from typing import Optional, Set, TypedDict
|
||||
|
||||
try:
|
||||
from typing import NotRequired
|
||||
except ImportError:
|
||||
from typing_extensions import NotRequired
|
||||
from types import ModuleType
|
||||
from dataclasses import dataclass
|
||||
import importlib
|
||||
import sys
|
||||
from packaging.version import Version, parse
|
||||
import builtins
|
||||
|
||||
# Add os and site for path checking
|
||||
import os
|
||||
import site
|
||||
|
||||
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor # type: ignore
|
||||
|
||||
from agentops.logging import logger
|
||||
from agentops.sdk.core import tracer
|
||||
from agentops.instrumentation.common import get_library_version
|
||||
|
||||
|
||||
# Define the structure for instrumentor configurations
|
||||
class InstrumentorConfig(TypedDict):
|
||||
module_name: str
|
||||
class_name: str
|
||||
min_version: str
|
||||
package_name: NotRequired[str] # Optional: actual pip package name if different from module
|
||||
|
||||
|
||||
# Configuration for supported LLM providers
|
||||
PROVIDERS: dict[str, InstrumentorConfig] = {
|
||||
"openai": {
|
||||
"module_name": "agentops.instrumentation.providers.openai",
|
||||
"class_name": "OpenaiInstrumentor",
|
||||
"min_version": "1.0.0",
|
||||
},
|
||||
"anthropic": {
|
||||
"module_name": "agentops.instrumentation.providers.anthropic",
|
||||
"class_name": "AnthropicInstrumentor",
|
||||
"min_version": "0.32.0",
|
||||
},
|
||||
"ibm_watsonx_ai": {
|
||||
"module_name": "agentops.instrumentation.providers.ibm_watsonx_ai",
|
||||
"class_name": "WatsonxInstrumentor",
|
||||
"min_version": "0.1.0",
|
||||
},
|
||||
"google.genai": {
|
||||
"module_name": "agentops.instrumentation.providers.google_genai",
|
||||
"class_name": "GoogleGenaiInstrumentor",
|
||||
"min_version": "0.1.0",
|
||||
"package_name": "google-genai", # Actual pip package name
|
||||
},
|
||||
"mem0": {
|
||||
"module_name": "agentops.instrumentation.providers.mem0",
|
||||
"class_name": "Mem0Instrumentor",
|
||||
"min_version": "0.1.0",
|
||||
"package_name": "mem0ai",
|
||||
},
|
||||
}
|
||||
|
||||
# Configuration for supported agentic libraries
|
||||
AGENTIC_LIBRARIES: dict[str, InstrumentorConfig] = {
|
||||
"crewai": {
|
||||
"module_name": "agentops.instrumentation.agentic.crewai",
|
||||
"class_name": "CrewaiInstrumentor",
|
||||
"min_version": "0.56.0",
|
||||
},
|
||||
"autogen": {
|
||||
"module_name": "agentops.instrumentation.agentic.ag2",
|
||||
"class_name": "AG2Instrumentor",
|
||||
"min_version": "0.3.2",
|
||||
},
|
||||
"agents": {
|
||||
"module_name": "agentops.instrumentation.agentic.openai_agents",
|
||||
"class_name": "OpenAIAgentsInstrumentor",
|
||||
"min_version": "0.0.1",
|
||||
},
|
||||
"google.adk": {
|
||||
"module_name": "agentops.instrumentation.agentic.google_adk",
|
||||
"class_name": "GooogleAdkInstrumentor",
|
||||
"min_version": "0.1.0",
|
||||
},
|
||||
"agno": {
|
||||
"module_name": "agentops.instrumentation.agentic.agno",
|
||||
"class_name": "AgnoInstrumentor",
|
||||
"min_version": "1.5.8",
|
||||
},
|
||||
"smolagents": {
|
||||
"module_name": "agentops.instrumentation.agentic.smolagents",
|
||||
"class_name": "SmolagentsInstrumentor",
|
||||
"min_version": "1.0.0",
|
||||
},
|
||||
"langgraph": {
|
||||
"module_name": "agentops.instrumentation.agentic.langgraph",
|
||||
"class_name": "LanggraphInstrumentor",
|
||||
"min_version": "0.2.0",
|
||||
},
|
||||
"xpander_sdk": {
|
||||
"module_name": "agentops.instrumentation.agentic.xpander",
|
||||
"class_name": "XpanderInstrumentor",
|
||||
"min_version": "1.0.0",
|
||||
"package_name": "xpander-sdk",
|
||||
},
|
||||
"haystack": {
|
||||
"module_name": "agentops.instrumentation.agentic.haystack",
|
||||
"class_name": "HaystackInstrumentor",
|
||||
"min_version": "2.0.0",
|
||||
"package_name": "haystack-ai",
|
||||
},
|
||||
}
|
||||
|
||||
# Combine all target packages for monitoring
|
||||
TARGET_PACKAGES = set(PROVIDERS.keys()) | set(AGENTIC_LIBRARIES.keys())
|
||||
|
||||
# Create a single instance of the manager
|
||||
# _manager = InstrumentationManager() # Removed
|
||||
|
||||
# Module-level state variables
|
||||
_active_instrumentors: list[BaseInstrumentor] = []
|
||||
_original_builtins_import = builtins.__import__ # Store original import
|
||||
_instrumenting_packages: Set[str] = set()
|
||||
_has_agentic_library: bool = False
|
||||
|
||||
|
||||
# New helper function to check module origin
|
||||
def _is_installed_package(module_obj: ModuleType, package_name_key: str) -> bool:
|
||||
"""
|
||||
Determines if the given module object corresponds to an installed site-package
|
||||
rather than a local module, especially when names might collide.
|
||||
`package_name_key` is the key from TARGET_PACKAGES (e.g., 'agents', 'google.adk').
|
||||
"""
|
||||
if not hasattr(module_obj, "__file__") or not module_obj.__file__:
|
||||
logger.debug(
|
||||
f"_is_installed_package: Module '{package_name_key}' has no __file__, assuming it might be an SDK namespace package. Returning True."
|
||||
)
|
||||
return True
|
||||
|
||||
module_path = os.path.normcase(os.path.realpath(os.path.abspath(module_obj.__file__)))
|
||||
|
||||
# Priority 1: Check if it's in any site-packages directory.
|
||||
site_packages_dirs = site.getsitepackages()
|
||||
if isinstance(site_packages_dirs, str):
|
||||
site_packages_dirs = [site_packages_dirs]
|
||||
|
||||
if hasattr(site, "USER_SITE") and site.USER_SITE and os.path.exists(site.USER_SITE):
|
||||
site_packages_dirs.append(site.USER_SITE)
|
||||
|
||||
normalized_site_packages_dirs = [
|
||||
os.path.normcase(os.path.realpath(p)) for p in site_packages_dirs if p and os.path.exists(p)
|
||||
]
|
||||
|
||||
for sp_dir in normalized_site_packages_dirs:
|
||||
if module_path.startswith(sp_dir):
|
||||
logger.debug(
|
||||
f"_is_installed_package: Module '{package_name_key}' is a library, instrumenting '{package_name_key}'."
|
||||
)
|
||||
return True
|
||||
|
||||
# Priority 2: If not in site-packages, it's highly likely a local module or not an SDK we target.
|
||||
logger.debug(f"_is_installed_package: Module '{package_name_key}' is a local module, skipping instrumentation.")
|
||||
return False
|
||||
|
||||
|
||||
def _is_package_instrumented(package_name: str) -> bool:
|
||||
"""Check if a package is already instrumented by looking at active instrumentors."""
|
||||
# Handle package.module names by converting dots to underscores for comparison
|
||||
normalized_target_name = package_name.replace(".", "_").lower()
|
||||
for instrumentor in _active_instrumentors:
|
||||
# Check based on the key it was registered with
|
||||
if (
|
||||
hasattr(instrumentor, "_agentops_instrumented_package_key")
|
||||
and instrumentor._agentops_instrumented_package_key == package_name
|
||||
):
|
||||
return True
|
||||
|
||||
# Fallback to class name check (existing logic, less precise)
|
||||
# We use split('.')[-1] for cases like 'google.genai' to match GenAIInstrumentor
|
||||
instrumentor_class_name_prefix = instrumentor.__class__.__name__.lower().replace("instrumentor", "")
|
||||
target_base_name = package_name.split(".")[-1].lower()
|
||||
normalized_class_name_match = (
|
||||
normalized_target_name.startswith(instrumentor_class_name_prefix)
|
||||
or target_base_name == instrumentor_class_name_prefix
|
||||
)
|
||||
|
||||
if normalized_class_name_match:
|
||||
# This fallback can be noisy, let's make it more specific or rely on the key above more
|
||||
# For now, if the key matches or this broad name match works, consider instrumented.
|
||||
# This helps if _agentops_instrumented_package_key was somehow not set.
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def _uninstrument_providers():
|
||||
"""Uninstrument all provider instrumentors while keeping agentic libraries active."""
|
||||
global _active_instrumentors
|
||||
new_active_instrumentors = []
|
||||
uninstrumented_any = False
|
||||
for instrumentor in _active_instrumentors:
|
||||
instrumented_key = getattr(instrumentor, "_agentops_instrumented_package_key", None)
|
||||
if instrumented_key and instrumented_key in PROVIDERS:
|
||||
try:
|
||||
instrumentor.uninstrument()
|
||||
logger.debug(
|
||||
f"AgentOps: Uninstrumented provider: {instrumentor.__class__.__name__} (for package '{instrumented_key}') due to agentic library activation."
|
||||
)
|
||||
uninstrumented_any = True
|
||||
except Exception as e:
|
||||
logger.error(f"Error uninstrumenting provider {instrumentor.__class__.__name__}: {e}")
|
||||
else:
|
||||
# Keep non-provider instrumentors or those without our key (shouldn't happen for managed ones)
|
||||
new_active_instrumentors.append(instrumentor)
|
||||
|
||||
if uninstrumented_any or not new_active_instrumentors and _active_instrumentors:
|
||||
logger.debug(
|
||||
f"_uninstrument_providers: Processed. Previous active: {len(_active_instrumentors)}, New active after filtering providers: {len(new_active_instrumentors)}"
|
||||
)
|
||||
_active_instrumentors = new_active_instrumentors
|
||||
|
||||
|
||||
def _should_instrument_package(package_name: str) -> bool:
|
||||
"""
|
||||
Determine if a package should be instrumented based on current state.
|
||||
Handles special cases for agentic libraries and providers.
|
||||
"""
|
||||
global _has_agentic_library
|
||||
|
||||
# If already instrumented by AgentOps (using our refined check), skip.
|
||||
if _is_package_instrumented(package_name):
|
||||
logger.debug(f"_should_instrument_package: '{package_name}' already instrumented by AgentOps. Skipping.")
|
||||
return False
|
||||
|
||||
is_target_agentic = package_name in AGENTIC_LIBRARIES
|
||||
is_target_provider = package_name in PROVIDERS
|
||||
|
||||
if not is_target_agentic and not is_target_provider:
|
||||
logger.debug(
|
||||
f"_should_instrument_package: '{package_name}' is not a targeted provider or agentic library. Skipping."
|
||||
)
|
||||
return False
|
||||
|
||||
if _has_agentic_library:
|
||||
# An agentic library is already active.
|
||||
if is_target_agentic:
|
||||
logger.debug(
|
||||
f"AgentOps: An agentic library is active. Skipping instrumentation for subsequent agentic library '{package_name}'."
|
||||
)
|
||||
return False
|
||||
if is_target_provider:
|
||||
logger.debug(
|
||||
f"AgentOps: An agentic library is active. Skipping instrumentation for provider '{package_name}'."
|
||||
)
|
||||
return False
|
||||
else:
|
||||
# No agentic library is active yet.
|
||||
if is_target_agentic:
|
||||
logger.debug(
|
||||
f"AgentOps: '{package_name}' is the first-targeted agentic library. Will uninstrument providers if any are/become active."
|
||||
)
|
||||
_uninstrument_providers()
|
||||
return True
|
||||
if is_target_provider:
|
||||
logger.debug(
|
||||
f"_should_instrument_package: '{package_name}' is a provider, no agentic library active. Allowing."
|
||||
)
|
||||
return True
|
||||
|
||||
logger.debug(
|
||||
f"_should_instrument_package: Defaulting to False for '{package_name}' (state: _has_agentic_library={_has_agentic_library})"
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def _perform_instrumentation(package_name: str):
|
||||
"""Helper function to perform instrumentation for a given package."""
|
||||
global _instrumenting_packages, _active_instrumentors, _has_agentic_library
|
||||
if not _should_instrument_package(package_name):
|
||||
return
|
||||
|
||||
# Get the appropriate configuration for the package
|
||||
# Ensure package_name is a key in either PROVIDERS or AGENTIC_LIBRARIES
|
||||
if package_name not in PROVIDERS and package_name not in AGENTIC_LIBRARIES:
|
||||
logger.debug(
|
||||
f"_perform_instrumentation: Package '{package_name}' not found in PROVIDERS or AGENTIC_LIBRARIES. Skipping."
|
||||
)
|
||||
return
|
||||
|
||||
config = PROVIDERS.get(package_name) or AGENTIC_LIBRARIES.get(package_name)
|
||||
loader = InstrumentorLoader(**config)
|
||||
|
||||
# instrument_one already checks loader.should_activate
|
||||
instrumentor_instance = instrument_one(loader)
|
||||
if instrumentor_instance is not None:
|
||||
# Check if it was *actually* instrumented by instrument_one by seeing if the instrument method was called successfully.
|
||||
# This relies on instrument_one returning None if its internal .instrument() call failed (if we revert that, this needs adjustment)
|
||||
# For now, assuming instrument_one returns instance only on full success.
|
||||
# User request was to return instrumentor even if .instrument() fails. So, we check if _agentops_instrumented_package_key was set by us.
|
||||
|
||||
# Let's assume instrument_one might return an instance whose .instrument() failed.
|
||||
# The key is set before _active_instrumentors.append, so if it's already there and matches, it means it's a re-attempt on the same package.
|
||||
# The _is_package_instrumented check at the start of _should_instrument_package should prevent most re-entry for the same package_name.
|
||||
|
||||
# Store the package key this instrumentor is for, to aid _is_package_instrumented
|
||||
instrumentor_instance._agentops_instrumented_package_key = package_name
|
||||
|
||||
# Add to active_instrumentors only if it's not a duplicate in terms of package_key being instrumented
|
||||
# This is a safeguard, _is_package_instrumented should catch this earlier.
|
||||
is_newly_added = True
|
||||
for existing_inst in _active_instrumentors:
|
||||
if (
|
||||
hasattr(existing_inst, "_agentops_instrumented_package_key")
|
||||
and existing_inst._agentops_instrumented_package_key == package_name
|
||||
):
|
||||
is_newly_added = False
|
||||
logger.debug(
|
||||
f"_perform_instrumentation: Instrumentor for '{package_name}' already in _active_instrumentors. Not adding again."
|
||||
)
|
||||
break
|
||||
if is_newly_added:
|
||||
_active_instrumentors.append(instrumentor_instance)
|
||||
|
||||
# If this was an agentic library AND it's newly effectively instrumented.
|
||||
if (
|
||||
package_name in AGENTIC_LIBRARIES and not _has_agentic_library
|
||||
): # Check _has_agentic_library to ensure this is the *first* one.
|
||||
# _uninstrument_providers() was already called in _should_instrument_package for the first agentic library.
|
||||
_has_agentic_library = True
|
||||
|
||||
# Special case: If mem0 is instrumented, also instrument concurrent.futures
|
||||
if (package_name == "mem0" or package_name == "autogen") and is_newly_added:
|
||||
try:
|
||||
# Check if concurrent.futures module is available
|
||||
|
||||
# Create config for concurrent.futures instrumentor
|
||||
concurrent_config = InstrumentorConfig(
|
||||
module_name="agentops.instrumentation.utilities.concurrent_futures",
|
||||
class_name="ConcurrentFuturesInstrumentor",
|
||||
min_version="3.7.0", # Python 3.7+ (concurrent.futures is stdlib)
|
||||
package_name="python", # Special case for stdlib modules
|
||||
)
|
||||
|
||||
# Create and instrument concurrent.futures
|
||||
concurrent_loader = InstrumentorLoader(**concurrent_config)
|
||||
concurrent_instrumentor = instrument_one(concurrent_loader)
|
||||
|
||||
if concurrent_instrumentor is not None:
|
||||
concurrent_instrumentor._agentops_instrumented_package_key = "concurrent.futures"
|
||||
_active_instrumentors.append(concurrent_instrumentor)
|
||||
logger.debug("AgentOps: Instrumented concurrent.futures as a dependency of mem0.")
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not instrument concurrent.futures for mem0: {e}")
|
||||
else:
|
||||
logger.debug(
|
||||
f"_perform_instrumentation: instrument_one for '{package_name}' returned None. Not added to active instrumentors."
|
||||
)
|
||||
|
||||
|
||||
def _import_monitor(name: str, globals_dict=None, locals_dict=None, fromlist=(), level=0):
|
||||
"""
|
||||
Monitor imports and instrument packages as they are imported.
|
||||
This replaces the built-in import function to intercept package imports.
|
||||
"""
|
||||
global _instrumenting_packages, _has_agentic_library
|
||||
|
||||
# If an agentic library is already instrumented, skip all further instrumentation
|
||||
if _has_agentic_library:
|
||||
return _original_builtins_import(name, globals_dict, locals_dict, fromlist, level)
|
||||
|
||||
# First, do the actual import
|
||||
module = _original_builtins_import(name, globals_dict, locals_dict, fromlist, level)
|
||||
|
||||
# Check for exact matches first (handles package.module like google.adk)
|
||||
packages_to_check = set()
|
||||
|
||||
# Check the imported module itself
|
||||
if name in TARGET_PACKAGES:
|
||||
packages_to_check.add(name)
|
||||
else:
|
||||
# Check if any target package is a prefix of the import name
|
||||
for target in TARGET_PACKAGES:
|
||||
if name.startswith(target + ".") or name == target:
|
||||
packages_to_check.add(target)
|
||||
|
||||
# For "from X import Y" style imports, also check submodules
|
||||
if fromlist:
|
||||
for item in fromlist:
|
||||
# Construct potential full name, e.g., "google.adk" from name="google", item="adk"
|
||||
# Or if name="os", item="path", full_name="os.path"
|
||||
# If the original name itself is a multi-part name like "a.b", and item is "c", then "a.b.c"
|
||||
# This logic needs to correctly identify the root package if 'name' is already a sub-package.
|
||||
# The existing TARGET_PACKAGES check is simpler: it checks against pre-defined full names.
|
||||
|
||||
# Check full name if item forms part of a target package name
|
||||
full_item_name_candidate = f"{name}.{item}"
|
||||
|
||||
if full_item_name_candidate in TARGET_PACKAGES:
|
||||
packages_to_check.add(full_item_name_candidate)
|
||||
else: # Fallback to checking if 'name' itself is a target
|
||||
for target in TARGET_PACKAGES:
|
||||
if name == target or name.startswith(target + "."):
|
||||
packages_to_check.add(target) # Check the base target if a submodule is imported from it.
|
||||
|
||||
# Instrument all matching packages
|
||||
for package_to_check in packages_to_check:
|
||||
if package_to_check not in _instrumenting_packages and not _is_package_instrumented(package_to_check):
|
||||
target_module_obj = sys.modules.get(package_to_check)
|
||||
|
||||
if target_module_obj:
|
||||
is_sdk = _is_installed_package(target_module_obj, package_to_check)
|
||||
if not is_sdk:
|
||||
logger.debug(
|
||||
f"AgentOps: Target '{package_to_check}' appears to be a local module/directory. Skipping AgentOps SDK instrumentation for it."
|
||||
)
|
||||
continue
|
||||
else:
|
||||
logger.debug(
|
||||
f"_import_monitor: No module object found in sys.modules for '{package_to_check}', proceeding with SDK instrumentation attempt."
|
||||
)
|
||||
|
||||
_instrumenting_packages.add(package_to_check)
|
||||
try:
|
||||
_perform_instrumentation(package_to_check)
|
||||
# If we just instrumented an agentic library, stop
|
||||
if _has_agentic_library:
|
||||
break
|
||||
except Exception as e:
|
||||
logger.error(f"Error instrumenting {package_to_check}: {str(e)}")
|
||||
finally:
|
||||
_instrumenting_packages.discard(package_to_check)
|
||||
|
||||
return module
|
||||
|
||||
|
||||
@dataclass
|
||||
class InstrumentorLoader:
|
||||
"""
|
||||
Represents a dynamically-loadable instrumentor.
|
||||
Handles version checking and instantiation of instrumentors.
|
||||
"""
|
||||
|
||||
module_name: str
|
||||
class_name: str
|
||||
min_version: str
|
||||
package_name: Optional[str] = None # Optional: actual pip package name
|
||||
|
||||
@property
|
||||
def module(self) -> ModuleType:
|
||||
"""Get the instrumentor module."""
|
||||
return importlib.import_module(self.module_name)
|
||||
|
||||
@property
|
||||
def should_activate(self) -> bool:
|
||||
"""Check if the package is available and meets version requirements."""
|
||||
try:
|
||||
# Special case for stdlib modules (like concurrent.futures)
|
||||
if self.package_name == "python":
|
||||
import sys
|
||||
|
||||
python_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
|
||||
return Version(python_version) >= parse(self.min_version)
|
||||
|
||||
# Use explicit package_name if provided, otherwise derive from module_name
|
||||
if self.package_name:
|
||||
provider_name = self.package_name
|
||||
else:
|
||||
provider_name = self.module_name.split(".")[-1]
|
||||
|
||||
# Use common version utility
|
||||
module_version = get_library_version(provider_name)
|
||||
return module_version != "unknown" and Version(module_version) >= parse(self.min_version)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def get_instance(self) -> BaseInstrumentor:
|
||||
"""Create and return a new instance of the instrumentor."""
|
||||
return getattr(self.module, self.class_name)()
|
||||
|
||||
|
||||
def instrument_one(loader: InstrumentorLoader) -> Optional[BaseInstrumentor]:
|
||||
"""
|
||||
Instrument a single package using the provided loader.
|
||||
Returns the instrumentor instance if successful, None otherwise.
|
||||
"""
|
||||
if not loader.should_activate:
|
||||
# This log is important for users to know why something wasn't instrumented.
|
||||
logger.debug(
|
||||
f"AgentOps: Package '{loader.package_name or loader.module_name}' not found or version is less than minimum required ('{loader.min_version}'). Skipping instrumentation."
|
||||
)
|
||||
return None
|
||||
|
||||
instrumentor = loader.get_instance()
|
||||
try:
|
||||
# Use the provider directly from the global tracer instance
|
||||
instrumentor.instrument(tracer_provider=tracer.provider)
|
||||
logger.debug(
|
||||
f"AgentOps: Successfully instrumented '{loader.class_name}' for package '{loader.package_name or loader.module_name}'."
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to instrument {loader.class_name} for {loader.package_name or loader.module_name}: {e}",
|
||||
exc_info=True,
|
||||
)
|
||||
return instrumentor
|
||||
|
||||
|
||||
def instrument_all():
|
||||
"""Start monitoring and instrumenting packages if not already started."""
|
||||
# Check if active_instrumentors is empty, as a proxy for not started.
|
||||
if not _active_instrumentors:
|
||||
builtins.__import__ = _import_monitor
|
||||
global _instrumenting_packages, _has_agentic_library
|
||||
|
||||
# If an agentic library is already instrumented, don't instrument anything else
|
||||
if _has_agentic_library:
|
||||
return
|
||||
|
||||
for name in list(sys.modules.keys()):
|
||||
# Stop if an agentic library gets instrumented during the loop
|
||||
if _has_agentic_library:
|
||||
break
|
||||
|
||||
module = sys.modules.get(name)
|
||||
if not isinstance(module, ModuleType):
|
||||
continue
|
||||
|
||||
# Check for exact matches first (handles package.module like google.adk)
|
||||
package_to_check = None
|
||||
if name in TARGET_PACKAGES:
|
||||
package_to_check = name
|
||||
else:
|
||||
# Check if any target package is a prefix of the module name
|
||||
for target in TARGET_PACKAGES:
|
||||
if name.startswith(target + ".") or name == target:
|
||||
package_to_check = target
|
||||
break
|
||||
|
||||
if (
|
||||
package_to_check
|
||||
and package_to_check not in _instrumenting_packages
|
||||
and not _is_package_instrumented(package_to_check)
|
||||
):
|
||||
target_module_obj = sys.modules.get(package_to_check)
|
||||
|
||||
if target_module_obj:
|
||||
is_sdk = _is_installed_package(target_module_obj, package_to_check)
|
||||
if not is_sdk:
|
||||
continue
|
||||
else:
|
||||
logger.debug(
|
||||
f"instrument_all: No module object found for '{package_to_check}' in sys.modules during startup scan. Proceeding cautiously."
|
||||
)
|
||||
|
||||
_instrumenting_packages.add(package_to_check)
|
||||
try:
|
||||
_perform_instrumentation(package_to_check)
|
||||
except Exception as e:
|
||||
logger.error(f"Error instrumenting {package_to_check}: {str(e)}")
|
||||
finally:
|
||||
_instrumenting_packages.discard(package_to_check)
|
||||
|
||||
|
||||
def uninstrument_all():
|
||||
"""Stop monitoring and uninstrument all packages."""
|
||||
global _active_instrumentors, _has_agentic_library
|
||||
builtins.__import__ = _original_builtins_import
|
||||
for instrumentor in _active_instrumentors:
|
||||
instrumentor.uninstrument()
|
||||
logger.debug(f"Uninstrumented {instrumentor.__class__.__name__}")
|
||||
_active_instrumentors = []
|
||||
_has_agentic_library = False
|
||||
|
||||
|
||||
def get_active_libraries() -> set[str]:
|
||||
"""
|
||||
Get all actively used libraries in the current execution context.
|
||||
Returns a set of package names that are currently imported and being monitored.
|
||||
"""
|
||||
active_libs = set()
|
||||
for name, module in sys.modules.items():
|
||||
if not isinstance(module, ModuleType):
|
||||
continue
|
||||
|
||||
# Check for exact matches first
|
||||
if name in TARGET_PACKAGES:
|
||||
active_libs.add(name)
|
||||
else:
|
||||
# Check if any target package is a prefix of the module name
|
||||
for target in TARGET_PACKAGES:
|
||||
if name.startswith(target + ".") or name == target:
|
||||
active_libs.add(target)
|
||||
break
|
||||
|
||||
return active_libs
|
||||
@@ -0,0 +1,18 @@
|
||||
"""AG2 Instrumentation for AgentOps
|
||||
|
||||
This module provides instrumentation for AG2 (AutoGen), adding telemetry to track agent
|
||||
interactions, conversation flows, and tool usage while focusing on summary-level data rather
|
||||
than individual message exchanges.
|
||||
"""
|
||||
|
||||
from agentops.instrumentation.common import LibraryInfo
|
||||
|
||||
# Library information
|
||||
_library_info = LibraryInfo(name="ag2")
|
||||
LIBRARY_NAME = _library_info.name
|
||||
LIBRARY_VERSION = _library_info.version
|
||||
|
||||
# Import after defining constants to avoid circular imports
|
||||
from agentops.instrumentation.agentic.ag2.instrumentor import AG2Instrumentor # noqa: E402
|
||||
|
||||
__all__ = ["AG2Instrumentor", "LIBRARY_NAME", "LIBRARY_VERSION"]
|
||||
@@ -0,0 +1,922 @@
|
||||
"""AG2 (AutoGen) Instrumentation Module
|
||||
|
||||
This module provides the main instrumentor class and wrapping functions for AG2 (AutoGen).
|
||||
It focuses on collecting summary-level telemetry rather than individual message events.
|
||||
"""
|
||||
|
||||
import json
|
||||
from typing import Dict, Any
|
||||
from wrapt import wrap_function_wrapper
|
||||
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.metrics import Meter
|
||||
from opentelemetry.instrumentation.utils import unwrap as otel_unwrap
|
||||
import contextvars
|
||||
import threading
|
||||
from opentelemetry import context as otel_context
|
||||
from agentops.logging import logger
|
||||
from agentops.instrumentation.common import (
|
||||
CommonInstrumentor,
|
||||
InstrumentorConfig,
|
||||
StandardMetrics,
|
||||
create_span,
|
||||
SpanAttributeManager,
|
||||
)
|
||||
from agentops.instrumentation.agentic.ag2 import LIBRARY_NAME, LIBRARY_VERSION
|
||||
from agentops.semconv.message import MessageAttributes
|
||||
from agentops.semconv.span_attributes import SpanAttributes
|
||||
from agentops.semconv.span_kinds import AgentOpsSpanKindValues
|
||||
from agentops.semconv.agent import AgentAttributes
|
||||
from agentops.semconv.workflow import WorkflowAttributes
|
||||
from agentops.semconv.tool import ToolAttributes
|
||||
|
||||
|
||||
class AG2Instrumentor(CommonInstrumentor):
|
||||
"""Instrumentor for AG2 (AutoGen)
|
||||
|
||||
This instrumentor captures high-level events from AG2's agent interactions,
|
||||
focusing on summaries rather than individual messages, and providing detailed
|
||||
tool usage information.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
config = InstrumentorConfig(
|
||||
library_name=LIBRARY_NAME,
|
||||
library_version=LIBRARY_VERSION,
|
||||
wrapped_methods=[], # We'll use custom wrapping
|
||||
metrics_enabled=True,
|
||||
dependencies=["ag2 >= 0.3.2"],
|
||||
)
|
||||
super().__init__(config)
|
||||
self._attribute_manager = None
|
||||
|
||||
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
|
||||
"""Create metrics for AG2 instrumentation."""
|
||||
return StandardMetrics.create_standard_metrics(meter)
|
||||
|
||||
def _initialize(self, **kwargs):
|
||||
"""Initialize attribute manager and AG2-specific concurrent.futures instrumentation."""
|
||||
self._attribute_manager = SpanAttributeManager(service_name="agentops", deployment_environment="production")
|
||||
|
||||
def _custom_wrap(self, **kwargs):
|
||||
"""Perform custom wrapping for AG2 methods."""
|
||||
|
||||
methods_to_wrap = [
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.__init__", self._agent_init_wrapper),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.run", self._agent_run_wrapper_with_context),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.initiate_chat", self._initiate_chat_wrapper),
|
||||
(
|
||||
"autogen.agentchat.conversable_agent",
|
||||
"ConversableAgent.a_initiate_chat",
|
||||
self._async_initiate_chat_wrapper,
|
||||
),
|
||||
(
|
||||
"autogen.agentchat.conversable_agent",
|
||||
"ConversableAgent._generate_oai_reply_from_client",
|
||||
self._generate_oai_reply_from_client_wrapper,
|
||||
),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.receive", self._receive_wrapper),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.a_receive", self._async_receive_wrapper),
|
||||
("autogen.agentchat.groupchat", "GroupChatManager.run_chat", self._group_chat_run_wrapper),
|
||||
("autogen.agentchat.groupchat", "GroupChatManager.a_run_chat", self._async_group_chat_run_wrapper),
|
||||
(
|
||||
"autogen.agentchat.conversable_agent",
|
||||
"ConversableAgent.execute_function",
|
||||
lambda tracer: self._tool_execution_wrapper(tracer, "function"),
|
||||
),
|
||||
(
|
||||
"autogen.agentchat.conversable_agent",
|
||||
"ConversableAgent.run_code",
|
||||
lambda tracer: self._tool_execution_wrapper(tracer, "code"),
|
||||
),
|
||||
("autogen.agentchat.groupchat", "GroupChat.select_speaker", self._group_chat_select_speaker_wrapper),
|
||||
]
|
||||
|
||||
for module, method, wrapper_factory in methods_to_wrap:
|
||||
try:
|
||||
wrap_function_wrapper(module, method, wrapper_factory(self._tracer))
|
||||
except (AttributeError, ModuleNotFoundError) as e:
|
||||
logger.debug(f"Failed to wrap {method}: {e}")
|
||||
|
||||
def _custom_unwrap(self, **kwargs):
|
||||
"""Remove instrumentation from AG2."""
|
||||
# Unwrap all instrumented methods
|
||||
methods_to_unwrap = [
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.__init__"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.run"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.initiate_chat"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.a_initiate_chat"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent._generate_oai_reply_from_client"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.receive"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.a_receive"),
|
||||
("autogen.agentchat.groupchat", "GroupChatManager.run_chat"),
|
||||
("autogen.agentchat.groupchat", "GroupChatManager.a_run_chat"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.execute_function"),
|
||||
("autogen.agentchat.conversable_agent", "ConversableAgent.run_code"),
|
||||
("autogen.agentchat.groupchat", "GroupChat.select_speaker"),
|
||||
]
|
||||
|
||||
try:
|
||||
for module, method in methods_to_unwrap:
|
||||
otel_unwrap(module, method)
|
||||
logger.debug("Successfully uninstrumented AG2")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to unwrap AG2 methods: {e}")
|
||||
|
||||
def _set_llm_config_attributes(self, span, llm_config):
|
||||
if not isinstance(llm_config, dict):
|
||||
return
|
||||
|
||||
if "model" in llm_config:
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, llm_config["model"])
|
||||
|
||||
for param, attr in [
|
||||
("temperature", SpanAttributes.LLM_REQUEST_TEMPERATURE),
|
||||
("top_p", SpanAttributes.LLM_REQUEST_TOP_P),
|
||||
("frequency_penalty", SpanAttributes.LLM_REQUEST_FREQUENCY_PENALTY),
|
||||
("presence_penalty", SpanAttributes.LLM_REQUEST_PRESENCE_PENALTY),
|
||||
]:
|
||||
if param in llm_config and llm_config[param] is not None:
|
||||
span.set_attribute(attr, llm_config[param])
|
||||
|
||||
def _agent_init_wrapper(self, tracer):
|
||||
"""Wrapper for capturing agent initialization."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
try:
|
||||
name = kwargs.get("name", "unnamed_agent")
|
||||
llm_config = kwargs.get("llm_config", {})
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
model = "unknown"
|
||||
if isinstance(llm_config, dict) and llm_config:
|
||||
model = llm_config.get("model", "unknown")
|
||||
|
||||
instance._agentops_metadata = {"name": name, "type": "ConversableAgent", "model": model}
|
||||
|
||||
return result
|
||||
except Exception:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
def _generate_oai_reply_from_client_wrapper(self, tracer):
|
||||
"""Wrapper for capturing _generate_oai_reply_from_client method calls with token metrics."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
agent_name = getattr(instance, "name", "unnamed_agent")
|
||||
|
||||
# Get model name from llm_client for span naming
|
||||
llm_client = args[0] if args else kwargs.get("llm_client")
|
||||
|
||||
# Extract model from _config_list
|
||||
model_name = "unknown"
|
||||
if hasattr(llm_client, "_config_list") and llm_client._config_list:
|
||||
if isinstance(llm_client._config_list, list) and len(llm_client._config_list) > 0:
|
||||
config = llm_client._config_list[0]
|
||||
if isinstance(config, dict) and "model" in config:
|
||||
model_name = config["model"]
|
||||
|
||||
span_name = f"{model_name}.llm"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.CLIENT, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
# Set span kind for actual LLM client call
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.LLM.value)
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "llm")
|
||||
span.set_attribute("llm.client_call", "true")
|
||||
|
||||
# Get messages from args
|
||||
messages = args[1] if len(args) > 1 else kwargs.get("messages", [])
|
||||
|
||||
# Extract input from messages and set gen_ai.prompt
|
||||
if messages and isinstance(messages, list) and len(messages) > 0:
|
||||
# Set gen_ai.prompt array with full conversation history
|
||||
prompt_index = 0
|
||||
for msg in messages:
|
||||
if isinstance(msg, dict) and msg.get("role") in ["user", "assistant", "system"]:
|
||||
role = msg.get("role")
|
||||
content = msg.get("content", "")
|
||||
if content and role:
|
||||
span.set_attribute(
|
||||
f"{SpanAttributes.LLM_PROMPTS}.{prompt_index}.content", self._safe_str(content)
|
||||
)
|
||||
span.set_attribute(f"{SpanAttributes.LLM_PROMPTS}.{prompt_index}.role", role)
|
||||
prompt_index += 1
|
||||
|
||||
# Set entity input to the latest user message (what triggered this LLM call)
|
||||
latest_user_message = None
|
||||
for msg in messages:
|
||||
if isinstance(msg, dict) and msg.get("role") == "user":
|
||||
content = msg.get("content", "")
|
||||
if content:
|
||||
latest_user_message = content
|
||||
|
||||
if latest_user_message:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, self._safe_str(latest_user_message))
|
||||
|
||||
# Call the wrapped method - this is where the actual LLM call happens
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Set the output and gen_ai.completion
|
||||
if result:
|
||||
if isinstance(result, dict):
|
||||
content = result.get("content", "")
|
||||
if content:
|
||||
content_str = self._safe_str(content)
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, content_str)
|
||||
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.content", content_str)
|
||||
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.role", "assistant")
|
||||
|
||||
# If model information is in the result
|
||||
if "model" in result:
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, result["model"])
|
||||
elif isinstance(result, str):
|
||||
# Handle string result (which is what AG2 returns)
|
||||
result_str = self._safe_str(result)
|
||||
|
||||
# Set entity output
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, result_str)
|
||||
|
||||
# Set gen_ai.completion with full content
|
||||
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.content", result_str)
|
||||
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.role", "assistant")
|
||||
|
||||
# Try to get token metrics from the client's usage tracking
|
||||
try:
|
||||
# The OpenAIWrapper tracks usage in actual_usage_summary and total_usage_summary
|
||||
if hasattr(llm_client, "actual_usage_summary") and llm_client.actual_usage_summary:
|
||||
# Get the latest usage
|
||||
for model, usage in llm_client.actual_usage_summary.items():
|
||||
if model != "total_cost" and isinstance(usage, dict):
|
||||
prompt_tokens = usage.get("prompt_tokens", 0)
|
||||
completion_tokens = usage.get("completion_tokens", 0)
|
||||
total_tokens = usage.get("total_tokens", 0)
|
||||
cost = usage.get("cost", 0.0)
|
||||
|
||||
# Set token usage metrics
|
||||
if prompt_tokens > 0:
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_PROMPT_TOKENS, str(prompt_tokens))
|
||||
if completion_tokens > 0:
|
||||
span.set_attribute(
|
||||
SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, str(completion_tokens)
|
||||
)
|
||||
if total_tokens > 0:
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, str(total_tokens))
|
||||
if cost > 0:
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_TOOL_COST, str(cost))
|
||||
|
||||
# Set request/response model
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model)
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, model)
|
||||
span.set_attribute(SpanAttributes.LLM_SYSTEM, "ag2")
|
||||
|
||||
break # Use the first model's metrics
|
||||
except Exception as e:
|
||||
logger.debug(f"[AG2 DEBUG] Could not extract token metrics: {e}")
|
||||
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _initiate_chat_wrapper(self, tracer):
|
||||
"""Wrapper for capturing individual chat initiation as a parent span."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
recipient_agent = args[0] if args else None
|
||||
if not recipient_agent:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
# Get agent names for span identification
|
||||
initiator_name = getattr(instance, "name", "unnamed_initiator")
|
||||
recipient_name = getattr(recipient_agent, "name", "unnamed_agent")
|
||||
|
||||
span_name = f"ag2.chat.{initiator_name}_to_{recipient_name}.workflow"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
# Set span kind as agent for proper categorization
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
|
||||
span.set_attribute(AgentAttributes.FROM_AGENT, initiator_name)
|
||||
span.set_attribute(AgentAttributes.TO_AGENT, recipient_name)
|
||||
span.set_attribute("agent.type", "individual")
|
||||
span.set_attribute("agent.initiator", initiator_name)
|
||||
span.set_attribute("agent.recipient", recipient_name)
|
||||
|
||||
# Set agentops entity attributes
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
|
||||
|
||||
# Extract initial message
|
||||
initial_message = kwargs.get("message", "")
|
||||
if initial_message:
|
||||
initial_message = self._safe_str(initial_message)
|
||||
span.set_attribute("agent.initial_message", initial_message)
|
||||
# Set entity input
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, initial_message)
|
||||
|
||||
# Extract system messages and put them in agent attributes
|
||||
initiator_system_msg = getattr(instance, "system_message", "")
|
||||
if initiator_system_msg:
|
||||
initiator_system_msg = self._safe_str(initiator_system_msg)
|
||||
span.set_attribute("agent.initiator_system_message", initiator_system_msg)
|
||||
|
||||
recipient_system_msg = getattr(recipient_agent, "system_message", "")
|
||||
if recipient_system_msg:
|
||||
recipient_system_msg = self._safe_str(recipient_system_msg)
|
||||
span.set_attribute("agent.system_instruction", recipient_system_msg)
|
||||
# Also set in gen_ai for compatibility
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION, recipient_system_msg)
|
||||
|
||||
# Extract LLM config and set gen_ai attributes
|
||||
recipient_llm_config = getattr(recipient_agent, "llm_config", {})
|
||||
|
||||
if isinstance(recipient_llm_config, dict) and recipient_llm_config:
|
||||
model = recipient_llm_config.get("model", "unknown")
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model)
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, model)
|
||||
span.set_attribute(SpanAttributes.LLM_SYSTEM, "ag2")
|
||||
|
||||
# Also set LLM config attributes
|
||||
self._set_llm_config_attributes(span, recipient_llm_config)
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Extract chat history after completion
|
||||
self._extract_chat_history(span, instance, recipient_agent)
|
||||
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _async_initiate_chat_wrapper(self, tracer):
|
||||
"""Wrapper for capturing async individual chat initiation as a parent span."""
|
||||
|
||||
async def wrapper(wrapped, instance, args, kwargs):
|
||||
recipient_agent = args[0] if args else None
|
||||
if not recipient_agent:
|
||||
return await wrapped(*args, **kwargs)
|
||||
|
||||
# Get agent names for span identification
|
||||
initiator_name = getattr(instance, "name", "unnamed_initiator")
|
||||
recipient_name = getattr(recipient_agent, "name", "unnamed_agent")
|
||||
|
||||
span_name = f"ag2.chat.{initiator_name}_to_{recipient_name}.workflow"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
# Set span kind as agent for proper categorization
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
|
||||
span.set_attribute(AgentAttributes.FROM_AGENT, initiator_name)
|
||||
span.set_attribute(AgentAttributes.TO_AGENT, recipient_name)
|
||||
span.set_attribute("agent.type", "individual_async")
|
||||
span.set_attribute("agent.initiator", initiator_name)
|
||||
span.set_attribute("agent.recipient", recipient_name)
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
|
||||
|
||||
# Extract initial message
|
||||
initial_message = kwargs.get("message", "")
|
||||
if initial_message:
|
||||
initial_message = self._safe_str(initial_message)
|
||||
span.set_attribute("agent.initial_message", initial_message)
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, initial_message)
|
||||
|
||||
# Extract system messages
|
||||
recipient_system_msg = getattr(recipient_agent, "system_message", "")
|
||||
if recipient_system_msg:
|
||||
recipient_system_msg = self._safe_str(recipient_system_msg)
|
||||
span.set_attribute("agent.system_instruction", recipient_system_msg)
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION, recipient_system_msg)
|
||||
|
||||
# Extract LLM config
|
||||
recipient_llm_config = getattr(recipient_agent, "llm_config", {})
|
||||
if isinstance(recipient_llm_config, dict) and recipient_llm_config:
|
||||
self._set_llm_config_attributes(span, recipient_llm_config)
|
||||
|
||||
result = await wrapped(*args, **kwargs)
|
||||
|
||||
# Extract chat history after completion
|
||||
self._extract_chat_history(span, instance, recipient_agent)
|
||||
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _receive_wrapper(self, tracer):
|
||||
"""Wrapper for capturing message receive events."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
agent_name = getattr(instance, "name", "unnamed_agent")
|
||||
span_name = f"ag2.agent.{agent_name}.receive"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
|
||||
|
||||
# Get message and sender
|
||||
message = args[0] if args else kwargs.get("message", "")
|
||||
sender = args[1] if len(args) > 1 else kwargs.get("sender")
|
||||
|
||||
if sender:
|
||||
sender_name = getattr(sender, "name", "unknown")
|
||||
span.set_attribute("agent.sender", sender_name)
|
||||
|
||||
if message:
|
||||
if isinstance(message, dict):
|
||||
content = message.get("content", "")
|
||||
if content:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, self._safe_str(content))
|
||||
elif isinstance(message, str):
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, self._safe_str(message))
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _async_receive_wrapper(self, tracer):
|
||||
"""Wrapper for async capturing message reception."""
|
||||
|
||||
async def wrapper(wrapped, instance, args, kwargs):
|
||||
agent_name = getattr(instance, "name", "unnamed_agent")
|
||||
span_name = f"ag2.agent.{agent_name}.async_receive"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
|
||||
|
||||
# Get message from the first argument
|
||||
message = args[0] if args else None
|
||||
|
||||
# Enhanced message processing
|
||||
if message:
|
||||
if isinstance(message, dict):
|
||||
# Dict message format
|
||||
sender_name = message.get("name", "unknown")
|
||||
content = self._extract_message_content(message)
|
||||
role = message.get("role", "user")
|
||||
|
||||
# Set sender and message attributes
|
||||
span.set_attribute("agent.sender", sender_name)
|
||||
span.set_attribute("message.role", role)
|
||||
|
||||
if content:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, content)
|
||||
span.set_attribute("message.content", content)
|
||||
|
||||
elif isinstance(message, str):
|
||||
# String message format
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, message)
|
||||
span.set_attribute("message.content", message)
|
||||
|
||||
# Get sender from the second argument if available
|
||||
sender = args[1] if len(args) > 1 else None
|
||||
if sender and hasattr(sender, "name"):
|
||||
span.set_attribute("agent.sender_name", sender.name)
|
||||
|
||||
return await wrapped(*args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
def _async_group_chat_run_wrapper(self, tracer):
|
||||
"""Wrapper for capturing async group chat execution."""
|
||||
|
||||
async def wrapper(wrapped, instance, args, kwargs):
|
||||
with create_span(
|
||||
tracer,
|
||||
"ag2.groupchat.run.task.async",
|
||||
kind=SpanKind.INTERNAL,
|
||||
attribute_manager=self._attribute_manager,
|
||||
) as span:
|
||||
group_chat = getattr(instance, "groupchat", None)
|
||||
agents = getattr(group_chat, "agents", []) if group_chat else []
|
||||
agent_names = [getattr(agent, "name", f"agent_{i}") for i, agent in enumerate(agents)]
|
||||
|
||||
span.set_attribute(AgentAttributes.AGENT_ROLE, "GroupChatManager")
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, getattr(instance, "name", "unnamed_manager"))
|
||||
span.set_attribute("groupchat.agents", ", ".join(agent_names))
|
||||
span.set_attribute("groupchat.agent_count", len(agents))
|
||||
|
||||
# Capture input message if available
|
||||
message = kwargs.get("message", "")
|
||||
if message:
|
||||
content_to_set = self._extract_message_content(message)
|
||||
span.set_attribute("groupchat.input_message", content_to_set)
|
||||
|
||||
result = await wrapped(*args, **kwargs)
|
||||
self._capture_group_chat_summary(span, instance, result)
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _agent_run_wrapper_with_context(self, tracer):
|
||||
"""Wrapper for capturing agent run with context propagation and proper span lifecycle."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
agent_name = getattr(instance, "name", "unnamed_agent")
|
||||
agent_type = getattr(instance, "_agentops_metadata", {}).get("type", "ConversableAgent")
|
||||
span_name = f"ag2.agent.{agent_name}.run.workflow"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
model = getattr(instance, "_agentops_metadata", {}).get("model", "unknown")
|
||||
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
|
||||
span.set_attribute(AgentAttributes.AGENT_ROLE, agent_type)
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model)
|
||||
|
||||
llm_config = getattr(instance, "llm_config", None)
|
||||
self._set_llm_config_attributes(span, llm_config)
|
||||
|
||||
# Capture input message if available
|
||||
message = kwargs.get("message", "")
|
||||
if message:
|
||||
content_to_set = self._extract_message_content(message)
|
||||
span.set_attribute("agent.run.input_message", content_to_set)
|
||||
|
||||
# Capture BOTH contextvars and OpenTelemetry context
|
||||
ctx = contextvars.copy_context()
|
||||
current_otel_context = otel_context.get_current()
|
||||
|
||||
# Thread tracking for proper span lifecycle
|
||||
active_threads = []
|
||||
|
||||
# Store the original Thread.__init__ and start methods
|
||||
original_thread_init = threading.Thread.__init__
|
||||
original_thread_start = threading.Thread.start
|
||||
|
||||
def context_aware_init(self, group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None):
|
||||
"""Modified Thread.__init__ that wraps the target to run in both captured contexts."""
|
||||
if kwargs is None:
|
||||
kwargs = {}
|
||||
if target and callable(target):
|
||||
original_target = target
|
||||
|
||||
def wrapped_target(*target_args, **target_kwargs):
|
||||
# Run in both contextvars AND OpenTelemetry context
|
||||
def run_with_otel_context():
|
||||
# Attach the OpenTelemetry context in the thread
|
||||
token = otel_context.attach(current_otel_context)
|
||||
try:
|
||||
return original_target(*target_args, **target_kwargs)
|
||||
finally:
|
||||
otel_context.detach(token)
|
||||
|
||||
# Run with contextvars context
|
||||
return ctx.run(run_with_otel_context)
|
||||
|
||||
target = wrapped_target
|
||||
|
||||
# Keep original daemon setting but ensure conversations don't run indefinitely
|
||||
# If daemon was not explicitly set, default to False (AG2's normal behavior)
|
||||
if daemon is None:
|
||||
daemon = False
|
||||
|
||||
original_thread_init(
|
||||
self, group=group, target=target, name=name, args=args, kwargs=kwargs, daemon=daemon
|
||||
)
|
||||
|
||||
def context_aware_start(self):
|
||||
"""Modified Thread.start that tracks the thread."""
|
||||
active_threads.append(self)
|
||||
return original_thread_start(self)
|
||||
|
||||
# Temporarily patch Thread.__init__ and start just for this run() call
|
||||
threading.Thread.__init__ = context_aware_init
|
||||
threading.Thread.start = context_aware_start
|
||||
try:
|
||||
response = wrapped(*args, **kwargs)
|
||||
except Exception as e:
|
||||
logger.error(f"[AG2 DEBUG] Error in agent.run execution: {e}")
|
||||
raise
|
||||
finally:
|
||||
# Always restore the original Thread methods
|
||||
try:
|
||||
threading.Thread.__init__ = original_thread_init
|
||||
threading.Thread.start = original_thread_start
|
||||
except Exception as e:
|
||||
logger.error(f"[AG2 DEBUG] Error restoring Thread methods: {e}")
|
||||
# Force restore
|
||||
threading.Thread.__init__ = (
|
||||
threading.Thread.__init__.__wrapped__
|
||||
if hasattr(threading.Thread.__init__, "__wrapped__")
|
||||
else original_thread_init
|
||||
)
|
||||
threading.Thread.start = (
|
||||
threading.Thread.start.__wrapped__
|
||||
if hasattr(threading.Thread.start, "__wrapped__")
|
||||
else original_thread_start
|
||||
)
|
||||
|
||||
# Try to get final results from response if available
|
||||
try:
|
||||
if hasattr(response, "get_chat_results"):
|
||||
chat_results = response.get_chat_results()
|
||||
if chat_results:
|
||||
self._capture_conversation_summary(span, instance, chat_results)
|
||||
elif hasattr(response, "chat_history"):
|
||||
self._capture_conversation_summary(span, instance, response)
|
||||
elif hasattr(response, "get") and callable(response.get):
|
||||
model_info = response.get("model", "")
|
||||
if model_info:
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, str(model_info))
|
||||
except Exception as e:
|
||||
logger.debug(f"[AG2 DEBUG] Could not extract final results: {e}")
|
||||
|
||||
span.set_attribute(WorkflowAttributes.WORKFLOW_STEP_STATUS, "completed")
|
||||
return response
|
||||
|
||||
return wrapper
|
||||
|
||||
def _group_chat_run_wrapper(self, tracer):
|
||||
"""Wrapper for capturing group chat execution."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
with create_span(
|
||||
tracer, "ag2.groupchat.run.task", kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
group_chat = getattr(instance, "groupchat", None)
|
||||
agents = getattr(group_chat, "agents", []) if group_chat else []
|
||||
agent_names = [getattr(agent, "name", f"agent_{i}") for i, agent in enumerate(agents)]
|
||||
|
||||
span.set_attribute(AgentAttributes.AGENT_ROLE, "GroupChatManager")
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, getattr(instance, "name", "unnamed_manager"))
|
||||
span.set_attribute("groupchat.agents", ", ".join(agent_names))
|
||||
span.set_attribute("groupchat.agent_count", len(agents))
|
||||
|
||||
# Capture input message if available
|
||||
message = kwargs.get("message", "")
|
||||
if message:
|
||||
content_to_set = self._extract_message_content(message)
|
||||
span.set_attribute("groupchat.input_message", content_to_set)
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
self._capture_group_chat_summary(span, instance, result)
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _tool_execution_wrapper(self, tracer, tool_type):
|
||||
"""Wrapper for capturing tool execution."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
span_name = f"ag2.tool.{tool_type}.tool_usage"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.CLIENT, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
# Set span kind and type as tool for proper categorization
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.TOOL.value)
|
||||
agent_name = getattr(instance, "name", "unnamed_agent")
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
|
||||
span.set_attribute(ToolAttributes.TOOL_NAME, tool_type)
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "tool")
|
||||
|
||||
if tool_type == "function" and args:
|
||||
func_call = args[0]
|
||||
if isinstance(func_call, dict):
|
||||
span.set_attribute(
|
||||
MessageAttributes.TOOL_CALL_NAME.format(i=0), func_call.get("name", "unknown")
|
||||
)
|
||||
if "arguments" in func_call:
|
||||
try:
|
||||
span.set_attribute(
|
||||
MessageAttributes.TOOL_CALL_ARGUMENTS.format(i=0),
|
||||
json.dumps(func_call["arguments"]),
|
||||
)
|
||||
except:
|
||||
pass
|
||||
|
||||
elif tool_type == "code" and args:
|
||||
code = args[0]
|
||||
if isinstance(code, str):
|
||||
span.set_attribute("tool.code.size", len(code))
|
||||
span.set_attribute("tool.code.language", kwargs.get("lang", "unknown"))
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
self._process_tool_result(span, result, tool_type)
|
||||
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _group_chat_select_speaker_wrapper(self, tracer):
|
||||
"""Wrapper for capturing which agent is selected to speak in a group chat."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
previous_speaker_name = "unknown"
|
||||
messages = getattr(instance, "messages", [])
|
||||
if messages and len(messages) > 0:
|
||||
previous_speaker_name = messages[-1].get("name", "unknown")
|
||||
|
||||
selected_speaker = wrapped(*args, **kwargs)
|
||||
|
||||
if not selected_speaker:
|
||||
return selected_speaker
|
||||
|
||||
current_speaker_name = getattr(selected_speaker, "name", "unnamed")
|
||||
|
||||
with create_span(
|
||||
tracer, "ag2.handoff", kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
|
||||
) as span:
|
||||
span.set_attribute(AgentAttributes.FROM_AGENT, previous_speaker_name)
|
||||
span.set_attribute(AgentAttributes.TO_AGENT, current_speaker_name)
|
||||
span.set_attribute(AgentAttributes.AGENT_NAME, current_speaker_name)
|
||||
span.set_attribute(AgentAttributes.AGENT_ROLE, selected_speaker.__class__.__name__)
|
||||
|
||||
system_message = getattr(selected_speaker, "system_message", "")
|
||||
if system_message:
|
||||
system_message = self._safe_str(system_message)
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION, system_message)
|
||||
|
||||
self._set_llm_config_attributes(span, getattr(selected_speaker, "llm_config", None))
|
||||
|
||||
if messages:
|
||||
for msg in reversed(messages):
|
||||
if msg.get("name") == current_speaker_name:
|
||||
if "metadata" in msg and isinstance(msg["metadata"], dict):
|
||||
meta = msg["metadata"]
|
||||
if "model" in meta:
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, meta["model"])
|
||||
break
|
||||
|
||||
span.set_attribute("groupchat.role", "participant")
|
||||
|
||||
return selected_speaker
|
||||
|
||||
return wrapper
|
||||
|
||||
# Helper methods
|
||||
def _safe_str(self, value):
|
||||
"""Safely convert value to string."""
|
||||
if value is None:
|
||||
return ""
|
||||
return str(value) if not isinstance(value, str) else value
|
||||
|
||||
def _extract_message_content(self, message):
|
||||
"""Extract content from various message formats."""
|
||||
if isinstance(message, dict):
|
||||
content = message.get("content", "")
|
||||
return self._safe_str(content)
|
||||
elif isinstance(message, str):
|
||||
return message
|
||||
else:
|
||||
return str(message)
|
||||
|
||||
def _extract_chat_history(self, span, initiator, recipient):
|
||||
"""Extract chat history information."""
|
||||
try:
|
||||
# Get recipient chat history
|
||||
recipient_chat_history = getattr(recipient, "chat_history", [])
|
||||
|
||||
if recipient_chat_history:
|
||||
message_count = len(recipient_chat_history)
|
||||
span.set_attribute("conversation.message_count", message_count)
|
||||
|
||||
# Record sample of conversation messages
|
||||
if message_count > 0:
|
||||
first_msg = recipient_chat_history[0]
|
||||
last_msg = recipient_chat_history[-1]
|
||||
|
||||
self._set_message_attributes(span, first_msg, 0, "prompt")
|
||||
self._set_message_attributes(span, last_msg, 0, "completion")
|
||||
|
||||
# Check for tool usage
|
||||
span.set_attribute("chat.used_tools", "tool_calls" in last_msg)
|
||||
|
||||
# Capture metadata
|
||||
if "metadata" in last_msg and isinstance(last_msg["metadata"], dict):
|
||||
meta = last_msg["metadata"]
|
||||
if "model" in meta:
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, meta["model"])
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not extract chat history: {e}")
|
||||
|
||||
def _set_message_attributes(self, span, message, index, prefix):
|
||||
"""Set message attributes on span."""
|
||||
if isinstance(message, dict):
|
||||
role = message.get("role", "unknown")
|
||||
content = message.get("content", "")
|
||||
name = message.get("name", "unknown")
|
||||
|
||||
span.set_attribute(f"messaging.{prefix}.role.{index}", role)
|
||||
content = self._safe_str(content)
|
||||
span.set_attribute(f"messaging.{prefix}.content.{index}", content)
|
||||
span.set_attribute(f"messaging.{prefix}.speaker.{index}", name)
|
||||
|
||||
def _process_tool_result(self, span, result, tool_type):
|
||||
"""Process and set tool execution result attributes."""
|
||||
if tool_type == "function" and isinstance(result, tuple) and len(result) > 0:
|
||||
success = result[0] if isinstance(result[0], bool) else False
|
||||
span.set_attribute(ToolAttributes.TOOL_STATUS, "success" if success else "error")
|
||||
|
||||
if len(result) > 1 and isinstance(result[1], dict):
|
||||
try:
|
||||
span.set_attribute(ToolAttributes.TOOL_RESULT, json.dumps(result[1]))
|
||||
except:
|
||||
pass
|
||||
|
||||
if tool_type == "code" and isinstance(result, tuple) and len(result) >= 3:
|
||||
exit_code = result[0]
|
||||
span.set_attribute("exit_code", exit_code)
|
||||
span.set_attribute(ToolAttributes.TOOL_STATUS, "success" if exit_code == 0 else "error")
|
||||
|
||||
if len(result) > 1 and result[1]:
|
||||
stdout = self._safe_str(result[1])
|
||||
span.set_attribute("tool.code.stdout", stdout)
|
||||
|
||||
if len(result) > 2 and result[2]:
|
||||
stderr = self._safe_str(result[2])
|
||||
span.set_attribute("tool.code.stderr", stderr)
|
||||
|
||||
def _capture_conversation_summary(self, span, agent, response):
|
||||
"""Extract and record conversation summary data."""
|
||||
if not hasattr(response, "chat_history"):
|
||||
return
|
||||
|
||||
try:
|
||||
chat_history = getattr(response, "chat_history", [])
|
||||
message_count = len(chat_history)
|
||||
user_messages = sum(1 for msg in chat_history if msg.get("role") == "user")
|
||||
assistant_messages = sum(1 for msg in chat_history if msg.get("role") == "assistant")
|
||||
|
||||
span.set_attribute("conversation.message_count", message_count)
|
||||
span.set_attribute("conversation.user_messages", user_messages)
|
||||
span.set_attribute("conversation.assistant_messages", assistant_messages)
|
||||
|
||||
# Set prompts and completions
|
||||
span.set_attribute(SpanAttributes.LLM_PROMPTS, user_messages)
|
||||
span.set_attribute(SpanAttributes.LLM_COMPLETIONS, assistant_messages)
|
||||
if message_count > 0:
|
||||
for i, msg in enumerate(chat_history[: min(2, message_count)]):
|
||||
self._set_message_attributes(span, msg, i, "prompt")
|
||||
|
||||
if message_count > 2:
|
||||
self._set_message_attributes(span, chat_history[-1], 0, "completion")
|
||||
except Exception as e:
|
||||
logger.error(f"[AG2 DEBUG] Error capturing conversation summary: {e}")
|
||||
|
||||
def _capture_group_chat_summary(self, span, manager, result):
|
||||
"""Extract and record group chat summary data."""
|
||||
try:
|
||||
messages = getattr(manager.groupchat, "messages", [])
|
||||
message_count = len(messages)
|
||||
|
||||
agent_message_counts = {}
|
||||
for message in messages:
|
||||
agent_name = message.get("name", "unknown")
|
||||
if agent_name not in agent_message_counts:
|
||||
agent_message_counts[agent_name] = 0
|
||||
agent_message_counts[agent_name] += 1
|
||||
|
||||
span.set_attribute("conversation.message_count", message_count)
|
||||
|
||||
for agent_name, count in agent_message_counts.items():
|
||||
span.set_attribute(f"conversation.agent_messages.{agent_name}", count)
|
||||
|
||||
if hasattr(manager.groupchat, "speaker_selection_method"):
|
||||
span.set_attribute(
|
||||
"groupchat.speaker_selection_method", str(manager.groupchat.speaker_selection_method)
|
||||
)
|
||||
|
||||
if message_count > 0:
|
||||
for i, msg in enumerate(messages[: min(2, message_count)]):
|
||||
role = msg.get("role", "unknown")
|
||||
content = msg.get("content", "")
|
||||
name = msg.get("name", "unknown")
|
||||
|
||||
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), role)
|
||||
content = self._safe_str(content)
|
||||
span.set_attribute(MessageAttributes.PROMPT_CONTENT.format(i=i), content)
|
||||
span.set_attribute(MessageAttributes.PROMPT_SPEAKER.format(i=i), name)
|
||||
|
||||
if message_count > 2:
|
||||
last_msg = messages[-1]
|
||||
role = last_msg.get("role", "unknown")
|
||||
content = last_msg.get("content", "")
|
||||
name = last_msg.get("name", "unknown")
|
||||
|
||||
span.set_attribute(MessageAttributes.COMPLETION_ROLE.format(i=0), role)
|
||||
content = self._safe_str(content)
|
||||
span.set_attribute(MessageAttributes.COMPLETION_CONTENT.format(i=0), content)
|
||||
span.set_attribute(MessageAttributes.COMPLETION_SPEAKER.format(i=0), name)
|
||||
|
||||
if "metadata" in last_msg and isinstance(last_msg["metadata"], dict):
|
||||
meta = last_msg["metadata"]
|
||||
if "model" in meta:
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, meta["model"])
|
||||
except Exception as e:
|
||||
logger.error(f"Error capturing group chat summary: {e}")
|
||||
@@ -0,0 +1,19 @@
|
||||
"""Agno Agent instrumentation package."""
|
||||
|
||||
import logging
|
||||
from agentops.instrumentation.common import LibraryInfo
|
||||
|
||||
from .instrumentor import AgnoInstrumentor
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Library information
|
||||
_library_info = LibraryInfo(name="agno", default_version="1.5.8")
|
||||
LIBRARY_NAME = _library_info.name
|
||||
LIBRARY_VERSION = _library_info.version
|
||||
|
||||
__all__ = [
|
||||
"AgnoInstrumentor",
|
||||
"LIBRARY_NAME",
|
||||
"LIBRARY_VERSION",
|
||||
]
|
||||
@@ -0,0 +1,20 @@
|
||||
"""Agno instrumentation attribute handlers."""
|
||||
|
||||
from .agent import get_agent_run_attributes
|
||||
from .metrics import get_metrics_attributes
|
||||
from .team import get_team_run_attributes
|
||||
from .tool import get_tool_execution_attributes
|
||||
from .workflow import get_workflow_run_attributes, get_workflow_session_attributes, get_workflow_cache_attributes
|
||||
from .storage import get_storage_read_attributes, get_storage_write_attributes
|
||||
|
||||
__all__ = [
|
||||
"get_agent_run_attributes",
|
||||
"get_metrics_attributes",
|
||||
"get_team_run_attributes",
|
||||
"get_tool_execution_attributes",
|
||||
"get_workflow_run_attributes",
|
||||
"get_workflow_session_attributes",
|
||||
"get_workflow_cache_attributes",
|
||||
"get_storage_read_attributes",
|
||||
"get_storage_write_attributes",
|
||||
]
|
||||
@@ -0,0 +1,250 @@
|
||||
"""Agno Agent run attributes handler."""
|
||||
|
||||
from typing import Optional, Tuple, Dict, Any
|
||||
from agentops.instrumentation.common.attributes import AttributeMap
|
||||
from agentops.semconv import SpanAttributes, AgentAttributes, ToolAttributes
|
||||
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
|
||||
import json
|
||||
|
||||
|
||||
def get_agent_run_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> AttributeMap:
|
||||
"""Extract span attributes for Agent.run/arun calls.
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to the run method (self, message, ...)
|
||||
kwargs: Keyword arguments passed to the run method
|
||||
return_value: The return value from the run method (RunResponse)
|
||||
|
||||
Returns:
|
||||
A dictionary of span attributes to be set on the agent span
|
||||
"""
|
||||
attributes: AttributeMap = {}
|
||||
|
||||
# Initialize variables to avoid UnboundLocalError
|
||||
agent_name = None
|
||||
|
||||
# Base attributes
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.AGENT
|
||||
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
|
||||
|
||||
# AgentOps entity attributes
|
||||
attributes[SpanAttributes.AGENTOPS_ENTITY_NAME] = "agent"
|
||||
|
||||
# Extract agent information from args[0] (self)
|
||||
if args and len(args) >= 1:
|
||||
agent = args[0]
|
||||
|
||||
# Core agent identification - set directly at root level
|
||||
if hasattr(agent, "agent_id") and agent.agent_id:
|
||||
agent_id = str(agent.agent_id)
|
||||
attributes[AgentAttributes.AGENT_ID] = agent_id
|
||||
|
||||
if hasattr(agent, "name") and agent.name:
|
||||
agent_name = str(agent.name)
|
||||
attributes[AgentAttributes.AGENT_NAME] = agent_name
|
||||
|
||||
if hasattr(agent, "role") and agent.role:
|
||||
agent_role = str(agent.role)
|
||||
attributes[AgentAttributes.AGENT_ROLE] = agent_role
|
||||
|
||||
# Check if agent is part of a team
|
||||
if hasattr(agent, "_team") and agent._team:
|
||||
team = agent._team
|
||||
if hasattr(team, "name") and team.name:
|
||||
attributes["agent.parent_team"] = str(team.name)
|
||||
attributes["agent.parent_team_display"] = f"Under {team.name}"
|
||||
if hasattr(team, "team_id") and team.team_id:
|
||||
attributes["agent.parent_team_id"] = str(team.team_id)
|
||||
|
||||
# Model information -
|
||||
if hasattr(agent, "model") and agent.model:
|
||||
model = agent.model
|
||||
if hasattr(model, "id"):
|
||||
model_id = str(model.id)
|
||||
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_id
|
||||
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = model_id
|
||||
|
||||
if hasattr(model, "provider"):
|
||||
model_provider = str(model.provider)
|
||||
attributes["agent.model_provider"] = model_provider
|
||||
|
||||
# Agent configuration details - set directly at root level
|
||||
if hasattr(agent, "description") and agent.description:
|
||||
attributes["agent.description"] = str(agent.description)
|
||||
if hasattr(agent, "goal") and agent.goal:
|
||||
attributes["agent.goal"] = str(agent.goal)
|
||||
|
||||
if hasattr(agent, "instructions") and agent.instructions:
|
||||
if isinstance(agent.instructions, list):
|
||||
attributes["agent.instruction"] = " | ".join(str(i) for i in agent.instructions)
|
||||
else:
|
||||
attributes["agent.instruction"] = str(agent.instructions)
|
||||
if hasattr(agent, "expected_output") and agent.expected_output:
|
||||
attributes["agent.expected_output"] = str(agent.expected_output)
|
||||
|
||||
if hasattr(agent, "markdown"):
|
||||
attributes["agent.markdown"] = str(agent.markdown)
|
||||
|
||||
if hasattr(agent, "reasoning"):
|
||||
attributes[AgentAttributes.AGENT_REASONING] = str(agent.reasoning)
|
||||
|
||||
if hasattr(agent, "stream"):
|
||||
attributes["agent.stream"] = str(agent.stream)
|
||||
|
||||
if hasattr(agent, "show_tool_calls"):
|
||||
attributes["agent.show_tool_calls"] = str(agent.show_tool_calls)
|
||||
|
||||
if hasattr(agent, "tool_call_limit") and agent.tool_call_limit:
|
||||
attributes["agent.tool_call_limit"] = str(agent.tool_call_limit)
|
||||
|
||||
# Tools information
|
||||
if hasattr(agent, "tools") and agent.tools:
|
||||
# Set tool count based on actual number of tools
|
||||
attributes["agent.tools_count"] = str(len(agent.tools))
|
||||
|
||||
# Collect all tool names for the AGENT_TOOLS attribute
|
||||
tool_names = []
|
||||
|
||||
if len(agent.tools) == 1:
|
||||
# Single tool - set directly at root level
|
||||
tool = agent.tools[0]
|
||||
tool_name = None
|
||||
if hasattr(tool, "name"):
|
||||
tool_name = str(tool.name)
|
||||
attributes[ToolAttributes.TOOL_NAME] = tool_name
|
||||
elif hasattr(tool, "__name__"):
|
||||
tool_name = str(tool.__name__)
|
||||
attributes[ToolAttributes.TOOL_NAME] = tool_name
|
||||
elif callable(tool):
|
||||
tool_name = getattr(tool, "__name__", "unknown_tool")
|
||||
attributes[ToolAttributes.TOOL_NAME] = tool_name
|
||||
|
||||
if tool_name:
|
||||
tool_names.append(tool_name)
|
||||
|
||||
if hasattr(tool, "description") and tool.description:
|
||||
attributes[ToolAttributes.TOOL_DESCRIPTION] = str(tool.description)
|
||||
elif hasattr(tool, "__doc__") and tool.__doc__:
|
||||
# Fallback to docstring if no description attribute
|
||||
attributes[ToolAttributes.TOOL_DESCRIPTION] = str(tool.__doc__).strip()
|
||||
else:
|
||||
# Multiple tools - use indexed format
|
||||
for i, tool in enumerate(agent.tools):
|
||||
tool_name = None
|
||||
if hasattr(tool, "name"):
|
||||
tool_name = str(tool.name)
|
||||
attributes[f"tool.{i}.name"] = tool_name
|
||||
elif hasattr(tool, "__name__"):
|
||||
tool_name = str(tool.__name__)
|
||||
attributes[f"tool.{i}.name"] = tool_name
|
||||
elif callable(tool):
|
||||
tool_name = getattr(tool, "__name__", "unknown_tool")
|
||||
attributes[f"tool.{i}.name"] = tool_name
|
||||
|
||||
if tool_name:
|
||||
tool_names.append(tool_name)
|
||||
|
||||
if hasattr(tool, "description") and tool.description:
|
||||
attributes[f"tool.{i}.description"] = str(tool.description)
|
||||
elif hasattr(tool, "__doc__") and tool.__doc__:
|
||||
# Fallback to docstring if no description attribute
|
||||
attributes[f"tool.{i}.description"] = str(tool.__doc__).strip()
|
||||
|
||||
# Set the AGENT_TOOLS attribute with all tool names
|
||||
if tool_names:
|
||||
attributes[AgentAttributes.AGENT_TOOLS] = json.dumps(tool_names)
|
||||
|
||||
if hasattr(agent, "knowledge") and agent.knowledge:
|
||||
knowledge_type = type(agent.knowledge).__name__
|
||||
attributes["agent.knowledge_type"] = knowledge_type
|
||||
|
||||
if hasattr(agent, "storage") and agent.storage:
|
||||
storage_type = type(agent.storage).__name__
|
||||
attributes["agent.storage_type"] = storage_type
|
||||
|
||||
# Session information
|
||||
if hasattr(agent, "session_id") and agent.session_id:
|
||||
session_id = str(agent.session_id)
|
||||
attributes["agent.session_id"] = session_id
|
||||
|
||||
if hasattr(agent, "user_id") and agent.user_id:
|
||||
user_id = str(agent.user_id)
|
||||
attributes["agent.user_id"] = user_id
|
||||
|
||||
# Output key if present
|
||||
if hasattr(agent, "output_key") and agent.output_key:
|
||||
attributes["agent.output_key"] = str(agent.output_key)
|
||||
|
||||
# Extract run input information
|
||||
if args and len(args) >= 2:
|
||||
message = args[1] # The message argument
|
||||
if message:
|
||||
message_str = str(message)
|
||||
attributes["agent.input"] = message_str
|
||||
# AgentOps entity input
|
||||
attributes[SpanAttributes.AGENTOPS_ENTITY_INPUT] = message_str
|
||||
|
||||
# Extract kwargs information
|
||||
if kwargs:
|
||||
if kwargs.get("stream") is not None:
|
||||
attributes[SpanAttributes.LLM_REQUEST_STREAMING] = str(kwargs["stream"])
|
||||
|
||||
if kwargs.get("session_id"):
|
||||
attributes["agent.run_session_id"] = str(kwargs["session_id"])
|
||||
|
||||
if kwargs.get("user_id"):
|
||||
attributes["agent.run_user_id"] = str(kwargs["user_id"])
|
||||
|
||||
# Extract return value information
|
||||
if return_value:
|
||||
if hasattr(return_value, "run_id") and return_value.run_id:
|
||||
run_id = str(return_value.run_id)
|
||||
attributes["agent.run_id"] = run_id
|
||||
|
||||
if hasattr(return_value, "content") and return_value.content:
|
||||
content = str(return_value.content)
|
||||
attributes["agent.output"] = content
|
||||
|
||||
if hasattr(return_value, "event") and return_value.event:
|
||||
event = str(return_value.event)
|
||||
attributes["agent.event"] = event
|
||||
|
||||
# Tool executions from the response
|
||||
if hasattr(return_value, "tools") and return_value.tools:
|
||||
# Track the number of tool executions
|
||||
attributes["agent.tool_executions_count"] = str(len(return_value.tools))
|
||||
|
||||
for i, tool_exec in enumerate(return_value.tools): # No limit - show all tools
|
||||
if hasattr(tool_exec, "tool_name") and tool_exec.tool_name:
|
||||
attributes[f"tool.{i}.name"] = str(tool_exec.tool_name)
|
||||
|
||||
if hasattr(tool_exec, "tool_args") and tool_exec.tool_args:
|
||||
try:
|
||||
args_str = json.dumps(tool_exec.tool_args)
|
||||
attributes[f"tool.{i}.parameters"] = args_str
|
||||
except:
|
||||
attributes[f"tool.{i}.parameters"] = str(tool_exec.tool_args)
|
||||
|
||||
if hasattr(tool_exec, "result") and tool_exec.result:
|
||||
result_str = str(tool_exec.result)
|
||||
attributes[f"tool.{i}.result"] = result_str
|
||||
|
||||
if hasattr(tool_exec, "tool_call_error") and tool_exec.tool_call_error:
|
||||
attributes[f"tool.{i}.error"] = str(tool_exec.tool_call_error)
|
||||
|
||||
attributes[f"tool.{i}.status"] = "success" # Default to success
|
||||
|
||||
# Add display name for better UI visualization
|
||||
if agent_name:
|
||||
# Check if we have parent team info
|
||||
parent_team = attributes.get("agent.parent_team")
|
||||
if parent_team:
|
||||
attributes["agent.display_name"] = f"{agent_name} (Agent under {parent_team})"
|
||||
else:
|
||||
attributes["agent.display_name"] = f"{agent_name}"
|
||||
|
||||
return attributes
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Agno Agent session metrics attributes handler."""
|
||||
|
||||
from typing import Optional, Tuple, Dict, Any
|
||||
|
||||
from agentops.instrumentation.common.attributes import AttributeMap
|
||||
from agentops.semconv import SpanAttributes
|
||||
|
||||
|
||||
def get_metrics_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> AttributeMap:
|
||||
"""Extract span attributes for Agent._set_session_metrics calls.
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to the _set_session_metrics method (self, run_messages)
|
||||
kwargs: Keyword arguments passed to the _set_session_metrics method
|
||||
return_value: The return value from the _set_session_metrics method
|
||||
|
||||
Returns:
|
||||
A dictionary of span attributes to be set on the metrics span
|
||||
"""
|
||||
attributes: AttributeMap = {}
|
||||
|
||||
# Base attributes
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = "llm"
|
||||
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
|
||||
attributes[SpanAttributes.AGENTOPS_ENTITY_NAME] = "LLM"
|
||||
|
||||
# Initialize usage tracking variables
|
||||
usage_data = {}
|
||||
|
||||
# Initialize counters for indexed messages
|
||||
prompt_count = 0
|
||||
completion_count = 0
|
||||
|
||||
# Extract agent and run_messages from args (self, run_messages)
|
||||
if args and len(args) >= 2:
|
||||
agent = args[0] # self (Agent instance)
|
||||
run_messages = args[1] # RunMessages object
|
||||
|
||||
# Add agent display name for LLM calls
|
||||
if hasattr(agent, "name") and agent.name:
|
||||
attributes["agno.llm.display_name"] = f"{agent.name} → LLM"
|
||||
|
||||
# Model information - get additional request parameters if available
|
||||
if hasattr(agent, "model") and agent.model:
|
||||
model = agent.model
|
||||
# Set model ID first
|
||||
if hasattr(model, "id"):
|
||||
attributes[SpanAttributes.LLM_REQUEST_MODEL] = str(model.id)
|
||||
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = str(model.id)
|
||||
# Additional model parameters
|
||||
if hasattr(model, "temperature") and model.temperature is not None:
|
||||
attributes[SpanAttributes.LLM_REQUEST_TEMPERATURE] = str(model.temperature)
|
||||
if hasattr(model, "max_tokens") and model.max_tokens is not None:
|
||||
attributes[SpanAttributes.LLM_REQUEST_MAX_TOKENS] = str(model.max_tokens)
|
||||
if hasattr(model, "top_p") and model.top_p is not None:
|
||||
attributes[SpanAttributes.LLM_REQUEST_TOP_P] = str(model.top_p)
|
||||
if hasattr(model, "provider"):
|
||||
attributes["agno.model.provider"] = str(model.provider)
|
||||
|
||||
# Add model class name for better identification (with null check)
|
||||
if hasattr(model, "__class__") and hasattr(model.__class__, "__name__"):
|
||||
model_class = model.__class__.__name__
|
||||
attributes["agno.model.class"] = model_class
|
||||
|
||||
if hasattr(run_messages, "messages") and run_messages.messages:
|
||||
messages = run_messages.messages
|
||||
|
||||
# Initialize token tracking
|
||||
total_prompt_tokens = 0
|
||||
total_completion_tokens = 0
|
||||
total_output_tokens = 0
|
||||
total_input_tokens = 0
|
||||
total_tokens = 0
|
||||
total_time = 0.0
|
||||
|
||||
# Process messages to create individual indexed gen_ai.prompt.{i} and gen_ai.completion.{i} attributes
|
||||
for i, msg in enumerate(messages):
|
||||
# Extract message content for prompts/completions
|
||||
if hasattr(msg, "role") and hasattr(msg, "content"):
|
||||
# Only process messages with actual content
|
||||
if msg.content is not None and str(msg.content).strip() != "" and str(msg.content) != "None":
|
||||
content = str(msg.content)
|
||||
# No truncation - keep full content for observability
|
||||
|
||||
if msg.role == "user":
|
||||
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.role"] = "user"
|
||||
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.content"] = content
|
||||
prompt_count += 1
|
||||
elif msg.role == "assistant":
|
||||
attributes[f"{SpanAttributes.LLM_COMPLETIONS}.{completion_count}.role"] = "assistant"
|
||||
attributes[f"{SpanAttributes.LLM_COMPLETIONS}.{completion_count}.content"] = content
|
||||
completion_count += 1
|
||||
elif msg.role == "system":
|
||||
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.role"] = "system"
|
||||
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.content"] = content
|
||||
prompt_count += 1
|
||||
|
||||
# Extract token metrics from message
|
||||
if hasattr(msg, "metrics") and msg.metrics:
|
||||
metrics = msg.metrics
|
||||
|
||||
# Handle different token metric patterns
|
||||
if hasattr(metrics, "prompt_tokens") and metrics.prompt_tokens > 0:
|
||||
total_prompt_tokens += metrics.prompt_tokens
|
||||
if hasattr(metrics, "completion_tokens") and metrics.completion_tokens > 0:
|
||||
total_completion_tokens += metrics.completion_tokens
|
||||
if hasattr(metrics, "total_tokens") and metrics.total_tokens > 0:
|
||||
total_tokens += metrics.total_tokens
|
||||
# For messages that only have output_tokens
|
||||
if hasattr(metrics, "output_tokens") and metrics.output_tokens > 0:
|
||||
total_output_tokens += metrics.output_tokens
|
||||
if hasattr(metrics, "input_tokens") and metrics.input_tokens > 0:
|
||||
total_input_tokens += metrics.input_tokens
|
||||
if hasattr(metrics, "time") and metrics.time:
|
||||
total_time += metrics.time
|
||||
|
||||
# Token metrics from agent session metrics
|
||||
if hasattr(agent, "session_metrics") and agent.session_metrics:
|
||||
session_metrics = agent.session_metrics
|
||||
|
||||
# Try to get model name from session metrics if not already set
|
||||
if SpanAttributes.LLM_REQUEST_MODEL not in attributes:
|
||||
if hasattr(session_metrics, "model") and session_metrics.model:
|
||||
model_id = str(session_metrics.model)
|
||||
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_id
|
||||
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = model_id
|
||||
|
||||
# Only set token variables if the attributes actually exist
|
||||
session_prompt_tokens = None
|
||||
session_completion_tokens = None
|
||||
session_output_tokens = None
|
||||
session_input_tokens = None
|
||||
session_total_tokens = None
|
||||
|
||||
if hasattr(session_metrics, "prompt_tokens"):
|
||||
session_prompt_tokens = session_metrics.prompt_tokens
|
||||
|
||||
if hasattr(session_metrics, "completion_tokens"):
|
||||
session_completion_tokens = session_metrics.completion_tokens
|
||||
|
||||
if hasattr(session_metrics, "output_tokens"):
|
||||
session_output_tokens = session_metrics.output_tokens
|
||||
|
||||
if hasattr(session_metrics, "input_tokens"):
|
||||
session_input_tokens = session_metrics.input_tokens
|
||||
|
||||
if hasattr(session_metrics, "total_tokens"):
|
||||
session_total_tokens = session_metrics.total_tokens
|
||||
|
||||
# For Anthropic, output_tokens represents completion tokens
|
||||
if session_output_tokens is not None and session_output_tokens > 0:
|
||||
if session_completion_tokens is None or session_completion_tokens == 0:
|
||||
session_completion_tokens = session_output_tokens
|
||||
|
||||
# For some providers, input_tokens represents prompt tokens
|
||||
if session_input_tokens is not None and session_input_tokens > 0:
|
||||
if session_prompt_tokens is None or session_prompt_tokens == 0:
|
||||
session_prompt_tokens = session_input_tokens
|
||||
|
||||
# Only set token attributes if we have actual values
|
||||
if session_total_tokens is not None and session_total_tokens > 0:
|
||||
usage_data["total_tokens"] = session_total_tokens
|
||||
|
||||
# Set breakdown if available
|
||||
if session_prompt_tokens is not None and session_prompt_tokens > 0:
|
||||
usage_data["prompt_tokens"] = session_prompt_tokens
|
||||
if session_completion_tokens is not None and session_completion_tokens > 0:
|
||||
usage_data["completion_tokens"] = session_completion_tokens
|
||||
|
||||
# Additional token types from session metrics - only set if present
|
||||
if hasattr(session_metrics, "cached_tokens") and session_metrics.cached_tokens > 0:
|
||||
usage_data["cache_read_input_tokens"] = session_metrics.cached_tokens
|
||||
if hasattr(session_metrics, "reasoning_tokens") and session_metrics.reasoning_tokens > 0:
|
||||
usage_data["reasoning_tokens"] = session_metrics.reasoning_tokens
|
||||
|
||||
# If we don't have token data from session metrics, try message aggregation
|
||||
if "total_tokens" not in usage_data:
|
||||
# Set aggregated token usage from messages
|
||||
if total_prompt_tokens > 0 or total_input_tokens > 0:
|
||||
usage_data["prompt_tokens"] = total_prompt_tokens or total_input_tokens
|
||||
if total_completion_tokens > 0 or total_output_tokens > 0:
|
||||
usage_data["completion_tokens"] = total_completion_tokens or total_output_tokens
|
||||
if total_tokens > 0:
|
||||
usage_data["total_tokens"] = total_tokens
|
||||
|
||||
# Extract user message info if available
|
||||
if hasattr(run_messages, "user_message") and run_messages.user_message:
|
||||
user_msg = run_messages.user_message
|
||||
if hasattr(user_msg, "content"):
|
||||
content = str(user_msg.content)
|
||||
attributes["agno.metrics.user_input"] = content
|
||||
|
||||
# Set individual LLM usage attributes only for values we actually have
|
||||
if "prompt_tokens" in usage_data:
|
||||
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = usage_data["prompt_tokens"]
|
||||
if "completion_tokens" in usage_data:
|
||||
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = usage_data["completion_tokens"]
|
||||
if "total_tokens" in usage_data:
|
||||
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = usage_data["total_tokens"]
|
||||
if "cache_read_input_tokens" in usage_data:
|
||||
attributes[SpanAttributes.LLM_USAGE_CACHE_READ_INPUT_TOKENS] = usage_data["cache_read_input_tokens"]
|
||||
if "reasoning_tokens" in usage_data:
|
||||
attributes[SpanAttributes.LLM_USAGE_REASONING_TOKENS] = usage_data["reasoning_tokens"]
|
||||
|
||||
# But only if we have any usage data
|
||||
if usage_data:
|
||||
for key, value in usage_data.items():
|
||||
attributes[f"gen_ai.usage.{key}"] = value
|
||||
|
||||
return attributes
|
||||
@@ -0,0 +1,158 @@
|
||||
"""Storage operation attribute handlers for Agno workflow instrumentation."""
|
||||
|
||||
import json
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
from opentelemetry.util.types import AttributeValue
|
||||
|
||||
from agentops.semconv.span_attributes import SpanAttributes
|
||||
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
|
||||
from agentops.instrumentation.common.attributes import get_common_attributes
|
||||
|
||||
|
||||
def get_storage_read_attributes(
|
||||
args: Tuple[Any, ...] = (),
|
||||
kwargs: Optional[Dict[str, Any]] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, AttributeValue]:
|
||||
"""Extract attributes from storage read operations.
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to read_from_storage
|
||||
kwargs: Keyword arguments passed to read_from_storage
|
||||
return_value: Return value from read_from_storage (the cached data or None)
|
||||
|
||||
Returns:
|
||||
Dictionary of OpenTelemetry attributes for storage read operations
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
kwargs = kwargs or {}
|
||||
|
||||
# Mark this as a storage operation within workflow context
|
||||
attributes["storage.operation"] = "read"
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.WORKFLOW
|
||||
|
||||
if args and len(args) > 0:
|
||||
workflow = args[0]
|
||||
|
||||
# Get workflow information
|
||||
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
|
||||
attributes["storage.workflow_id"] = str(workflow.workflow_id)
|
||||
if hasattr(workflow, "session_id") and workflow.session_id:
|
||||
attributes["storage.session_id"] = str(workflow.session_id)
|
||||
|
||||
# Get storage type
|
||||
if hasattr(workflow, "storage") and workflow.storage:
|
||||
storage_type = type(workflow.storage).__name__
|
||||
attributes["storage.backend"] = storage_type
|
||||
|
||||
# Get session state info for context
|
||||
if hasattr(workflow, "session_state") and isinstance(workflow.session_state, dict):
|
||||
# Get all cache keys
|
||||
cache_keys = list(workflow.session_state.keys())
|
||||
attributes["storage.cache_size"] = len(cache_keys)
|
||||
if cache_keys:
|
||||
attributes["storage.cache_keys"] = json.dumps(cache_keys)
|
||||
|
||||
# Analyze the return value to determine cache hit/miss
|
||||
if return_value is not None:
|
||||
# Cache hit
|
||||
attributes["storage.cache_hit"] = True
|
||||
attributes["storage.result"] = "hit"
|
||||
|
||||
# Get data type and size
|
||||
data_type = type(return_value).__name__
|
||||
attributes["storage.data_type"] = data_type
|
||||
|
||||
# For dict/list, show structure
|
||||
if isinstance(return_value, dict):
|
||||
attributes["storage.data_keys"] = json.dumps(list(return_value.keys()))
|
||||
attributes["storage.data_size"] = len(return_value)
|
||||
elif isinstance(return_value, (list, tuple)):
|
||||
attributes["storage.data_size"] = len(return_value)
|
||||
elif isinstance(return_value, str):
|
||||
attributes["storage.data_size"] = len(return_value)
|
||||
# Show full string data without truncation
|
||||
attributes["storage.data_preview"] = return_value
|
||||
else:
|
||||
# Cache miss
|
||||
attributes["storage.cache_hit"] = False
|
||||
attributes["storage.result"] = "miss"
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_storage_write_attributes(
|
||||
args: Tuple[Any, ...] = (),
|
||||
kwargs: Optional[Dict[str, Any]] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, AttributeValue]:
|
||||
"""Extract attributes from storage write operations.
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to write_to_storage
|
||||
kwargs: Keyword arguments passed to write_to_storage
|
||||
return_value: Return value from write_to_storage (usually None or success indicator)
|
||||
|
||||
Returns:
|
||||
Dictionary of OpenTelemetry attributes for storage write operations
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
kwargs = kwargs or {}
|
||||
|
||||
# Mark this as a storage operation within workflow context
|
||||
attributes["storage.operation"] = "write"
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.WORKFLOW
|
||||
|
||||
if args and len(args) > 0:
|
||||
workflow = args[0]
|
||||
|
||||
# Get workflow information
|
||||
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
|
||||
attributes["storage.workflow_id"] = str(workflow.workflow_id)
|
||||
if hasattr(workflow, "session_id") and workflow.session_id:
|
||||
attributes["storage.session_id"] = str(workflow.session_id)
|
||||
|
||||
# Get storage type
|
||||
if hasattr(workflow, "storage") and workflow.storage:
|
||||
storage_type = type(workflow.storage).__name__
|
||||
attributes["storage.backend"] = storage_type
|
||||
|
||||
# Get session state info to see what's being written
|
||||
if hasattr(workflow, "session_state") and isinstance(workflow.session_state, dict):
|
||||
# Get cache state after write
|
||||
cache_keys = list(workflow.session_state.keys())
|
||||
attributes["storage.cache_size"] = len(cache_keys)
|
||||
if cache_keys:
|
||||
attributes["storage.cache_keys"] = json.dumps(cache_keys)
|
||||
|
||||
# Try to identify what was written (the newest/changed data)
|
||||
# This is a heuristic - in practice you might need to track state changes
|
||||
if cache_keys:
|
||||
# Show the last key as likely the one just written
|
||||
last_key = cache_keys[-1]
|
||||
attributes["storage.written_key"] = last_key
|
||||
|
||||
# Get value preview
|
||||
value = workflow.session_state.get(last_key)
|
||||
if value is not None:
|
||||
value_type = type(value).__name__
|
||||
attributes["storage.written_value_type"] = value_type
|
||||
|
||||
if isinstance(value, str):
|
||||
if len(value) > 100:
|
||||
attributes["storage.written_value_preview"] = value[:100] + "..."
|
||||
else:
|
||||
attributes["storage.written_value_preview"] = value
|
||||
attributes["storage.written_value_size"] = len(value)
|
||||
elif isinstance(value, (dict, list)):
|
||||
attributes["storage.written_value_size"] = len(value)
|
||||
attributes["storage.written_value_preview"] = f"{value_type} with {len(value)} items"
|
||||
|
||||
# Check write result
|
||||
if return_value is not None:
|
||||
attributes["storage.write_success"] = True
|
||||
else:
|
||||
# Most storage writes return None on success, so this is normal
|
||||
attributes["storage.write_success"] = True
|
||||
|
||||
return attributes
|
||||
@@ -0,0 +1,195 @@
|
||||
"""Agno Team run attributes handler."""
|
||||
|
||||
from typing import Optional, Tuple, Dict, Any
|
||||
import json
|
||||
from agentops.instrumentation.common.attributes import AttributeMap
|
||||
from agentops.semconv import SpanAttributes, WorkflowAttributes
|
||||
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
|
||||
|
||||
|
||||
def get_team_run_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> AttributeMap:
|
||||
"""Extract span attributes for Team method calls (both internal and public).
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to the Team method
|
||||
kwargs: Keyword arguments passed to the Team method
|
||||
return_value: The return value from the Team method
|
||||
|
||||
Returns:
|
||||
A dictionary of span attributes to be set on the workflow span
|
||||
"""
|
||||
attributes: AttributeMap = {}
|
||||
|
||||
# Base attributes
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.WORKFLOW
|
||||
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
|
||||
attributes[WorkflowAttributes.WORKFLOW_TYPE] = "team_run"
|
||||
|
||||
# Extract team information from instance
|
||||
if args and len(args) > 0:
|
||||
team = args[0] # self (Team instance)
|
||||
|
||||
# Team identification
|
||||
if hasattr(team, "name") and team.name:
|
||||
attributes["team.name"] = str(team.name)
|
||||
attributes["team.display_name"] = f"{team.name} (Team)"
|
||||
|
||||
if hasattr(team, "team_id") and team.team_id:
|
||||
attributes["team.team_id"] = str(team.team_id)
|
||||
|
||||
if hasattr(team, "mode") and team.mode:
|
||||
attributes["team.mode"] = str(team.mode)
|
||||
|
||||
if hasattr(team, "members") and team.members:
|
||||
attributes["team.members_count"] = str(len(team.members))
|
||||
|
||||
# Add detailed member information
|
||||
member_agents = []
|
||||
for i, member in enumerate(team.members):
|
||||
member_info = {}
|
||||
if hasattr(member, "name") and member.name:
|
||||
member_info["name"] = str(member.name)
|
||||
if hasattr(member, "agent_id") and member.agent_id:
|
||||
member_info["id"] = str(member.agent_id)
|
||||
if hasattr(member, "role") and member.role:
|
||||
member_info["role"] = str(member.role)
|
||||
if hasattr(member, "model") and member.model:
|
||||
if hasattr(member.model, "id"):
|
||||
member_info["model"] = str(member.model.id)
|
||||
|
||||
# Add member info to list
|
||||
if member_info:
|
||||
member_agents.append(member_info)
|
||||
|
||||
# Also add individual member attributes
|
||||
for key, value in member_info.items():
|
||||
attributes[f"team.member.{i}.{key}"] = value
|
||||
|
||||
# Add aggregated member list
|
||||
if member_agents:
|
||||
try:
|
||||
attributes["team.members"] = json.dumps(member_agents)
|
||||
# Also add a simple list of member names
|
||||
member_names = [m.get("name", "Unknown") for m in member_agents]
|
||||
attributes["team.member_names"] = ", ".join(member_names)
|
||||
except:
|
||||
attributes["team.members"] = str(member_agents)
|
||||
|
||||
# Process input arguments - handle both internal and public method signatures
|
||||
if args and len(args) >= 2:
|
||||
input_arg = args[1]
|
||||
|
||||
# Check if it's internal method (has run_messages) or public method (direct message)
|
||||
if hasattr(input_arg, "messages"):
|
||||
# Internal method: args[1] is run_messages
|
||||
run_messages = input_arg
|
||||
# Get the user message for workflow input
|
||||
user_messages = [msg for msg in run_messages.messages if hasattr(msg, "role") and msg.role == "user"]
|
||||
if user_messages:
|
||||
last_user_msg = user_messages[-1]
|
||||
if hasattr(last_user_msg, "content"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT] = str(last_user_msg.content)
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT_TYPE] = "message"
|
||||
# Count total messages
|
||||
attributes["team.messages_count"] = str(len(run_messages.messages))
|
||||
else:
|
||||
# Public method: args[1] is the message directly
|
||||
message = input_arg
|
||||
if message is not None:
|
||||
if isinstance(message, str):
|
||||
message_content = message
|
||||
elif hasattr(message, "content"):
|
||||
message_content = str(message.content)
|
||||
elif hasattr(message, "get_content_string"):
|
||||
message_content = message.get_content_string()
|
||||
else:
|
||||
message_content = str(message)
|
||||
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT] = message_content
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT_TYPE] = "message"
|
||||
|
||||
# Process keyword arguments
|
||||
if kwargs:
|
||||
if kwargs.get("user_id"):
|
||||
attributes[SpanAttributes.LLM_USER] = kwargs["user_id"]
|
||||
|
||||
if kwargs.get("session_id"):
|
||||
attributes["team.session_id"] = kwargs["session_id"]
|
||||
|
||||
if kwargs.get("response_format"):
|
||||
attributes["team.response_format"] = str(type(kwargs["response_format"]).__name__)
|
||||
|
||||
if kwargs.get("stream"):
|
||||
attributes["team.streaming"] = str(kwargs["stream"])
|
||||
|
||||
if kwargs.get("stream_intermediate_steps"):
|
||||
attributes["team.stream_intermediate_steps"] = str(kwargs["stream_intermediate_steps"])
|
||||
|
||||
if kwargs.get("retries"):
|
||||
attributes["team.retries"] = str(kwargs["retries"])
|
||||
|
||||
# Media attachments
|
||||
if kwargs.get("audio"):
|
||||
attributes["team.has_audio"] = "true"
|
||||
if kwargs.get("images"):
|
||||
attributes["team.has_images"] = "true"
|
||||
if kwargs.get("videos"):
|
||||
attributes["team.has_videos"] = "true"
|
||||
if kwargs.get("files"):
|
||||
attributes["team.has_files"] = "true"
|
||||
|
||||
if kwargs.get("knowledge_filters"):
|
||||
attributes["team.has_knowledge_filters"] = "true"
|
||||
|
||||
# Process return value (TeamRunResponse or Iterator)
|
||||
if return_value:
|
||||
# Handle both single response and iterator
|
||||
if hasattr(return_value, "__iter__") and not isinstance(return_value, str):
|
||||
# It's an iterator for streaming
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = "team_run_response_stream"
|
||||
attributes["team.is_streaming"] = "true"
|
||||
else:
|
||||
# Non-streaming response
|
||||
if hasattr(return_value, "content"):
|
||||
# It's a TeamRunResponse with content
|
||||
content = str(return_value.content)
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT] = content
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = "team_run_response"
|
||||
else:
|
||||
# Unknown return type or response without content
|
||||
output = str(return_value)
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT] = output
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = type(return_value).__name__
|
||||
|
||||
# Set additional team response attributes (for both streaming and non-streaming)
|
||||
if hasattr(return_value, "run_id"):
|
||||
attributes["team.run_id"] = str(return_value.run_id)
|
||||
|
||||
if hasattr(return_value, "session_id"):
|
||||
attributes["team.response_session_id"] = str(return_value.session_id)
|
||||
|
||||
if hasattr(return_value, "team_id"):
|
||||
attributes["team.response_team_id"] = str(return_value.team_id)
|
||||
|
||||
if hasattr(return_value, "model"):
|
||||
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = str(return_value.model)
|
||||
|
||||
if hasattr(return_value, "model_provider"):
|
||||
attributes["team.model_provider"] = str(return_value.model_provider)
|
||||
|
||||
if hasattr(return_value, "event"):
|
||||
attributes["team.event"] = str(return_value.event)
|
||||
|
||||
# Team-specific attributes
|
||||
if hasattr(return_value, "content_type"):
|
||||
attributes["team.response_content_type"] = str(return_value.content_type)
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
# Keep the public function as an alias for backward compatibility
|
||||
get_team_public_run_attributes = get_team_run_attributes
|
||||
@@ -0,0 +1,210 @@
|
||||
"""Agno tool execution attributes handler."""
|
||||
|
||||
import json
|
||||
from typing import Optional, Tuple, Dict, Any
|
||||
import time
|
||||
from agentops.instrumentation.common.attributes import AttributeMap
|
||||
from agentops.semconv import SpanAttributes
|
||||
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
|
||||
from agentops.semconv.tool import ToolAttributes
|
||||
|
||||
|
||||
def get_tool_execution_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> AttributeMap:
|
||||
"""Extract span attributes for tool execution calls (FunctionCall.execute/aexecute).
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to the execute method (self)
|
||||
kwargs: Keyword arguments passed to the execute method
|
||||
return_value: The return value from the execute method (FunctionExecutionResult)
|
||||
|
||||
Returns:
|
||||
A dictionary of span attributes to be set on the tool execution span
|
||||
"""
|
||||
attributes: AttributeMap = {}
|
||||
|
||||
# Base attributes
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.TOOL
|
||||
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
|
||||
|
||||
# AgentOps entity attributes
|
||||
attributes[SpanAttributes.AGENTOPS_ENTITY_NAME] = "tool"
|
||||
|
||||
# Process the FunctionCall object (self in execute method)
|
||||
if args and len(args) > 0:
|
||||
function_call = args[0]
|
||||
|
||||
# Add detailed function call information
|
||||
attributes["tool.function_call_type"] = str(type(function_call).__name__)
|
||||
|
||||
# Extract tool information
|
||||
if hasattr(function_call, "function") and function_call.function:
|
||||
function = function_call.function
|
||||
|
||||
# Get function name and add display name
|
||||
if hasattr(function, "__name__"):
|
||||
func_name = function.__name__
|
||||
attributes["tool.function_name"] = func_name
|
||||
attributes["tool.display_name"] = f"{func_name} (Tool)"
|
||||
|
||||
tool_name = getattr(function, "name", "unknown_tool")
|
||||
|
||||
# Set span attributes for the tool execution span
|
||||
attributes[ToolAttributes.TOOL_NAME] = tool_name
|
||||
attributes["tool.function_name"] = tool_name
|
||||
|
||||
# Function details and context
|
||||
if hasattr(function, "description"):
|
||||
description = getattr(function, "description", "")
|
||||
if description:
|
||||
attributes[ToolAttributes.TOOL_DESCRIPTION] = description
|
||||
attributes["tool.function_description"] = description
|
||||
|
||||
# Function source information
|
||||
if hasattr(function, "entrypoint") and function.entrypoint:
|
||||
entrypoint = function.entrypoint
|
||||
if hasattr(entrypoint, "__module__"):
|
||||
attributes["tool.function_module"] = str(entrypoint.__module__)
|
||||
if hasattr(entrypoint, "__name__"):
|
||||
attributes["tool.function_method"] = str(entrypoint.__name__)
|
||||
if hasattr(entrypoint, "__qualname__"):
|
||||
attributes["tool.function_qualname"] = str(entrypoint.__qualname__)
|
||||
|
||||
# Tool capabilities
|
||||
if hasattr(function, "requires_confirmation"):
|
||||
attributes["tool.requires_confirmation"] = str(function.requires_confirmation)
|
||||
if hasattr(function, "show_result"):
|
||||
attributes["tool.show_result"] = str(function.show_result)
|
||||
if hasattr(function, "stop_after_tool_call"):
|
||||
attributes["tool.stop_after_tool_call"] = str(function.stop_after_tool_call)
|
||||
|
||||
# Extract tool arguments with better formatting
|
||||
if hasattr(function_call, "arguments") and function_call.arguments:
|
||||
try:
|
||||
if isinstance(function_call.arguments, str):
|
||||
args_dict = json.loads(function_call.arguments)
|
||||
else:
|
||||
args_dict = function_call.arguments
|
||||
|
||||
# Format arguments nicely
|
||||
formatted_args = []
|
||||
for key, value in args_dict.items():
|
||||
value_str = str(value)
|
||||
formatted_args.append(f"{key}={value_str}")
|
||||
|
||||
attributes[ToolAttributes.TOOL_PARAMETERS] = json.dumps(args_dict)
|
||||
attributes["tool.formatted_args"] = ", ".join(formatted_args)
|
||||
attributes["tool.args_count"] = str(len(args_dict))
|
||||
except Exception as e:
|
||||
attributes[ToolAttributes.TOOL_PARAMETERS] = str(function_call.arguments)
|
||||
attributes["tool.args_parse_error"] = str(e)
|
||||
|
||||
# Extract call ID and metadata
|
||||
if hasattr(function_call, "tool_call_id"):
|
||||
attributes["tool.call_id"] = str(function_call.tool_call_id)
|
||||
|
||||
# Check for any agent context
|
||||
if hasattr(function_call, "_agent") and function_call._agent:
|
||||
agent = function_call._agent
|
||||
if hasattr(agent, "name"):
|
||||
attributes["tool.calling_agent_name"] = str(agent.name)
|
||||
if hasattr(agent, "agent_id"):
|
||||
attributes["tool.calling_agent_id"] = str(agent.agent_id)
|
||||
|
||||
# Process return value
|
||||
if return_value is not None:
|
||||
# Add timing information
|
||||
attributes["tool.execution_timestamp"] = str(int(time.time() * 1000))
|
||||
|
||||
# Determine execution status and result information
|
||||
if hasattr(return_value, "value"):
|
||||
# FunctionExecutionResult with value
|
||||
result_value = return_value.value
|
||||
attributes["tool.execution_status"] = "success"
|
||||
else:
|
||||
# Direct return value
|
||||
result_value = return_value
|
||||
attributes["tool.execution_status"] = "success"
|
||||
|
||||
# Process result value
|
||||
if result_value is not None:
|
||||
result_type = type(result_value).__name__
|
||||
attributes["tool.execution_result_status"] = str(result_type)
|
||||
|
||||
# Handle FunctionExecutionResult objects specifically
|
||||
if hasattr(result_value, "status") and hasattr(result_value, "result"):
|
||||
# This looks like a FunctionExecutionResult
|
||||
status = getattr(result_value, "status", "unknown")
|
||||
actual_result = getattr(result_value, "result", None)
|
||||
error = getattr(result_value, "error", None)
|
||||
|
||||
attributes["tool.execution_result_status"] = str(status)
|
||||
attributes[ToolAttributes.TOOL_STATUS] = str(status)
|
||||
|
||||
if error:
|
||||
attributes["tool.execution_error"] = str(error)
|
||||
attributes["tool.error"] = str(error)
|
||||
|
||||
if actual_result is not None:
|
||||
actual_result_type = type(actual_result).__name__
|
||||
attributes["tool.actual_result_type"] = actual_result_type
|
||||
|
||||
# Enhanced generator handling
|
||||
if hasattr(actual_result, "__iter__") and hasattr(actual_result, "__next__"):
|
||||
attributes["tool.result_is_generator"] = "true"
|
||||
|
||||
# Try to get more meaningful information about the generator
|
||||
generator_info = []
|
||||
|
||||
# Get function name from the generator
|
||||
if hasattr(actual_result, "gi_code"):
|
||||
func_name = actual_result.gi_code.co_name
|
||||
attributes["tool.generator_function"] = func_name
|
||||
generator_info.append(f"function={func_name}")
|
||||
|
||||
if generator_info:
|
||||
result_str = f"Generator<{actual_result_type}>({', '.join(generator_info)})"
|
||||
else:
|
||||
result_str = f"Generator<{actual_result_type}> - {str(actual_result)}"
|
||||
else:
|
||||
# Regular result
|
||||
result_str = str(actual_result)
|
||||
else:
|
||||
result_str = str(status)
|
||||
else:
|
||||
# Not a FunctionExecutionResult, handle as direct result
|
||||
if hasattr(result_value, "__iter__") and hasattr(result_value, "__next__"):
|
||||
# It's a generator
|
||||
attributes["tool.result_is_generator"] = "true"
|
||||
|
||||
if hasattr(result_value, "gi_code"):
|
||||
func_name = result_value.gi_code.co_name
|
||||
attributes["tool.generator_function"] = func_name
|
||||
result_str = f"Generator<{result_type}> function={func_name} - {str(result_value)}"
|
||||
else:
|
||||
result_str = f"Generator<{result_type}> - {str(result_value)}"
|
||||
else:
|
||||
# Regular result
|
||||
result_str = str(result_value)
|
||||
else:
|
||||
result_str = "None"
|
||||
|
||||
# Set the main result attribute
|
||||
attributes[ToolAttributes.TOOL_RESULT] = result_str
|
||||
|
||||
# Add additional analysis attributes
|
||||
attributes["tool.result_length"] = str(len(result_str))
|
||||
|
||||
# Set final execution status
|
||||
if not attributes.get(ToolAttributes.TOOL_STATUS):
|
||||
attributes[ToolAttributes.TOOL_STATUS] = "success"
|
||||
|
||||
# Add execution summary for debugging
|
||||
tool_name = attributes.get(ToolAttributes.TOOL_NAME, "unknown")
|
||||
call_type = attributes.get("tool.transfer_type", "unknown")
|
||||
attributes["tool.execution_summary"] = f"Tool '{tool_name}' executed with type '{call_type}'"
|
||||
|
||||
return attributes
|
||||
@@ -0,0 +1,254 @@
|
||||
"""Workflow attribute extraction for agno workflow instrumentation."""
|
||||
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
from opentelemetry.util.types import AttributeValue
|
||||
import json
|
||||
|
||||
from agentops.semconv.instrumentation import InstrumentationAttributes
|
||||
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
|
||||
from agentops.semconv.workflow import WorkflowAttributes
|
||||
from agentops.instrumentation.common.attributes import get_common_attributes
|
||||
|
||||
|
||||
def get_workflow_run_attributes(
|
||||
args: Tuple[Any, ...] = (),
|
||||
kwargs: Optional[Dict[str, Any]] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, AttributeValue]:
|
||||
"""Extract attributes from workflow run operations.
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to the workflow run method
|
||||
kwargs: Keyword arguments passed to the workflow run method
|
||||
return_value: Return value from the workflow run method
|
||||
|
||||
Returns:
|
||||
Dictionary of OpenTelemetry attributes for workflow runs
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
kwargs = kwargs or {}
|
||||
|
||||
if args and len(args) > 0:
|
||||
workflow = args[0]
|
||||
|
||||
# Core workflow attributes
|
||||
if hasattr(workflow, "name") and workflow.name:
|
||||
attributes[WorkflowAttributes.WORKFLOW_NAME] = str(workflow.name)
|
||||
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_ID] = str(workflow.workflow_id)
|
||||
if hasattr(workflow, "description") and workflow.description:
|
||||
attributes[WorkflowAttributes.WORKFLOW_DESCRIPTION] = str(workflow.description)
|
||||
if hasattr(workflow, "app_id") and workflow.app_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_APP_ID] = str(workflow.app_id)
|
||||
|
||||
# Set workflow type
|
||||
attributes[WorkflowAttributes.WORKFLOW_TYPE] = "agno_workflow"
|
||||
|
||||
# Session and user attributes
|
||||
if hasattr(workflow, "session_id") and workflow.session_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_ID] = str(workflow.session_id)
|
||||
if hasattr(workflow, "session_name") and workflow.session_name:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_NAME] = str(workflow.session_name)
|
||||
if hasattr(workflow, "user_id") and workflow.user_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_USER_ID] = str(workflow.user_id)
|
||||
|
||||
# Run-specific attributes
|
||||
if hasattr(workflow, "run_id") and workflow.run_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_RUN_ID] = str(workflow.run_id)
|
||||
|
||||
# Configuration attributes
|
||||
if hasattr(workflow, "debug_mode"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_DEBUG_MODE] = bool(workflow.debug_mode)
|
||||
if hasattr(workflow, "monitoring"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_MONITORING] = bool(workflow.monitoring)
|
||||
if hasattr(workflow, "telemetry"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_TELEMETRY] = bool(workflow.telemetry)
|
||||
|
||||
# Memory and storage attributes
|
||||
if hasattr(workflow, "memory") and workflow.memory:
|
||||
memory_type = type(workflow.memory).__name__
|
||||
attributes[WorkflowAttributes.WORKFLOW_MEMORY_TYPE] = memory_type
|
||||
|
||||
if hasattr(workflow, "storage") and workflow.storage:
|
||||
storage_type = type(workflow.storage).__name__
|
||||
attributes[WorkflowAttributes.WORKFLOW_STORAGE_TYPE] = storage_type
|
||||
|
||||
# Input parameters from kwargs
|
||||
if kwargs:
|
||||
# Store workflow input
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT] = str(kwargs)
|
||||
|
||||
# Count and types of input parameters
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT_PARAMETER_COUNT] = len(kwargs)
|
||||
param_types = list(set(type(v).__name__ for v in kwargs.values()))
|
||||
if param_types:
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT_TYPE] = str(param_types)
|
||||
|
||||
# Store specific input keys (without values for privacy)
|
||||
input_keys = list(kwargs.keys())
|
||||
if input_keys:
|
||||
attributes[WorkflowAttributes.WORKFLOW_INPUT_PARAMETER_KEYS] = str(input_keys)
|
||||
|
||||
# Workflow method parameters if available
|
||||
if hasattr(workflow, "_run_parameters") and workflow._run_parameters:
|
||||
param_count = len(workflow._run_parameters)
|
||||
attributes[WorkflowAttributes.WORKFLOW_METHOD_PARAMETER_COUNT] = param_count
|
||||
|
||||
if hasattr(workflow, "_run_return_type") and workflow._run_return_type:
|
||||
attributes[WorkflowAttributes.WORKFLOW_METHOD_RETURN_TYPE] = str(workflow._run_return_type)
|
||||
|
||||
# Process return value attributes
|
||||
if return_value is not None:
|
||||
return_type = type(return_value).__name__
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = return_type
|
||||
|
||||
# Handle RunResponse objects
|
||||
if hasattr(return_value, "content"):
|
||||
# Store workflow output
|
||||
if return_value.content:
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT] = str(return_value.content)
|
||||
|
||||
if hasattr(return_value, "content_type"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_CONTENT_TYPE] = str(return_value.content_type)
|
||||
if hasattr(return_value, "event"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_EVENT] = str(return_value.event)
|
||||
if hasattr(return_value, "model"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_MODEL] = (
|
||||
str(return_value.model) if return_value.model else ""
|
||||
)
|
||||
if hasattr(return_value, "model_provider"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_MODEL_PROVIDER] = (
|
||||
str(return_value.model_provider) if return_value.model_provider else ""
|
||||
)
|
||||
|
||||
# Count various response components
|
||||
if hasattr(return_value, "messages") and return_value.messages:
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_MESSAGE_COUNT] = len(return_value.messages)
|
||||
if hasattr(return_value, "tools") and return_value.tools:
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TOOL_COUNT] = len(return_value.tools)
|
||||
if hasattr(return_value, "images") and return_value.images:
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_IMAGE_COUNT] = len(return_value.images)
|
||||
if hasattr(return_value, "videos") and return_value.videos:
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_VIDEO_COUNT] = len(return_value.videos)
|
||||
if hasattr(return_value, "audio") and return_value.audio:
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_AUDIO_COUNT] = len(return_value.audio)
|
||||
|
||||
# Handle generators/iterators
|
||||
elif hasattr(return_value, "__iter__") and not isinstance(return_value, (str, bytes)):
|
||||
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_IS_STREAMING] = True
|
||||
|
||||
# Set span kind - AgentOpsSpanKind.WORKFLOW is already a string
|
||||
attributes[InstrumentationAttributes.INSTRUMENTATION_TYPE] = AgentOpsSpanKind.WORKFLOW
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_workflow_session_attributes(
|
||||
args: Tuple[Any, ...] = (),
|
||||
kwargs: Optional[Dict[str, Any]] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, AttributeValue]:
|
||||
"""Extract attributes from workflow session operations.
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to the session method
|
||||
kwargs: Keyword arguments passed to the session method
|
||||
return_value: Return value from the session method
|
||||
|
||||
Returns:
|
||||
Dictionary of OpenTelemetry attributes for workflow sessions
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
kwargs = kwargs or {}
|
||||
|
||||
if args and len(args) > 0:
|
||||
workflow = args[0]
|
||||
|
||||
# Session attributes
|
||||
if hasattr(workflow, "session_id") and workflow.session_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_ID] = str(workflow.session_id)
|
||||
if hasattr(workflow, "session_name") and workflow.session_name:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_NAME] = str(workflow.session_name)
|
||||
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_WORKFLOW_ID] = str(workflow.workflow_id)
|
||||
if hasattr(workflow, "user_id") and workflow.user_id:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_USER_ID] = str(workflow.user_id)
|
||||
|
||||
# Session state attributes
|
||||
if hasattr(workflow, "session_state") and workflow.session_state:
|
||||
if isinstance(workflow.session_state, dict):
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_STATE_KEYS] = str(list(workflow.session_state.keys()))
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_STATE_SIZE] = len(workflow.session_state)
|
||||
|
||||
# Storage attributes
|
||||
if hasattr(workflow, "storage") and workflow.storage:
|
||||
storage_type = type(workflow.storage).__name__
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_STORAGE_TYPE] = storage_type
|
||||
|
||||
# Process session return value if it's a WorkflowSession
|
||||
if return_value is not None and hasattr(return_value, "session_id"):
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_RETURNED_SESSION_ID] = str(return_value.session_id)
|
||||
if hasattr(return_value, "created_at") and return_value.created_at:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_CREATED_AT] = int(return_value.created_at)
|
||||
if hasattr(return_value, "updated_at") and return_value.updated_at:
|
||||
attributes[WorkflowAttributes.WORKFLOW_SESSION_UPDATED_AT] = int(return_value.updated_at)
|
||||
|
||||
# Set span kind - AgentOpsSpanKind.WORKFLOW is already a string
|
||||
attributes[InstrumentationAttributes.INSTRUMENTATION_TYPE] = AgentOpsSpanKind.WORKFLOW
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_workflow_cache_attributes(
|
||||
args: Tuple[Any, ...] = (),
|
||||
kwargs: Optional[Dict[str, Any]] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, AttributeValue]:
|
||||
"""Extract attributes from workflow cache operations.
|
||||
|
||||
Args:
|
||||
args: Positional arguments passed to the cache method
|
||||
kwargs: Keyword arguments passed to the cache method
|
||||
return_value: Return value from the cache method
|
||||
|
||||
Returns:
|
||||
Dictionary of OpenTelemetry attributes for cache operations
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
kwargs = kwargs or {}
|
||||
|
||||
if args and len(args) > 0:
|
||||
workflow = args[0]
|
||||
|
||||
# Get workflow information
|
||||
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
|
||||
attributes["cache.workflow_id"] = str(workflow.workflow_id)
|
||||
if hasattr(workflow, "session_id") and workflow.session_id:
|
||||
attributes["cache.session_id"] = str(workflow.session_id)
|
||||
|
||||
# Get cache state
|
||||
if hasattr(workflow, "session_state") and isinstance(workflow.session_state, dict):
|
||||
attributes["cache.size"] = len(workflow.session_state)
|
||||
attributes["cache.keys"] = json.dumps(list(workflow.session_state.keys()))
|
||||
|
||||
# Determine cache operation type and result
|
||||
if len(args) > 1:
|
||||
cache_key = str(args[1])
|
||||
attributes["cache.key"] = cache_key
|
||||
|
||||
if return_value is not None:
|
||||
attributes["cache.hit"] = True
|
||||
attributes["cache.result"] = "hit"
|
||||
|
||||
# Add value info
|
||||
if isinstance(return_value, str):
|
||||
attributes["cache.value_size"] = len(return_value)
|
||||
if len(return_value) <= 100:
|
||||
attributes["cache.value"] = return_value
|
||||
else:
|
||||
attributes["cache.value_preview"] = return_value[:100] + "..."
|
||||
else:
|
||||
attributes["cache.hit"] = False
|
||||
attributes["cache.result"] = "miss"
|
||||
|
||||
return attributes
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright 2023 openllmetry
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
@@ -0,0 +1,10 @@
|
||||
This package contains code derived from the OpenLLMetry project, which is licensed under the Apache License, Version 2.0.
|
||||
|
||||
Original repository: https://github.com/traceloop/openllmetry
|
||||
|
||||
Copyright notice from the original project:
|
||||
Copyright (c) Traceloop (https://traceloop.com)
|
||||
|
||||
The Apache 2.0 license can be found in the LICENSE file in this directory.
|
||||
|
||||
This code has been modified and adapted for use in the AgentOps project.
|
||||
@@ -0,0 +1,6 @@
|
||||
"""OpenTelemetry CrewAI instrumentation"""
|
||||
|
||||
from agentops.instrumentation.agentic.crewai.version import __version__
|
||||
from agentops.instrumentation.agentic.crewai.instrumentation import CrewaiInstrumentor
|
||||
|
||||
__all__ = ["CrewaiInstrumentor", "__version__"]
|
||||
@@ -0,0 +1,335 @@
|
||||
"""OpenTelemetry instrumentation for CrewAI."""
|
||||
|
||||
import json
|
||||
import logging
|
||||
from typing import Any
|
||||
from opentelemetry.trace import Span
|
||||
|
||||
from agentops.semconv.span_attributes import SpanAttributes
|
||||
from agentops.semconv.agent import AgentAttributes
|
||||
from agentops.semconv.message import MessageAttributes
|
||||
|
||||
# Initialize logger for logging potential issues and operations
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _parse_tools(tools):
|
||||
"""Parse tools into a JSON string with name and description."""
|
||||
result = []
|
||||
for tool in tools:
|
||||
res = {}
|
||||
if hasattr(tool, "name") and tool.name is not None:
|
||||
res["name"] = tool.name
|
||||
if hasattr(tool, "description") and tool.description is not None:
|
||||
res["description"] = tool.description
|
||||
if res:
|
||||
result.append(res)
|
||||
return result
|
||||
|
||||
|
||||
def set_span_attribute(span: Span, key: str, value: Any) -> None:
|
||||
"""Set a single attribute on a span."""
|
||||
if value is not None and value != "":
|
||||
if hasattr(value, "__str__"):
|
||||
value = str(value)
|
||||
span.set_attribute(key, value)
|
||||
|
||||
|
||||
class CrewAISpanAttributes:
|
||||
"""Manages span attributes for CrewAI instrumentation."""
|
||||
|
||||
def __init__(self, span: Span, instance, skip_agent_processing=False) -> None:
|
||||
self.span = span
|
||||
self.instance = instance
|
||||
self.skip_agent_processing = skip_agent_processing
|
||||
self.process_instance()
|
||||
|
||||
def process_instance(self):
|
||||
"""Process the instance based on its type."""
|
||||
instance_type = self.instance.__class__.__name__
|
||||
self._set_attribute(SpanAttributes.LLM_SYSTEM, "crewai")
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, instance_type)
|
||||
|
||||
method_mapping = {
|
||||
"Crew": self._process_crew,
|
||||
"Agent": self._process_agent,
|
||||
"Task": self._process_task,
|
||||
"LLM": self._process_llm,
|
||||
}
|
||||
method = method_mapping.get(instance_type)
|
||||
if method:
|
||||
method()
|
||||
|
||||
def _process_crew(self):
|
||||
"""Process a Crew instance."""
|
||||
crew_id = getattr(self.instance, "id", "")
|
||||
self._set_attribute("crewai.crew.id", str(crew_id))
|
||||
self._set_attribute("crewai.crew.type", "crewai.crew")
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "workflow")
|
||||
|
||||
logger.debug(f"CrewAI: Processing crew with id {crew_id}")
|
||||
|
||||
for key, value in self.instance.__dict__.items():
|
||||
if value is None:
|
||||
continue
|
||||
|
||||
if key == "tasks":
|
||||
if isinstance(value, list):
|
||||
self._set_attribute("crewai.crew.max_turns", str(len(value)))
|
||||
logger.debug(f"CrewAI: Found {len(value)} tasks")
|
||||
elif key == "agents":
|
||||
if isinstance(value, list):
|
||||
logger.debug(f"CrewAI: Found {len(value)} agents in crew")
|
||||
|
||||
if not self.skip_agent_processing:
|
||||
self._parse_agents(value)
|
||||
elif key == "llms":
|
||||
self._parse_llms(value)
|
||||
elif key == "result":
|
||||
self._set_attribute("crewai.crew.final_output", str(value))
|
||||
self._set_attribute("crewai.crew.output", str(value))
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, str(value))
|
||||
else:
|
||||
self._set_attribute(f"crewai.crew.{key}", str(value))
|
||||
|
||||
def _process_agent(self):
|
||||
"""Process an Agent instance."""
|
||||
agent = {}
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "agent")
|
||||
|
||||
for key, value in self.instance.__dict__.items():
|
||||
if key == "tools":
|
||||
parsed_tools = _parse_tools(value)
|
||||
for i, tool in enumerate(parsed_tools):
|
||||
tool_prefix = f"crewai.agent.tool.{i}."
|
||||
for tool_key, tool_value in tool.items():
|
||||
self._set_attribute(f"{tool_prefix}{tool_key}", str(tool_value))
|
||||
|
||||
agent[key] = json.dumps(parsed_tools)
|
||||
|
||||
if value is None:
|
||||
continue
|
||||
|
||||
if key != "tools":
|
||||
agent[key] = str(value)
|
||||
|
||||
self._set_attribute(AgentAttributes.AGENT_ID, agent.get("id", ""))
|
||||
self._set_attribute(AgentAttributes.AGENT_ROLE, agent.get("role", ""))
|
||||
self._set_attribute(AgentAttributes.AGENT_NAME, agent.get("name", ""))
|
||||
self._set_attribute(AgentAttributes.AGENT_TOOLS, agent.get("tools", ""))
|
||||
|
||||
if "reasoning" in agent:
|
||||
self._set_attribute(AgentAttributes.AGENT_REASONING, agent.get("reasoning", ""))
|
||||
|
||||
if "goal" in agent:
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, agent.get("goal", ""))
|
||||
|
||||
self._set_attribute("crewai.agent.goal", agent.get("goal", ""))
|
||||
self._set_attribute("crewai.agent.backstory", agent.get("backstory", ""))
|
||||
self._set_attribute("crewai.agent.cache", agent.get("cache", ""))
|
||||
self._set_attribute("crewai.agent.allow_delegation", agent.get("allow_delegation", ""))
|
||||
self._set_attribute("crewai.agent.allow_code_execution", agent.get("allow_code_execution", ""))
|
||||
self._set_attribute("crewai.agent.max_retry_limit", agent.get("max_retry_limit", ""))
|
||||
|
||||
if hasattr(self.instance, "llm") and self.instance.llm is not None:
|
||||
model_name = (
|
||||
getattr(self.instance.llm, "model", None) or getattr(self.instance.llm, "model_name", None) or ""
|
||||
)
|
||||
temp = getattr(self.instance.llm, "temperature", None)
|
||||
max_tokens = getattr(self.instance.llm, "max_tokens", None)
|
||||
top_p = getattr(self.instance.llm, "top_p", None)
|
||||
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model_name)
|
||||
if temp is not None:
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_TEMPERATURE, str(temp))
|
||||
if max_tokens is not None:
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_MAX_TOKENS, str(max_tokens))
|
||||
if top_p is not None:
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_TOP_P, str(top_p))
|
||||
|
||||
self._set_attribute("crewai.agent.llm", str(model_name))
|
||||
self._set_attribute(AgentAttributes.AGENT_MODELS, str(model_name))
|
||||
|
||||
def _process_task(self):
|
||||
"""Process a Task instance."""
|
||||
task = {}
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "workflow.step")
|
||||
|
||||
for key, value in self.instance.__dict__.items():
|
||||
if value is None:
|
||||
continue
|
||||
if key == "tools":
|
||||
parsed_tools = _parse_tools(value)
|
||||
for i, tool in enumerate(parsed_tools):
|
||||
tool_prefix = f"crewai.task.tool.{i}."
|
||||
for tool_key, tool_value in tool.items():
|
||||
self._set_attribute(f"{tool_prefix}{tool_key}", str(tool_value))
|
||||
|
||||
task[key] = json.dumps(parsed_tools)
|
||||
|
||||
elif key == "agent":
|
||||
task[key] = value.role if value else None
|
||||
if value:
|
||||
agent_id = getattr(value, "id", "")
|
||||
self._set_attribute(AgentAttributes.FROM_AGENT, str(agent_id))
|
||||
else:
|
||||
task[key] = str(value)
|
||||
|
||||
self._set_attribute("crewai.task.name", task.get("description", ""))
|
||||
self._set_attribute("crewai.task.type", "task")
|
||||
self._set_attribute("crewai.task.input", task.get("context", ""))
|
||||
self._set_attribute("crewai.task.expected_output", task.get("expected_output", ""))
|
||||
|
||||
if "description" in task:
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, task.get("description", ""))
|
||||
if "output" in task:
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, task.get("output", ""))
|
||||
self._set_attribute("crewai.task.output", task.get("output", ""))
|
||||
|
||||
if "id" in task:
|
||||
self._set_attribute("crewai.task.id", str(task.get("id", "")))
|
||||
|
||||
if "status" in task:
|
||||
self._set_attribute("crewai.task.status", task.get("status", ""))
|
||||
|
||||
self._set_attribute("crewai.task.agent", task.get("agent", ""))
|
||||
self._set_attribute("crewai.task.human_input", task.get("human_input", ""))
|
||||
self._set_attribute("crewai.task.processed_by_agents", str(task.get("processed_by_agents", "")))
|
||||
|
||||
if "tools" in task and task["tools"]:
|
||||
try:
|
||||
tools = json.loads(task["tools"])
|
||||
for i, tool in enumerate(tools):
|
||||
self._set_attribute(MessageAttributes.TOOL_CALL_NAME.format(i=i), tool.get("name", ""))
|
||||
self._set_attribute(
|
||||
MessageAttributes.TOOL_CALL_DESCRIPTION.format(i=i), tool.get("description", "")
|
||||
)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
logger.warning(f"Failed to parse tools for task: {task.get('id', 'unknown')}")
|
||||
|
||||
def _process_llm(self):
|
||||
"""Process an LLM instance."""
|
||||
llm = {}
|
||||
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "llm")
|
||||
|
||||
for key, value in self.instance.__dict__.items():
|
||||
if value is None:
|
||||
continue
|
||||
llm[key] = str(value)
|
||||
|
||||
model_name = llm.get("model_name", "") or llm.get("model", "")
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model_name)
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_TEMPERATURE, llm.get("temperature", ""))
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_MAX_TOKENS, llm.get("max_tokens", ""))
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_TOP_P, llm.get("top_p", ""))
|
||||
|
||||
if "frequency_penalty" in llm:
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_FREQUENCY_PENALTY, llm.get("frequency_penalty", ""))
|
||||
if "presence_penalty" in llm:
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_PRESENCE_PENALTY, llm.get("presence_penalty", ""))
|
||||
if "streaming" in llm:
|
||||
self._set_attribute(SpanAttributes.LLM_REQUEST_STREAMING, llm.get("streaming", ""))
|
||||
|
||||
if "api_key" in llm:
|
||||
self._set_attribute("gen_ai.request.api_key_present", "true")
|
||||
|
||||
if "base_url" in llm:
|
||||
self._set_attribute(SpanAttributes.LLM_OPENAI_API_BASE, llm.get("base_url", ""))
|
||||
|
||||
if "api_version" in llm:
|
||||
self._set_attribute(SpanAttributes.LLM_OPENAI_API_VERSION, llm.get("api_version", ""))
|
||||
|
||||
def _parse_agents(self, agents):
|
||||
"""Parse agents into a list of dictionaries."""
|
||||
if not agents:
|
||||
logger.debug("CrewAI: No agents to parse")
|
||||
return
|
||||
|
||||
agent_count = len(agents)
|
||||
logger.debug(f"CrewAI: Parsing {agent_count} agents")
|
||||
|
||||
# Pre-process all agents to collect their data first
|
||||
agent_data_list = []
|
||||
|
||||
for idx, agent in enumerate(agents):
|
||||
if agent is None:
|
||||
logger.debug(f"CrewAI: Agent at index {idx} is None, skipping")
|
||||
agent_data_list.append(None)
|
||||
continue
|
||||
|
||||
logger.debug(f"CrewAI: Processing agent at index {idx}")
|
||||
try:
|
||||
agent_data = self._extract_agent_data(agent)
|
||||
agent_data_list.append(agent_data)
|
||||
except Exception as e:
|
||||
logger.error(f"CrewAI: Error extracting data for agent at index {idx}: {str(e)}")
|
||||
agent_data_list.append(None)
|
||||
|
||||
# Now set all attributes at once for each agent
|
||||
for idx, agent_data in enumerate(agent_data_list):
|
||||
if agent_data is None:
|
||||
continue
|
||||
|
||||
for key, value in agent_data.items():
|
||||
if key == "tools" and isinstance(value, list):
|
||||
for tool_idx, tool in enumerate(value):
|
||||
for tool_key, tool_value in tool.items():
|
||||
self._set_attribute(f"crewai.agents.{idx}.tools.{tool_idx}.{tool_key}", str(tool_value))
|
||||
else:
|
||||
self._set_attribute(f"crewai.agents.{idx}.{key}", value)
|
||||
|
||||
def _parse_llms(self, llms):
|
||||
"""Parse LLMs into a list of dictionaries."""
|
||||
for idx, llm in enumerate(llms):
|
||||
if llm is not None:
|
||||
model_name = getattr(llm, "model", None) or getattr(llm, "model_name", None) or ""
|
||||
llm_data = {
|
||||
"model": model_name,
|
||||
"temperature": llm.temperature,
|
||||
"max_tokens": llm.max_tokens,
|
||||
"max_completion_tokens": llm.max_completion_tokens,
|
||||
"top_p": llm.top_p,
|
||||
"n": llm.n,
|
||||
"seed": llm.seed,
|
||||
"base_url": llm.base_url,
|
||||
"api_version": llm.api_version,
|
||||
}
|
||||
|
||||
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_MODEL}.{idx}", model_name)
|
||||
if hasattr(llm, "temperature"):
|
||||
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_TEMPERATURE}.{idx}", str(llm.temperature))
|
||||
if hasattr(llm, "max_tokens"):
|
||||
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_MAX_TOKENS}.{idx}", str(llm.max_tokens))
|
||||
if hasattr(llm, "top_p"):
|
||||
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_TOP_P}.{idx}", str(llm.top_p))
|
||||
|
||||
for key, value in llm_data.items():
|
||||
if value is not None:
|
||||
self._set_attribute(f"crewai.llms.{idx}.{key}", str(value))
|
||||
|
||||
def _extract_agent_data(self, agent):
|
||||
"""Extract data from an agent."""
|
||||
model = getattr(agent.llm, "model", None) or getattr(agent.llm, "model_name", None) or ""
|
||||
|
||||
tools_list = []
|
||||
if hasattr(agent, "tools") and agent.tools:
|
||||
tools_list = _parse_tools(agent.tools)
|
||||
|
||||
return {
|
||||
"id": str(agent.id),
|
||||
"role": agent.role,
|
||||
"goal": agent.goal,
|
||||
"backstory": agent.backstory,
|
||||
"cache": agent.cache,
|
||||
"config": agent.config,
|
||||
"verbose": agent.verbose,
|
||||
"allow_delegation": agent.allow_delegation,
|
||||
"tools": tools_list,
|
||||
"max_iter": agent.max_iter,
|
||||
"llm": str(model),
|
||||
}
|
||||
|
||||
def _set_attribute(self, key, value):
|
||||
"""Set an attribute on the span."""
|
||||
if value is not None and value != "":
|
||||
set_span_attribute(self.span, key, value)
|
||||
@@ -0,0 +1,535 @@
|
||||
import os
|
||||
import time
|
||||
import logging
|
||||
from typing import Dict, Any
|
||||
from contextlib import contextmanager
|
||||
|
||||
from opentelemetry.trace import SpanKind, get_current_span
|
||||
from opentelemetry.metrics import Meter
|
||||
from opentelemetry.instrumentation.utils import unwrap
|
||||
|
||||
from agentops.instrumentation.common import (
|
||||
CommonInstrumentor,
|
||||
InstrumentorConfig,
|
||||
StandardMetrics,
|
||||
create_wrapper_factory,
|
||||
create_span,
|
||||
SpanAttributeManager,
|
||||
safe_set_attribute,
|
||||
set_token_usage_attributes,
|
||||
TokenUsageExtractor,
|
||||
)
|
||||
from agentops.instrumentation.agentic.crewai.version import __version__
|
||||
from agentops.semconv import SpanAttributes, AgentOpsSpanKindValues, ToolAttributes, MessageAttributes
|
||||
from agentops.semconv.core import CoreAttributes
|
||||
from agentops.instrumentation.agentic.crewai.crewai_span_attributes import CrewAISpanAttributes, set_span_attribute
|
||||
from agentops import get_client
|
||||
|
||||
# Initialize logger
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_instruments = ("crewai >= 0.70.0",)
|
||||
|
||||
# Global context to store tool executions by parent span ID
|
||||
_tool_executions_by_agent = {}
|
||||
|
||||
|
||||
@contextmanager
|
||||
def store_tool_execution():
|
||||
"""Context manager to store tool execution details for later attachment to agent spans."""
|
||||
parent_span = get_current_span()
|
||||
parent_span_id = getattr(parent_span.get_span_context(), "span_id", None)
|
||||
|
||||
if parent_span_id:
|
||||
if parent_span_id not in _tool_executions_by_agent:
|
||||
_tool_executions_by_agent[parent_span_id] = []
|
||||
|
||||
tool_details = {}
|
||||
|
||||
try:
|
||||
yield tool_details
|
||||
|
||||
if tool_details:
|
||||
_tool_executions_by_agent[parent_span_id].append(tool_details)
|
||||
finally:
|
||||
pass
|
||||
|
||||
|
||||
def attach_tool_executions_to_agent_span(span):
|
||||
"""Attach stored tool executions to the agent span."""
|
||||
span_id = getattr(span.get_span_context(), "span_id", None)
|
||||
|
||||
if span_id and span_id in _tool_executions_by_agent:
|
||||
for idx, tool_execution in enumerate(_tool_executions_by_agent[span_id]):
|
||||
for key, value in tool_execution.items():
|
||||
if value is not None:
|
||||
span.set_attribute(f"crewai.agent.tool_execution.{idx}.{key}", str(value))
|
||||
|
||||
del _tool_executions_by_agent[span_id]
|
||||
|
||||
|
||||
class CrewaiInstrumentor(CommonInstrumentor):
|
||||
"""Instrumentor for CrewAI framework."""
|
||||
|
||||
def __init__(self):
|
||||
config = InstrumentorConfig(
|
||||
library_name="crewai",
|
||||
library_version=__version__,
|
||||
wrapped_methods=[], # We'll use custom wrapping for CrewAI
|
||||
metrics_enabled=is_metrics_enabled(),
|
||||
dependencies=_instruments,
|
||||
)
|
||||
super().__init__(config)
|
||||
self._attribute_manager = None
|
||||
|
||||
def _initialize(self, **kwargs):
|
||||
"""Initialize attribute manager."""
|
||||
application_name = kwargs.get("application_name", "default_application")
|
||||
environment = kwargs.get("environment", "default_environment")
|
||||
self._attribute_manager = SpanAttributeManager(
|
||||
service_name=application_name, deployment_environment=environment
|
||||
)
|
||||
|
||||
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
|
||||
"""Create metrics for CrewAI instrumentation."""
|
||||
return StandardMetrics.create_standard_metrics(meter)
|
||||
|
||||
def _custom_wrap(self, **kwargs):
|
||||
"""Perform custom wrapping for CrewAI methods."""
|
||||
from wrapt import wrap_function_wrapper
|
||||
|
||||
# Get attribute manager for all wrappers
|
||||
attr_manager = self._attribute_manager
|
||||
|
||||
# Define wrappers using the new create_wrapper_factory
|
||||
wrap_function_wrapper(
|
||||
"crewai.crew",
|
||||
"Crew.kickoff",
|
||||
create_wrapper_factory(wrap_kickoff_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
wrap_function_wrapper(
|
||||
"crewai.agent",
|
||||
"Agent.execute_task",
|
||||
create_wrapper_factory(wrap_agent_execute_task_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
wrap_function_wrapper(
|
||||
"crewai.task",
|
||||
"Task.execute_sync",
|
||||
create_wrapper_factory(wrap_task_execute_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
wrap_function_wrapper(
|
||||
"crewai.llm",
|
||||
"LLM.call",
|
||||
create_wrapper_factory(wrap_llm_call_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
wrap_function_wrapper(
|
||||
"crewai.utilities.tool_utils",
|
||||
"execute_tool_and_check_finality",
|
||||
create_wrapper_factory(wrap_tool_execution_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
wrap_function_wrapper(
|
||||
"crewai.tools.tool_usage",
|
||||
"ToolUsage.use",
|
||||
create_wrapper_factory(wrap_tool_usage_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
def _custom_unwrap(self, **kwargs):
|
||||
"""Perform custom unwrapping for CrewAI methods."""
|
||||
unwrap("crewai.crew", "Crew.kickoff")
|
||||
unwrap("crewai.agent", "Agent.execute_task")
|
||||
unwrap("crewai.task", "Task.execute_sync")
|
||||
unwrap("crewai.llm", "LLM.call")
|
||||
unwrap("crewai.utilities.tool_utils", "execute_tool_and_check_finality")
|
||||
unwrap("crewai.tools.tool_usage", "ToolUsage.use")
|
||||
|
||||
|
||||
# Implementation functions for wrappers
|
||||
def wrap_kickoff_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
"""Implementation of kickoff wrapper."""
|
||||
logger.debug(
|
||||
f"CrewAI: Starting workflow instrumentation for Crew with {len(getattr(instance, 'agents', []))} agents"
|
||||
)
|
||||
|
||||
config = get_client().config
|
||||
attributes = {
|
||||
SpanAttributes.LLM_SYSTEM: "crewai",
|
||||
}
|
||||
|
||||
if config.default_tags and len(config.default_tags) > 0:
|
||||
tag_list = list(config.default_tags)
|
||||
attributes[CoreAttributes.TAGS] = tag_list
|
||||
|
||||
# Use trace_name from config if available, otherwise default to "crewai.workflow"
|
||||
span_name = config.trace_name if config.trace_name else "crewai.workflow"
|
||||
|
||||
with create_span(
|
||||
tracer, span_name, kind=SpanKind.INTERNAL, attributes=attributes, attribute_manager=attr_manager
|
||||
) as span:
|
||||
logger.debug("CrewAI: Processing crew instance attributes")
|
||||
|
||||
# First set general crew attributes but skip agent processing
|
||||
crew_attrs = CrewAISpanAttributes(span=span, instance=instance, skip_agent_processing=True)
|
||||
|
||||
# Prioritize agent processing before task execution
|
||||
if hasattr(instance, "agents") and instance.agents:
|
||||
logger.debug(f"CrewAI: Explicitly processing {len(instance.agents)} agents before task execution")
|
||||
crew_attrs._parse_agents(instance.agents)
|
||||
|
||||
logger.debug("CrewAI: Executing wrapped crew kickoff function")
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
if result:
|
||||
class_name = instance.__class__.__name__
|
||||
span.set_attribute(f"crewai.{class_name.lower()}.result", str(result))
|
||||
|
||||
if class_name == "Crew":
|
||||
_process_crew_result(span, instance, result)
|
||||
|
||||
# Set token usage using common utilities
|
||||
set_token_usage_attributes(span, result)
|
||||
_calculate_efficiency_metrics(span, result)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _process_crew_result(span, instance, result):
|
||||
"""Process crew execution result."""
|
||||
if hasattr(result, "usage_metrics"):
|
||||
span.set_attribute("crewai.crew.usage_metrics", str(getattr(result, "usage_metrics")))
|
||||
|
||||
if hasattr(result, "tasks_output") and result.tasks_output:
|
||||
span.set_attribute("crewai.crew.tasks_output", str(result.tasks_output))
|
||||
|
||||
try:
|
||||
task_details_by_description = _build_task_details_map(instance)
|
||||
_process_task_outputs(span, result.tasks_output, task_details_by_description)
|
||||
except Exception as ex:
|
||||
logger.warning(f"Failed to parse task outputs: {ex}")
|
||||
|
||||
|
||||
def _build_task_details_map(instance):
|
||||
"""Build a map of task descriptions to task details."""
|
||||
task_details_by_description = {}
|
||||
if hasattr(instance, "tasks"):
|
||||
for task in instance.tasks:
|
||||
if task is not None:
|
||||
agent_id = ""
|
||||
agent_role = ""
|
||||
if hasattr(task, "agent") and task.agent:
|
||||
agent_id = str(getattr(task.agent, "id", ""))
|
||||
agent_role = getattr(task.agent, "role", "")
|
||||
|
||||
tools = []
|
||||
if hasattr(task, "tools") and task.tools:
|
||||
for tool in task.tools:
|
||||
tool_info = {}
|
||||
if hasattr(tool, "name"):
|
||||
tool_info["name"] = tool.name
|
||||
if hasattr(tool, "description"):
|
||||
tool_info["description"] = tool.description
|
||||
if tool_info:
|
||||
tools.append(tool_info)
|
||||
|
||||
task_details_by_description[task.description] = {
|
||||
"agent_id": agent_id,
|
||||
"agent_role": agent_role,
|
||||
"async_execution": getattr(task, "async_execution", False),
|
||||
"human_input": getattr(task, "human_input", False),
|
||||
"output_file": getattr(task, "output_file", ""),
|
||||
"tools": tools,
|
||||
}
|
||||
return task_details_by_description
|
||||
|
||||
|
||||
def _process_task_outputs(span, tasks_output, task_details_by_description):
|
||||
"""Process task outputs and set attributes."""
|
||||
for idx, task_output in enumerate(tasks_output):
|
||||
task_prefix = f"crewai.crew.tasks.{idx}"
|
||||
|
||||
task_attrs = {
|
||||
"description": getattr(task_output, "description", ""),
|
||||
"name": getattr(task_output, "name", ""),
|
||||
"expected_output": getattr(task_output, "expected_output", ""),
|
||||
"summary": getattr(task_output, "summary", ""),
|
||||
"raw": getattr(task_output, "raw", ""),
|
||||
"agent": getattr(task_output, "agent", ""),
|
||||
"output_format": str(getattr(task_output, "output_format", "")),
|
||||
}
|
||||
|
||||
for attr_name, attr_value in task_attrs.items():
|
||||
if attr_value:
|
||||
safe_set_attribute(span, f"{task_prefix}.{attr_name}", attr_value, max_length=1000)
|
||||
|
||||
span.set_attribute(f"{task_prefix}.status", "completed")
|
||||
span.set_attribute(f"{task_prefix}.id", str(idx))
|
||||
|
||||
description = task_attrs.get("description", "")
|
||||
if description and description in task_details_by_description:
|
||||
details = task_details_by_description[description]
|
||||
|
||||
span.set_attribute(f"{task_prefix}.agent_id", details["agent_id"])
|
||||
span.set_attribute(f"{task_prefix}.async_execution", str(details["async_execution"]))
|
||||
span.set_attribute(f"{task_prefix}.human_input", str(details["human_input"]))
|
||||
|
||||
if details["output_file"]:
|
||||
span.set_attribute(f"{task_prefix}.output_file", details["output_file"])
|
||||
|
||||
for tool_idx, tool in enumerate(details["tools"]):
|
||||
for tool_key, tool_value in tool.items():
|
||||
span.set_attribute(f"{task_prefix}.tools.{tool_idx}.{tool_key}", str(tool_value))
|
||||
|
||||
|
||||
def _calculate_efficiency_metrics(span, result):
|
||||
"""Calculate and set efficiency metrics."""
|
||||
if hasattr(result, "token_usage"):
|
||||
try:
|
||||
usage = TokenUsageExtractor.extract_from_response(result)
|
||||
|
||||
# Calculate efficiency
|
||||
if usage.prompt_tokens and usage.completion_tokens and usage.prompt_tokens > 0:
|
||||
efficiency = usage.completion_tokens / usage.prompt_tokens
|
||||
span.set_attribute("crewai.crew.token_efficiency", f"{efficiency:.4f}")
|
||||
|
||||
# Calculate cache efficiency
|
||||
if usage.cached_prompt_tokens and usage.prompt_tokens and usage.prompt_tokens > 0:
|
||||
cache_ratio = usage.cached_prompt_tokens / usage.prompt_tokens
|
||||
span.set_attribute("crewai.crew.cache_efficiency", f"{cache_ratio:.4f}")
|
||||
|
||||
except Exception as ex:
|
||||
logger.warning(f"Failed to calculate efficiency metrics: {ex}")
|
||||
|
||||
|
||||
def wrap_agent_execute_task_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
"""Implementation of agent execute task wrapper."""
|
||||
agent_name = instance.role if hasattr(instance, "role") else "agent"
|
||||
|
||||
with create_span(
|
||||
tracer,
|
||||
f"{agent_name}.agent",
|
||||
kind=SpanKind.CLIENT,
|
||||
attributes={
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: AgentOpsSpanKindValues.AGENT.value,
|
||||
},
|
||||
attribute_manager=attr_manager,
|
||||
) as span:
|
||||
CrewAISpanAttributes(span=span, instance=instance)
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
attach_tool_executions_to_agent_span(span)
|
||||
|
||||
# Record token metrics if available
|
||||
if metrics.get("token_histogram") and hasattr(instance, "_token_process"):
|
||||
token_process = instance._token_process.get_summary()
|
||||
if hasattr(token_process, "prompt_tokens"):
|
||||
metrics["token_histogram"].record(
|
||||
token_process.prompt_tokens,
|
||||
attributes={
|
||||
SpanAttributes.LLM_SYSTEM: "crewai",
|
||||
SpanAttributes.LLM_TOKEN_TYPE: "input",
|
||||
SpanAttributes.LLM_RESPONSE_MODEL: str(instance.llm.model),
|
||||
},
|
||||
)
|
||||
if hasattr(token_process, "completion_tokens"):
|
||||
metrics["token_histogram"].record(
|
||||
token_process.completion_tokens,
|
||||
attributes={
|
||||
SpanAttributes.LLM_SYSTEM: "crewai",
|
||||
SpanAttributes.LLM_TOKEN_TYPE: "output",
|
||||
SpanAttributes.LLM_RESPONSE_MODEL: str(instance.llm.model),
|
||||
},
|
||||
)
|
||||
|
||||
if hasattr(instance, "llm") and hasattr(instance.llm, "model"):
|
||||
set_span_attribute(span, SpanAttributes.LLM_REQUEST_MODEL, str(instance.llm.model))
|
||||
set_span_attribute(span, SpanAttributes.LLM_RESPONSE_MODEL, str(instance.llm.model))
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def wrap_task_execute_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
"""Implementation of task execute wrapper."""
|
||||
task_name = instance.description if hasattr(instance, "description") else "task"
|
||||
|
||||
config = get_client().config
|
||||
attributes = {
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: AgentOpsSpanKindValues.TASK.value,
|
||||
}
|
||||
|
||||
if config.default_tags and len(config.default_tags) > 0:
|
||||
tag_list = list(config.default_tags)
|
||||
attributes[CoreAttributes.TAGS] = tag_list
|
||||
|
||||
with create_span(
|
||||
tracer, f"{task_name}.task", kind=SpanKind.CLIENT, attributes=attributes, attribute_manager=attr_manager
|
||||
) as span:
|
||||
CrewAISpanAttributes(span=span, instance=instance)
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
set_span_attribute(span, SpanAttributes.AGENTOPS_ENTITY_OUTPUT, str(result))
|
||||
return result
|
||||
|
||||
|
||||
def wrap_llm_call_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
"""Implementation of LLM call wrapper."""
|
||||
llm = instance.model if hasattr(instance, "model") else "llm"
|
||||
start_time = time.time()
|
||||
|
||||
with create_span(tracer, f"{llm}.llm", kind=SpanKind.CLIENT, attribute_manager=attr_manager) as span:
|
||||
CrewAISpanAttributes(span=span, instance=instance)
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Set prompt attributes from args
|
||||
if args and isinstance(args[0], list):
|
||||
for i, message in enumerate(args[0]):
|
||||
if isinstance(message, dict):
|
||||
if "role" in message:
|
||||
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), message["role"])
|
||||
if "content" in message:
|
||||
span.set_attribute(MessageAttributes.PROMPT_CONTENT.format(i=i), message["content"])
|
||||
|
||||
# Set completion attributes from result
|
||||
if result:
|
||||
span.set_attribute(MessageAttributes.COMPLETION_CONTENT.format(i=0), str(result))
|
||||
span.set_attribute(MessageAttributes.COMPLETION_ROLE.format(i=0), "assistant")
|
||||
|
||||
# Set token usage attributes from callbacks
|
||||
if "callbacks" in kwargs and kwargs["callbacks"] and hasattr(kwargs["callbacks"][0], "token_cost_process"):
|
||||
token_process = kwargs["callbacks"][0].token_cost_process
|
||||
if hasattr(token_process, "completion_tokens"):
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, token_process.completion_tokens)
|
||||
if hasattr(token_process, "prompt_tokens"):
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_PROMPT_TOKENS, token_process.prompt_tokens)
|
||||
if hasattr(token_process, "total_tokens"):
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, token_process.total_tokens)
|
||||
|
||||
# Record duration metric
|
||||
if metrics.get("duration_histogram"):
|
||||
duration = time.time() - start_time
|
||||
metrics["duration_histogram"].record(
|
||||
duration,
|
||||
attributes={
|
||||
SpanAttributes.LLM_SYSTEM: "crewai",
|
||||
SpanAttributes.LLM_RESPONSE_MODEL: str(instance.model),
|
||||
},
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def wrap_tool_execution_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
"""Implementation of tool execution wrapper."""
|
||||
agent_action = args[0] if args else None
|
||||
tools = args[1] if len(args) > 1 else []
|
||||
|
||||
if not agent_action:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
tool_name = getattr(agent_action, "tool", "unknown_tool")
|
||||
tool_input = getattr(agent_action, "tool_input", "")
|
||||
|
||||
with store_tool_execution() as tool_details:
|
||||
tool_details["name"] = tool_name
|
||||
tool_details["parameters"] = str(tool_input)
|
||||
|
||||
matching_tool = next((tool for tool in tools if hasattr(tool, "name") and tool.name == tool_name), None)
|
||||
if matching_tool and hasattr(matching_tool, "description"):
|
||||
tool_details["description"] = str(matching_tool.description)
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
with create_span(
|
||||
tracer,
|
||||
f"{tool_name}.tool",
|
||||
kind=SpanKind.CLIENT,
|
||||
attributes={
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "tool",
|
||||
ToolAttributes.TOOL_NAME: tool_name,
|
||||
ToolAttributes.TOOL_PARAMETERS: str(tool_input),
|
||||
},
|
||||
attribute_manager=attr_manager,
|
||||
) as span:
|
||||
if matching_tool and hasattr(matching_tool, "description"):
|
||||
span.set_attribute(ToolAttributes.TOOL_DESCRIPTION, str(matching_tool.description))
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Record duration metric
|
||||
if metrics.get("duration_histogram"):
|
||||
duration = time.time() - start_time
|
||||
metrics["duration_histogram"].record(
|
||||
duration,
|
||||
attributes={
|
||||
SpanAttributes.LLM_SYSTEM: "crewai",
|
||||
ToolAttributes.TOOL_NAME: tool_name,
|
||||
},
|
||||
)
|
||||
|
||||
if hasattr(result, "result"):
|
||||
tool_result = str(result.result)
|
||||
span.set_attribute(ToolAttributes.TOOL_RESULT, tool_result)
|
||||
tool_details["result"] = tool_result
|
||||
|
||||
tool_status = "success" if not hasattr(result, "error") or not result.error else "error"
|
||||
span.set_attribute(ToolAttributes.TOOL_STATUS, tool_status)
|
||||
tool_details["status"] = tool_status
|
||||
|
||||
if hasattr(result, "error") and result.error:
|
||||
tool_details["error"] = str(result.error)
|
||||
|
||||
duration = time.time() - start_time
|
||||
tool_details["duration"] = f"{duration:.3f}"
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def wrap_tool_usage_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
"""Implementation of tool usage wrapper."""
|
||||
calling = args[0] if args else None
|
||||
|
||||
if not calling:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
tool_name = getattr(calling, "tool_name", "unknown_tool")
|
||||
|
||||
with store_tool_execution() as tool_details:
|
||||
tool_details["name"] = tool_name
|
||||
|
||||
if hasattr(calling, "arguments") and calling.arguments:
|
||||
tool_details["parameters"] = str(calling.arguments)
|
||||
|
||||
with create_span(
|
||||
tracer,
|
||||
f"{tool_name}.tool_usage",
|
||||
kind=SpanKind.INTERNAL,
|
||||
attributes={
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "tool.usage",
|
||||
ToolAttributes.TOOL_NAME: tool_name,
|
||||
},
|
||||
attribute_manager=attr_manager,
|
||||
) as span:
|
||||
if hasattr(calling, "arguments") and calling.arguments:
|
||||
span.set_attribute(ToolAttributes.TOOL_PARAMETERS, str(calling.arguments))
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
tool_result = str(result)
|
||||
span.set_attribute(ToolAttributes.TOOL_RESULT, tool_result)
|
||||
tool_details["result"] = tool_result
|
||||
|
||||
tool_status = "success"
|
||||
span.set_attribute(ToolAttributes.TOOL_STATUS, tool_status)
|
||||
tool_details["status"] = tool_status
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def is_metrics_enabled() -> bool:
|
||||
return (os.getenv("AGENTOPS_METRICS_ENABLED") or "true").lower() == "true"
|
||||
@@ -0,0 +1 @@
|
||||
__version__ = "0.36.0"
|
||||
@@ -0,0 +1,19 @@
|
||||
"""Google ADK Instrumentation for AgentOps
|
||||
|
||||
This module provides instrumentation for Google's Agent Development Kit (ADK),
|
||||
capturing agent execution, LLM calls, tool calls, and other ADK-specific events.
|
||||
"""
|
||||
|
||||
from agentops.instrumentation.common import LibraryInfo
|
||||
|
||||
# Library information
|
||||
_library_info = LibraryInfo(
|
||||
name="agentops.instrumentation.google_adk", package_name="google-adk", default_version="0.0.0"
|
||||
)
|
||||
LIBRARY_NAME = _library_info.name
|
||||
LIBRARY_VERSION = _library_info.version
|
||||
|
||||
from agentops.instrumentation.agentic.google_adk.instrumentor import GooogleAdkInstrumentor # noqa: E402
|
||||
from agentops.instrumentation.agentic.google_adk import patch # noqa: E402
|
||||
|
||||
__all__ = ["LIBRARY_NAME", "LIBRARY_VERSION", "GooogleAdkInstrumentor", "patch"]
|
||||
@@ -0,0 +1,68 @@
|
||||
"""Google ADK Instrumentation for AgentOps
|
||||
|
||||
This module provides instrumentation for Google's Agent Development Kit (ADK).
|
||||
It uses a patching approach to:
|
||||
1. Disable ADK's built-in telemetry to prevent duplicate spans
|
||||
2. Create AgentOps spans that mirror ADK's telemetry structure
|
||||
3. Extract and properly index LLM messages and tool calls
|
||||
"""
|
||||
|
||||
from typing import Dict, Any
|
||||
|
||||
from agentops.logging import logger
|
||||
from opentelemetry.metrics import Meter
|
||||
from agentops.instrumentation.common import CommonInstrumentor, StandardMetrics, InstrumentorConfig
|
||||
from agentops.instrumentation.agentic.google_adk.patch import patch_adk, unpatch_adk
|
||||
|
||||
# Library info for tracer/meter
|
||||
LIBRARY_NAME = "agentops.instrumentation.google_adk"
|
||||
LIBRARY_VERSION = "0.1.0"
|
||||
|
||||
|
||||
class GooogleAdkInstrumentor(CommonInstrumentor):
|
||||
"""An instrumentor for Google Agent Development Kit (ADK).
|
||||
|
||||
This instrumentor patches Google ADK to:
|
||||
- Prevent ADK from creating its own telemetry spans
|
||||
- Create AgentOps spans for agent runs, LLM calls, and tool calls
|
||||
- Properly extract and index message content and tool interactions
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the Google ADK instrumentor."""
|
||||
# Create instrumentor config
|
||||
config = InstrumentorConfig(
|
||||
library_name=LIBRARY_NAME,
|
||||
library_version=LIBRARY_VERSION,
|
||||
wrapped_methods=[], # We use patching instead of wrapping
|
||||
metrics_enabled=True,
|
||||
dependencies=["google-adk >= 0.1.0"],
|
||||
)
|
||||
|
||||
super().__init__(config)
|
||||
|
||||
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
|
||||
"""Create metrics for the instrumentor.
|
||||
|
||||
Returns a dictionary of metric name to metric instance.
|
||||
"""
|
||||
# Create standard metrics for LLM operations
|
||||
return StandardMetrics.create_standard_metrics(meter)
|
||||
|
||||
def _custom_wrap(self, **kwargs):
|
||||
"""Apply custom patching for Google ADK.
|
||||
|
||||
This is called after normal wrapping, but we use it for patching
|
||||
since we don't have normal wrapped methods.
|
||||
"""
|
||||
# Apply patches with our tracer
|
||||
patch_adk(self._tracer)
|
||||
logger.info("Google ADK instrumentation enabled")
|
||||
|
||||
def _custom_unwrap(self, **kwargs):
|
||||
"""Remove custom patching from Google ADK.
|
||||
|
||||
This method removes all patches and restores ADK's original behavior.
|
||||
"""
|
||||
unpatch_adk()
|
||||
logger.info("Google ADK instrumentation disabled")
|
||||
@@ -0,0 +1,767 @@
|
||||
"""Patch functions for Google ADK instrumentation.
|
||||
|
||||
This module patches key methods in Google ADK to:
|
||||
1. Prevent ADK from creating its own spans
|
||||
2. Create AgentOps spans that mirror ADK's telemetry
|
||||
3. Extract and set proper attributes on spans
|
||||
"""
|
||||
|
||||
import json
|
||||
import wrapt
|
||||
from typing import Any
|
||||
from opentelemetry import trace as opentelemetry_api_trace
|
||||
from opentelemetry.trace import SpanKind as SpanKind
|
||||
|
||||
from agentops.logging import logger
|
||||
from agentops.semconv import SpanAttributes, ToolAttributes, MessageAttributes, AgentAttributes
|
||||
|
||||
|
||||
_wrapped_methods = []
|
||||
|
||||
|
||||
class NoOpSpan:
|
||||
"""A no-op span that does nothing."""
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *args):
|
||||
pass
|
||||
|
||||
def set_attribute(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def set_attributes(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def add_event(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def set_status(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def update_name(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def is_recording(self):
|
||||
return False
|
||||
|
||||
def end(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
def record_exception(self, *args, **kwargs):
|
||||
pass
|
||||
|
||||
|
||||
class NoOpTracer:
|
||||
"""A tracer that creates no-op spans to prevent ADK from creating real spans."""
|
||||
|
||||
def start_as_current_span(self, *args, **kwargs):
|
||||
"""Return a no-op context manager."""
|
||||
return NoOpSpan()
|
||||
|
||||
def start_span(self, *args, **kwargs):
|
||||
"""Return a no-op span."""
|
||||
return NoOpSpan()
|
||||
|
||||
def use_span(self, *args, **kwargs):
|
||||
"""Return a no-op context manager."""
|
||||
return NoOpSpan()
|
||||
|
||||
|
||||
def _build_llm_request_for_trace(llm_request) -> dict:
|
||||
"""Build a dictionary representation of the LLM request for tracing."""
|
||||
from google.genai import types
|
||||
|
||||
result = {
|
||||
"model": llm_request.model,
|
||||
"config": llm_request.config.model_dump(exclude_none=True, exclude="response_schema"),
|
||||
"contents": [],
|
||||
}
|
||||
|
||||
for content in llm_request.contents:
|
||||
parts = [part for part in content.parts if not hasattr(part, "inline_data") or not part.inline_data]
|
||||
result["contents"].append(types.Content(role=content.role, parts=parts).model_dump(exclude_none=True))
|
||||
return result
|
||||
|
||||
|
||||
def _extract_messages_from_contents(contents: list) -> dict:
|
||||
"""Extract messages from LLM contents for proper indexing."""
|
||||
attributes = {}
|
||||
|
||||
for i, content in enumerate(contents):
|
||||
# Get role and normalize it
|
||||
raw_role = content.get("role", "user")
|
||||
|
||||
# Hardcode role mapping for consistency
|
||||
if raw_role == "model":
|
||||
role = "assistant"
|
||||
elif raw_role == "user":
|
||||
role = "user"
|
||||
elif raw_role == "system":
|
||||
role = "system"
|
||||
else:
|
||||
role = raw_role # Keep original if not recognized
|
||||
|
||||
parts = content.get("parts", [])
|
||||
|
||||
# Set role
|
||||
attributes[MessageAttributes.PROMPT_ROLE.format(i=i)] = role
|
||||
|
||||
# Extract content from parts
|
||||
text_parts = []
|
||||
for part in parts:
|
||||
if "text" in part and part.get("text") is not None:
|
||||
text_parts.append(str(part["text"]))
|
||||
elif "function_call" in part:
|
||||
# Function calls in prompts are typically from the model's previous responses
|
||||
func_call = part["function_call"]
|
||||
# Store as a generic attribute since MessageAttributes doesn't have prompt tool calls
|
||||
attributes[f"gen_ai.prompt.{i}.function_call.name"] = func_call.get("name", "")
|
||||
attributes[f"gen_ai.prompt.{i}.function_call.args"] = json.dumps(func_call.get("args", {}))
|
||||
if "id" in func_call:
|
||||
attributes[f"gen_ai.prompt.{i}.function_call.id"] = func_call["id"]
|
||||
elif "function_response" in part:
|
||||
# Function responses are typically user messages with tool results
|
||||
func_resp = part["function_response"]
|
||||
attributes[f"gen_ai.prompt.{i}.function_response.name"] = func_resp.get("name", "")
|
||||
attributes[f"gen_ai.prompt.{i}.function_response.result"] = json.dumps(func_resp.get("response", {}))
|
||||
if "id" in func_resp:
|
||||
attributes[f"gen_ai.prompt.{i}.function_response.id"] = func_resp["id"]
|
||||
|
||||
# Combine text parts
|
||||
if text_parts:
|
||||
attributes[MessageAttributes.PROMPT_CONTENT.format(i=i)] = "\n".join(text_parts)
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def _extract_llm_attributes(llm_request_dict: dict, llm_response: Any) -> dict:
|
||||
"""Extract attributes from LLM request and response."""
|
||||
attributes = {}
|
||||
|
||||
# Model
|
||||
if "model" in llm_request_dict:
|
||||
attributes[SpanAttributes.LLM_REQUEST_MODEL] = llm_request_dict["model"]
|
||||
|
||||
# Config
|
||||
if "config" in llm_request_dict:
|
||||
config = llm_request_dict["config"]
|
||||
|
||||
# System instruction - commented out, now handled as a system role message
|
||||
# if "system_instruction" in config:
|
||||
# attributes[SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION] = config["system_instruction"]
|
||||
|
||||
# Temperature
|
||||
if "temperature" in config:
|
||||
attributes[SpanAttributes.LLM_REQUEST_TEMPERATURE] = config["temperature"]
|
||||
|
||||
# Max output tokens
|
||||
if "max_output_tokens" in config:
|
||||
attributes[SpanAttributes.LLM_REQUEST_MAX_TOKENS] = config["max_output_tokens"]
|
||||
|
||||
# Top P
|
||||
if "top_p" in config:
|
||||
attributes[SpanAttributes.LLM_REQUEST_TOP_P] = config["top_p"]
|
||||
|
||||
# Top K
|
||||
if "top_k" in config:
|
||||
attributes[SpanAttributes.LLM_REQUEST_TOP_K] = config["top_k"]
|
||||
|
||||
# Candidate count
|
||||
if "candidate_count" in config:
|
||||
attributes[SpanAttributes.LLM_REQUEST_CANDIDATE_COUNT] = config["candidate_count"]
|
||||
|
||||
# Stop sequences
|
||||
if "stop_sequences" in config:
|
||||
attributes[SpanAttributes.LLM_REQUEST_STOP_SEQUENCES] = json.dumps(config["stop_sequences"])
|
||||
|
||||
# Response MIME type
|
||||
if "response_mime_type" in config:
|
||||
attributes["gen_ai.request.response_mime_type"] = config["response_mime_type"]
|
||||
|
||||
# Tools/Functions
|
||||
if "tools" in config:
|
||||
# Extract tool definitions
|
||||
for i, tool in enumerate(config["tools"]):
|
||||
if "function_declarations" in tool:
|
||||
for j, func in enumerate(tool["function_declarations"]):
|
||||
attributes[f"gen_ai.request.tools.{j}.name"] = func.get("name", "")
|
||||
attributes[f"gen_ai.request.tools.{j}.description"] = func.get("description", "")
|
||||
|
||||
# Messages - handle system instruction and regular contents
|
||||
message_index = 0
|
||||
|
||||
# First, add system instruction as a system role message if present
|
||||
# TODO: This is not Chat Completions format but doing this for frontend rendering consistency
|
||||
if "config" in llm_request_dict and "system_instruction" in llm_request_dict["config"]:
|
||||
system_instruction = llm_request_dict["config"]["system_instruction"]
|
||||
attributes[MessageAttributes.PROMPT_ROLE.format(i=message_index)] = "system"
|
||||
attributes[MessageAttributes.PROMPT_CONTENT.format(i=message_index)] = system_instruction
|
||||
message_index += 1
|
||||
|
||||
# Then add regular contents with proper indexing
|
||||
if "contents" in llm_request_dict:
|
||||
for content in llm_request_dict["contents"]:
|
||||
# Get role and normalize it
|
||||
raw_role = content.get("role", "user")
|
||||
|
||||
# Hardcode role mapping for consistency
|
||||
if raw_role == "model":
|
||||
role = "assistant"
|
||||
elif raw_role == "user":
|
||||
role = "user"
|
||||
elif raw_role == "system":
|
||||
role = "system"
|
||||
else:
|
||||
role = raw_role # Keep original if not recognized
|
||||
|
||||
parts = content.get("parts", [])
|
||||
|
||||
# Set role
|
||||
attributes[MessageAttributes.PROMPT_ROLE.format(i=message_index)] = role
|
||||
|
||||
# Extract content from parts
|
||||
text_parts = []
|
||||
for part in parts:
|
||||
if "text" in part and part.get("text") is not None:
|
||||
text_parts.append(str(part["text"]))
|
||||
elif "function_call" in part:
|
||||
# Function calls in prompts are typically from the model's previous responses
|
||||
func_call = part["function_call"]
|
||||
# Store as a generic attribute since MessageAttributes doesn't have prompt tool calls
|
||||
attributes[f"gen_ai.prompt.{message_index}.function_call.name"] = func_call.get("name", "")
|
||||
attributes[f"gen_ai.prompt.{message_index}.function_call.args"] = json.dumps(
|
||||
func_call.get("args", {})
|
||||
)
|
||||
if "id" in func_call:
|
||||
attributes[f"gen_ai.prompt.{message_index}.function_call.id"] = func_call["id"]
|
||||
elif "function_response" in part:
|
||||
# Function responses are typically user messages with tool results
|
||||
func_resp = part["function_response"]
|
||||
attributes[f"gen_ai.prompt.{message_index}.function_response.name"] = func_resp.get("name", "")
|
||||
attributes[f"gen_ai.prompt.{message_index}.function_response.result"] = json.dumps(
|
||||
func_resp.get("response", {})
|
||||
)
|
||||
if "id" in func_resp:
|
||||
attributes[f"gen_ai.prompt.{message_index}.function_response.id"] = func_resp["id"]
|
||||
|
||||
# Combine text parts
|
||||
if text_parts:
|
||||
attributes[MessageAttributes.PROMPT_CONTENT.format(i=message_index)] = "\n".join(text_parts)
|
||||
|
||||
message_index += 1
|
||||
|
||||
# Response
|
||||
if llm_response:
|
||||
try:
|
||||
response_dict = json.loads(llm_response) if isinstance(llm_response, str) else llm_response
|
||||
|
||||
# Response model
|
||||
if "model" in response_dict:
|
||||
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = response_dict["model"]
|
||||
|
||||
# Usage metadata
|
||||
if "usage_metadata" in response_dict:
|
||||
usage = response_dict["usage_metadata"]
|
||||
if "prompt_token_count" in usage:
|
||||
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = usage["prompt_token_count"]
|
||||
if "candidates_token_count" in usage:
|
||||
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = usage["candidates_token_count"]
|
||||
if "total_token_count" in usage:
|
||||
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = usage["total_token_count"]
|
||||
|
||||
# Additional token details if available
|
||||
if "prompt_tokens_details" in usage:
|
||||
for detail in usage["prompt_tokens_details"]:
|
||||
if "modality" in detail and "token_count" in detail:
|
||||
attributes[f"gen_ai.usage.prompt_tokens.{detail['modality'].lower()}"] = detail[
|
||||
"token_count"
|
||||
]
|
||||
|
||||
if "candidates_tokens_details" in usage:
|
||||
for detail in usage["candidates_tokens_details"]:
|
||||
if "modality" in detail and "token_count" in detail:
|
||||
attributes[f"gen_ai.usage.completion_tokens.{detail['modality'].lower()}"] = detail[
|
||||
"token_count"
|
||||
]
|
||||
|
||||
# Response content
|
||||
if "content" in response_dict and "parts" in response_dict["content"]:
|
||||
parts = response_dict["content"]["parts"]
|
||||
|
||||
# Set completion role and content - hardcode role as 'assistant' for consistency
|
||||
attributes[MessageAttributes.COMPLETION_ROLE.format(i=0)] = "assistant"
|
||||
|
||||
text_parts = []
|
||||
tool_call_index = 0
|
||||
for part in parts:
|
||||
if "text" in part and part.get("text") is not None:
|
||||
text_parts.append(str(part["text"]))
|
||||
elif "function_call" in part:
|
||||
# This is a function call in the response
|
||||
func_call = part["function_call"]
|
||||
attributes[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=tool_call_index)] = (
|
||||
func_call.get("name", "")
|
||||
)
|
||||
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=tool_call_index)] = (
|
||||
json.dumps(func_call.get("args", {}))
|
||||
)
|
||||
if "id" in func_call:
|
||||
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=tool_call_index)] = (
|
||||
func_call["id"]
|
||||
)
|
||||
tool_call_index += 1
|
||||
|
||||
if text_parts:
|
||||
attributes[MessageAttributes.COMPLETION_CONTENT.format(i=0)] = "\n".join(text_parts)
|
||||
|
||||
# Finish reason
|
||||
if "finish_reason" in response_dict:
|
||||
attributes[SpanAttributes.LLM_RESPONSE_FINISH_REASON] = response_dict["finish_reason"]
|
||||
|
||||
# Response ID
|
||||
if "id" in response_dict:
|
||||
attributes[SpanAttributes.LLM_RESPONSE_ID] = response_dict["id"]
|
||||
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to extract response attributes: {e}")
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
# Wrapper for Runner.run_async - REMOVED per user request
|
||||
# We just pass through without creating a span
|
||||
def _runner_run_async_wrapper(agentops_tracer):
|
||||
def actual_decorator(wrapped, instance, args, kwargs):
|
||||
async def new_function():
|
||||
# Just pass through without creating a span
|
||||
async_gen = wrapped(*args, **kwargs)
|
||||
async for item in async_gen:
|
||||
yield item
|
||||
|
||||
return new_function()
|
||||
|
||||
return actual_decorator
|
||||
|
||||
|
||||
def extract_agent_attributes(instance):
|
||||
attributes = {}
|
||||
# Use AgentAttributes from semconv
|
||||
attributes[AgentAttributes.AGENT_NAME] = instance.name
|
||||
if hasattr(instance, "description"):
|
||||
attributes["agent.description"] = instance.description
|
||||
if hasattr(instance, "model"):
|
||||
attributes["agent.model"] = instance.model
|
||||
if hasattr(instance, "instruction"):
|
||||
attributes["agent.instruction"] = instance.instruction
|
||||
if hasattr(instance, "tools"):
|
||||
for tool in instance.tools:
|
||||
if hasattr(tool, "name"):
|
||||
attributes[ToolAttributes.TOOL_NAME] = tool.name
|
||||
if hasattr(tool, "description"):
|
||||
attributes[ToolAttributes.TOOL_DESCRIPTION] = tool.description
|
||||
if hasattr(instance, "output_key"):
|
||||
attributes["agent.output_key"] = instance.output_key
|
||||
# Subagents
|
||||
if hasattr(instance, "sub_agents"):
|
||||
# recursively extract attributes from subagents but add a prefix to the keys, also with indexing, because we can have multiple subagents, also subagent can have subagents, So have to index them even if they are not in the same level
|
||||
for i, sub_agent in enumerate(instance.sub_agents):
|
||||
sub_agent_attributes = extract_agent_attributes(sub_agent)
|
||||
for key, value in sub_agent_attributes.items():
|
||||
attributes[f"agent.sub_agents.{i}.{key}"] = value
|
||||
return attributes
|
||||
|
||||
|
||||
# Wrapper for BaseAgent.run_async
|
||||
def _base_agent_run_async_wrapper(agentops_tracer):
|
||||
def actual_decorator(wrapped, instance, args, kwargs):
|
||||
async def new_function():
|
||||
agent_name = instance.name if hasattr(instance, "name") else "unknown"
|
||||
span_name = f"adk.agent.{agent_name}"
|
||||
|
||||
with agentops_tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "agent")
|
||||
span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
|
||||
|
||||
span.set_attributes(extract_agent_attributes(instance))
|
||||
# # Extract invocation context if available
|
||||
if len(args) > 0 and hasattr(args[0], "invocation_id"):
|
||||
span.set_attribute("adk.invocation_id", args[0].invocation_id)
|
||||
|
||||
async_gen = wrapped(*args, **kwargs)
|
||||
async for item in async_gen:
|
||||
yield item
|
||||
|
||||
return new_function()
|
||||
|
||||
return actual_decorator
|
||||
|
||||
|
||||
# Wrapper for BaseLlmFlow._call_llm_async
|
||||
def _base_llm_flow_call_llm_async_wrapper(agentops_tracer):
|
||||
def actual_decorator(wrapped, instance, args, kwargs):
|
||||
async def new_function():
|
||||
# Extract model info and llm_request if available
|
||||
model_name = "unknown"
|
||||
llm_request = None
|
||||
|
||||
if len(args) > 1:
|
||||
llm_request = args[1]
|
||||
if hasattr(llm_request, "model"):
|
||||
model_name = llm_request.model
|
||||
|
||||
span_name = f"adk.llm.{model_name}"
|
||||
|
||||
with agentops_tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "request")
|
||||
span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "request")
|
||||
|
||||
# Extract and set attributes from llm_request before the call
|
||||
if llm_request:
|
||||
llm_request_dict = _build_llm_request_for_trace(llm_request)
|
||||
# Only extract request attributes here, response will be set later by _finalize_model_response_event
|
||||
llm_attrs = _extract_llm_attributes(llm_request_dict, None)
|
||||
for key, value in llm_attrs.items():
|
||||
span.set_attribute(key, value)
|
||||
|
||||
# Note: The actual LLM response attributes will be set by
|
||||
# _finalize_model_response_event_wrapper when ADK finalizes the response
|
||||
|
||||
async_gen = wrapped(*args, **kwargs)
|
||||
async for item in async_gen:
|
||||
yield item
|
||||
|
||||
return new_function()
|
||||
|
||||
return actual_decorator
|
||||
|
||||
|
||||
# Wrapper for ADK telemetry functions - these add attributes to current span
|
||||
def _adk_trace_tool_call_wrapper(agentops_tracer):
|
||||
@wrapt.decorator
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
# Call original to preserve ADK behavior
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
tool_args = args[0] if args else kwargs.get("args")
|
||||
current_span = opentelemetry_api_trace.get_current_span()
|
||||
if current_span.is_recording() and tool_args is not None:
|
||||
current_span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
|
||||
current_span.set_attribute("gcp.vertex.agent.tool_call_args", json.dumps(tool_args))
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
def _adk_trace_tool_response_wrapper(agentops_tracer):
|
||||
@wrapt.decorator
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
# Call original to preserve ADK behavior
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
invocation_context = args[0] if len(args) > 0 else kwargs.get("invocation_context")
|
||||
event_id = args[1] if len(args) > 1 else kwargs.get("event_id")
|
||||
function_response_event = args[2] if len(args) > 2 else kwargs.get("function_response_event")
|
||||
|
||||
current_span = opentelemetry_api_trace.get_current_span()
|
||||
if current_span.is_recording():
|
||||
current_span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
|
||||
if invocation_context:
|
||||
current_span.set_attribute("gcp.vertex.agent.invocation_id", invocation_context.invocation_id)
|
||||
if event_id:
|
||||
current_span.set_attribute("gcp.vertex.agent.event_id", event_id)
|
||||
if function_response_event:
|
||||
current_span.set_attribute(
|
||||
"gcp.vertex.agent.tool_response", function_response_event.model_dump_json(exclude_none=True)
|
||||
)
|
||||
current_span.set_attribute("gcp.vertex.agent.llm_request", "{}")
|
||||
current_span.set_attribute("gcp.vertex.agent.llm_response", "{}")
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
def _adk_trace_call_llm_wrapper(agentops_tracer):
|
||||
@wrapt.decorator
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
# Call the original first to ensure ADK's behavior is preserved
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
invocation_context = args[0] if len(args) > 0 else kwargs.get("invocation_context")
|
||||
event_id = args[1] if len(args) > 1 else kwargs.get("event_id")
|
||||
llm_request = args[2] if len(args) > 2 else kwargs.get("llm_request")
|
||||
llm_response = args[3] if len(args) > 3 else kwargs.get("llm_response")
|
||||
|
||||
current_span = opentelemetry_api_trace.get_current_span()
|
||||
if current_span.is_recording():
|
||||
current_span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
|
||||
if llm_request:
|
||||
current_span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, llm_request.model)
|
||||
if invocation_context:
|
||||
current_span.set_attribute("gcp.vertex.agent.invocation_id", invocation_context.invocation_id)
|
||||
current_span.set_attribute("gcp.vertex.agent.session_id", invocation_context.session.id)
|
||||
if event_id:
|
||||
current_span.set_attribute("gcp.vertex.agent.event_id", event_id)
|
||||
|
||||
if llm_request:
|
||||
llm_request_dict = _build_llm_request_for_trace(llm_request)
|
||||
current_span.set_attribute("gcp.vertex.agent.llm_request", json.dumps(llm_request_dict))
|
||||
|
||||
# Extract and set all attributes including usage
|
||||
llm_response_json = None
|
||||
if llm_response:
|
||||
llm_response_json = llm_response.model_dump_json(exclude_none=True)
|
||||
current_span.set_attribute("gcp.vertex.agent.llm_response", llm_response_json)
|
||||
|
||||
llm_attrs = _extract_llm_attributes(llm_request_dict, llm_response_json)
|
||||
for key, value in llm_attrs.items():
|
||||
current_span.set_attribute(key, value)
|
||||
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
def _adk_trace_send_data_wrapper(agentops_tracer):
|
||||
@wrapt.decorator
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
# Call original to preserve ADK behavior
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
invocation_context = args[0] if len(args) > 0 else kwargs.get("invocation_context")
|
||||
event_id = args[1] if len(args) > 1 else kwargs.get("event_id")
|
||||
data = args[2] if len(args) > 2 else kwargs.get("data")
|
||||
|
||||
current_span = opentelemetry_api_trace.get_current_span()
|
||||
if current_span.is_recording():
|
||||
if invocation_context:
|
||||
current_span.set_attribute("gcp.vertex.agent.invocation_id", invocation_context.invocation_id)
|
||||
if event_id:
|
||||
current_span.set_attribute("gcp.vertex.agent.event_id", event_id)
|
||||
if data:
|
||||
from google.genai import types
|
||||
|
||||
current_span.set_attribute(
|
||||
"gcp.vertex.agent.data",
|
||||
json.dumps(
|
||||
[
|
||||
types.Content(role=content.role, parts=content.parts).model_dump(exclude_none=True)
|
||||
for content in data
|
||||
]
|
||||
),
|
||||
)
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
# Wrapper for _finalize_model_response_event to capture response attributes
|
||||
def _finalize_model_response_event_wrapper(agentops_tracer):
|
||||
def actual_decorator(wrapped, instance, args, kwargs):
|
||||
# Call the original method
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Extract llm_request and llm_response from args
|
||||
llm_request = args[0] if len(args) > 0 else kwargs.get("llm_request")
|
||||
llm_response = args[1] if len(args) > 1 else kwargs.get("llm_response")
|
||||
|
||||
# Get the current span and set response attributes
|
||||
current_span = opentelemetry_api_trace.get_current_span()
|
||||
if current_span.is_recording() and llm_request and llm_response:
|
||||
span_name = getattr(current_span, "name", "")
|
||||
if "adk.llm" in span_name:
|
||||
# Build request dict
|
||||
llm_request_dict = _build_llm_request_for_trace(llm_request)
|
||||
|
||||
# Extract response attributes
|
||||
llm_response_json = llm_response.model_dump_json(exclude_none=True)
|
||||
llm_attrs = _extract_llm_attributes(llm_request_dict, llm_response_json)
|
||||
|
||||
# Only set response-related attributes (request attrs already set)
|
||||
for key, value in llm_attrs.items():
|
||||
if "usage" in key or "completion" in key or "response" in key:
|
||||
current_span.set_attribute(key, value)
|
||||
|
||||
return result
|
||||
|
||||
return actual_decorator
|
||||
|
||||
|
||||
# Wrapper for tool execution that creates a single merged span
|
||||
def _call_tool_async_wrapper(agentops_tracer):
|
||||
"""Wrapper that creates a single span for tool call and response."""
|
||||
|
||||
def actual_decorator(wrapped, instance, args, kwargs):
|
||||
async def new_function():
|
||||
# Extract tool info from args
|
||||
tool = args[0] if args else kwargs.get("tool")
|
||||
tool_args = args[1] if len(args) > 1 else kwargs.get("args", {})
|
||||
tool_context = args[2] if len(args) > 2 else kwargs.get("tool_context")
|
||||
|
||||
tool_name = getattr(tool, "name", "unknown_tool")
|
||||
span_name = f"adk.tool.{tool_name}"
|
||||
|
||||
with agentops_tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "tool")
|
||||
span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
|
||||
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "tool")
|
||||
|
||||
# Set tool call attributes
|
||||
span.set_attribute(ToolAttributes.TOOL_NAME, tool_name)
|
||||
if hasattr(tool, "description"):
|
||||
span.set_attribute(ToolAttributes.TOOL_DESCRIPTION, tool.description)
|
||||
if hasattr(tool, "is_long_running"):
|
||||
span.set_attribute("tool.is_long_running", tool.is_long_running)
|
||||
span.set_attribute(ToolAttributes.TOOL_PARAMETERS, json.dumps(tool_args))
|
||||
|
||||
if tool_context and hasattr(tool_context, "function_call_id"):
|
||||
span.set_attribute("tool.call_id", tool_context.function_call_id)
|
||||
if tool_context and hasattr(tool_context, "invocation_context"):
|
||||
span.set_attribute("adk.invocation_id", tool_context.invocation_context.invocation_id)
|
||||
|
||||
# Execute the tool
|
||||
result = await wrapped(*args, **kwargs)
|
||||
|
||||
# Set tool response attributes
|
||||
if result:
|
||||
if isinstance(result, dict):
|
||||
span.set_attribute(ToolAttributes.TOOL_RESULT, json.dumps(result))
|
||||
else:
|
||||
span.set_attribute(ToolAttributes.TOOL_RESULT, str(result))
|
||||
|
||||
return result
|
||||
|
||||
return new_function()
|
||||
|
||||
return actual_decorator
|
||||
|
||||
|
||||
def _patch(module_name: str, object_name: str, method_name: str, wrapper_function, agentops_tracer):
|
||||
"""Helper to apply a patch and keep track of it."""
|
||||
try:
|
||||
module = __import__(module_name, fromlist=[object_name])
|
||||
obj = getattr(module, object_name)
|
||||
wrapt.wrap_function_wrapper(obj, method_name, wrapper_function(agentops_tracer))
|
||||
_wrapped_methods.append((obj, method_name))
|
||||
logger.debug(f"Successfully wrapped {module_name}.{object_name}.{method_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not wrap {module_name}.{object_name}.{method_name}: {e}")
|
||||
|
||||
|
||||
def _patch_module_function(module_name: str, function_name: str, wrapper_function, agentops_tracer):
|
||||
"""Helper to patch module-level functions."""
|
||||
try:
|
||||
module = __import__(module_name, fromlist=[function_name])
|
||||
wrapt.wrap_function_wrapper(module, function_name, wrapper_function(agentops_tracer))
|
||||
_wrapped_methods.append((module, function_name))
|
||||
logger.debug(f"Successfully wrapped {module_name}.{function_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not wrap {module_name}.{function_name}: {e}")
|
||||
|
||||
|
||||
def patch_adk(agentops_tracer):
|
||||
"""Apply all patches to Google ADK modules."""
|
||||
logger.debug("Applying Google ADK patches for AgentOps instrumentation")
|
||||
|
||||
# First, disable ADK's own tracer by replacing it with our NoOpTracer
|
||||
noop_tracer = NoOpTracer()
|
||||
try:
|
||||
import google.adk.telemetry as adk_telemetry
|
||||
|
||||
# Replace the tracer with our no-op version
|
||||
adk_telemetry.tracer = noop_tracer
|
||||
logger.debug("Replaced ADK's tracer with NoOpTracer")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to replace ADK tracer: {e}")
|
||||
|
||||
# Also replace the tracer in all modules that have already imported it
|
||||
modules_to_patch = [
|
||||
"google.adk.runners",
|
||||
"google.adk.agents.base_agent",
|
||||
"google.adk.flows.llm_flows.base_llm_flow",
|
||||
"google.adk.flows.llm_flows.functions",
|
||||
]
|
||||
|
||||
import sys
|
||||
|
||||
for module_name in modules_to_patch:
|
||||
if module_name in sys.modules:
|
||||
try:
|
||||
module = sys.modules[module_name]
|
||||
if hasattr(module, "tracer"):
|
||||
module.tracer = noop_tracer
|
||||
logger.debug(f"Replaced tracer in {module_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to replace tracer in {module_name}: {e}")
|
||||
|
||||
# Patch methods that create top-level AgentOps spans
|
||||
# Skip runner patching - we don't want adk.runner spans
|
||||
_patch("google.adk.agents.base_agent", "BaseAgent", "run_async", _base_agent_run_async_wrapper, agentops_tracer)
|
||||
|
||||
# Patch ADK's telemetry functions to add attributes to AgentOps spans
|
||||
_patch_module_function("google.adk.telemetry", "trace_tool_call", _adk_trace_tool_call_wrapper, agentops_tracer)
|
||||
_patch_module_function(
|
||||
"google.adk.telemetry", "trace_tool_response", _adk_trace_tool_response_wrapper, agentops_tracer
|
||||
)
|
||||
_patch_module_function("google.adk.telemetry", "trace_call_llm", _adk_trace_call_llm_wrapper, agentops_tracer)
|
||||
|
||||
_patch_module_function("google.adk.telemetry", "trace_send_data", _adk_trace_send_data_wrapper, agentops_tracer)
|
||||
|
||||
# Patch method that creates nested spans
|
||||
_patch(
|
||||
"google.adk.flows.llm_flows.base_llm_flow",
|
||||
"BaseLlmFlow",
|
||||
"_call_llm_async",
|
||||
_base_llm_flow_call_llm_async_wrapper,
|
||||
agentops_tracer,
|
||||
)
|
||||
|
||||
# Also patch _finalize_model_response_event to capture response attributes
|
||||
_patch(
|
||||
"google.adk.flows.llm_flows.base_llm_flow",
|
||||
"BaseLlmFlow",
|
||||
"_finalize_model_response_event",
|
||||
_finalize_model_response_event_wrapper,
|
||||
agentops_tracer,
|
||||
)
|
||||
|
||||
# Patch tool execution to create merged tool spans
|
||||
_patch_module_function(
|
||||
"google.adk.flows.llm_flows.functions", "__call_tool_async", _call_tool_async_wrapper, agentops_tracer
|
||||
)
|
||||
|
||||
logger.info("Google ADK patching complete")
|
||||
|
||||
|
||||
def unpatch_adk():
|
||||
"""Remove all patches from Google ADK modules."""
|
||||
logger.debug("Removing Google ADK patches")
|
||||
|
||||
# Restore ADK's tracer
|
||||
try:
|
||||
import google.adk.telemetry as adk_telemetry
|
||||
from opentelemetry import trace
|
||||
|
||||
adk_telemetry.tracer = trace.get_tracer("gcp.vertex.agent")
|
||||
logger.debug("Restored ADK's built-in tracer")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to restore ADK tracer: {e}")
|
||||
|
||||
# Unwrap all methods
|
||||
for obj, method_name in _wrapped_methods:
|
||||
try:
|
||||
if hasattr(getattr(obj, method_name), "__wrapped__"):
|
||||
original = getattr(obj, method_name).__wrapped__
|
||||
setattr(obj, method_name, original)
|
||||
logger.debug(f"Successfully unwrapped {obj}.{method_name}")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to unwrap {obj}.{method_name}: {e}")
|
||||
|
||||
_wrapped_methods.clear()
|
||||
logger.info("Google ADK unpatching complete")
|
||||
@@ -0,0 +1 @@
|
||||
from .instrumentor import HaystackInstrumentor # noqa: F401
|
||||
@@ -0,0 +1,186 @@
|
||||
from typing import Any, Dict
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.instrumentation.utils import unwrap
|
||||
from wrapt import wrap_function_wrapper
|
||||
|
||||
from agentops.instrumentation.common import (
|
||||
CommonInstrumentor,
|
||||
InstrumentorConfig,
|
||||
StandardMetrics,
|
||||
create_wrapper_factory,
|
||||
create_span,
|
||||
SpanAttributeManager,
|
||||
)
|
||||
from agentops.semconv import SpanAttributes
|
||||
|
||||
|
||||
_instruments = ("haystack-ai >= 2.0.0",)
|
||||
|
||||
|
||||
class HaystackInstrumentor(CommonInstrumentor):
|
||||
def __init__(self):
|
||||
config = InstrumentorConfig(
|
||||
library_name="haystack",
|
||||
library_version="2",
|
||||
wrapped_methods=[],
|
||||
metrics_enabled=False,
|
||||
dependencies=_instruments,
|
||||
)
|
||||
super().__init__(config)
|
||||
self._attribute_manager = None
|
||||
|
||||
def _initialize(self, **kwargs):
|
||||
application_name = kwargs.get("application_name", "default_application")
|
||||
environment = kwargs.get("environment", "default_environment")
|
||||
self._attribute_manager = SpanAttributeManager(
|
||||
service_name=application_name, deployment_environment=environment
|
||||
)
|
||||
|
||||
def _create_metrics(self, meter) -> Dict[str, Any]:
|
||||
return StandardMetrics.create_standard_metrics(meter)
|
||||
|
||||
def _custom_wrap(self, **kwargs):
|
||||
attr_manager = self._attribute_manager
|
||||
|
||||
wrap_function_wrapper(
|
||||
"haystack.components.generators.openai",
|
||||
"OpenAIGenerator.run",
|
||||
create_wrapper_factory(_wrap_haystack_run_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
wrap_function_wrapper(
|
||||
"haystack.components.generators.chat",
|
||||
"AzureOpenAIChatGenerator.run",
|
||||
create_wrapper_factory(_wrap_haystack_run_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
|
||||
try:
|
||||
wrap_function_wrapper(
|
||||
"haystack.components.generators.openai",
|
||||
"OpenAIGenerator.stream",
|
||||
create_wrapper_factory(_wrap_haystack_stream_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
try:
|
||||
wrap_function_wrapper(
|
||||
"haystack.components.generators.chat",
|
||||
"AzureOpenAIChatGenerator.stream",
|
||||
create_wrapper_factory(_wrap_haystack_stream_impl, self._metrics, attr_manager)(self._tracer),
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _custom_unwrap(self, **kwargs):
|
||||
unwrap("haystack.components.generators.openai", "OpenAIGenerator.run")
|
||||
unwrap("haystack.components.generators.chat", "AzureOpenAIChatGenerator.run")
|
||||
try:
|
||||
unwrap("haystack.components.generators.openai", "OpenAIGenerator.stream")
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
unwrap("haystack.components.generators.chat", "AzureOpenAIChatGenerator.stream")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _first_non_empty_text(value):
|
||||
if isinstance(value, list) and value:
|
||||
return _first_non_empty_text(value[0])
|
||||
if isinstance(value, dict):
|
||||
if "content" in value:
|
||||
return str(value["content"])
|
||||
if "text" in value:
|
||||
return str(value["text"])
|
||||
if "replies" in value and value["replies"]:
|
||||
return str(value["replies"][0])
|
||||
if value is None:
|
||||
return None
|
||||
return str(value)
|
||||
|
||||
|
||||
def _extract_prompt(args, kwargs):
|
||||
if "prompt" in kwargs:
|
||||
return kwargs.get("prompt")
|
||||
if "messages" in kwargs:
|
||||
return kwargs.get("messages")
|
||||
if args:
|
||||
return args[0]
|
||||
return None
|
||||
|
||||
|
||||
def _get_model_name(instance):
|
||||
for attr in ("model", "model_name", "deployment_name", "deployment"):
|
||||
if hasattr(instance, attr):
|
||||
val = getattr(instance, attr)
|
||||
if val:
|
||||
return str(val)
|
||||
return None
|
||||
|
||||
|
||||
def _wrap_haystack_run_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
model = _get_model_name(instance)
|
||||
with create_span(
|
||||
tracer,
|
||||
"haystack.generator.run",
|
||||
kind=SpanKind.CLIENT,
|
||||
attributes={
|
||||
SpanAttributes.LLM_SYSTEM: "haystack",
|
||||
"gen_ai.model": model,
|
||||
SpanAttributes.LLM_REQUEST_STREAMING: False,
|
||||
},
|
||||
attribute_manager=attr_manager,
|
||||
) as span:
|
||||
prompt = _extract_prompt(args, kwargs)
|
||||
prompt_text = _first_non_empty_text(prompt)
|
||||
if prompt_text:
|
||||
span.set_attribute("gen_ai.prompt.0.content", prompt_text[:500])
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
reply_text = None
|
||||
if isinstance(result, dict):
|
||||
reply_text = _first_non_empty_text(result.get("replies"))
|
||||
if not reply_text:
|
||||
reply_text = _first_non_empty_text(result)
|
||||
else:
|
||||
reply_text = _first_non_empty_text(result)
|
||||
|
||||
if reply_text:
|
||||
span.set_attribute("gen_ai.response.0.content", str(reply_text)[:500])
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def _wrap_haystack_stream_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
|
||||
model = _get_model_name(instance)
|
||||
with create_span(
|
||||
tracer,
|
||||
"haystack.generator.stream",
|
||||
kind=SpanKind.CLIENT,
|
||||
attributes={
|
||||
SpanAttributes.LLM_SYSTEM: "haystack",
|
||||
"gen_ai.model": model,
|
||||
SpanAttributes.LLM_REQUEST_STREAMING: True,
|
||||
},
|
||||
attribute_manager=attr_manager,
|
||||
) as span:
|
||||
prompt = _extract_prompt(args, kwargs)
|
||||
prompt_text = _first_non_empty_text(prompt)
|
||||
if prompt_text:
|
||||
span.set_attribute("gen_ai.prompt.0.content", prompt_text[:500])
|
||||
|
||||
out = wrapped(*args, **kwargs)
|
||||
|
||||
try:
|
||||
chunk_count = 0
|
||||
for chunk in out:
|
||||
chunk_count += 1
|
||||
last_text = _first_non_empty_text(chunk)
|
||||
if last_text:
|
||||
span.set_attribute("gen_ai.response.0.content", str(last_text)[:500])
|
||||
yield chunk
|
||||
span.set_attribute("gen_ai.response.chunk_count", chunk_count)
|
||||
except TypeError:
|
||||
return out
|
||||
@@ -0,0 +1,3 @@
|
||||
from agentops.instrumentation.agentic.langgraph.instrumentation import LanggraphInstrumentor
|
||||
|
||||
__all__ = ["LanggraphInstrumentor"]
|
||||
@@ -0,0 +1,54 @@
|
||||
from typing import Dict, Any
|
||||
import json
|
||||
|
||||
|
||||
def ensure_no_none_values(attributes: Dict[str, Any]) -> Dict[str, Any]:
|
||||
return {k: v for k, v in attributes.items() if v is not None}
|
||||
|
||||
|
||||
def set_graph_attributes(span: Any, graph_nodes: list = None, graph_edges: list = None) -> None:
|
||||
if graph_nodes:
|
||||
span.set_attribute("langgraph.graph.nodes", json.dumps(graph_nodes))
|
||||
span.set_attribute("langgraph.graph.node_count", len(graph_nodes))
|
||||
|
||||
for i, node in enumerate(graph_nodes):
|
||||
span.set_attribute(f"langgraph.node.{i}.name", node)
|
||||
span.set_attribute(f"langgraph.node.{i}.type", "unknown")
|
||||
|
||||
if graph_edges:
|
||||
span.set_attribute("langgraph.graph.edges", json.dumps(graph_edges))
|
||||
span.set_attribute("langgraph.graph.edge_count", len(graph_edges))
|
||||
|
||||
for i, edge in enumerate(graph_edges):
|
||||
parts = edge.split("->")
|
||||
if len(parts) == 2:
|
||||
span.set_attribute(f"langgraph.edge.{i}.source", parts[0])
|
||||
span.set_attribute(f"langgraph.edge.{i}.target", parts[1])
|
||||
|
||||
|
||||
def extract_messages_from_input(input_data: Any) -> list:
|
||||
if isinstance(input_data, dict) and "messages" in input_data:
|
||||
return input_data["messages"]
|
||||
return []
|
||||
|
||||
|
||||
def extract_messages_from_output(output_data: Any) -> list:
|
||||
if isinstance(output_data, dict) and "messages" in output_data:
|
||||
return output_data["messages"]
|
||||
return []
|
||||
|
||||
|
||||
def get_message_content(message: Any) -> str:
|
||||
if hasattr(message, "content"):
|
||||
return str(message.content)
|
||||
return ""
|
||||
|
||||
|
||||
def get_message_role(message: Any) -> str:
|
||||
if hasattr(message, "role"):
|
||||
return message.role
|
||||
elif hasattr(message, "type"):
|
||||
return message.type
|
||||
elif hasattr(message, "__class__"):
|
||||
return message.__class__.__name__.replace("Message", "").lower()
|
||||
return "unknown"
|
||||
@@ -0,0 +1,598 @@
|
||||
from functools import wraps
|
||||
from typing import Any, Callable, Collection, Dict, Optional, Tuple
|
||||
import json
|
||||
import inspect
|
||||
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.trace import SpanKind, Status, StatusCode, get_tracer
|
||||
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
|
||||
from opentelemetry.instrumentation.utils import unwrap
|
||||
from wrapt import wrap_function_wrapper
|
||||
|
||||
from agentops.semconv import (
|
||||
SpanAttributes,
|
||||
WorkflowAttributes,
|
||||
MessageAttributes,
|
||||
)
|
||||
from .attributes import (
|
||||
ensure_no_none_values,
|
||||
set_graph_attributes,
|
||||
extract_messages_from_input,
|
||||
get_message_content,
|
||||
get_message_role,
|
||||
)
|
||||
|
||||
import sys
|
||||
|
||||
if "typing_extensions" not in sys.modules:
|
||||
from unittest import mock
|
||||
|
||||
sys.modules["typing_extensions"] = mock.MagicMock()
|
||||
|
||||
|
||||
class LanggraphInstrumentor(BaseInstrumentor):
|
||||
def __init__(self, config: Optional[Dict[str, Any]] = None):
|
||||
super().__init__()
|
||||
self.config = config or {}
|
||||
self._tracer = None
|
||||
|
||||
def instrumentation_dependencies(self) -> Collection[str]:
|
||||
return ["langgraph >= 0.0.1"]
|
||||
|
||||
def _instrument(self, **kwargs):
|
||||
tracer_provider = kwargs.get("tracer_provider")
|
||||
self._tracer = get_tracer("agentops.instrumentation.agentic.langgraph", "0.1.0", tracer_provider)
|
||||
|
||||
# Initialize context variable for tracking graph executions
|
||||
import contextvars
|
||||
|
||||
self._current_graph_execution = contextvars.ContextVar("current_graph_execution", default=None)
|
||||
|
||||
wrap_function_wrapper("langgraph.graph.state", "StateGraph.__init__", self._wrap_state_graph_init)
|
||||
|
||||
wrap_function_wrapper("langgraph.graph.state", "StateGraph.compile", self._wrap_state_graph_compile)
|
||||
|
||||
wrap_function_wrapper("langgraph.pregel", "Pregel.invoke", self._wrap_invoke)
|
||||
|
||||
wrap_function_wrapper("langgraph.pregel", "Pregel.stream", self._wrap_stream)
|
||||
|
||||
wrap_function_wrapper("langgraph.graph.state", "StateGraph.add_node", self._wrap_add_node)
|
||||
|
||||
def _uninstrument(self, **kwargs):
|
||||
unwrap("langgraph.graph.state", "StateGraph.__init__")
|
||||
unwrap("langgraph.graph.state", "StateGraph.compile")
|
||||
unwrap("langgraph.pregel", "Pregel.invoke")
|
||||
unwrap("langgraph.pregel", "Pregel.stream")
|
||||
unwrap("langgraph.graph.state", "StateGraph.add_node")
|
||||
|
||||
def _wrap_state_graph_init(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
|
||||
if not self._tracer:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
with self._tracer.start_as_current_span("langgraph.graph.initialize", kind=SpanKind.INTERNAL) as span:
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
|
||||
WorkflowAttributes.WORKFLOW_TYPE: "graph_initialization",
|
||||
SpanAttributes.AGENTOPS_ENTITY_NAME: "Graph Initialization",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
instance._langgraph_instrumented = True
|
||||
instance._langgraph_nodes = []
|
||||
instance._langgraph_edges = []
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
|
||||
def _wrap_state_graph_compile(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
|
||||
if not self._tracer:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
with self._tracer.start_as_current_span("langgraph.graph.compile", kind=SpanKind.INTERNAL) as span:
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
|
||||
WorkflowAttributes.WORKFLOW_TYPE: "graph_compilation",
|
||||
SpanAttributes.AGENTOPS_ENTITY_NAME: "Graph Compilation",
|
||||
SpanAttributes.LLM_SYSTEM: "langgraph",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
nodes = []
|
||||
edges = []
|
||||
|
||||
if hasattr(instance, "nodes"):
|
||||
nodes = list(instance.nodes.keys()) if hasattr(instance.nodes, "keys") else []
|
||||
|
||||
if hasattr(instance, "edges") and hasattr(instance.edges, "items"):
|
||||
for source, targets in instance.edges.items():
|
||||
if isinstance(targets, dict):
|
||||
for target in targets.values():
|
||||
edges.append(f"{source}->{target}")
|
||||
elif isinstance(targets, list):
|
||||
for target in targets:
|
||||
edges.append(f"{source}->{target}")
|
||||
|
||||
set_graph_attributes(span, nodes, edges)
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
|
||||
def _wrap_invoke(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
|
||||
if not self._tracer:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
current_span = trace.get_current_span()
|
||||
if current_span and current_span.name == "langgraph.workflow.execute":
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
with self._tracer.start_as_current_span("langgraph.workflow.execute", kind=SpanKind.INTERNAL) as span:
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
|
||||
WorkflowAttributes.WORKFLOW_TYPE: "langgraph_invoke",
|
||||
SpanAttributes.AGENTOPS_ENTITY_NAME: "Workflow Execution",
|
||||
SpanAttributes.LLM_REQUEST_STREAMING: False,
|
||||
"langgraph.execution.mode": "invoke",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
execution_state = {"executed_nodes": [], "message_count": 0, "final_response": None}
|
||||
|
||||
# Set the current execution state in context
|
||||
token = self._current_graph_execution.set(execution_state)
|
||||
|
||||
try:
|
||||
input_data = args[0] if args else kwargs.get("input", {})
|
||||
messages = extract_messages_from_input(input_data)
|
||||
if messages:
|
||||
execution_state["message_count"] = len(messages)
|
||||
for i, msg in enumerate(messages[:3]):
|
||||
content = get_message_content(msg)
|
||||
role = get_message_role(msg)
|
||||
if content:
|
||||
span.set_attribute(f"gen_ai.prompt.{i}.content", content[:500])
|
||||
span.set_attribute(f"gen_ai.prompt.{i}.role", role)
|
||||
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Extract execution information from result
|
||||
if isinstance(result, dict):
|
||||
# Check for messages in result
|
||||
if "messages" in result:
|
||||
output_messages = result["messages"]
|
||||
if isinstance(output_messages, list):
|
||||
# Count all messages in the result
|
||||
total_messages = len([msg for msg in output_messages if hasattr(msg, "content")])
|
||||
execution_state["message_count"] = total_messages
|
||||
|
||||
if output_messages:
|
||||
# Find the last non-tool message
|
||||
for msg in reversed(output_messages):
|
||||
if hasattr(msg, "content") and not hasattr(msg, "tool_call_id"):
|
||||
content = get_message_content(msg)
|
||||
if content:
|
||||
execution_state["final_response"] = content
|
||||
span.set_attribute("gen_ai.response.0.content", content[:500])
|
||||
break
|
||||
|
||||
# Capture final execution state before returning
|
||||
final_executed_nodes = list(execution_state["executed_nodes"]) # Copy the list
|
||||
final_node_count = len(final_executed_nodes)
|
||||
final_message_count = execution_state["message_count"]
|
||||
final_response = execution_state["final_response"]
|
||||
|
||||
span.set_status(Status(StatusCode.OK))
|
||||
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
"langgraph.graph.executed_nodes": json.dumps(final_executed_nodes),
|
||||
"langgraph.graph.node_execution_count": final_node_count,
|
||||
"langgraph.graph.message_count": final_message_count,
|
||||
"langgraph.graph.final_response": final_response,
|
||||
"langgraph.graph.status": "success",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
finally:
|
||||
# Reset the context
|
||||
self._current_graph_execution.reset(token)
|
||||
|
||||
def _wrap_stream(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
|
||||
if not self._tracer:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
current_span = trace.get_current_span()
|
||||
if current_span and current_span.name == "langgraph.workflow.execute":
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
span = self._tracer.start_span("langgraph.workflow.execute", kind=SpanKind.INTERNAL)
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
|
||||
WorkflowAttributes.WORKFLOW_TYPE: "langgraph_stream",
|
||||
SpanAttributes.AGENTOPS_ENTITY_NAME: "Workflow Stream",
|
||||
SpanAttributes.LLM_REQUEST_STREAMING: True,
|
||||
"langgraph.execution.mode": "stream",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
execution_state = {"executed_nodes": [], "message_count": 0, "chunk_count": 0, "final_response": None}
|
||||
|
||||
# Set the current execution state in context
|
||||
token = self._current_graph_execution.set(execution_state)
|
||||
|
||||
try:
|
||||
# Extract input messages
|
||||
input_data = args[0] if args else kwargs.get("input", {})
|
||||
messages = extract_messages_from_input(input_data)
|
||||
if messages:
|
||||
execution_state["message_count"] = len(messages)
|
||||
for i, msg in enumerate(messages[:3]):
|
||||
content = get_message_content(msg)
|
||||
role = get_message_role(msg)
|
||||
if content:
|
||||
span.set_attribute(f"gen_ai.prompt.{i}.content", content[:500])
|
||||
span.set_attribute(f"gen_ai.prompt.{i}.role", role)
|
||||
|
||||
stream_gen = wrapped(*args, **kwargs)
|
||||
|
||||
def stream_wrapper():
|
||||
try:
|
||||
for chunk in stream_gen:
|
||||
execution_state["chunk_count"] += 1
|
||||
|
||||
if isinstance(chunk, dict):
|
||||
# Debug: print chunk structure
|
||||
# print(f"DEBUG: Chunk keys: {list(chunk.keys())}")
|
||||
|
||||
for key in chunk:
|
||||
# Track node executions (excluding special keys)
|
||||
if (
|
||||
key not in ["__start__", "__end__", "__interrupt__"]
|
||||
and key not in execution_state["executed_nodes"]
|
||||
):
|
||||
execution_state["executed_nodes"].append(key)
|
||||
|
||||
# Track messages in the chunk value
|
||||
chunk_value = chunk[key]
|
||||
if isinstance(chunk_value, dict):
|
||||
# Check for messages in the chunk value
|
||||
if "messages" in chunk_value:
|
||||
msg_list = chunk_value["messages"]
|
||||
if isinstance(msg_list, list):
|
||||
execution_state["message_count"] += len(msg_list)
|
||||
for msg in msg_list:
|
||||
content = get_message_content(msg)
|
||||
if content:
|
||||
execution_state["final_response"] = content
|
||||
elif key == "messages" and isinstance(chunk_value, list):
|
||||
# Sometimes messages might be directly in the chunk
|
||||
execution_state["message_count"] += len(chunk_value)
|
||||
for msg in chunk_value:
|
||||
content = get_message_content(msg)
|
||||
if content:
|
||||
execution_state["final_response"] = content
|
||||
|
||||
yield chunk
|
||||
|
||||
# Capture final execution state before ending
|
||||
final_executed_nodes = list(execution_state["executed_nodes"])
|
||||
final_node_count = len(final_executed_nodes)
|
||||
final_message_count = execution_state["message_count"]
|
||||
final_chunk_count = execution_state["chunk_count"]
|
||||
final_response = execution_state["final_response"]
|
||||
|
||||
span.set_status(Status(StatusCode.OK))
|
||||
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
"langgraph.graph.executed_nodes": json.dumps(final_executed_nodes),
|
||||
"langgraph.graph.node_execution_count": final_node_count,
|
||||
"langgraph.graph.message_count": final_message_count,
|
||||
"langgraph.graph.total_chunks": final_chunk_count,
|
||||
"langgraph.graph.final_response": final_response,
|
||||
"langgraph.graph.status": "success",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
finally:
|
||||
span.end()
|
||||
|
||||
return stream_wrapper()
|
||||
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
span.end()
|
||||
raise
|
||||
finally:
|
||||
# Reset the context
|
||||
self._current_graph_execution.reset(token)
|
||||
|
||||
def _wrap_add_node(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
|
||||
if not self._tracer:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
# Get node name and function
|
||||
if args:
|
||||
key = args[0]
|
||||
action = args[1] if len(args) > 1 else kwargs.get("action")
|
||||
else:
|
||||
key = kwargs.get("key")
|
||||
action = kwargs.get("action")
|
||||
|
||||
if not action:
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
# Create wrapped node function that instruments LLM calls
|
||||
def create_wrapped_node(original_func):
|
||||
if inspect.iscoroutinefunction(original_func):
|
||||
|
||||
@wraps(original_func)
|
||||
async def wrapped_node_async(state):
|
||||
# Track node execution in parent graph span
|
||||
self._track_node_execution(key)
|
||||
|
||||
# Check if this node contains an LLM call
|
||||
is_llm_node = self._detect_llm_node(original_func)
|
||||
|
||||
if is_llm_node:
|
||||
with self._tracer.start_as_current_span("langgraph.node.execute", kind=SpanKind.CLIENT) as span:
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "llm",
|
||||
SpanAttributes.AGENTOPS_ENTITY_NAME: f"Node: {key}",
|
||||
SpanAttributes.LLM_SYSTEM: "langgraph",
|
||||
"langgraph.node.name": key,
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
# Call the original function
|
||||
result = await original_func(state)
|
||||
|
||||
# Extract LLM information from the result
|
||||
self._extract_llm_info_from_result(span, state, result)
|
||||
|
||||
span.set_status(Status(StatusCode.OK))
|
||||
return result
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
else:
|
||||
# Non-LLM node, just execute normally
|
||||
return await original_func(state)
|
||||
else:
|
||||
|
||||
@wraps(original_func)
|
||||
def wrapped_node_sync(state):
|
||||
# Track node execution in parent graph span
|
||||
self._track_node_execution(key)
|
||||
|
||||
# Check if this node contains an LLM call
|
||||
is_llm_node = self._detect_llm_node(original_func)
|
||||
|
||||
if is_llm_node:
|
||||
with self._tracer.start_as_current_span("langgraph.node.execute", kind=SpanKind.CLIENT) as span:
|
||||
span.set_attributes(
|
||||
ensure_no_none_values(
|
||||
{
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: "llm",
|
||||
SpanAttributes.AGENTOPS_ENTITY_NAME: f"Node: {key}",
|
||||
SpanAttributes.LLM_SYSTEM: "langgraph",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
# Call the original function
|
||||
result = original_func(state)
|
||||
|
||||
# Extract LLM information from the result
|
||||
self._extract_llm_info_from_result(span, state, result)
|
||||
|
||||
span.set_status(Status(StatusCode.OK))
|
||||
return result
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
else:
|
||||
# Non-LLM node, just execute normally
|
||||
return original_func(state)
|
||||
|
||||
return wrapped_node_sync
|
||||
|
||||
return wrapped_node_async if inspect.iscoroutinefunction(original_func) else wrapped_node_sync
|
||||
|
||||
# Wrap the action function
|
||||
wrapped_action = create_wrapped_node(action)
|
||||
|
||||
# Call the original add_node with the wrapped action
|
||||
if args and len(args) > 1:
|
||||
new_args = (args[0], wrapped_action) + args[2:]
|
||||
return wrapped(*new_args, **kwargs)
|
||||
else:
|
||||
kwargs["action"] = wrapped_action
|
||||
return wrapped(*args, **kwargs)
|
||||
|
||||
def _track_node_execution(self, node_name: str) -> None:
|
||||
"""Track node execution in the active graph span."""
|
||||
# Use context variable to track the current execution
|
||||
if hasattr(self, "_current_graph_execution"):
|
||||
execution_state = self._current_graph_execution.get()
|
||||
if execution_state and node_name not in execution_state["executed_nodes"]:
|
||||
execution_state["executed_nodes"].append(node_name)
|
||||
|
||||
def _detect_llm_node(self, func: Callable) -> bool:
|
||||
"""Detect if a node function contains LLM calls."""
|
||||
try:
|
||||
# Get the source code of the function
|
||||
source = inspect.getsource(func)
|
||||
|
||||
# Check for common LLM patterns
|
||||
llm_patterns = [
|
||||
"ChatOpenAI",
|
||||
"ChatAnthropic",
|
||||
"ChatGoogleGenerativeAI",
|
||||
".invoke(",
|
||||
".ainvoke(",
|
||||
".stream(",
|
||||
".astream(",
|
||||
"llm.",
|
||||
"model.",
|
||||
"chat.",
|
||||
]
|
||||
|
||||
for pattern in llm_patterns:
|
||||
if pattern in source:
|
||||
return True
|
||||
|
||||
# Check if function has 'llm' or 'model' in its local variables
|
||||
if hasattr(func, "__code__"):
|
||||
local_vars = func.__code__.co_varnames
|
||||
if any(var in ["llm", "model", "chat"] for var in local_vars):
|
||||
return True
|
||||
|
||||
except Exception:
|
||||
# If we can't inspect the source, assume it might be an LLM node
|
||||
pass
|
||||
|
||||
return False
|
||||
|
||||
def _extract_llm_info_from_result(self, span: Any, state: Dict, result: Any) -> None:
|
||||
"""Extract LLM information from the node execution result."""
|
||||
try:
|
||||
# Extract messages from state
|
||||
if isinstance(state, dict) and "messages" in state:
|
||||
messages = state["messages"]
|
||||
# Set input messages
|
||||
for i, msg in enumerate(messages[-5:]): # Last 5 messages as context
|
||||
if hasattr(msg, "content"):
|
||||
span.set_attribute(MessageAttributes.PROMPT_CONTENT.format(i=i), str(msg.content)[:1000])
|
||||
if hasattr(msg, "role"):
|
||||
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), msg.role)
|
||||
elif hasattr(msg, "type"):
|
||||
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), msg.type)
|
||||
|
||||
# Extract messages from result
|
||||
if isinstance(result, dict) and "messages" in result:
|
||||
output_messages = result["messages"]
|
||||
if output_messages:
|
||||
last_msg = output_messages[-1] if isinstance(output_messages, list) else output_messages
|
||||
|
||||
# Extract model information from message if available
|
||||
if hasattr(last_msg, "response_metadata"):
|
||||
metadata = last_msg.response_metadata
|
||||
if isinstance(metadata, dict):
|
||||
if "model_name" in metadata:
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, metadata["model_name"])
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, metadata["model_name"])
|
||||
elif "model" in metadata:
|
||||
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, metadata["model"])
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, metadata["model"])
|
||||
|
||||
# Token usage
|
||||
if "token_usage" in metadata:
|
||||
usage = metadata["token_usage"]
|
||||
if isinstance(usage, dict):
|
||||
if "prompt_tokens" in usage:
|
||||
span.set_attribute(
|
||||
SpanAttributes.LLM_USAGE_PROMPT_TOKENS, usage["prompt_tokens"]
|
||||
)
|
||||
if "completion_tokens" in usage:
|
||||
span.set_attribute(
|
||||
SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, usage["completion_tokens"]
|
||||
)
|
||||
if "total_tokens" in usage:
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, usage["total_tokens"])
|
||||
|
||||
# Response ID
|
||||
if "id" in metadata and metadata["id"] is not None:
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_ID, metadata["id"])
|
||||
|
||||
# Finish reason
|
||||
if "finish_reason" in metadata:
|
||||
span.set_attribute(
|
||||
MessageAttributes.COMPLETION_FINISH_REASON.format(i=0), metadata["finish_reason"]
|
||||
)
|
||||
|
||||
# Content
|
||||
if hasattr(last_msg, "content"):
|
||||
span.set_attribute(
|
||||
MessageAttributes.COMPLETION_CONTENT.format(i=0), str(last_msg.content)[:1000]
|
||||
)
|
||||
if hasattr(last_msg, "role"):
|
||||
span.set_attribute(MessageAttributes.COMPLETION_ROLE.format(i=0), last_msg.role)
|
||||
|
||||
# Check for tool calls
|
||||
if hasattr(last_msg, "tool_calls") and last_msg.tool_calls:
|
||||
for j, tool_call in enumerate(last_msg.tool_calls[:5]):
|
||||
if hasattr(tool_call, "name"):
|
||||
span.set_attribute(
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=j), tool_call.name
|
||||
)
|
||||
if hasattr(tool_call, "args"):
|
||||
span.set_attribute(
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=j),
|
||||
json.dumps(tool_call.args)[:500],
|
||||
)
|
||||
if hasattr(tool_call, "id"):
|
||||
span.set_attribute(
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=j), tool_call.id
|
||||
)
|
||||
|
||||
# Additional attributes from message
|
||||
if hasattr(last_msg, "id") and last_msg.id is not None:
|
||||
span.set_attribute(SpanAttributes.LLM_RESPONSE_ID, last_msg.id)
|
||||
|
||||
# Usage information might be on the message itself
|
||||
if hasattr(last_msg, "usage_metadata"):
|
||||
usage = last_msg.usage_metadata
|
||||
if hasattr(usage, "input_tokens"):
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_PROMPT_TOKENS, usage.input_tokens)
|
||||
if hasattr(usage, "output_tokens"):
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, usage.output_tokens)
|
||||
if hasattr(usage, "total_tokens"):
|
||||
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, usage.total_tokens)
|
||||
except Exception:
|
||||
# Don't fail the span if we can't extract info
|
||||
pass
|
||||
@@ -0,0 +1 @@
|
||||
__version__ = "0.1.0"
|
||||
@@ -0,0 +1,156 @@
|
||||
# OpenAI Agents SDK Instrumentation
|
||||
|
||||
This module provides automatic instrumentation for the OpenAI Agents SDK, adding telemetry that follows OpenTelemetry semantic conventions for Generative AI systems.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The OpenAI Agents SDK instrumentor works by:
|
||||
|
||||
1. Intercepting the Agents SDK's trace processor interface to capture Agent, Function, Generation, and other span types
|
||||
2. Monkey-patching the Agents SDK `Runner` class to capture the full execution lifecycle, including streaming operations
|
||||
3. Converting all captured data to OpenTelemetry spans and metrics following semantic conventions
|
||||
|
||||
The instrumentation is organized into several key components:
|
||||
|
||||
1. **Instrumentor (`instrumentor.py`)**: The entry point that patches the Agents SDK and configures trace capture
|
||||
2. **Processor (`processor.py`)**: Receives events from the SDK and prepares them for export
|
||||
3. **Exporter (`exporter.py`)**: Converts SDK spans to OpenTelemetry spans and exports them
|
||||
4. **Attributes Module (`attributes/`)**: Specialized modules for extracting and formatting span attributes
|
||||
|
||||
## Attribute Processing Modules
|
||||
|
||||
The attribute modules extract and format OpenTelemetry-compatible attributes from span data:
|
||||
|
||||
- **Common (`attributes/common.py`)**: Core attribute extraction functions for all span types and utility functions
|
||||
- **Completion (`attributes/completion.py`)**: Handles different completion content formats (Chat Completions API, Response API, Agents SDK)
|
||||
- **Model (`attributes/model.py`)**: Extracts model information and parameters
|
||||
- **Tokens (`attributes/tokens.py`)**: Processes token usage data and metrics
|
||||
- **Response (`attributes/response.py`)**: Handles interpretation of Response API objects
|
||||
|
||||
Each getter function in these modules is focused on a single responsibility and does not modify global state. Functions are designed to be composable, allowing different attribute types to be combined as needed in the exporter.
|
||||
|
||||
## Span Types
|
||||
|
||||
The instrumentor captures the following span types:
|
||||
|
||||
- **Trace**: The root span representing an entire agent workflow execution
|
||||
- Created using `get_base_trace_attributes()` to initialize with standard fields
|
||||
- Captures workflow name, trace ID, and workflow-level metadata
|
||||
|
||||
- **Agent**: Represents an agent's execution lifecycle
|
||||
- Processed using `get_agent_span_attributes()` with `AGENT_SPAN_ATTRIBUTES` mapping
|
||||
- Uses `SpanKind.CONSUMER` to indicate an agent receiving a request
|
||||
- Captures agent name, input, output, tools, and other metadata
|
||||
|
||||
- **Function**: Represents a tool/function call
|
||||
- Processed using `get_function_span_attributes()` with `FUNCTION_SPAN_ATTRIBUTES` mapping
|
||||
- Uses `SpanKind.CLIENT` to indicate an outbound call to a function
|
||||
- Captures function name, input arguments, output results, and from_agent information
|
||||
|
||||
- **Generation**: Captures details of model generation
|
||||
- Processed using `get_generation_span_attributes()` with `GENERATION_SPAN_ATTRIBUTES` mapping
|
||||
- Uses `SpanKind.CLIENT` to indicate an outbound call to an LLM
|
||||
- Captures model name, configuration, usage statistics, and response content
|
||||
|
||||
- **Response**: Lightweight span for tracking model response data
|
||||
- Processed using `get_response_span_attributes()` with `RESPONSE_SPAN_ATTRIBUTES` mapping
|
||||
- Extracts response content and metadata from different API formats
|
||||
|
||||
- **Handoff**: Represents control transfer between agents
|
||||
- Processed using `get_handoff_span_attributes()` with `HANDOFF_SPAN_ATTRIBUTES` mapping
|
||||
- Tracks from_agent and to_agent information
|
||||
|
||||
## Span Lifecycle Management
|
||||
|
||||
The exporter (`exporter.py`) handles the full span lifecycle:
|
||||
|
||||
1. **Start Events**:
|
||||
- Create spans but DO NOT END them
|
||||
- Store span references in tracking dictionaries
|
||||
- Use OpenTelemetry's start_span to control when spans end
|
||||
- Leave status as UNSET to indicate in-progress
|
||||
|
||||
2. **End Events**:
|
||||
- Look up existing span by ID in tracking dictionaries
|
||||
- If found and not ended:
|
||||
- Update span with all final attributes
|
||||
- Set status to OK or ERROR based on task outcome
|
||||
- End the span manually
|
||||
- If not found or already ended:
|
||||
- Create a new complete span with all data
|
||||
- End it immediately
|
||||
|
||||
3. **Error Handling**:
|
||||
- Check if spans are already ended before attempting updates
|
||||
- Provide informative log messages about span lifecycle
|
||||
- Properly clean up tracking resources
|
||||
|
||||
This approach is essential because:
|
||||
- Agents SDK sends separate start and end events for each task
|
||||
- We need to maintain a single span for the entire task lifecycle to get accurate timing
|
||||
- Final data (outputs, token usage, etc.) is only available at the end event
|
||||
- We want to avoid creating duplicate spans for the same task
|
||||
- Spans must be properly created and ended to avoid leaks
|
||||
|
||||
The span lifecycle management ensures spans have:
|
||||
- Accurate start and end times (preserving the actual task duration)
|
||||
- Complete attribute data from both start and end events
|
||||
- Proper status reflecting task completion
|
||||
- All final outputs, errors, and metrics
|
||||
- Clean resource management with no memory leaks
|
||||
|
||||
## Key Design Patterns
|
||||
|
||||
### Semantic Conventions
|
||||
|
||||
All attribute names follow the OpenTelemetry semantic conventions defined in `agentops.semconv`:
|
||||
|
||||
```python
|
||||
# Using constants from semconv module
|
||||
attributes[CoreAttributes.TRACE_ID] = trace_id
|
||||
attributes[WorkflowAttributes.WORKFLOW_NAME] = trace.name
|
||||
attributes[SpanAttributes.LLM_SYSTEM] = "openai"
|
||||
attributes[MessageAttributes.COMPLETION_CONTENT.format(i=0)] = content
|
||||
```
|
||||
|
||||
### Target → Source Attribute Mapping
|
||||
|
||||
We use a consistent pattern for attribute extraction with typed mapping dictionaries:
|
||||
|
||||
```python
|
||||
# Attribute mapping example
|
||||
AGENT_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
# target_attribute: source_attribute
|
||||
AgentAttributes.AGENT_NAME: "name",
|
||||
WorkflowAttributes.WORKFLOW_INPUT: "input",
|
||||
WorkflowAttributes.FINAL_OUTPUT: "output",
|
||||
# ...
|
||||
}
|
||||
```
|
||||
|
||||
### Structured Attribute Handling
|
||||
|
||||
- Always use MessageAttributes semantic conventions for content and tool calls
|
||||
- For chat completions, use MessageAttributes.COMPLETION_CONTENT.format(i=0)
|
||||
- For tool calls, use MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=0), etc.
|
||||
- Never try to combine or aggregate contents into a single attribute
|
||||
- Each message component should have its own properly formatted attribute
|
||||
- This ensures proper display in OpenTelemetry backends and dashboards
|
||||
|
||||
### Serialization Rules
|
||||
|
||||
1. We do not serialize data structures arbitrarily; everything has a semantic convention
|
||||
2. Span attributes should use semantic conventions and avoid complex serialized structures
|
||||
3. Keep all string data in its original form - do not parse JSON within strings
|
||||
4. If a function has JSON attributes for its arguments, do not parse that JSON - keep as string
|
||||
5. If a completion or response body text/content contains JSON, keep it as a string
|
||||
7. Function arguments and tool call arguments should remain in their raw string form
|
||||
|
||||
### Critical Notes for Attribute Handling
|
||||
|
||||
- NEVER manually set the root completion attributes (`SpanAttributes.LLM_COMPLETIONS` or "gen_ai.completion")
|
||||
- Let OpenTelemetry backend derive these values from the detailed attributes
|
||||
- Setting root completion attributes creates duplication and inconsistency
|
||||
- Tests should verify attribute existence using MessageAttributes constants
|
||||
- Do not check for the presence of SpanAttributes.LLM_COMPLETIONS
|
||||
- Verify individual content/tool attributes instead of root attributes
|
||||
@@ -0,0 +1,145 @@
|
||||
# OpenAI Agents Spans and Traces
|
||||
|
||||
This document describes the span types, naming conventions, and attribute patterns used by the AgentOps instrumentation for the OpenAI Agents SDK.
|
||||
|
||||
## Span Types and Classes
|
||||
|
||||
The instrumentation works with these specific span data classes:
|
||||
|
||||
1. **AgentSpanData**: Represents a single agent's operation
|
||||
- Has attributes for name, input, output, tools, and handoffs
|
||||
- Processed by `get_agent_span_attributes()` using `AGENT_SPAN_ATTRIBUTES` mapping
|
||||
|
||||
2. **FunctionSpanData**: Represents tool or function calls
|
||||
- Has attributes for name, input, output, and from_agent
|
||||
- Processed by `get_function_span_attributes()` using `FUNCTION_SPAN_ATTRIBUTES` mapping
|
||||
|
||||
3. **GenerationSpanData**: Represents LLM model invocations
|
||||
- Has attributes for model, input, output, tools, and from_agent
|
||||
- Processed by `get_generation_span_attributes()` using `GENERATION_SPAN_ATTRIBUTES` mapping
|
||||
|
||||
4. **HandoffSpanData**: Represents agent-to-agent handoffs
|
||||
- Has attributes for from_agent and to_agent
|
||||
- Processed by `get_handoff_span_attributes()` using `HANDOFF_SPAN_ATTRIBUTES` mapping
|
||||
|
||||
5. **ResponseSpanData**: Represents model response data
|
||||
- Has attributes for input and response
|
||||
- Processed by `get_response_span_attributes()` using `RESPONSE_SPAN_ATTRIBUTES` mapping
|
||||
|
||||
## Span Naming Conventions
|
||||
|
||||
Spans are named according to these conventions:
|
||||
|
||||
1. **Trace Spans**: `agents.trace.{workflow_name}`
|
||||
- Represents the entire agent workflow
|
||||
- Named after the workflow or trace name
|
||||
|
||||
2. **Agent Spans**: `agents.agent`
|
||||
- Represents a single agent's operation
|
||||
- Uses `SpanKind.CONSUMER`
|
||||
|
||||
3. **Function Spans**: `agents.function`
|
||||
- Represents tool or function calls
|
||||
- Uses `SpanKind.CLIENT`
|
||||
|
||||
4. **Generation Spans**: `agents.generation`
|
||||
- Represents LLM model invocations
|
||||
- Uses `SpanKind.CLIENT`
|
||||
|
||||
5. **Handoff Spans**: `agents.handoff`
|
||||
- Represents agent-to-agent handoffs
|
||||
- Uses `SpanKind.INTERNAL`
|
||||
|
||||
6. **Response Spans**: `agents.response`
|
||||
- Represents model response data
|
||||
- Uses `SpanKind.CLIENT`
|
||||
|
||||
## Span Hierarchy
|
||||
|
||||
The spans follow a parent-child relationship that reflects the execution flow:
|
||||
|
||||
```
|
||||
agents.trace.{workflow_name}
|
||||
└── agents.agent
|
||||
├── agents.generation
|
||||
├── agents.function
|
||||
├── agents.response
|
||||
└── agents.handoff
|
||||
```
|
||||
|
||||
## Semantic Conventions and Attributes
|
||||
|
||||
Each span type has attributes following OpenTelemetry semantic conventions:
|
||||
|
||||
### Common Attributes (All Spans)
|
||||
|
||||
- `trace.id`: OpenTelemetry trace ID
|
||||
- `span.id`: OpenTelemetry span ID
|
||||
- `parent.id`: Parent span ID (if applicable)
|
||||
- `instrumentation.name`: "agentops"
|
||||
- `instrumentation.version`: AgentOps library version
|
||||
- `instrumentation.library.name`: "openai_agents"
|
||||
- `instrumentation.library.version`: Library version
|
||||
|
||||
### Workflow and Trace Attributes
|
||||
|
||||
- `workflow.name`: Name of the workflow or trace
|
||||
- `workflow.step_type`: "trace" for trace spans
|
||||
- `workflow.input`: Input to the workflow
|
||||
- `workflow.final_output`: Final output from the workflow
|
||||
|
||||
### Agent Attributes
|
||||
|
||||
- `agent.name`: The name of the agent
|
||||
- `agent.tools`: Comma-separated list of available tools
|
||||
- `agent.handoffs`: Comma-separated list of handoff targets
|
||||
- `agent.from`: Source agent in handoffs (used in HandoffSpanData)
|
||||
- `agent.to`: Destination agent in handoffs (used in HandoffSpanData)
|
||||
|
||||
### LLM Attributes
|
||||
|
||||
- `gen_ai.system`: "openai" for all OpenAI spans
|
||||
- `gen_ai.request.model`: Model used for generation
|
||||
- `gen_ai.response.model`: Model that provided the response
|
||||
- `gen_ai.prompt`: Input prompt or message
|
||||
- `gen_ai.completion.0.role`: Role of the completion message (usually "assistant")
|
||||
- `gen_ai.completion.0.content`: Content of the completion message
|
||||
- `gen_ai.tool_call.0.0.name`: Name of the tool called (if applicable)
|
||||
- `gen_ai.tool_call.0.0.arguments`: Arguments for the tool call (if applicable)
|
||||
|
||||
### Token Usage Attributes
|
||||
|
||||
- `gen_ai.usage.prompt_tokens`: Number of input tokens
|
||||
- `gen_ai.usage.completion_tokens`: Number of output tokens
|
||||
- `gen_ai.usage.total_tokens`: Total number of tokens
|
||||
- `gen_ai.usage.reasoning_tokens`: Tokens used for reasoning (Response API)
|
||||
- `gen_ai.usage.cache_read.input_tokens`: Cached input tokens (Response API)
|
||||
|
||||
## Span Lifecycle Management
|
||||
|
||||
The exporter handles span lifecycle with these stages:
|
||||
|
||||
1. **Start Events**:
|
||||
- Create spans with `start_span()` (not using context manager)
|
||||
- Store span references in tracking dictionaries
|
||||
- Leave status as UNSET to indicate in-progress
|
||||
|
||||
2. **End Events**:
|
||||
- Look up existing span by ID
|
||||
- Update with final attributes
|
||||
- Set appropriate status and end the span manually
|
||||
|
||||
3. **Error Handling**:
|
||||
- Set status to ERROR for spans with errors
|
||||
- Add error type and message as attributes
|
||||
- Record exceptions with `record_exception()`
|
||||
|
||||
## OpenTelemetry Span Kinds
|
||||
|
||||
Span kinds map to OpenTelemetry concepts:
|
||||
|
||||
- `AgentSpanData` → `SpanKind.CONSUMER`
|
||||
- `FunctionSpanData` → `SpanKind.CLIENT`
|
||||
- `GenerationSpanData` → `SpanKind.CLIENT`
|
||||
- `ResponseSpanData` → `SpanKind.CLIENT`
|
||||
- `HandoffSpanData` → `SpanKind.INTERNAL`
|
||||
@@ -0,0 +1,144 @@
|
||||
# OpenAI Agents Tracing API Integration
|
||||
|
||||
This document provides an overview of how AgentOps integrates with the OpenAI Agents SDK tracing system.
|
||||
|
||||
## OpenAI Agents Tracing API Overview
|
||||
|
||||
The OpenAI Agents SDK provides a comprehensive tracing system that allows you to monitor and instrument agent activities. AgentOps integrates with this system to capture and forward trace data to its backend.
|
||||
|
||||
## Core Integration Methods
|
||||
|
||||
### 1. `add_trace_processor(processor)`
|
||||
|
||||
The main integration point that allows external systems like AgentOps to receive trace events:
|
||||
|
||||
```python
|
||||
from agents import add_trace_processor
|
||||
from agentops.instrumentation.openai_agents.processor import OpenAIAgentsProcessor
|
||||
|
||||
processor = OpenAIAgentsProcessor()
|
||||
add_trace_processor(processor)
|
||||
```
|
||||
|
||||
### 2. `set_trace_processors(processors)`
|
||||
|
||||
Replaces all current processors with a new list:
|
||||
|
||||
```python
|
||||
from agents import set_trace_processors
|
||||
set_trace_processors([my_processor1, my_processor2])
|
||||
```
|
||||
|
||||
### 3. `set_tracing_disabled(disabled)`
|
||||
|
||||
Globally enables/disables tracing:
|
||||
|
||||
```python
|
||||
from agents import set_tracing_disabled
|
||||
set_tracing_disabled(True) # Disable tracing
|
||||
```
|
||||
|
||||
### 4. `set_tracing_export_api_key(api_key)`
|
||||
|
||||
Sets the API key for the backend exporter:
|
||||
|
||||
```python
|
||||
from agents import set_tracing_export_api_key
|
||||
set_tracing_export_api_key("your-api-key")
|
||||
```
|
||||
|
||||
## Span Creation Methods
|
||||
|
||||
The SDK provides specialized methods for creating different types of spans:
|
||||
|
||||
1. **`agent_span(name, handoffs, tools, output_type, ...)`**
|
||||
- Creates spans for agent operations
|
||||
- Tracks agent name, available tools, potential handoffs
|
||||
|
||||
2. **`function_span(name, input, output, ...)`**
|
||||
- Creates spans for function/tool calls
|
||||
- Records function name, input arguments, and results
|
||||
|
||||
3. **`generation_span(input, output, model, model_config, usage, ...)`**
|
||||
- Creates spans for LLM generations
|
||||
- Records prompts, completions, model details, and token usage
|
||||
|
||||
4. **`response_span(response, ...)`**
|
||||
- Lightweight span for capturing OpenAI API response metadata
|
||||
|
||||
5. **`handoff_span(from_agent, to_agent, ...)`**
|
||||
- Tracks agent-to-agent handoffs
|
||||
|
||||
6. **`guardrail_span(name, triggered, ...)`**
|
||||
- Records guardrail evaluations
|
||||
|
||||
7. **`custom_span(name, data, ...)`**
|
||||
- Creates user-defined spans with arbitrary data
|
||||
|
||||
## Trace and Context Management
|
||||
|
||||
1. **`trace(workflow_name, trace_id, group_id, metadata, ...)`**
|
||||
- Creates and manages a trace context
|
||||
- Groups related spans into a logical trace/session
|
||||
|
||||
2. **`get_current_span()`**
|
||||
- Returns the current active span
|
||||
|
||||
3. **`get_current_trace()`**
|
||||
- Returns the current active trace
|
||||
|
||||
## How AgentOps Implements Integration
|
||||
|
||||
AgentOps integrates with this API through:
|
||||
|
||||
1. The `OpenAIAgentsProcessor` class that implements the `TracingProcessor` interface
|
||||
2. The `create_span` context manager that ensures proper parent-child relationships between spans
|
||||
3. The `AgentsInstrumentor` which registers the processor and adds additional instrumentation
|
||||
|
||||
This integration allows AgentOps to capture detailed information about agent execution, including:
|
||||
- Agent operations and tool usage
|
||||
- LLM requests and responses
|
||||
- Token usage metrics
|
||||
- Error information
|
||||
- Agent-to-agent handoffs
|
||||
|
||||
### Trace Context Propagation
|
||||
|
||||
Our implementation ensures proper parent-child relationships between spans through:
|
||||
|
||||
1. **Context Manager Pattern**: Using `start_as_current_span()` to maintain the OpenTelemetry span context
|
||||
2. **Parent Reference Tracking**: Storing parent span relationships and using them to create proper span hierarchies
|
||||
3. **Trace Correlation Attributes**: Adding consistent attributes to help with querying:
|
||||
- `agentops.original_trace_id`: Original trace ID from the Agents SDK
|
||||
- `agentops.original_span_id`: Original span ID from the Agents SDK
|
||||
- `agentops.parent_span_id`: Parent span ID for child spans
|
||||
- `agentops.trace_hash`: Consistent hash based on the original trace ID
|
||||
- `agentops.is_root_span`: "true" for spans without a parent
|
||||
|
||||
When querying spans for analysis:
|
||||
1. Group spans by `agentops.original_trace_id` to find all spans in the same trace
|
||||
2. Use `agentops.parent_span_id` to reconstruct the parent-child hierarchy
|
||||
|
||||
## Span Data Types
|
||||
|
||||
Several specialized span data types exist in the OpenAI Agents SDK to capture different operations:
|
||||
|
||||
- **AgentSpanData**: Captures agent execution data
|
||||
- **FunctionSpanData**: Records tool/function calls
|
||||
- **GenerationSpanData**: Records LLM generation details
|
||||
- **ResponseSpanData**: Captures model response information
|
||||
- **HandoffSpanData**: Tracks agent-to-agent handoffs
|
||||
- **GuardrailSpanData**: Records guardrail evaluations
|
||||
- **CustomSpanData**: For user-defined spans
|
||||
|
||||
## Processor Interface
|
||||
|
||||
The `TracingProcessor` interface defines methods processors must implement:
|
||||
- `on_trace_start`: Called when a trace begins
|
||||
- `on_trace_end`: Called when a trace ends
|
||||
- `on_span_start`: Called when a span begins
|
||||
- `on_span_end`: Called when a span completes
|
||||
- `shutdown`: Called during application shutdown
|
||||
- `force_flush`: Forces immediate processing of pending spans
|
||||
|
||||
The processor receives events from OpenAI Agents SDK's tracing system through these callback methods, translates them to OpenTelemetry spans, and sends them to the AgentOps backend for analysis and visualization.
|
||||
@@ -0,0 +1,30 @@
|
||||
"""
|
||||
AgentOps Instrumentor for OpenAI Agents SDK
|
||||
|
||||
This module provides automatic instrumentation for the OpenAI Agents SDK when AgentOps is imported.
|
||||
It implements a clean, maintainable implementation that follows semantic conventions.
|
||||
|
||||
IMPORTANT DISTINCTION BETWEEN OPENAI API FORMATS:
|
||||
1. OpenAI Completions API - The traditional API format using prompt_tokens/completion_tokens
|
||||
2. OpenAI Response API - The newer format used by the Agents SDK using input_tokens/output_tokens
|
||||
3. Agents SDK - The framework that uses Response API format
|
||||
|
||||
The Agents SDK uses the Response API format, which we handle using shared utilities from
|
||||
agentops.instrumentation.openai.
|
||||
"""
|
||||
|
||||
from agentops.instrumentation.common import LibraryInfo
|
||||
|
||||
# Library information
|
||||
_library_info = LibraryInfo(name="openai-agents")
|
||||
LIBRARY_NAME = _library_info.name
|
||||
LIBRARY_VERSION = _library_info.version
|
||||
|
||||
# Import after defining constants to avoid circular imports
|
||||
from agentops.instrumentation.agentic.openai_agents.instrumentor import OpenAIAgentsInstrumentor # noqa: E402
|
||||
|
||||
__all__ = [
|
||||
"LIBRARY_NAME",
|
||||
"LIBRARY_VERSION",
|
||||
"OpenAIAgentsInstrumentor",
|
||||
]
|
||||
@@ -0,0 +1,549 @@
|
||||
"""Common utilities and constants for attribute processing.
|
||||
|
||||
This module contains shared constants, attribute mappings, and utility functions for processing
|
||||
trace and span attributes in OpenAI Agents instrumentation. It provides the core functionality
|
||||
for extracting and formatting attributes according to OpenTelemetry semantic conventions.
|
||||
"""
|
||||
|
||||
from typing import Any, List, Dict, Optional
|
||||
from agentops.logging import logger
|
||||
from agentops.semconv import (
|
||||
AgentAttributes,
|
||||
WorkflowAttributes,
|
||||
SpanAttributes,
|
||||
InstrumentationAttributes,
|
||||
ToolAttributes,
|
||||
AgentOpsSpanKindValues,
|
||||
ToolStatus,
|
||||
)
|
||||
from agentops.helpers import safe_serialize # Import safe_serialize
|
||||
|
||||
from agentops.instrumentation.common import AttributeMap, _extract_attributes_from_mapping
|
||||
from agentops.instrumentation.common.attributes import get_common_attributes
|
||||
from agentops.instrumentation.common.objects import get_uploaded_object_attributes
|
||||
from agentops.instrumentation.providers.openai.attributes.response import get_response_response_attributes
|
||||
from agentops.instrumentation.agentic.openai_agents import LIBRARY_NAME, LIBRARY_VERSION
|
||||
|
||||
from agentops.instrumentation.agentic.openai_agents.attributes.model import (
|
||||
get_model_attributes,
|
||||
get_model_config_attributes,
|
||||
)
|
||||
from agentops.instrumentation.agentic.openai_agents.attributes.completion import get_generation_output_attributes
|
||||
|
||||
|
||||
# Attribute mapping for AgentSpanData
|
||||
AGENT_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
AgentAttributes.AGENT_NAME: "name",
|
||||
AgentAttributes.AGENT_TOOLS: "tools",
|
||||
AgentAttributes.HANDOFFS: "handoffs",
|
||||
WorkflowAttributes.WORKFLOW_INPUT: "input",
|
||||
WorkflowAttributes.WORKFLOW_OUTPUT: "output",
|
||||
}
|
||||
|
||||
|
||||
# Attribute mapping for FunctionSpanData
|
||||
FUNCTION_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
ToolAttributes.TOOL_NAME: "name",
|
||||
ToolAttributes.TOOL_PARAMETERS: "input",
|
||||
ToolAttributes.TOOL_RESULT: "output",
|
||||
# AgentAttributes.AGENT_NAME: "name",
|
||||
AgentAttributes.FROM_AGENT: "from_agent",
|
||||
}
|
||||
|
||||
|
||||
# Attribute mapping for HandoffSpanData
|
||||
HANDOFF_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
AgentAttributes.FROM_AGENT: "from_agent",
|
||||
AgentAttributes.TO_AGENT: "to_agent",
|
||||
}
|
||||
|
||||
|
||||
# Attribute mapping for GenerationSpanData
|
||||
GENERATION_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
SpanAttributes.LLM_PROMPTS: "input",
|
||||
}
|
||||
|
||||
|
||||
# Attribute mapping for ResponseSpanData
|
||||
RESPONSE_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
# Don't map input here as it causes double serialization
|
||||
# We handle prompts manually in get_response_span_attributes
|
||||
SpanAttributes.LLM_RESPONSE_MODEL: "model",
|
||||
}
|
||||
|
||||
|
||||
# Attribute mapping for TranscriptionSpanData
|
||||
TRANSCRIPTION_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
# `input` and `input_format` are handled below
|
||||
WorkflowAttributes.WORKFLOW_OUTPUT: "output",
|
||||
}
|
||||
|
||||
|
||||
# Attribute mapping for SpeechSpanData
|
||||
SPEECH_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
WorkflowAttributes.WORKFLOW_INPUT: "input",
|
||||
# `output` and `output_format` are handled below
|
||||
# TODO `first_content_at` is not converted
|
||||
}
|
||||
|
||||
|
||||
# Attribute mapping for SpeechGroupSpanData
|
||||
SPEECH_GROUP_SPAN_ATTRIBUTES: AttributeMap = {
|
||||
WorkflowAttributes.WORKFLOW_INPUT: "input",
|
||||
}
|
||||
|
||||
|
||||
def _get_llm_messages_attributes(messages: Optional[List[Dict]], attribute_base: str) -> AttributeMap:
|
||||
"""
|
||||
Extracts attributes from a list of message dictionaries (e.g., prompts or completions).
|
||||
Uses the attribute_base to format the specific attribute keys.
|
||||
"""
|
||||
attributes: AttributeMap = {}
|
||||
if not messages:
|
||||
return attributes
|
||||
if not isinstance(messages, list):
|
||||
logger.warning(
|
||||
f"[_get_llm_messages_attributes] Expected a list of messages for base '{attribute_base}', got {type(messages)}. Value: {safe_serialize(messages)}. Returning empty."
|
||||
)
|
||||
return attributes
|
||||
|
||||
for i, msg_dict in enumerate(messages):
|
||||
if isinstance(msg_dict, dict):
|
||||
role = msg_dict.get("role")
|
||||
content = msg_dict.get("content")
|
||||
name = msg_dict.get("name")
|
||||
tool_calls = msg_dict.get("tool_calls")
|
||||
tool_call_id = msg_dict.get("tool_call_id")
|
||||
|
||||
# Common role and content
|
||||
if role:
|
||||
attributes[f"{attribute_base}.{i}.role"] = str(role)
|
||||
if content is not None:
|
||||
attributes[f"{attribute_base}.{i}.content"] = safe_serialize(content)
|
||||
|
||||
# Optional name for some roles
|
||||
if name:
|
||||
attributes[f"{attribute_base}.{i}.name"] = str(name)
|
||||
|
||||
# Tool calls (specific to assistant messages)
|
||||
if tool_calls and isinstance(tool_calls, list):
|
||||
for tc_idx, tc_dict in enumerate(tool_calls):
|
||||
if isinstance(tc_dict, dict):
|
||||
tc_id = tc_dict.get("id")
|
||||
tc_type = tc_dict.get("type")
|
||||
tc_function_data = tc_dict.get("function")
|
||||
|
||||
if tc_function_data and isinstance(tc_function_data, dict):
|
||||
tc_func_name = tc_function_data.get("name")
|
||||
tc_func_args = tc_function_data.get("arguments")
|
||||
|
||||
base_tool_call_key_formatted = f"{attribute_base}.{i}.tool_calls.{tc_idx}"
|
||||
if tc_id:
|
||||
attributes[f"{base_tool_call_key_formatted}.id"] = str(tc_id)
|
||||
if tc_type:
|
||||
attributes[f"{base_tool_call_key_formatted}.type"] = str(tc_type)
|
||||
if tc_func_name:
|
||||
attributes[f"{base_tool_call_key_formatted}.function.name"] = str(tc_func_name)
|
||||
if tc_func_args is not None:
|
||||
attributes[f"{base_tool_call_key_formatted}.function.arguments"] = safe_serialize(
|
||||
tc_func_args
|
||||
)
|
||||
|
||||
# Tool call ID (specific to tool_call_output messages)
|
||||
if tool_call_id:
|
||||
attributes[f"{attribute_base}.{i}.tool_call_id"] = str(tool_call_id)
|
||||
else:
|
||||
# If a message is not a dict, serialize its representation
|
||||
attributes[f"{attribute_base}.{i}.content"] = safe_serialize(msg_dict)
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_common_instrumentation_attributes() -> AttributeMap:
|
||||
"""Get common instrumentation attributes for the OpenAI Agents instrumentation.
|
||||
|
||||
This combines the generic AgentOps attributes with OpenAI Agents specific library attributes.
|
||||
|
||||
Returns:
|
||||
Dictionary of common instrumentation attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
attributes.update(
|
||||
{
|
||||
InstrumentationAttributes.LIBRARY_NAME: LIBRARY_NAME,
|
||||
InstrumentationAttributes.LIBRARY_VERSION: LIBRARY_VERSION,
|
||||
}
|
||||
)
|
||||
return attributes
|
||||
|
||||
|
||||
def get_agent_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from an AgentSpanData object.
|
||||
|
||||
Agents are requests made to the `openai.agents` endpoint.
|
||||
|
||||
Args:
|
||||
span_data: The AgentSpanData object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for agent span
|
||||
"""
|
||||
attributes = {}
|
||||
attributes.update(get_common_attributes())
|
||||
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.AGENT.value
|
||||
|
||||
# Get agent name directly from span_data
|
||||
if hasattr(span_data, "name") and span_data.name:
|
||||
attributes[AgentAttributes.AGENT_NAME] = str(span_data.name)
|
||||
|
||||
# Get handoffs directly from span_data
|
||||
if hasattr(span_data, "handoffs") and span_data.handoffs:
|
||||
attributes[AgentAttributes.HANDOFFS] = safe_serialize(span_data.handoffs)
|
||||
|
||||
if hasattr(span_data, "tools") and span_data.tools:
|
||||
attributes[AgentAttributes.AGENT_TOOLS] = safe_serialize([str(getattr(t, "name", t)) for t in span_data.tools])
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_function_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from a FunctionSpanData object.
|
||||
|
||||
Functions are requests made to the `openai.functions` endpoint.
|
||||
|
||||
Args:
|
||||
span_data: The FunctionSpanData object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for function span
|
||||
"""
|
||||
attributes = _extract_attributes_from_mapping(span_data, FUNCTION_SPAN_ATTRIBUTES)
|
||||
attributes.update(get_common_attributes())
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.TOOL.value
|
||||
|
||||
# Determine tool status based on presence of error
|
||||
if hasattr(span_data, "error") and span_data.error:
|
||||
attributes[ToolAttributes.TOOL_STATUS] = ToolStatus.FAILED.value
|
||||
else:
|
||||
if hasattr(span_data, "output") and span_data.output is not None:
|
||||
attributes[ToolAttributes.TOOL_STATUS] = ToolStatus.SUCCEEDED.value
|
||||
else:
|
||||
# Status will be set by exporter based on span lifecycle
|
||||
pass
|
||||
|
||||
if hasattr(span_data, "from_agent") and span_data.from_agent:
|
||||
attributes["agent.calling_tool.name"] = str(span_data.from_agent)
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_handoff_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from a HandoffSpanData object.
|
||||
|
||||
Handoffs are requests made to the `openai.handoffs` endpoint.
|
||||
|
||||
Args:
|
||||
span_data: The HandoffSpanData object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for handoff span
|
||||
"""
|
||||
attributes = _extract_attributes_from_mapping(span_data, HANDOFF_SPAN_ATTRIBUTES)
|
||||
attributes.update(get_common_attributes())
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def _extract_text_from_content(content: Any) -> Optional[str]:
|
||||
"""Extract text from various content formats used in the Responses API.
|
||||
|
||||
Args:
|
||||
content: Content in various formats (str, dict, list)
|
||||
|
||||
Returns:
|
||||
Extracted text or None if no text found
|
||||
"""
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
|
||||
if isinstance(content, dict):
|
||||
# Direct text field
|
||||
if "text" in content:
|
||||
return content["text"]
|
||||
# Output text type
|
||||
if content.get("type") == "output_text":
|
||||
return content.get("text", "")
|
||||
|
||||
if isinstance(content, list):
|
||||
text_parts = []
|
||||
for item in content:
|
||||
extracted = _extract_text_from_content(item)
|
||||
if extracted:
|
||||
text_parts.append(extracted)
|
||||
return " ".join(text_parts) if text_parts else None
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _build_prompt_messages_from_input(input_data: Any) -> List[Dict[str, Any]]:
|
||||
"""Build prompt messages from various input formats.
|
||||
|
||||
Args:
|
||||
input_data: Input data from span_data.input
|
||||
|
||||
Returns:
|
||||
List of message dictionaries with role and content
|
||||
"""
|
||||
messages = []
|
||||
|
||||
if isinstance(input_data, str):
|
||||
# Single string input - assume it's a user message
|
||||
messages.append({"role": "user", "content": input_data})
|
||||
|
||||
elif isinstance(input_data, list):
|
||||
for msg in input_data:
|
||||
if isinstance(msg, dict):
|
||||
role = msg.get("role")
|
||||
content = msg.get("content")
|
||||
|
||||
if role and content is not None:
|
||||
extracted_text = _extract_text_from_content(content)
|
||||
if extracted_text:
|
||||
messages.append({"role": role, "content": extracted_text})
|
||||
|
||||
return messages
|
||||
|
||||
|
||||
def get_response_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from a ResponseSpanData object with full LLM response processing.
|
||||
|
||||
Responses are requests made to the `openai.responses` endpoint.
|
||||
|
||||
This function extracts not just the basic input/response mapping but also processes
|
||||
the rich response object to extract LLM-specific attributes like token usage,
|
||||
model information, content, etc.
|
||||
|
||||
TODO tool calls arrive from this span type; need to figure out why that is.
|
||||
|
||||
Args:
|
||||
span_data: The ResponseSpanData object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for response span
|
||||
"""
|
||||
# Get basic attributes from mapping
|
||||
attributes = _extract_attributes_from_mapping(span_data, RESPONSE_SPAN_ATTRIBUTES)
|
||||
attributes.update(get_common_attributes())
|
||||
|
||||
# Process response attributes first to get all response data including instructions
|
||||
if span_data.response:
|
||||
response_attrs = get_response_response_attributes(span_data.response)
|
||||
|
||||
# Extract system prompt if present
|
||||
system_prompt = response_attrs.get(SpanAttributes.LLM_OPENAI_RESPONSE_INSTRUCTIONS)
|
||||
|
||||
prompt_messages = []
|
||||
# Add system prompt as first message if available
|
||||
if system_prompt:
|
||||
prompt_messages.append({"role": "system", "content": system_prompt})
|
||||
# Remove from response attrs to avoid duplication
|
||||
response_attrs.pop(SpanAttributes.LLM_OPENAI_RESPONSE_INSTRUCTIONS, None)
|
||||
|
||||
# Add conversation history from input
|
||||
if hasattr(span_data, "input") and span_data.input:
|
||||
prompt_messages.extend(_build_prompt_messages_from_input(span_data.input))
|
||||
|
||||
# Format prompts using existing function
|
||||
if prompt_messages:
|
||||
attributes.update(_get_llm_messages_attributes(prompt_messages, "gen_ai.prompt"))
|
||||
|
||||
# Remove any prompt-related attributes that might have been set by response processing
|
||||
response_attrs = {
|
||||
k: v for k, v in response_attrs.items() if not k.startswith("gen_ai.prompt") and k != "gen_ai.request.tools"
|
||||
}
|
||||
|
||||
# Add remaining response attributes
|
||||
attributes.update(response_attrs)
|
||||
else:
|
||||
# No response object, just process input as prompts
|
||||
if hasattr(span_data, "input") and span_data.input:
|
||||
prompt_messages = _build_prompt_messages_from_input(span_data.input)
|
||||
if prompt_messages:
|
||||
attributes.update(_get_llm_messages_attributes(prompt_messages, "gen_ai.prompt"))
|
||||
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.LLM.value
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_generation_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from a GenerationSpanData object.
|
||||
|
||||
Generations are requests made to the `openai.completions` endpoint.
|
||||
|
||||
Args:
|
||||
span_data: The GenerationSpanData object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for generation span
|
||||
"""
|
||||
attributes = _extract_attributes_from_mapping(span_data, GENERATION_SPAN_ATTRIBUTES)
|
||||
attributes.update(get_common_attributes())
|
||||
|
||||
if SpanAttributes.LLM_PROMPTS in attributes:
|
||||
raw_prompt_input = attributes.pop(SpanAttributes.LLM_PROMPTS)
|
||||
formatted_prompt_for_llm = []
|
||||
if isinstance(raw_prompt_input, str):
|
||||
formatted_prompt_for_llm.append({"role": "user", "content": raw_prompt_input})
|
||||
elif isinstance(raw_prompt_input, list):
|
||||
temp_formatted_list = []
|
||||
all_strings_or_dicts = True
|
||||
for item in raw_prompt_input:
|
||||
if isinstance(item, str):
|
||||
temp_formatted_list.append({"role": "user", "content": item})
|
||||
elif isinstance(item, dict):
|
||||
temp_formatted_list.append(item)
|
||||
else:
|
||||
all_strings_or_dicts = False
|
||||
break
|
||||
if all_strings_or_dicts:
|
||||
formatted_prompt_for_llm = temp_formatted_list
|
||||
else:
|
||||
logger.warning(
|
||||
f"[get_generation_span_attributes] span_data.input was a list with mixed/unexpected content: {safe_serialize(raw_prompt_input)}"
|
||||
)
|
||||
|
||||
if formatted_prompt_for_llm:
|
||||
attributes.update(_get_llm_messages_attributes(formatted_prompt_for_llm, "gen_ai.prompt"))
|
||||
|
||||
if span_data.model:
|
||||
attributes.update(get_model_attributes(span_data.model))
|
||||
|
||||
if span_data.output:
|
||||
attributes.update(get_generation_output_attributes(span_data.output))
|
||||
|
||||
if span_data.model_config:
|
||||
attributes.update(get_model_config_attributes(span_data.model_config))
|
||||
|
||||
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.LLM.value
|
||||
return attributes
|
||||
|
||||
|
||||
def get_transcription_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from a TranscriptionSpanData object.
|
||||
|
||||
This represents a conversion from audio to text.
|
||||
|
||||
Args:
|
||||
span_data: The TranscriptionSpanData object
|
||||
Returns:
|
||||
Dictionary of attributes for transcription span
|
||||
"""
|
||||
from agentops import get_client
|
||||
from agentops.client.api.types import UploadedObjectResponse
|
||||
|
||||
client = get_client()
|
||||
|
||||
attributes = _extract_attributes_from_mapping(span_data, TRANSCRIPTION_SPAN_ATTRIBUTES)
|
||||
attributes.update(get_common_attributes())
|
||||
|
||||
if span_data.input:
|
||||
prefix = WorkflowAttributes.WORKFLOW_INPUT
|
||||
uploaded_object: UploadedObjectResponse = client.api.v4.upload_object(span_data.input)
|
||||
attributes.update(get_uploaded_object_attributes(uploaded_object, prefix))
|
||||
|
||||
if span_data.model:
|
||||
attributes.update(get_model_attributes(span_data.model))
|
||||
|
||||
if span_data.model_config:
|
||||
attributes.update(get_model_config_attributes(span_data.model_config))
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_speech_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from a SpeechSpanData object.
|
||||
|
||||
This represents a conversion from audio to text.
|
||||
|
||||
Args:
|
||||
span_data: The SpeechSpanData object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for speech span
|
||||
"""
|
||||
from agentops import get_client
|
||||
from agentops.client.api.types import UploadedObjectResponse
|
||||
|
||||
client = get_client()
|
||||
|
||||
attributes = _extract_attributes_from_mapping(span_data, SPEECH_SPAN_ATTRIBUTES)
|
||||
attributes.update(get_common_attributes())
|
||||
|
||||
if span_data.output:
|
||||
prefix = WorkflowAttributes.WORKFLOW_OUTPUT
|
||||
uploaded_object: UploadedObjectResponse = client.api.v4.upload_object(span_data.output)
|
||||
attributes.update(get_uploaded_object_attributes(uploaded_object, prefix))
|
||||
|
||||
if span_data.model:
|
||||
attributes.update(get_model_attributes(span_data.model))
|
||||
|
||||
if span_data.model_config:
|
||||
attributes.update(get_model_config_attributes(span_data.model_config))
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_speech_group_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Extract attributes from a SpeechGroupSpanData object.
|
||||
|
||||
This represents a conversion from audio to text.
|
||||
|
||||
Args:
|
||||
span_data: The SpeechGroupSpanData object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for speech group span
|
||||
"""
|
||||
attributes = _extract_attributes_from_mapping(span_data, SPEECH_GROUP_SPAN_ATTRIBUTES)
|
||||
attributes.update(get_common_attributes())
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_span_attributes(span_data: Any) -> AttributeMap:
|
||||
"""Get attributes for a span based on its type.
|
||||
|
||||
This function centralizes attribute extraction by delegating to type-specific
|
||||
getter functions.
|
||||
|
||||
Args:
|
||||
span_data: The span data object
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes for the span
|
||||
"""
|
||||
span_type = span_data.__class__.__name__
|
||||
|
||||
if span_type == "AgentSpanData":
|
||||
attributes = get_agent_span_attributes(span_data)
|
||||
elif span_type == "FunctionSpanData":
|
||||
attributes = get_function_span_attributes(span_data)
|
||||
elif span_type == "GenerationSpanData":
|
||||
attributes = get_generation_span_attributes(span_data)
|
||||
elif span_type == "HandoffSpanData":
|
||||
attributes = get_handoff_span_attributes(span_data)
|
||||
elif span_type == "ResponseSpanData":
|
||||
attributes = get_response_span_attributes(span_data)
|
||||
elif span_type == "TranscriptionSpanData":
|
||||
attributes = get_transcription_span_attributes(span_data)
|
||||
elif span_type == "SpeechSpanData":
|
||||
attributes = get_speech_span_attributes(span_data)
|
||||
elif span_type == "SpeechGroupSpanData":
|
||||
attributes = get_speech_group_span_attributes(span_data)
|
||||
else:
|
||||
logger.debug(f"[agentops.instrumentation.openai_agents.attributes] Unknown span type: {span_type}")
|
||||
attributes = {}
|
||||
|
||||
return attributes
|
||||
@@ -0,0 +1,172 @@
|
||||
"""Completion processing utilities for OpenAI Agents instrumentation.
|
||||
|
||||
This module handles completion content processing from both the Chat Completions API
|
||||
and the OpenAI Response API formats, extracting messages, tool calls, function calls, etc.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict
|
||||
|
||||
from agentops.instrumentation.common.attributes import AttributeMap
|
||||
|
||||
from agentops.logging import logger
|
||||
from agentops.helpers.serialization import model_to_dict
|
||||
from agentops.semconv import (
|
||||
SpanAttributes,
|
||||
MessageAttributes,
|
||||
)
|
||||
from agentops.instrumentation.agentic.openai_agents.attributes.tokens import process_token_usage
|
||||
|
||||
|
||||
def get_generation_output_attributes(output: Any) -> Dict[str, Any]:
|
||||
"""Extract LLM response attributes from an `openai/completions` object.
|
||||
|
||||
Args:
|
||||
output: The response object (can be dict, Response object, or other format)
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes extracted from the response in a consistent format
|
||||
"""
|
||||
# Convert model to dictionary for easier processing
|
||||
response_dict = model_to_dict(output)
|
||||
result: AttributeMap = {}
|
||||
|
||||
if not response_dict:
|
||||
# Handle output as string if it's not a dict
|
||||
if isinstance(output, str):
|
||||
# For string output, just return the minimal set of attributes
|
||||
return {}
|
||||
return result
|
||||
|
||||
# Check for OpenAI Agents SDK response format (has raw_responses array)
|
||||
if "raw_responses" in response_dict and isinstance(response_dict["raw_responses"], list):
|
||||
result.update(get_raw_response_attributes(response_dict))
|
||||
else:
|
||||
# TODO base attributes for completion type
|
||||
|
||||
# Get completions or response API output attributes first
|
||||
if "choices" in response_dict:
|
||||
result.update(get_chat_completions_attributes(response_dict))
|
||||
|
||||
# Extract token usage from dictionary for standard formats
|
||||
usage_attributes: AttributeMap = {}
|
||||
if "usage" in response_dict:
|
||||
process_token_usage(response_dict["usage"], usage_attributes)
|
||||
result.update(usage_attributes)
|
||||
|
||||
# Extract token usage from Response object directly if dict conversion didn't work
|
||||
if hasattr(output, "usage") and output.usage:
|
||||
direct_usage_attributes: AttributeMap = {}
|
||||
process_token_usage(output.usage, direct_usage_attributes)
|
||||
result.update(direct_usage_attributes)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def get_raw_response_attributes(response: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Extract attributes from OpenAI Agents SDK response format (with raw_responses).
|
||||
|
||||
This function handles the specific structure of OpenAI Agents SDK responses,
|
||||
which include a raw_responses array containing the actual API responses.
|
||||
This is the format used specifically by the Agents SDK, not the standard OpenAI API.
|
||||
|
||||
Args:
|
||||
response: The OpenAI Agents SDK response dictionary (containing raw_responses array)
|
||||
|
||||
Returns:
|
||||
Dictionary of attributes extracted from the Agents SDK response
|
||||
"""
|
||||
result: AttributeMap = {}
|
||||
|
||||
# Set the LLM system to OpenAI
|
||||
result[SpanAttributes.LLM_SYSTEM] = "openai"
|
||||
|
||||
# Process raw responses
|
||||
if "raw_responses" in response and isinstance(response["raw_responses"], list):
|
||||
for i, raw_response in enumerate(response["raw_responses"]):
|
||||
# Extract token usage from the first raw response
|
||||
if "usage" in raw_response and isinstance(raw_response["usage"], dict):
|
||||
usage_attrs: AttributeMap = {}
|
||||
process_token_usage(raw_response["usage"], usage_attrs)
|
||||
result.update(usage_attrs)
|
||||
logger.debug(f"Extracted token usage from raw_responses[{i}]: {usage_attrs}")
|
||||
|
||||
# Extract output content
|
||||
if "output" in raw_response and isinstance(raw_response["output"], list):
|
||||
for j, output_item in enumerate(raw_response["output"]):
|
||||
# Process message content
|
||||
if "content" in output_item and isinstance(output_item["content"], list):
|
||||
for content_item in output_item["content"]:
|
||||
if content_item.get("type") == "output_text" and "text" in content_item:
|
||||
# Set message content attribute using the standard convention
|
||||
result[MessageAttributes.COMPLETION_CONTENT.format(i=j)] = content_item["text"]
|
||||
|
||||
# Process role
|
||||
if "role" in output_item:
|
||||
result[MessageAttributes.COMPLETION_ROLE.format(i=j)] = output_item["role"]
|
||||
|
||||
# Process tool calls
|
||||
if "tool_calls" in output_item and isinstance(output_item["tool_calls"], list):
|
||||
for k, tool_call in enumerate(output_item["tool_calls"]):
|
||||
tool_id = tool_call.get("id", "")
|
||||
# Handle function format
|
||||
if "function" in tool_call and isinstance(tool_call["function"], dict):
|
||||
function = tool_call["function"]
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=j, j=k)] = tool_id
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=j, j=k)] = function.get(
|
||||
"name", ""
|
||||
)
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=j, j=k)] = (
|
||||
function.get("arguments", "")
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def get_chat_completions_attributes(response: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Get attributes from OpenAI Chat Completions API format (with choices array).
|
||||
|
||||
This function specifically handles the original Chat Completions API format
|
||||
that uses a 'choices' array with 'message' objects, as opposed to the newer
|
||||
Response API format that uses an 'output' array.
|
||||
|
||||
Args:
|
||||
response: The response dictionary containing chat completions (with choices array)
|
||||
|
||||
Returns:
|
||||
Dictionary of chat completion attributes
|
||||
"""
|
||||
result: AttributeMap = {}
|
||||
|
||||
if "choices" not in response:
|
||||
return result
|
||||
|
||||
for i, choice in enumerate(response["choices"]):
|
||||
if "finish_reason" in choice:
|
||||
result[MessageAttributes.COMPLETION_FINISH_REASON.format(i=i)] = choice["finish_reason"]
|
||||
|
||||
message = choice.get("message", {})
|
||||
|
||||
if "role" in message:
|
||||
result[MessageAttributes.COMPLETION_ROLE.format(i=i)] = message["role"]
|
||||
|
||||
if "content" in message:
|
||||
content = message["content"] if message["content"] is not None else ""
|
||||
result[MessageAttributes.COMPLETION_CONTENT.format(i=i)] = content
|
||||
|
||||
if "tool_calls" in message and message["tool_calls"] is not None:
|
||||
tool_calls = message["tool_calls"]
|
||||
for j, tool_call in enumerate(tool_calls):
|
||||
if "function" in tool_call:
|
||||
function = tool_call["function"]
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=i, j=j)] = tool_call.get("id")
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=i, j=j)] = function.get("name")
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=i, j=j)] = function.get(
|
||||
"arguments"
|
||||
)
|
||||
|
||||
if "function_call" in message and message["function_call"] is not None:
|
||||
function_call = message["function_call"]
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=i)] = function_call.get("name")
|
||||
result[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=i)] = function_call.get("arguments")
|
||||
|
||||
return result
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Model information extraction for OpenAI Agents instrumentation.
|
||||
|
||||
This module provides utilities for extracting model information and parameters
|
||||
from various object types, centralizing model attribute handling logic.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict
|
||||
from agentops.semconv import SpanAttributes
|
||||
from agentops.instrumentation.common.attributes import AttributeMap, _extract_attributes_from_mapping
|
||||
|
||||
|
||||
# Parameter mapping dictionary for model parameters
|
||||
MODEL_CONFIG_ATTRIBUTES: AttributeMap = {
|
||||
SpanAttributes.LLM_REQUEST_TEMPERATURE: "temperature",
|
||||
SpanAttributes.LLM_REQUEST_TOP_P: "top_p",
|
||||
SpanAttributes.LLM_REQUEST_FREQUENCY_PENALTY: "frequency_penalty",
|
||||
SpanAttributes.LLM_REQUEST_PRESENCE_PENALTY: "presence_penalty",
|
||||
SpanAttributes.LLM_REQUEST_MAX_TOKENS: "max_tokens",
|
||||
# TODO we need to establish semantic conventions for the following:
|
||||
# tool_choice
|
||||
# parallel_tool_calls
|
||||
# truncation
|
||||
# store
|
||||
# language
|
||||
# prompt
|
||||
# turn_detection
|
||||
SpanAttributes.LLM_REQUEST_INSTRUCTIONS: "instructions",
|
||||
SpanAttributes.LLM_REQUEST_VOICE: "voice",
|
||||
SpanAttributes.LLM_REQUEST_SPEED: "speed",
|
||||
}
|
||||
|
||||
|
||||
def get_model_attributes(model_name: str) -> Dict[str, Any]:
|
||||
"""Get model name attributes for both request and response for consistency.
|
||||
|
||||
Args:
|
||||
model_name: The model name to set
|
||||
|
||||
Returns:
|
||||
Dictionary of model name attributes
|
||||
"""
|
||||
return {
|
||||
SpanAttributes.LLM_REQUEST_MODEL: model_name,
|
||||
SpanAttributes.LLM_RESPONSE_MODEL: model_name,
|
||||
SpanAttributes.LLM_SYSTEM: "openai",
|
||||
}
|
||||
|
||||
|
||||
def get_model_config_attributes(model_config: Any) -> Dict[str, Any]:
|
||||
"""Extract model configuration attributes using the model parameter mapping.
|
||||
|
||||
Args:
|
||||
model_config: The model configuration object
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted model configuration attributes
|
||||
"""
|
||||
return _extract_attributes_from_mapping(model_config, MODEL_CONFIG_ATTRIBUTES)
|
||||
@@ -0,0 +1,275 @@
|
||||
"""Token processing and metrics for the OpenAI Agents instrumentation.
|
||||
|
||||
This module contains functions for processing token usage data from OpenAI responses,
|
||||
including standardized handling of different API formats (Chat Completions API vs Response API)
|
||||
and recording token usage metrics.
|
||||
"""
|
||||
|
||||
import json
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from agentops.semconv import SpanAttributes
|
||||
from agentops.logging import logger
|
||||
|
||||
|
||||
def safe_parse(content: str) -> Optional[Dict[str, Any]]:
|
||||
"""Safely parse JSON content from a string.
|
||||
|
||||
Args:
|
||||
content: String content that might contain JSON
|
||||
|
||||
Returns:
|
||||
Parsed dictionary if content is valid JSON, None otherwise
|
||||
"""
|
||||
if not isinstance(content, str):
|
||||
return None
|
||||
|
||||
try:
|
||||
# Try to parse the string as JSON
|
||||
return json.loads(content)
|
||||
except (json.JSONDecodeError, TypeError, ValueError):
|
||||
# If parsing fails, log a debug message and return None
|
||||
logger.debug(f"Failed to parse JSON content: {content[:100]}...")
|
||||
return None
|
||||
|
||||
|
||||
def extract_nested_usage(content: Any) -> Optional[Dict[str, Any]]:
|
||||
"""Recursively extract usage data from potentially nested response structures.
|
||||
|
||||
Handles multiple nesting patterns:
|
||||
1. Direct usage field at the top level
|
||||
2. Usage nested in completion content JSON string
|
||||
3. Usage nested in response.output[].content[].text
|
||||
|
||||
Args:
|
||||
content: Any content object that might contain usage data
|
||||
|
||||
Returns:
|
||||
Extracted usage dictionary or None if not found
|
||||
"""
|
||||
# Case: direct dictionary with usage field
|
||||
if isinstance(content, dict) and "usage" in content:
|
||||
return content["usage"]
|
||||
|
||||
# Case: JSON string that might contain usage
|
||||
if isinstance(content, str):
|
||||
parsed_data = safe_parse(content)
|
||||
if parsed_data:
|
||||
# Direct usage field in parsed JSON
|
||||
if "usage" in parsed_data and isinstance(parsed_data["usage"], dict):
|
||||
return parsed_data["usage"]
|
||||
|
||||
# Response API format with nested output structure
|
||||
if "output" in parsed_data and isinstance(parsed_data["output"], list):
|
||||
# Usage at top level in Response format
|
||||
if "usage" in parsed_data:
|
||||
return parsed_data["usage"]
|
||||
|
||||
# Case: complex nested structure with output array
|
||||
# This handles the Response API format where usage is at the top level
|
||||
if isinstance(content, dict):
|
||||
if "output" in content and isinstance(content["output"], list):
|
||||
if "usage" in content:
|
||||
return content["usage"]
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def process_token_usage(
|
||||
usage: Dict[str, Any], attributes: Dict[str, Any], completion_content: Optional[str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""Process token usage data from OpenAI responses using standardized attribute naming.
|
||||
|
||||
Args:
|
||||
usage: Dictionary containing token usage data
|
||||
attributes: Dictionary where attributes will be set
|
||||
completion_content: Optional JSON string that may contain token usage info
|
||||
|
||||
Returns:
|
||||
Dictionary mapping token types to counts for metrics
|
||||
"""
|
||||
# Result dictionary for metric recording
|
||||
result = {}
|
||||
|
||||
# If usage is empty or None, use completion_content to find usage data
|
||||
if not usage or (isinstance(usage, dict) and len(usage) == 0):
|
||||
if completion_content:
|
||||
logger.debug("TOKENS: Usage is empty, trying to extract from completion content")
|
||||
extracted_usage = extract_nested_usage(completion_content)
|
||||
if extracted_usage:
|
||||
usage = extracted_usage
|
||||
|
||||
# Always set token usage attributes directly on the span to ensure they're captured
|
||||
# For both Chat Completions API and Response API formats
|
||||
|
||||
# Helper to get an attribute from either a dict or an object
|
||||
def get_value(obj, key):
|
||||
if isinstance(obj, dict) and key in obj:
|
||||
return obj[key]
|
||||
elif hasattr(obj, key):
|
||||
return getattr(obj, key)
|
||||
return None
|
||||
|
||||
# Helper to check if an object has an attribute
|
||||
def has_key(obj, key):
|
||||
if isinstance(obj, dict):
|
||||
return key in obj
|
||||
return hasattr(obj, key)
|
||||
|
||||
# Process prompt/input tokens
|
||||
if has_key(usage, "prompt_tokens"):
|
||||
prompt_tokens = get_value(usage, "prompt_tokens")
|
||||
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = prompt_tokens
|
||||
result["prompt_tokens"] = prompt_tokens
|
||||
elif has_key(usage, "input_tokens"):
|
||||
input_tokens = get_value(usage, "input_tokens")
|
||||
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = input_tokens
|
||||
result["prompt_tokens"] = input_tokens
|
||||
|
||||
# Process completion/output tokens
|
||||
if has_key(usage, "completion_tokens"):
|
||||
completion_tokens = get_value(usage, "completion_tokens")
|
||||
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = completion_tokens
|
||||
result["completion_tokens"] = completion_tokens
|
||||
elif has_key(usage, "output_tokens"):
|
||||
output_tokens = get_value(usage, "output_tokens")
|
||||
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = output_tokens
|
||||
result["completion_tokens"] = output_tokens
|
||||
|
||||
# Process total tokens
|
||||
if has_key(usage, "total_tokens"):
|
||||
total_tokens = get_value(usage, "total_tokens")
|
||||
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = total_tokens
|
||||
result["total_tokens"] = total_tokens
|
||||
|
||||
# Process Response API specific token details using defined semantic conventions
|
||||
|
||||
# Process reasoning tokens (from Response API output_tokens_details)
|
||||
output_tokens_details = None
|
||||
if has_key(usage, "output_tokens_details"):
|
||||
output_tokens_details = get_value(usage, "output_tokens_details")
|
||||
|
||||
if output_tokens_details:
|
||||
# Handle both dict and object types
|
||||
if isinstance(output_tokens_details, dict):
|
||||
details = output_tokens_details
|
||||
if "reasoning_tokens" in details:
|
||||
attributes[SpanAttributes.LLM_USAGE_REASONING_TOKENS] = details["reasoning_tokens"]
|
||||
result["reasoning_tokens"] = details["reasoning_tokens"]
|
||||
elif hasattr(output_tokens_details, "reasoning_tokens"):
|
||||
reasoning_tokens = output_tokens_details.reasoning_tokens
|
||||
attributes[SpanAttributes.LLM_USAGE_REASONING_TOKENS] = reasoning_tokens
|
||||
result["reasoning_tokens"] = reasoning_tokens
|
||||
|
||||
# Process cached tokens (from Response API input_tokens_details)
|
||||
input_tokens_details = None
|
||||
if has_key(usage, "input_tokens_details"):
|
||||
input_tokens_details = get_value(usage, "input_tokens_details")
|
||||
|
||||
if input_tokens_details:
|
||||
# Handle both dict and object types
|
||||
if isinstance(input_tokens_details, dict):
|
||||
details = input_tokens_details
|
||||
if "cached_tokens" in details:
|
||||
attributes[SpanAttributes.LLM_USAGE_CACHE_READ_INPUT_TOKENS] = details["cached_tokens"]
|
||||
result["cached_input_tokens"] = details["cached_tokens"]
|
||||
# Handle object with cached_tokens attribute
|
||||
elif hasattr(input_tokens_details, "cached_tokens"):
|
||||
cached_tokens = input_tokens_details.cached_tokens
|
||||
attributes[SpanAttributes.LLM_USAGE_CACHE_READ_INPUT_TOKENS] = cached_tokens
|
||||
result["cached_input_tokens"] = cached_tokens
|
||||
|
||||
# Log all token-related attributes that were set
|
||||
token_attrs = {k: v for k, v in attributes.items() if k.startswith("gen_ai.usage")}
|
||||
|
||||
# If we still have no token attributes, try one more approach - look for nested output structure
|
||||
if not token_attrs and completion_content:
|
||||
try:
|
||||
# Parse the completion content to see if we can find more deeply nested usage data
|
||||
parsed_content = safe_parse(completion_content)
|
||||
if parsed_content and isinstance(parsed_content, dict):
|
||||
# If this is a Response API format, check for nested output structure
|
||||
if "output" in parsed_content and isinstance(parsed_content["output"], list):
|
||||
for output_item in parsed_content["output"]:
|
||||
# Check if this has nested content with usage
|
||||
if "content" in output_item and isinstance(output_item["content"], list):
|
||||
for content_item in output_item["content"]:
|
||||
if "text" in content_item:
|
||||
# Try to parse this text for usage data
|
||||
parsed_text = safe_parse(content_item["text"])
|
||||
if parsed_text and "usage" in parsed_text:
|
||||
logger.debug(f"Found deeply nested usage data: {parsed_text['usage']}")
|
||||
# Process this usage data recursively
|
||||
return process_token_usage(parsed_text["usage"], attributes)
|
||||
except Exception as e:
|
||||
logger.debug(f"Error during deep token extraction: {e}")
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def map_token_type_to_metric_name(token_type: str) -> str:
|
||||
"""Maps token type names from SpanAttributes to simplified metric names.
|
||||
|
||||
Args:
|
||||
token_type: Token type name, could be a full semantic convention or a simple name
|
||||
|
||||
Returns:
|
||||
Simplified token type name for metrics
|
||||
"""
|
||||
# If token_type is a semantic convention (contains a dot), extract the last part
|
||||
if isinstance(token_type, str) and "." in token_type:
|
||||
parts = token_type.split(".")
|
||||
token_type = parts[-1]
|
||||
|
||||
# Map to simplified metric names
|
||||
if token_type == "prompt_tokens":
|
||||
return "input"
|
||||
elif token_type == "completion_tokens":
|
||||
return "output"
|
||||
elif token_type == "reasoning_tokens":
|
||||
return "reasoning"
|
||||
|
||||
# Return as-is if no mapping needed
|
||||
return token_type
|
||||
|
||||
|
||||
def get_token_metric_attributes(usage: Dict[str, Any], model_name: str) -> Dict[str, Dict[str, Any]]:
|
||||
"""Get token usage metric attributes from usage data.
|
||||
|
||||
Args:
|
||||
usage: Dictionary containing token usage data
|
||||
model_name: Name of the model used
|
||||
|
||||
Returns:
|
||||
Dictionary mapping token types to metric data including value and attributes
|
||||
"""
|
||||
# Process all token types using our standardized processor
|
||||
token_counts = process_token_usage(usage, {})
|
||||
|
||||
# Common attributes for all metrics
|
||||
common_attributes = {
|
||||
"model": model_name,
|
||||
SpanAttributes.LLM_REQUEST_MODEL: model_name,
|
||||
SpanAttributes.LLM_SYSTEM: "openai",
|
||||
}
|
||||
|
||||
# Prepare metrics data for each token type
|
||||
metrics_data = {}
|
||||
for token_type, count in token_counts.items():
|
||||
# Skip if no count
|
||||
if not count:
|
||||
continue
|
||||
|
||||
# Map token type to simplified metric name
|
||||
metric_token_type = map_token_type_to_metric_name(token_type)
|
||||
|
||||
# Prepare the metric data
|
||||
metrics_data[token_type] = {
|
||||
"value": count,
|
||||
"attributes": {
|
||||
"token_type": metric_token_type,
|
||||
**common_attributes,
|
||||
},
|
||||
}
|
||||
|
||||
return metrics_data
|
||||
@@ -0,0 +1,469 @@
|
||||
"""OpenAI Agents SDK Instrumentation Exporter for AgentOps
|
||||
|
||||
This module handles the conversion of Agents SDK spans to OpenTelemetry spans.
|
||||
It manages the complete span lifecycle, attribute application, and proper span hierarchy.
|
||||
|
||||
See the README.md in this directory for complete documentation on:
|
||||
- Span lifecycle management approach
|
||||
- Serialization rules for attributes
|
||||
- Structured attribute handling
|
||||
- Semantic conventions usage
|
||||
|
||||
IMPORTANT FOR TESTING:
|
||||
- Tests should verify attribute existence using MessageAttributes constants
|
||||
- Do not check for the presence of SpanAttributes.LLM_COMPLETIONS
|
||||
- Verify individual content/tool attributes instead of root attributes
|
||||
"""
|
||||
|
||||
import json
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from opentelemetry import trace, context as context_api
|
||||
from opentelemetry.trace import get_tracer, SpanKind, Status, StatusCode, NonRecordingSpan
|
||||
from opentelemetry import trace as trace_api
|
||||
from opentelemetry.sdk.trace import Span
|
||||
|
||||
from agentops.logging import logger
|
||||
from agentops.semconv import (
|
||||
CoreAttributes,
|
||||
)
|
||||
|
||||
from agentops.instrumentation.common.attributes import (
|
||||
get_base_trace_attributes,
|
||||
get_base_span_attributes,
|
||||
)
|
||||
|
||||
from agentops.instrumentation.agentic.openai_agents import LIBRARY_NAME, LIBRARY_VERSION
|
||||
from agentops.instrumentation.agentic.openai_agents.attributes.common import (
|
||||
get_span_attributes,
|
||||
)
|
||||
|
||||
|
||||
def log_otel_trace_id(span_type):
|
||||
"""Log the OpenTelemetry trace ID for debugging and correlation purposes.
|
||||
|
||||
The hexadecimal OTel trace ID is essential for querying the backend database
|
||||
and correlating local debugging logs with server-side trace data. This ID
|
||||
is different from the Agents SDK trace_id and is the primary key used in
|
||||
observability systems and the AgentOps dashboard.
|
||||
|
||||
This function retrieves the current OpenTelemetry trace ID directly from the
|
||||
active span context and formats it as a 32-character hex string.
|
||||
|
||||
Args:
|
||||
span_type: The type of span being exported for logging context
|
||||
|
||||
Returns:
|
||||
str or None: The OpenTelemetry trace ID as a hex string, or None if unavailable
|
||||
"""
|
||||
current_span = trace.get_current_span()
|
||||
if hasattr(current_span, "get_span_context"):
|
||||
ctx = current_span.get_span_context()
|
||||
if hasattr(ctx, "trace_id") and ctx.trace_id:
|
||||
# Convert trace_id to 32-character hex string as shown in the API
|
||||
otel_trace_id = f"{ctx.trace_id:032x}" if isinstance(ctx.trace_id, int) else str(ctx.trace_id)
|
||||
logger.debug(f"[SPAN] Export | Type: {span_type} | TRACE ID: {otel_trace_id}")
|
||||
return otel_trace_id
|
||||
|
||||
logger.debug(f"[SPAN] Export | Type: {span_type} | NO TRACE ID AVAILABLE")
|
||||
return None
|
||||
|
||||
|
||||
def get_span_kind(span: Any) -> SpanKind:
|
||||
"""Determine the appropriate span kind based on span type."""
|
||||
span_data = span.span_data
|
||||
span_type = span_data.__class__.__name__
|
||||
|
||||
if span_type == "AgentSpanData":
|
||||
return SpanKind.CONSUMER
|
||||
elif span_type in ["FunctionSpanData", "GenerationSpanData", "ResponseSpanData"]:
|
||||
return SpanKind.CLIENT
|
||||
else:
|
||||
return SpanKind.INTERNAL
|
||||
|
||||
|
||||
def get_span_name(span: Any) -> str:
|
||||
"""Get the name of the span based on its type and attributes."""
|
||||
span_data = span.span_data
|
||||
span_type = span_data.__class__.__name__
|
||||
|
||||
if hasattr(span_data, "name") and span_data.name:
|
||||
return span_data.name
|
||||
else:
|
||||
return span_type.replace("SpanData", "").lower() # fallback
|
||||
|
||||
|
||||
def _get_span_lookup_key(trace_id: str, span_id: str) -> str:
|
||||
"""Generate a unique lookup key for spans based on trace and span IDs.
|
||||
|
||||
This key is used to track spans in the exporter and allows for efficient
|
||||
lookups and management of spans during their lifecycle.
|
||||
|
||||
Args:
|
||||
trace_id: The trace ID for the current span
|
||||
span_id: The span ID for the current span
|
||||
|
||||
Returns:
|
||||
str: A unique lookup key for the span
|
||||
"""
|
||||
return f"span:{trace_id}:{span_id}"
|
||||
|
||||
|
||||
class OpenAIAgentsExporter:
|
||||
"""Exporter for Agents SDK traces and spans that forwards them to OpenTelemetry.
|
||||
|
||||
This exporter is responsible for:
|
||||
1. Creating and configuring spans
|
||||
2. Setting span attributes based on data from the processor
|
||||
3. Managing the span lifecycle
|
||||
4. Using semantic conventions for attribute naming
|
||||
5. Interacting with the OpenTelemetry API
|
||||
6. Tracking spans to allow updating them when tasks complete
|
||||
"""
|
||||
|
||||
def __init__(self, tracer_provider=None):
|
||||
self.tracer_provider = tracer_provider
|
||||
# Dictionary to track active spans by their SDK span ID
|
||||
# Allows us to reference spans later during task completion
|
||||
self._active_spans = {}
|
||||
# Dictionary to track spans by trace/span ID for faster lookups
|
||||
self._span_map = {}
|
||||
|
||||
def export_trace(self, trace: Any) -> None:
|
||||
"""
|
||||
Handle exporting the trace.
|
||||
"""
|
||||
tracer = get_tracer(LIBRARY_NAME, LIBRARY_VERSION, self.tracer_provider)
|
||||
trace_id = getattr(trace, "trace_id", "unknown")
|
||||
|
||||
if not hasattr(trace, "trace_id"):
|
||||
logger.debug("Cannot export trace: missing trace_id")
|
||||
return
|
||||
|
||||
# Determine if this is a trace end event using status field
|
||||
# We use the status field to determine if this is an end event
|
||||
is_end_event = hasattr(trace, "status") and trace.status == StatusCode.OK.name
|
||||
trace_lookup_key = _get_span_lookup_key(trace_id, trace_id)
|
||||
attributes = get_base_trace_attributes(trace)
|
||||
|
||||
# For end events, check if we already have the span
|
||||
if is_end_event and trace_lookup_key in self._span_map:
|
||||
existing_span = self._span_map[trace_lookup_key]
|
||||
|
||||
span_is_ended = False
|
||||
if isinstance(existing_span, Span) and hasattr(existing_span, "_end_time"):
|
||||
span_is_ended = existing_span._end_time is not None
|
||||
|
||||
if not span_is_ended:
|
||||
# Update with core attributes
|
||||
for key, value in attributes.items():
|
||||
existing_span.set_attribute(key, value)
|
||||
|
||||
# Handle error if present
|
||||
if hasattr(trace, "error") and trace.error:
|
||||
self._handle_span_error(trace, existing_span)
|
||||
# Set status to OK if no error
|
||||
else:
|
||||
existing_span.set_status(Status(StatusCode.OK))
|
||||
|
||||
existing_span.end()
|
||||
|
||||
# Clean up our tracking resources
|
||||
self._active_spans.pop(trace_id, None)
|
||||
self._span_map.pop(trace_lookup_key, None)
|
||||
return
|
||||
|
||||
# Create span directly instead of using context manager
|
||||
span = tracer.start_span(name=trace.name, kind=SpanKind.INTERNAL, attributes=attributes)
|
||||
|
||||
# Add any additional trace attributes
|
||||
if hasattr(trace, "group_id") and trace.group_id:
|
||||
span.set_attribute(CoreAttributes.GROUP_ID, trace.group_id)
|
||||
|
||||
if hasattr(trace, "metadata") and trace.metadata:
|
||||
for key, value in trace.metadata.items():
|
||||
if isinstance(value, (str, int, float, bool)):
|
||||
span.set_attribute(f"trace.metadata.{key}", value)
|
||||
|
||||
# Record error if present
|
||||
if hasattr(trace, "error") and trace.error:
|
||||
self._handle_span_error(trace, span)
|
||||
|
||||
# For start events, store the span for later reference
|
||||
if not is_end_event:
|
||||
self._span_map[trace_lookup_key] = span
|
||||
self._active_spans[trace_id] = {
|
||||
"span": span,
|
||||
"span_type": "TraceSpan",
|
||||
"trace_id": trace_id,
|
||||
"parent_id": None,
|
||||
}
|
||||
else:
|
||||
span.end()
|
||||
|
||||
def _get_parent_context(self, trace_id: str, span_id: str, parent_id: Optional[str] = None) -> Any:
|
||||
"""Find the parent span context for proper span nesting.
|
||||
|
||||
This method checks:
|
||||
1. First for an explicit parent ID in our span tracking dictionary
|
||||
2. Then checks if the trace span is the parent
|
||||
3. Falls back to the current active span context if no parent is found
|
||||
|
||||
Args:
|
||||
trace_id: The trace ID for the current span
|
||||
span_id: The span ID for the current span
|
||||
parent_id: Optional parent span ID to look up
|
||||
|
||||
Returns:
|
||||
The OpenTelemetry span context to use as parent
|
||||
"""
|
||||
parent_span_ctx = None
|
||||
|
||||
if parent_id:
|
||||
# Try to find the parent span in our tracking dictionary
|
||||
parent_lookup_key = f"span:{trace_id}:{parent_id}"
|
||||
if parent_lookup_key in self._span_map:
|
||||
parent_span = self._span_map[parent_lookup_key]
|
||||
# Get the context from the parent span if it exists
|
||||
if hasattr(parent_span, "get_span_context"):
|
||||
parent_span_ctx = parent_span.get_span_context()
|
||||
|
||||
# If parent not found by span ID, check if trace span should be the parent
|
||||
if not parent_span_ctx and parent_id is None:
|
||||
# Try using the trace span as parent
|
||||
trace_lookup_key = _get_span_lookup_key(trace_id, trace_id)
|
||||
|
||||
if trace_lookup_key in self._span_map:
|
||||
trace_span = self._span_map[trace_lookup_key]
|
||||
if hasattr(trace_span, "get_span_context"):
|
||||
parent_span_ctx = trace_span.get_span_context()
|
||||
|
||||
# If we couldn't find the parent by ID, use the current span context as parent
|
||||
if not parent_span_ctx:
|
||||
# Get the current span context from the context API
|
||||
ctx = context_api.get_current()
|
||||
parent_span_ctx = trace_api.get_current_span(ctx).get_span_context()
|
||||
|
||||
return parent_span_ctx
|
||||
|
||||
def _create_span_with_parent(
|
||||
self, name: str, kind: SpanKind, attributes: Dict[str, Any], parent_ctx: Any, end_immediately: bool = False
|
||||
) -> Any:
|
||||
"""Create a span with the specified parent context.
|
||||
|
||||
This centralizes span creation with proper parent nesting.
|
||||
|
||||
Args:
|
||||
name: The name for the new span
|
||||
kind: The span kind (CLIENT, SERVER, etc.)
|
||||
attributes: The attributes to set on the span
|
||||
parent_ctx: The parent context to use for nesting
|
||||
end_immediately: Whether to end the span immediately
|
||||
|
||||
Returns:
|
||||
The newly created span
|
||||
"""
|
||||
# Get tracer from provider
|
||||
tracer = get_tracer(LIBRARY_NAME, LIBRARY_VERSION, self.tracer_provider)
|
||||
|
||||
# Create span with context so we get proper nesting
|
||||
with trace_api.use_span(NonRecordingSpan(parent_ctx), end_on_exit=False):
|
||||
span = tracer.start_span(name=name, kind=kind, attributes=attributes)
|
||||
|
||||
# Optionally end the span immediately
|
||||
if end_immediately:
|
||||
span.end()
|
||||
|
||||
return span
|
||||
|
||||
def export_span(self, span: Any) -> None:
|
||||
"""Export a span to OpenTelemetry, creating or updating as needed.
|
||||
|
||||
This method decides whether to create a new span or update an existing one
|
||||
based on whether this is a start or end event for a given span ID.
|
||||
|
||||
For start events:
|
||||
- Create a new span and store it for later updates
|
||||
- Leave status as UNSET (in progress)
|
||||
- Do not end the span
|
||||
- Properly set parent span reference for nesting
|
||||
|
||||
For end events:
|
||||
- Look for an existing span to update
|
||||
- If found and not ended, update with final data and end it
|
||||
- If not found or already ended, create a new complete span with all data
|
||||
- End the span with proper status
|
||||
"""
|
||||
if not hasattr(span, "span_data"):
|
||||
return
|
||||
|
||||
span_data = span.span_data
|
||||
span_type = span_data.__class__.__name__
|
||||
span_id = getattr(span, "span_id", "unknown")
|
||||
trace_id = getattr(span, "trace_id", "unknown")
|
||||
parent_id = getattr(span, "parent_id", None)
|
||||
|
||||
# Check if this is a span end event
|
||||
is_end_event = hasattr(span, "status") and span.status == StatusCode.OK.name
|
||||
|
||||
# Unique lookup key for this span
|
||||
span_lookup_key = _get_span_lookup_key(trace_id, span_id)
|
||||
|
||||
attributes = get_base_span_attributes(span)
|
||||
span_attributes = get_span_attributes(span_data)
|
||||
attributes.update(span_attributes)
|
||||
|
||||
if is_end_event:
|
||||
# Update all attributes for end events
|
||||
attributes.update(span_attributes)
|
||||
|
||||
# Log the trace ID for debugging and correlation with AgentOps API
|
||||
log_otel_trace_id(span_type)
|
||||
|
||||
# For start events, create a new span and store it (don't end it)
|
||||
if not is_end_event:
|
||||
# Process the span based on its type
|
||||
# TODO span_name should come from the attributes module
|
||||
span_name = get_span_name(span)
|
||||
span_kind = get_span_kind(span)
|
||||
|
||||
# Get parent context for proper nesting
|
||||
parent_span_ctx = self._get_parent_context(trace_id, span_id, parent_id)
|
||||
|
||||
# Create the span with proper parent context
|
||||
otel_span = self._create_span_with_parent(
|
||||
name=span_name, kind=span_kind, attributes=attributes, parent_ctx=parent_span_ctx
|
||||
)
|
||||
|
||||
# Store the span for later reference
|
||||
if not isinstance(otel_span, NonRecordingSpan):
|
||||
self._span_map[span_lookup_key] = otel_span
|
||||
self._active_spans[span_id] = {
|
||||
"span": otel_span,
|
||||
"span_type": span_type,
|
||||
"trace_id": trace_id,
|
||||
"parent_id": parent_id,
|
||||
}
|
||||
|
||||
# Handle any error information
|
||||
self._handle_span_error(span, otel_span)
|
||||
# DO NOT end the span for start events - we want to keep it open for updates
|
||||
return
|
||||
|
||||
# For end events, check if we already have the span
|
||||
if span_lookup_key in self._span_map:
|
||||
existing_span = self._span_map[span_lookup_key]
|
||||
|
||||
span_is_ended = False
|
||||
if isinstance(existing_span, Span) and hasattr(existing_span, "_end_time"):
|
||||
span_is_ended = existing_span._end_time is not None
|
||||
|
||||
if not span_is_ended:
|
||||
# Update with final attributes
|
||||
for key, value in attributes.items():
|
||||
existing_span.set_attribute(key, value)
|
||||
|
||||
existing_span.set_status(
|
||||
Status(StatusCode.OK if getattr(span, "status", "OK") == "OK" else StatusCode.ERROR)
|
||||
)
|
||||
self._handle_span_error(span, existing_span)
|
||||
existing_span.end()
|
||||
# Span already ended, create a new one (should be rare if logic is correct)
|
||||
else:
|
||||
logger.warning(
|
||||
f"[Exporter] SDK span_id: {span_id} (END event) - Attempting to end an ALREADY ENDED span: {span_lookup_key}. Creating a new one instead."
|
||||
)
|
||||
self.create_span(span, span_type, attributes, is_already_ended=True)
|
||||
# No existing span found for end event, create a new one
|
||||
else:
|
||||
logger.warning(
|
||||
f"[Exporter] SDK span_id: {span_id} (END event) - No active span found for end event: {span_lookup_key}. Creating a new one."
|
||||
)
|
||||
self.create_span(span, span_type, attributes, is_already_ended=True)
|
||||
|
||||
self._active_spans.pop(span_id, None)
|
||||
self._span_map.pop(span_lookup_key, None)
|
||||
|
||||
def create_span(
|
||||
self, span: Any, span_type: str, attributes: Dict[str, Any], is_already_ended: bool = False
|
||||
) -> None:
|
||||
"""Create a new span with the provided data and end it immediately.
|
||||
|
||||
This method creates a span using the appropriate parent context, applies
|
||||
all attributes, and ends it immediately since it's for spans that are
|
||||
already in an ended state.
|
||||
|
||||
Args:
|
||||
span: The span data from the Agents SDK
|
||||
span_type: The type of span being created
|
||||
attributes: The attributes to set on the span
|
||||
"""
|
||||
parent_ctx = None
|
||||
if hasattr(span, "parent_id") and span.parent_id:
|
||||
parent_ctx = self._get_parent_context(
|
||||
getattr(span, "trace_id", "unknown"), getattr(span, "id", "unknown"), span.parent_id
|
||||
)
|
||||
|
||||
name = get_span_name(span)
|
||||
kind = get_span_kind(span)
|
||||
|
||||
self._create_span_with_parent(
|
||||
name=name, kind=kind, attributes=attributes, parent_ctx=parent_ctx, end_immediately=True
|
||||
)
|
||||
|
||||
def _handle_span_error(self, span: Any, otel_span: Any) -> None:
|
||||
"""Handle error information from spans."""
|
||||
if hasattr(span, "error") and span.error:
|
||||
# Set status to error
|
||||
status = Status(StatusCode.ERROR)
|
||||
otel_span.set_status(status)
|
||||
|
||||
# Determine error message - handle various error formats
|
||||
error_message = "Unknown error"
|
||||
error_data = {}
|
||||
error_type = "AgentError"
|
||||
|
||||
# Handle different error formats
|
||||
if isinstance(span.error, dict):
|
||||
error_message = span.error.get("message", span.error.get("error", "Unknown error"))
|
||||
error_data = span.error.get("data", {})
|
||||
# Extract error type if available
|
||||
if "type" in span.error:
|
||||
error_type = span.error["type"]
|
||||
elif "code" in span.error:
|
||||
error_type = span.error["code"]
|
||||
elif isinstance(span.error, str):
|
||||
error_message = span.error
|
||||
elif hasattr(span.error, "message"):
|
||||
error_message = span.error.message
|
||||
# Use type() for more reliable class name access
|
||||
error_type = type(span.error).__name__
|
||||
elif hasattr(span.error, "__str__"):
|
||||
# Fallback to string representation
|
||||
error_message = str(span.error)
|
||||
|
||||
# Record the exception with proper error data
|
||||
try:
|
||||
exception = Exception(error_message)
|
||||
error_data_json = json.dumps(error_data) if error_data else "{}"
|
||||
otel_span.record_exception(
|
||||
exception=exception,
|
||||
attributes={"error.data": error_data_json},
|
||||
)
|
||||
except Exception as e:
|
||||
# If JSON serialization fails, use simpler approach
|
||||
logger.warning(f"Error serializing error data: {e}")
|
||||
otel_span.record_exception(Exception(error_message))
|
||||
|
||||
# Set error attributes
|
||||
otel_span.set_attribute(CoreAttributes.ERROR_TYPE, error_type)
|
||||
otel_span.set_attribute(CoreAttributes.ERROR_MESSAGE, error_message)
|
||||
|
||||
def cleanup(self):
|
||||
"""Clean up any outstanding spans during shutdown.
|
||||
|
||||
This ensures we don't leak span resources when the exporter is shutdown.
|
||||
"""
|
||||
# Clear all tracking dictionaries
|
||||
self._active_spans.clear()
|
||||
self._span_map.clear()
|
||||
@@ -0,0 +1,107 @@
|
||||
"""OpenAI Agents SDK Instrumentation for AgentOps
|
||||
|
||||
This module provides instrumentation for the OpenAI Agents SDK, leveraging its built-in
|
||||
tracing API for observability. It captures detailed information about agent execution,
|
||||
tool usage, LLM requests, and token metrics.
|
||||
|
||||
The implementation uses a clean separation between exporters and processors. The exporter
|
||||
translates Agent spans into OpenTelemetry spans with appropriate semantic conventions.
|
||||
|
||||
The processor implements the tracing interface, collects metrics, and manages timing data.
|
||||
|
||||
We use the built-in add_trace_processor hook for all functionality. Streaming support
|
||||
would require monkey-patching the run method of `Runner`, but doesn't really get us
|
||||
more data than we already have, since the `Response` object is always passed to us
|
||||
from the `agents.tracing` module.
|
||||
|
||||
TODO Calls to the OpenAI API are not available in this tracing context, so we may
|
||||
need to monkey-patch the `openai` from here to get that data. While we do have
|
||||
separate instrumentation for the OpenAI API, in order to get it to nest with the
|
||||
spans we create here, it's probably easier (or even required) that we incorporate
|
||||
that here as well.
|
||||
"""
|
||||
|
||||
from typing import Collection
|
||||
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor # type: ignore
|
||||
from agentops.instrumentation.agentic.openai_agents import LIBRARY_VERSION
|
||||
|
||||
from agentops.logging import logger
|
||||
from agentops.instrumentation.agentic.openai_agents.processor import OpenAIAgentsProcessor
|
||||
from agentops.instrumentation.agentic.openai_agents.exporter import OpenAIAgentsExporter
|
||||
|
||||
|
||||
class OpenAIAgentsInstrumentor(BaseInstrumentor):
|
||||
"""An instrumentor for OpenAI Agents SDK that uses the built-in tracing API."""
|
||||
|
||||
_processor = None
|
||||
_exporter = None
|
||||
_default_processor = None
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self._tracer = None
|
||||
self._is_instrumented_instance_flag = False
|
||||
|
||||
def instrumentation_dependencies(self) -> Collection[str]:
|
||||
"""Return packages required for instrumentation."""
|
||||
return ["openai-agents >= 0.0.1"]
|
||||
|
||||
def _instrument(self, **kwargs):
|
||||
"""Instrument the OpenAI Agents SDK."""
|
||||
if self._is_instrumented_instance_flag:
|
||||
logger.debug("OpenAI Agents SDK already instrumented. Skipping.")
|
||||
return
|
||||
|
||||
tracer_provider = kwargs.get("tracer_provider")
|
||||
if self._tracer is None:
|
||||
logger.debug("OpenAI Agents SDK tracer is None, creating new tracer.")
|
||||
self._tracer = trace.get_tracer("agentops.instrumentation.openai_agents", LIBRARY_VERSION)
|
||||
|
||||
try:
|
||||
self._exporter = OpenAIAgentsExporter(tracer_provider=tracer_provider)
|
||||
self._processor = OpenAIAgentsProcessor(
|
||||
exporter=self._exporter,
|
||||
)
|
||||
|
||||
# Replace the default processor with our processor
|
||||
from agents import set_trace_processors
|
||||
from agents.tracing.processors import default_processor
|
||||
|
||||
if getattr(self, "_default_processor", None) is None:
|
||||
self._default_processor = default_processor()
|
||||
|
||||
# Store reference to default processor for later restoration
|
||||
set_trace_processors([self._processor])
|
||||
self._is_instrumented_instance_flag = True
|
||||
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to instrument OpenAI Agents SDK: {e}", exc_info=True)
|
||||
|
||||
def _uninstrument(self, **kwargs):
|
||||
"""Remove instrumentation from OpenAI Agents SDK."""
|
||||
if not self._is_instrumented_instance_flag:
|
||||
logger.debug("OpenAI Agents SDK not currently instrumented. Skipping uninstrument.")
|
||||
return
|
||||
try:
|
||||
# Clean up any active spans in the exporter
|
||||
if hasattr(self, "_exporter") and self._exporter:
|
||||
if hasattr(self._exporter, "cleanup"):
|
||||
self._exporter.cleanup()
|
||||
|
||||
# Put back the default processor
|
||||
from agents import set_trace_processors
|
||||
|
||||
if hasattr(self, "_default_processor") and self._default_processor:
|
||||
set_trace_processors([self._default_processor])
|
||||
self._default_processor = None
|
||||
else:
|
||||
logger.warning("OpenAI Agents SDK has no default processor to restore.")
|
||||
self._processor = None
|
||||
self._exporter = None
|
||||
|
||||
self._is_instrumented_instance_flag = False
|
||||
logger.info("Successfully removed OpenAI Agents SDK instrumentation")
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to uninstrument OpenAI Agents SDK: {e}", exc_info=True)
|
||||
@@ -0,0 +1,58 @@
|
||||
from typing import Any
|
||||
from opentelemetry.trace import StatusCode
|
||||
from agentops.logging import logger
|
||||
|
||||
|
||||
class OpenAIAgentsProcessor:
|
||||
"""Processor for OpenAI Agents SDK traces.
|
||||
|
||||
This processor implements the TracingProcessor interface from the Agents SDK
|
||||
and converts trace events to OpenTelemetry spans and metrics.
|
||||
|
||||
The processor does NOT directly create OpenTelemetry spans.
|
||||
It delegates span creation to the OpenAIAgentsExporter.
|
||||
"""
|
||||
|
||||
def __init__(self, exporter=None):
|
||||
self.exporter = exporter
|
||||
|
||||
def on_trace_start(self, sdk_trace: Any) -> None:
|
||||
"""Called when a trace starts in the Agents SDK."""
|
||||
|
||||
logger.debug(f"[agentops.instrumentation.openai_agents] Trace started: {sdk_trace}")
|
||||
self.exporter.export_trace(sdk_trace)
|
||||
|
||||
def on_trace_end(self, sdk_trace: Any) -> None:
|
||||
"""Called when a trace ends in the Agents SDK."""
|
||||
|
||||
# Mark this as an end event
|
||||
# This is used by the exporter to determine whether to create or update a trace
|
||||
sdk_trace.status = StatusCode.OK.name
|
||||
|
||||
logger.debug(f"[agentops.instrumentation.openai_agents] Trace ended: {sdk_trace}")
|
||||
self.exporter.export_trace(sdk_trace)
|
||||
|
||||
def on_span_start(self, span: Any) -> None:
|
||||
"""Called when a span starts in the Agents SDK."""
|
||||
|
||||
logger.debug(f"[agentops.instrumentation.openai_agents] Span started: {span}")
|
||||
self.exporter.export_span(span)
|
||||
|
||||
def on_span_end(self, span: Any) -> None:
|
||||
"""Called when a span ends in the Agents SDK."""
|
||||
|
||||
# Mark this as an end event
|
||||
# This is used by the exporter to determine whether to create or update a span
|
||||
span.status = StatusCode.OK.name
|
||||
|
||||
logger.debug(f"[agentops.instrumentation.openai_agents] Span ended: {span}")
|
||||
self.exporter.export_span(span)
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""Called when the application stops."""
|
||||
pass
|
||||
|
||||
def force_flush(self) -> None:
|
||||
"""Forces an immediate flush of all queued spans/traces."""
|
||||
# We don't queue spans so this is a no-op
|
||||
pass
|
||||
@@ -0,0 +1,88 @@
|
||||
# SmoLAgents Instrumentation
|
||||
|
||||
This module provides OpenTelemetry instrumentation for the SmoLAgents framework. It captures telemetry data from model operations, agent executions, and tool usage.
|
||||
|
||||
## Features
|
||||
|
||||
- Model operation tracking
|
||||
- Text generation
|
||||
- Token usage
|
||||
- Streaming responses
|
||||
- Latency metrics
|
||||
|
||||
- Agent execution monitoring
|
||||
- Step-by-step execution
|
||||
- Planning phases
|
||||
- Tool usage
|
||||
- Execution time
|
||||
|
||||
- Tool usage analytics
|
||||
- Tool call patterns
|
||||
- Success/failure rates
|
||||
- Execution time
|
||||
- Error tracking
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from agentops import init
|
||||
from agentops.instrumentation.smolagents import SmolagentsInstrumentor
|
||||
|
||||
# Initialize AgentOps with your API key
|
||||
init(api_key="your-api-key")
|
||||
|
||||
# The instrumentation will be automatically activated
|
||||
# All SmoLAgents operations will now be tracked
|
||||
```
|
||||
|
||||
## Metrics Collected
|
||||
|
||||
1. Token Usage
|
||||
- Input tokens
|
||||
- Output tokens
|
||||
- Total tokens per operation
|
||||
|
||||
2. Timing Metrics
|
||||
- Operation duration
|
||||
- Time to first token (streaming)
|
||||
- Tool execution time
|
||||
- Planning phase duration
|
||||
|
||||
3. Agent Metrics
|
||||
- Step counts
|
||||
- Planning steps
|
||||
- Tools used
|
||||
- Success/failure rates
|
||||
|
||||
4. Error Tracking
|
||||
- Generation errors
|
||||
- Tool execution errors
|
||||
- Parsing errors
|
||||
|
||||
## Architecture
|
||||
|
||||
The instrumentation is built on OpenTelemetry and follows the same pattern as other AgentOps instrumentors:
|
||||
|
||||
1. Attribute Extractors
|
||||
- Model attributes
|
||||
- Agent attributes
|
||||
- Tool call attributes
|
||||
|
||||
2. Wrappers
|
||||
- Method wrappers for sync operations
|
||||
- Stream wrappers for async operations
|
||||
- Context propagation handling
|
||||
|
||||
3. Metrics
|
||||
- Histograms for distributions
|
||||
- Counters for events
|
||||
- Custom attributes for filtering
|
||||
|
||||
## Contributing
|
||||
|
||||
When adding new features or modifying existing ones:
|
||||
|
||||
1. Follow the established pattern for attribute extraction
|
||||
2. Maintain context propagation
|
||||
3. Add appropriate error handling
|
||||
4. Update tests and documentation
|
||||
@@ -0,0 +1,12 @@
|
||||
"""SmoLAgents instrumentation for AgentOps."""
|
||||
|
||||
from agentops.instrumentation.common import LibraryInfo
|
||||
|
||||
# Library information
|
||||
_library_info = LibraryInfo(name="smolagents", default_version="1.16.0")
|
||||
LIBRARY_NAME = _library_info.name
|
||||
LIBRARY_VERSION = _library_info.version
|
||||
|
||||
from agentops.instrumentation.agentic.smolagents.instrumentor import SmolagentsInstrumentor # noqa: E402
|
||||
|
||||
__all__ = ["SmolagentsInstrumentor"]
|
||||
@@ -0,0 +1,354 @@
|
||||
"""Attribute extractors for SmoLAgents agent operations."""
|
||||
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
import uuid
|
||||
import json
|
||||
|
||||
from agentops.instrumentation.common.attributes import get_common_attributes
|
||||
from agentops.semconv.agent import AgentAttributes
|
||||
from agentops.semconv.tool import ToolAttributes
|
||||
|
||||
|
||||
def get_agent_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from an agent execution call.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Extract agent instance information
|
||||
agent_instance = None
|
||||
if args and len(args) > 0:
|
||||
agent_instance = args[0]
|
||||
elif kwargs and "self" in kwargs:
|
||||
agent_instance = kwargs["self"]
|
||||
|
||||
if agent_instance:
|
||||
# Extract agent name
|
||||
agent_name = getattr(agent_instance, "name", agent_instance.__class__.__name__)
|
||||
attributes[AgentAttributes.AGENT_NAME] = agent_name
|
||||
|
||||
# Generate agent ID if not present
|
||||
agent_id = getattr(agent_instance, "id", str(uuid.uuid4()))
|
||||
attributes[AgentAttributes.AGENT_ID] = agent_id
|
||||
|
||||
# Extract agent role/type
|
||||
attributes[AgentAttributes.AGENT_ROLE] = "executor"
|
||||
|
||||
# Extract tools information
|
||||
tools = getattr(agent_instance, "tools", [])
|
||||
if tools:
|
||||
tool_names = []
|
||||
for tool in tools:
|
||||
tool_name = getattr(tool, "name", str(tool))
|
||||
tool_names.append(tool_name)
|
||||
attributes[AgentAttributes.AGENT_TOOLS] = json.dumps(tool_names)
|
||||
else:
|
||||
attributes[AgentAttributes.AGENT_TOOLS] = "[]"
|
||||
|
||||
# Extract managed agents information
|
||||
managed_agents = getattr(agent_instance, "managed_agents", [])
|
||||
if managed_agents:
|
||||
managed_agent_names = []
|
||||
for managed_agent in managed_agents:
|
||||
agent_name = getattr(managed_agent, "name", managed_agent.__class__.__name__)
|
||||
managed_agent_names.append(agent_name)
|
||||
attributes[AgentAttributes.AGENT_MANAGED_AGENTS] = json.dumps(managed_agent_names)
|
||||
else:
|
||||
attributes[AgentAttributes.AGENT_MANAGED_AGENTS] = "[]"
|
||||
|
||||
# Extract input/task from args or kwargs
|
||||
task_input = None
|
||||
if args and len(args) > 1:
|
||||
task_input = args[1]
|
||||
elif kwargs and "task" in kwargs:
|
||||
task_input = kwargs["task"]
|
||||
elif kwargs and "prompt" in kwargs:
|
||||
task_input = kwargs["prompt"]
|
||||
|
||||
if task_input:
|
||||
attributes["agent.task"] = str(task_input)
|
||||
|
||||
# Extract return value/output
|
||||
if return_value is not None:
|
||||
attributes["agentops.entity.output"] = str(return_value)
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, continue with basic attributes
|
||||
pass
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_agent_stream_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from an agent streaming call.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Extract reasoning/task information
|
||||
if kwargs:
|
||||
if "max_steps" in kwargs:
|
||||
attributes["agent.max_steps"] = str(kwargs["max_steps"])
|
||||
|
||||
# Extract task/reasoning from various parameter names
|
||||
task_info = None
|
||||
for param_name in ["task", "prompt", "reasoning", "query"]:
|
||||
if param_name in kwargs:
|
||||
task_info = kwargs[param_name]
|
||||
break
|
||||
|
||||
if task_info:
|
||||
attributes["agent.reasoning"] = str(task_info)
|
||||
|
||||
# Extract from args
|
||||
if args and len(args) > 1:
|
||||
task_info = args[1]
|
||||
attributes["agent.reasoning"] = str(task_info)
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, continue with basic attributes
|
||||
pass
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_agent_step_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from an agent step execution.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Try to extract step information
|
||||
step_number = getattr(args[0] if args else None, "step_count", None)
|
||||
if step_number is not None:
|
||||
attributes["agent.step_number"] = str(step_number)
|
||||
|
||||
# Extract step name/type
|
||||
step_name = "ActionStep" # Default for smolagents
|
||||
attributes["agent.name"] = step_name
|
||||
|
||||
# Extract return value
|
||||
if return_value is not None:
|
||||
attributes["agentops.entity.output"] = str(return_value)
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, continue with basic attributes
|
||||
pass
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_tool_call_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from a tool call execution.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Generate tool execution ID
|
||||
tool_id = str(uuid.uuid4())
|
||||
attributes[ToolAttributes.TOOL_ID] = tool_id
|
||||
|
||||
# Extract tool information from various sources
|
||||
tool_name = "unknown"
|
||||
tool_description = "unknown"
|
||||
tool_parameters = "{}"
|
||||
|
||||
# Try to extract from instance (first arg)
|
||||
if args and len(args) > 0:
|
||||
instance = args[0]
|
||||
if hasattr(instance, "name"):
|
||||
tool_name = instance.name
|
||||
if hasattr(instance, "description"):
|
||||
tool_description = instance.description
|
||||
|
||||
# Try to extract from kwargs
|
||||
if kwargs:
|
||||
if "tool_call" in kwargs:
|
||||
tool_call = kwargs["tool_call"]
|
||||
if hasattr(tool_call, "function"):
|
||||
tool_name = tool_call.function.name
|
||||
if hasattr(tool_call.function, "arguments"):
|
||||
tool_parameters = tool_call.function.arguments
|
||||
elif "name" in kwargs:
|
||||
tool_name = kwargs["name"]
|
||||
elif "function_name" in kwargs:
|
||||
tool_name = kwargs["function_name"]
|
||||
|
||||
# Extract parameters
|
||||
if "parameters" in kwargs:
|
||||
tool_parameters = json.dumps(kwargs["parameters"])
|
||||
elif "arguments" in kwargs:
|
||||
tool_parameters = json.dumps(kwargs["arguments"])
|
||||
elif "args" in kwargs:
|
||||
tool_parameters = json.dumps(kwargs["args"])
|
||||
|
||||
# Set tool attributes
|
||||
attributes[ToolAttributes.TOOL_NAME] = tool_name
|
||||
attributes[ToolAttributes.TOOL_DESCRIPTION] = tool_description
|
||||
attributes[ToolAttributes.TOOL_PARAMETERS] = tool_parameters
|
||||
attributes[ToolAttributes.TOOL_STATUS] = "pending"
|
||||
attributes[ToolAttributes.TOOL_OUTPUT_TYPE] = "unknown"
|
||||
attributes[ToolAttributes.TOOL_INPUTS] = "{}"
|
||||
|
||||
# Extract return value
|
||||
if return_value is not None:
|
||||
attributes["tool.result"] = str(return_value)
|
||||
attributes[ToolAttributes.TOOL_STATUS] = "success"
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, set basic attributes
|
||||
attributes[ToolAttributes.TOOL_NAME] = "unknown"
|
||||
attributes[ToolAttributes.TOOL_DESCRIPTION] = "unknown"
|
||||
attributes[ToolAttributes.TOOL_ID] = str(uuid.uuid4())
|
||||
attributes[ToolAttributes.TOOL_PARAMETERS] = "{}"
|
||||
attributes[ToolAttributes.TOOL_STATUS] = "pending"
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_planning_step_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from a planning step execution.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Extract planning information
|
||||
if kwargs:
|
||||
if "planning_step" in kwargs:
|
||||
step = kwargs["planning_step"]
|
||||
attributes["agent.planning.step"] = str(step)
|
||||
if "reasoning" in kwargs:
|
||||
attributes["agent.planning.reasoning"] = str(kwargs["reasoning"])
|
||||
|
||||
# Extract return value
|
||||
if return_value is not None:
|
||||
attributes["agentops.entity.output"] = str(return_value)
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, continue with basic attributes
|
||||
pass
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_managed_agent_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from a managed agent call.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Extract managed agent information
|
||||
agent_instance = None
|
||||
if args and len(args) > 0:
|
||||
agent_instance = args[0]
|
||||
elif kwargs and "agent" in kwargs:
|
||||
agent_instance = kwargs["agent"]
|
||||
|
||||
if agent_instance:
|
||||
# Extract agent details
|
||||
agent_name = getattr(agent_instance, "name", agent_instance.__class__.__name__)
|
||||
agent_id = getattr(agent_instance, "id", str(uuid.uuid4()))
|
||||
agent_description = getattr(agent_instance, "description", "")
|
||||
|
||||
attributes[AgentAttributes.AGENT_NAME] = agent_name
|
||||
attributes[AgentAttributes.AGENT_ID] = agent_id
|
||||
attributes[AgentAttributes.AGENT_ROLE] = "managed"
|
||||
attributes[AgentAttributes.AGENT_TYPE] = agent_instance.__class__.__name__
|
||||
|
||||
if agent_description:
|
||||
attributes[AgentAttributes.AGENT_DESCRIPTION] = agent_description
|
||||
|
||||
# Check if this agent provides run summaries
|
||||
attributes["agent.provide_run_summary"] = "false" # Default for smolagents
|
||||
|
||||
# Extract task information
|
||||
task = None
|
||||
if args and len(args) > 1:
|
||||
task = args[1]
|
||||
elif kwargs and "task" in kwargs:
|
||||
task = kwargs["task"]
|
||||
|
||||
if task:
|
||||
attributes["agent.task"] = str(task)
|
||||
|
||||
# Extract return value
|
||||
if return_value is not None:
|
||||
attributes["agentops.entity.output"] = str(return_value)
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, continue with basic attributes
|
||||
pass
|
||||
|
||||
return attributes
|
||||
@@ -0,0 +1,205 @@
|
||||
"""Attribute extractors for SmoLAgents model operations."""
|
||||
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
import json
|
||||
|
||||
from agentops.instrumentation.common.attributes import (
|
||||
get_common_attributes,
|
||||
)
|
||||
from agentops.semconv.message import MessageAttributes
|
||||
from agentops.semconv.span_attributes import SpanAttributes
|
||||
|
||||
|
||||
def get_model_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from a model generation call.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Extract model name from various sources
|
||||
model_name = "unknown"
|
||||
|
||||
# Try to get from kwargs
|
||||
if kwargs:
|
||||
if "model" in kwargs:
|
||||
model_name = kwargs["model"]
|
||||
elif kwargs.get("self") and hasattr(kwargs["self"], "model_id"):
|
||||
model_name = kwargs["self"].model_id
|
||||
|
||||
# Try to get from args (instance is usually first arg in methods)
|
||||
if model_name == "unknown" and args and len(args) > 0:
|
||||
instance = args[0]
|
||||
if hasattr(instance, "model_id"):
|
||||
model_name = instance.model_id
|
||||
|
||||
# Set model attributes
|
||||
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_name
|
||||
|
||||
# Extract messages from kwargs
|
||||
if kwargs and "messages" in kwargs:
|
||||
messages = kwargs["messages"]
|
||||
if isinstance(messages, list):
|
||||
for i, message in enumerate(messages):
|
||||
message_dict = message
|
||||
if hasattr(message, "to_dict"):
|
||||
message_dict = message.to_dict()
|
||||
elif hasattr(message, "__dict__"):
|
||||
message_dict = message.__dict__
|
||||
|
||||
if isinstance(message_dict, dict):
|
||||
# Set role
|
||||
role = message_dict.get("role", "user")
|
||||
attributes[MessageAttributes.PROMPT_ROLE.format(i=i)] = role
|
||||
|
||||
# Set content
|
||||
content = message_dict.get("content", "")
|
||||
if content:
|
||||
attributes[MessageAttributes.PROMPT_CONTENT.format(i=i)] = str(content)
|
||||
|
||||
# Extract tools from kwargs
|
||||
if kwargs and "tools_to_call_from" in kwargs:
|
||||
tools = kwargs["tools_to_call_from"]
|
||||
if tools and isinstance(tools, list):
|
||||
for i, tool in enumerate(tools):
|
||||
tool_name = getattr(tool, "name", "unknown")
|
||||
tool_description = getattr(tool, "description", "")
|
||||
|
||||
attributes[MessageAttributes.TOOL_CALL_NAME.format(i=i)] = tool_name
|
||||
if tool_description:
|
||||
attributes[MessageAttributes.TOOL_CALL_DESCRIPTION.format(i=i)] = tool_description
|
||||
|
||||
# Extract additional parameters
|
||||
if kwargs:
|
||||
if "temperature" in kwargs:
|
||||
attributes[SpanAttributes.LLM_REQUEST_TEMPERATURE] = kwargs["temperature"]
|
||||
if "max_tokens" in kwargs:
|
||||
attributes[SpanAttributes.LLM_REQUEST_MAX_TOKENS] = kwargs["max_tokens"]
|
||||
if "stop_sequences" in kwargs:
|
||||
attributes[SpanAttributes.LLM_REQUEST_STOP_SEQUENCES] = json.dumps(kwargs["stop_sequences"])
|
||||
|
||||
# Extract response attributes
|
||||
if return_value:
|
||||
try:
|
||||
# Handle ChatMessage response
|
||||
if hasattr(return_value, "content"):
|
||||
attributes[MessageAttributes.COMPLETION_CONTENT.format(i=0)] = str(return_value.content)
|
||||
if hasattr(return_value, "role"):
|
||||
attributes[MessageAttributes.COMPLETION_ROLE.format(i=0)] = return_value.role
|
||||
|
||||
# Handle tool calls in response
|
||||
if hasattr(return_value, "tool_calls") and return_value.tool_calls:
|
||||
for j, tool_call in enumerate(return_value.tool_calls):
|
||||
if hasattr(tool_call, "function"):
|
||||
attributes[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=j)] = (
|
||||
tool_call.function.name
|
||||
)
|
||||
if hasattr(tool_call.function, "arguments"):
|
||||
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=j)] = (
|
||||
tool_call.function.arguments
|
||||
)
|
||||
if hasattr(tool_call, "id"):
|
||||
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=j)] = tool_call.id
|
||||
|
||||
# Extract token usage
|
||||
if hasattr(return_value, "token_usage") and return_value.token_usage:
|
||||
token_usage = return_value.token_usage
|
||||
if hasattr(token_usage, "input_tokens"):
|
||||
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = token_usage.input_tokens
|
||||
if hasattr(token_usage, "output_tokens"):
|
||||
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = token_usage.output_tokens
|
||||
if hasattr(token_usage, "total_tokens"):
|
||||
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = token_usage.total_tokens
|
||||
|
||||
# Extract response ID
|
||||
if hasattr(return_value, "raw") and return_value.raw:
|
||||
raw_response = return_value.raw
|
||||
if hasattr(raw_response, "id"):
|
||||
attributes[SpanAttributes.LLM_RESPONSE_ID] = raw_response.id
|
||||
|
||||
except Exception:
|
||||
# If we can't extract response attributes, continue with what we have
|
||||
pass
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, return basic attributes
|
||||
pass
|
||||
|
||||
return attributes
|
||||
|
||||
|
||||
def get_stream_attributes(
|
||||
args: Optional[Tuple] = None,
|
||||
kwargs: Optional[Dict] = None,
|
||||
return_value: Optional[Any] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Extract attributes from a streaming model generation call.
|
||||
|
||||
Args:
|
||||
args: Optional tuple of positional arguments
|
||||
kwargs: Optional dict of keyword arguments
|
||||
return_value: Optional return value from the function
|
||||
|
||||
Returns:
|
||||
Dictionary of extracted attributes
|
||||
"""
|
||||
attributes = get_common_attributes()
|
||||
|
||||
try:
|
||||
# Extract model name
|
||||
model_name = "unknown"
|
||||
if kwargs and kwargs.get("self") and hasattr(kwargs["self"], "model_id"):
|
||||
model_name = kwargs["self"].model_id
|
||||
elif args and len(args) > 0 and hasattr(args[0], "model_id"):
|
||||
model_name = args[0].model_id
|
||||
|
||||
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_name
|
||||
attributes["gen_ai.request.streaming"] = True
|
||||
|
||||
# Extract messages for streaming
|
||||
if kwargs and "messages" in kwargs:
|
||||
messages = kwargs["messages"]
|
||||
if isinstance(messages, list):
|
||||
for i, message in enumerate(messages):
|
||||
message_dict = message
|
||||
if hasattr(message, "to_dict"):
|
||||
message_dict = message.to_dict()
|
||||
elif hasattr(message, "__dict__"):
|
||||
message_dict = message.__dict__
|
||||
|
||||
if isinstance(message_dict, dict):
|
||||
role = message_dict.get("role", "user")
|
||||
attributes[MessageAttributes.PROMPT_ROLE.format(i=i)] = role
|
||||
|
||||
content = message_dict.get("content", "")
|
||||
if content:
|
||||
attributes[MessageAttributes.PROMPT_CONTENT.format(i=i)] = str(content)
|
||||
|
||||
# Extract tools for streaming
|
||||
if kwargs and "tools_to_call_from" in kwargs:
|
||||
tools = kwargs["tools_to_call_from"]
|
||||
if tools and isinstance(tools, list):
|
||||
for i, tool in enumerate(tools):
|
||||
tool_name = getattr(tool, "name", "unknown")
|
||||
tool_description = getattr(tool, "description", "")
|
||||
|
||||
attributes[MessageAttributes.TOOL_CALL_NAME.format(i=i)] = tool_name
|
||||
if tool_description:
|
||||
attributes[MessageAttributes.TOOL_CALL_DESCRIPTION.format(i=i)] = tool_description
|
||||
|
||||
except Exception:
|
||||
# If extraction fails, return basic attributes
|
||||
pass
|
||||
|
||||
return attributes
|
||||
@@ -0,0 +1,286 @@
|
||||
"""SmoLAgents instrumentation for AgentOps."""
|
||||
|
||||
from typing import Dict, Any
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.metrics import Meter
|
||||
from wrapt import wrap_function_wrapper
|
||||
|
||||
from agentops.instrumentation.common import CommonInstrumentor, StandardMetrics, InstrumentorConfig
|
||||
from agentops.logging import logger
|
||||
|
||||
# Library info for tracer/meter
|
||||
LIBRARY_NAME = "agentops.instrumentation.smolagents"
|
||||
LIBRARY_VERSION = "0.1.0"
|
||||
|
||||
# Import attribute handlers
|
||||
try:
|
||||
from agentops.instrumentation.agentic.smolagents.attributes.agent import (
|
||||
get_agent_attributes,
|
||||
get_tool_call_attributes,
|
||||
get_planning_step_attributes,
|
||||
get_agent_step_attributes,
|
||||
get_agent_stream_attributes,
|
||||
get_managed_agent_attributes,
|
||||
)
|
||||
from agentops.instrumentation.agentic.smolagents.attributes.model import (
|
||||
get_model_attributes,
|
||||
get_stream_attributes,
|
||||
)
|
||||
except ImportError:
|
||||
# Fallback functions if imports fail
|
||||
def get_agent_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
def get_tool_call_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
def get_planning_step_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
def get_agent_step_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
def get_agent_stream_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
def get_managed_agent_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
def get_model_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
def get_stream_attributes(*args, **kwargs):
|
||||
return {}
|
||||
|
||||
|
||||
class SmolagentsInstrumentor(CommonInstrumentor):
|
||||
"""Instrumentor for SmoLAgents library."""
|
||||
|
||||
def __init__(self):
|
||||
"""Initialize the SmoLAgents instrumentor."""
|
||||
# Create instrumentor config
|
||||
config = InstrumentorConfig(
|
||||
library_name=LIBRARY_NAME,
|
||||
library_version=LIBRARY_VERSION,
|
||||
wrapped_methods=[], # We use custom wrapping
|
||||
metrics_enabled=True,
|
||||
dependencies=["smolagents >= 1.0.0", "litellm"],
|
||||
)
|
||||
|
||||
super().__init__(config)
|
||||
|
||||
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
|
||||
"""Create metrics for the instrumentor.
|
||||
|
||||
Returns a dictionary of metric name to metric instance.
|
||||
"""
|
||||
# Create standard metrics for LLM operations
|
||||
return StandardMetrics.create_standard_metrics(meter)
|
||||
|
||||
def _custom_wrap(self, **kwargs):
|
||||
"""Apply custom wrapping for SmoLAgents.
|
||||
|
||||
This is called after normal wrapping, but we use it for all wrapping
|
||||
since we don't have normal wrapped methods.
|
||||
"""
|
||||
# Core agent operations
|
||||
wrap_function_wrapper("smolagents.agents", "CodeAgent.run", self._agent_run_wrapper(self._tracer))
|
||||
wrap_function_wrapper("smolagents.agents", "ToolCallingAgent.run", self._agent_run_wrapper(self._tracer))
|
||||
|
||||
# Tool calling operations
|
||||
wrap_function_wrapper(
|
||||
"smolagents.agents", "ToolCallingAgent.execute_tool_call", self._tool_execution_wrapper(self._tracer)
|
||||
)
|
||||
|
||||
# Model operations with proper model name extraction
|
||||
wrap_function_wrapper("smolagents.models", "LiteLLMModel.generate", self._llm_wrapper(self._tracer))
|
||||
wrap_function_wrapper("smolagents.models", "LiteLLMModel.generate_stream", self._llm_wrapper(self._tracer))
|
||||
|
||||
logger.info("SmoLAgents instrumentation enabled")
|
||||
|
||||
def _agent_run_wrapper(self, tracer):
|
||||
"""Wrapper for agent run methods."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
# Get proper agent name - handle None case
|
||||
agent_name = getattr(instance, "name", None)
|
||||
if not agent_name: # Handle None, empty string, or missing attribute
|
||||
agent_name = instance.__class__.__name__
|
||||
|
||||
span_name = f"{agent_name}.run"
|
||||
|
||||
with tracer.start_as_current_span(
|
||||
span_name,
|
||||
kind=SpanKind.CLIENT,
|
||||
) as span:
|
||||
# Extract attributes
|
||||
attributes = get_agent_attributes(args=(instance,) + args, kwargs=kwargs)
|
||||
|
||||
# Fix managed agents attribute
|
||||
if hasattr(instance, "managed_agents") and instance.managed_agents:
|
||||
managed_agent_names = []
|
||||
for agent in instance.managed_agents:
|
||||
name = getattr(agent, "name", None)
|
||||
if not name: # Handle None case for managed agents too
|
||||
name = agent.__class__.__name__
|
||||
managed_agent_names.append(name)
|
||||
attributes["agent.managed_agents"] = str(managed_agent_names)
|
||||
else:
|
||||
attributes["agent.managed_agents"] = "[]"
|
||||
|
||||
for key, value in attributes.items():
|
||||
if value is not None:
|
||||
span.set_attribute(key, value)
|
||||
|
||||
try:
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Set output attribute
|
||||
if result is not None:
|
||||
span.set_attribute("agentops.entity.output", str(result))
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
def _tool_execution_wrapper(self, tracer):
|
||||
"""Wrapper for tool execution methods."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
# Extract tool name for better span naming
|
||||
tool_name = "unknown"
|
||||
if args and len(args) > 0:
|
||||
tool_call = args[0]
|
||||
if hasattr(tool_call, "function"):
|
||||
tool_name = tool_call.function.name
|
||||
|
||||
span_name = f"tool.{tool_name}" if tool_name != "unknown" else "tool.execute"
|
||||
|
||||
with tracer.start_as_current_span(
|
||||
span_name,
|
||||
kind=SpanKind.CLIENT,
|
||||
) as span:
|
||||
# Extract tool information from kwargs or args
|
||||
tool_params = "{}"
|
||||
|
||||
# Try to extract tool call information
|
||||
if args and len(args) > 0:
|
||||
tool_call = args[0]
|
||||
if hasattr(tool_call, "function"):
|
||||
if hasattr(tool_call.function, "arguments"):
|
||||
tool_params = str(tool_call.function.arguments)
|
||||
|
||||
# Extract attributes
|
||||
attributes = get_tool_call_attributes(args=(instance,) + args, kwargs=kwargs)
|
||||
|
||||
# Override with better tool information if available
|
||||
if tool_name != "unknown":
|
||||
attributes["tool.name"] = tool_name
|
||||
attributes["tool.parameters"] = tool_params
|
||||
|
||||
for key, value in attributes.items():
|
||||
if value is not None:
|
||||
span.set_attribute(key, value)
|
||||
|
||||
try:
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Set success status and result
|
||||
span.set_attribute("tool.status", "success")
|
||||
if result is not None:
|
||||
span.set_attribute("tool.result", str(result))
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
span.set_attribute("tool.status", "error")
|
||||
span.record_exception(e)
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
def _llm_wrapper(self, tracer):
|
||||
"""Wrapper for LLM generation methods with proper model name extraction."""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
# Extract model name from instance
|
||||
model_name = getattr(instance, "model_id", "unknown")
|
||||
|
||||
# Determine if this is streaming
|
||||
is_streaming = "generate_stream" in wrapped.__name__
|
||||
operation = "generate_stream" if is_streaming else "generate"
|
||||
span_name = f"litellm.{operation} ({model_name})" if model_name != "unknown" else f"litellm.{operation}"
|
||||
|
||||
with tracer.start_as_current_span(
|
||||
span_name,
|
||||
kind=SpanKind.CLIENT,
|
||||
) as span:
|
||||
# Extract attributes
|
||||
if is_streaming:
|
||||
attributes = get_stream_attributes(args=(instance,) + args, kwargs=kwargs)
|
||||
else:
|
||||
attributes = get_model_attributes(args=(instance,) + args, kwargs=kwargs)
|
||||
|
||||
# Ensure model name is properly set
|
||||
attributes["gen_ai.request.model"] = model_name
|
||||
|
||||
for key, value in attributes.items():
|
||||
if value is not None:
|
||||
span.set_attribute(key, value)
|
||||
|
||||
try:
|
||||
result = wrapped(*args, **kwargs)
|
||||
|
||||
# Extract response attributes if available
|
||||
if result and hasattr(result, "content"):
|
||||
span.set_attribute("gen_ai.completion.0.content", str(result.content))
|
||||
if result and hasattr(result, "token_usage"):
|
||||
token_usage = result.token_usage
|
||||
if hasattr(token_usage, "input_tokens"):
|
||||
span.set_attribute("gen_ai.usage.prompt_tokens", token_usage.input_tokens)
|
||||
if hasattr(token_usage, "output_tokens"):
|
||||
span.set_attribute("gen_ai.usage.completion_tokens", token_usage.output_tokens)
|
||||
|
||||
return result
|
||||
except Exception as e:
|
||||
span.record_exception(e)
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
def _custom_unwrap(self, **kwargs):
|
||||
"""Remove custom wrapping from SmoLAgents.
|
||||
|
||||
This method removes all custom wrapping we applied.
|
||||
"""
|
||||
# Unwrap all instrumented methods
|
||||
from opentelemetry.instrumentation.utils import unwrap
|
||||
|
||||
try:
|
||||
unwrap("smolagents.agents", "CodeAgent.run")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to unwrap CodeAgent.run: {e}")
|
||||
|
||||
try:
|
||||
unwrap("smolagents.agents", "ToolCallingAgent.run")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to unwrap ToolCallingAgent.run: {e}")
|
||||
|
||||
try:
|
||||
unwrap("smolagents.agents", "ToolCallingAgent.execute_tool_call")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to unwrap ToolCallingAgent.execute_tool_call: {e}")
|
||||
|
||||
try:
|
||||
unwrap("smolagents.models", "LiteLLMModel.generate")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to unwrap LiteLLMModel.generate: {e}")
|
||||
|
||||
try:
|
||||
unwrap("smolagents.models", "LiteLLMModel.generate_stream")
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to unwrap LiteLLMModel.generate_stream: {e}")
|
||||
|
||||
logger.info("SmoLAgents instrumentation disabled")
|
||||
@@ -0,0 +1,258 @@
|
||||
"""Stream wrapper for SmoLAgents model streaming responses."""
|
||||
|
||||
import time
|
||||
import uuid
|
||||
from typing import Any, Generator, Optional
|
||||
from opentelemetry.trace import Status, StatusCode, Span
|
||||
|
||||
from agentops.semconv.message import MessageAttributes
|
||||
from agentops.semconv.agent import AgentAttributes
|
||||
from agentops.semconv.tool import ToolAttributes
|
||||
from .attributes.model import get_stream_attributes
|
||||
from agentops.semconv.span_attributes import SpanAttributes
|
||||
|
||||
|
||||
def model_stream_wrapper(tracer):
|
||||
"""Wrapper for model streaming methods.
|
||||
|
||||
Args:
|
||||
tracer: OpenTelemetry tracer
|
||||
|
||||
Returns:
|
||||
Wrapped function
|
||||
"""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
messages = kwargs.get("messages", [])
|
||||
model_id = instance.model_id if hasattr(instance, "model_id") else "unknown"
|
||||
|
||||
with tracer.start_as_current_span(
|
||||
name=f"{model_id}.generate_stream", attributes=get_stream_attributes(model_id=model_id, messages=messages)
|
||||
) as span:
|
||||
try:
|
||||
# Start streaming
|
||||
stream = wrapped(*args, **kwargs)
|
||||
first_token_received = False
|
||||
start_time = time.time()
|
||||
accumulated_text = ""
|
||||
|
||||
# Process stream
|
||||
for chunk in stream:
|
||||
if not first_token_received:
|
||||
first_token_received = True
|
||||
span.set_attribute("gen_ai.time_to_first_token", time.time() - start_time)
|
||||
|
||||
# Accumulate text and update attributes
|
||||
if hasattr(chunk, "content") and chunk.content:
|
||||
accumulated_text += chunk.content
|
||||
span.set_attribute(MessageAttributes.COMPLETION_CONTENT.format(i=0), accumulated_text)
|
||||
span.set_attribute(MessageAttributes.COMPLETION_TYPE.format(i=0), "text")
|
||||
|
||||
yield chunk
|
||||
|
||||
# Set final attributes
|
||||
span.set_attribute("gen_ai.streaming_duration", time.time() - start_time)
|
||||
span.set_status(Status(StatusCode.OK))
|
||||
|
||||
except Exception as e:
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
def agent_stream_wrapper(tracer):
|
||||
"""Wrapper for agent streaming methods.
|
||||
|
||||
Args:
|
||||
tracer: OpenTelemetry tracer
|
||||
|
||||
Returns:
|
||||
Wrapped function
|
||||
"""
|
||||
|
||||
def wrapper(wrapped, instance, args, kwargs):
|
||||
task = kwargs.get("task", args[0] if args else "unknown")
|
||||
agent_type = instance.__class__.__name__
|
||||
agent_id = str(uuid.uuid4())
|
||||
|
||||
with tracer.start_as_current_span(
|
||||
name=f"{agent_type}.run_stream",
|
||||
attributes={
|
||||
AgentAttributes.AGENT_ID: agent_id,
|
||||
AgentAttributes.AGENT_NAME: agent_type,
|
||||
AgentAttributes.AGENT_ROLE: "executor",
|
||||
AgentAttributes.AGENT_REASONING: task,
|
||||
},
|
||||
) as span:
|
||||
try:
|
||||
# Initialize counters
|
||||
step_count = 0
|
||||
planning_steps = 0
|
||||
tools_used = set()
|
||||
start_time = time.time()
|
||||
|
||||
# Process stream
|
||||
stream = wrapped(*args, **kwargs)
|
||||
for step in stream:
|
||||
step_count += 1
|
||||
|
||||
# Track step types
|
||||
if hasattr(step, "type"):
|
||||
if step.type == "planning":
|
||||
planning_steps += 1
|
||||
elif step.type == "tool_call":
|
||||
tools_used.add(step.tool_name)
|
||||
# Add tool-specific attributes
|
||||
span.set_attribute(ToolAttributes.TOOL_NAME, step.tool_name)
|
||||
if hasattr(step, "arguments"):
|
||||
span.set_attribute(ToolAttributes.TOOL_PARAMETERS, step.arguments)
|
||||
|
||||
# Update span attributes
|
||||
span.set_attribute("agent.step_count", step_count)
|
||||
span.set_attribute("agent.planning_steps", planning_steps)
|
||||
span.set_attribute(AgentAttributes.AGENT_TOOLS, list(tools_used))
|
||||
|
||||
yield step
|
||||
|
||||
# Set final attributes
|
||||
span.set_attribute("agent.execution_time", time.time() - start_time)
|
||||
span.set_status(Status(StatusCode.OK))
|
||||
|
||||
except Exception as e:
|
||||
span.set_status(Status(StatusCode.ERROR, str(e)))
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
class SmoLAgentsStreamWrapper:
|
||||
"""Wrapper for streaming responses from SmoLAgents models."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
stream: Generator,
|
||||
span: Span,
|
||||
model_id: Optional[str] = None,
|
||||
):
|
||||
"""Initialize the stream wrapper.
|
||||
|
||||
Args:
|
||||
stream: The original generator from the model
|
||||
span: The OpenTelemetry span to track the stream
|
||||
model_id: Optional model identifier
|
||||
"""
|
||||
self._stream = stream
|
||||
self._span = span
|
||||
self._model_id = model_id
|
||||
self._chunks_received = 0
|
||||
self._full_content = []
|
||||
self._tool_calls = []
|
||||
self._current_tool_call = None
|
||||
self._token_count = 0
|
||||
|
||||
def __iter__(self):
|
||||
"""Iterate over the stream."""
|
||||
return self
|
||||
|
||||
def __next__(self):
|
||||
"""Get the next chunk from the stream."""
|
||||
try:
|
||||
chunk = next(self._stream)
|
||||
self._process_chunk(chunk)
|
||||
return chunk
|
||||
except StopIteration:
|
||||
self._finalize_stream()
|
||||
raise
|
||||
|
||||
def _process_chunk(self, chunk: Any) -> None:
|
||||
"""Process a chunk from the stream.
|
||||
|
||||
Args:
|
||||
chunk: The chunk to process
|
||||
"""
|
||||
self._chunks_received += 1
|
||||
|
||||
# Handle ChatMessageStreamDelta objects
|
||||
if hasattr(chunk, "content") and chunk.content:
|
||||
self._full_content.append(chunk.content)
|
||||
|
||||
# Handle tool calls in chunks
|
||||
if hasattr(chunk, "tool_calls") and chunk.tool_calls:
|
||||
for tool_call in chunk.tool_calls:
|
||||
if tool_call.id not in [tc["id"] for tc in self._tool_calls]:
|
||||
self._tool_calls.append(
|
||||
{
|
||||
"id": tool_call.id,
|
||||
"type": tool_call.type,
|
||||
"name": tool_call.function.name,
|
||||
"arguments": tool_call.function.arguments,
|
||||
}
|
||||
)
|
||||
|
||||
# Track token usage if available
|
||||
if hasattr(chunk, "token_usage") and chunk.token_usage:
|
||||
if hasattr(chunk.token_usage, "output_tokens"):
|
||||
self._token_count += chunk.token_usage.output_tokens
|
||||
|
||||
# Update span with chunk information
|
||||
self._span.add_event(
|
||||
"stream_chunk_received",
|
||||
{
|
||||
"chunk_number": self._chunks_received,
|
||||
"chunk_content_length": len(chunk.content) if hasattr(chunk, "content") and chunk.content else 0,
|
||||
},
|
||||
)
|
||||
|
||||
def _finalize_stream(self) -> None:
|
||||
"""Finalize the stream and update span attributes."""
|
||||
# Combine all content chunks
|
||||
full_content = "".join(self._full_content)
|
||||
|
||||
# Set final attributes on the span
|
||||
attributes = {
|
||||
MessageAttributes.COMPLETION_CONTENT.format(i=0): full_content,
|
||||
"stream.chunks_received": self._chunks_received,
|
||||
"stream.total_content_length": len(full_content),
|
||||
}
|
||||
|
||||
# Add tool calls if any
|
||||
if self._tool_calls:
|
||||
for j, tool_call in enumerate(self._tool_calls):
|
||||
attributes.update(
|
||||
{
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=j): tool_call["id"],
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_TYPE.format(i=0, j=j): tool_call["type"],
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=j): tool_call["name"],
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=j): str(tool_call["arguments"]),
|
||||
}
|
||||
)
|
||||
|
||||
# Add token usage if tracked
|
||||
if self._token_count > 0:
|
||||
attributes[SpanAttributes.LLM_USAGE_STREAMING_TOKENS] = self._token_count
|
||||
|
||||
self._span.set_attributes(attributes)
|
||||
|
||||
def close(self) -> None:
|
||||
"""Close the stream wrapper."""
|
||||
if hasattr(self._stream, "close"):
|
||||
self._stream.close()
|
||||
|
||||
|
||||
def wrap_stream(
|
||||
stream: Generator,
|
||||
span: Span,
|
||||
model_id: Optional[str] = None,
|
||||
) -> SmoLAgentsStreamWrapper:
|
||||
"""Wrap a streaming response from a SmoLAgents model.
|
||||
|
||||
Args:
|
||||
stream: The original generator from the model
|
||||
span: The OpenTelemetry span to track the stream
|
||||
model_id: Optional model identifier
|
||||
|
||||
Returns:
|
||||
SmoLAgentsStreamWrapper: The wrapped stream
|
||||
"""
|
||||
return SmoLAgentsStreamWrapper(stream, span, model_id)
|
||||
@@ -0,0 +1,15 @@
|
||||
"""Xpander SDK instrumentation for AgentOps."""
|
||||
|
||||
from agentops.instrumentation.agentic.xpander.instrumentor import XpanderInstrumentor
|
||||
from agentops.instrumentation.agentic.xpander.trace_probe import (
|
||||
wrap_openai_call_for_xpander,
|
||||
is_xpander_session_active,
|
||||
get_active_xpander_session,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"XpanderInstrumentor",
|
||||
"wrap_openai_call_for_xpander",
|
||||
"is_xpander_session_active",
|
||||
"get_active_xpander_session",
|
||||
]
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Xpander context management for session tracking."""
|
||||
|
||||
import time
|
||||
import threading
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
|
||||
class XpanderContext:
|
||||
"""Context manager for Xpander sessions with nested conversation spans."""
|
||||
|
||||
def __init__(self):
|
||||
self._sessions = {} # session_id -> session_data
|
||||
self._workflow_spans = {} # session_id -> active workflow span
|
||||
self._agent_spans = {} # session_id -> active agent span
|
||||
self._conversation_spans = {} # session_id -> active conversation span
|
||||
self._conversation_counters = {} # session_id -> conversation counter
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def start_session(self, session_id: str, agent_info: Dict[str, Any], workflow_span=None, agent_span=None) -> None:
|
||||
"""Start a new session with agent info."""
|
||||
with self._lock:
|
||||
self._sessions[session_id] = {
|
||||
"agent_name": agent_info.get("agent_name", "unknown"),
|
||||
"agent_id": agent_info.get("agent_id", "unknown"),
|
||||
"task_input": agent_info.get("task_input"),
|
||||
"phase": "planning",
|
||||
"step_count": 0,
|
||||
"total_tokens": 0,
|
||||
"tools_executed": [],
|
||||
"start_time": time.time(),
|
||||
}
|
||||
if workflow_span:
|
||||
self._workflow_spans[session_id] = workflow_span
|
||||
if agent_span:
|
||||
self._agent_spans[session_id] = agent_span
|
||||
|
||||
# Initialize conversation counter
|
||||
self._conversation_counters[session_id] = 0
|
||||
|
||||
def start_conversation(self, session_id: str, conversation_span) -> None:
|
||||
"""Start a new conversation within the session."""
|
||||
with self._lock:
|
||||
self._conversation_spans[session_id] = conversation_span
|
||||
self._conversation_counters[session_id] = self._conversation_counters.get(session_id, 0) + 1
|
||||
|
||||
def end_conversation(self, session_id: str) -> None:
|
||||
"""End the current conversation."""
|
||||
with self._lock:
|
||||
if session_id in self._conversation_spans:
|
||||
del self._conversation_spans[session_id]
|
||||
|
||||
def has_active_conversation(self, session_id: str) -> bool:
|
||||
"""Check if there's an active conversation for this session."""
|
||||
with self._lock:
|
||||
return session_id in self._conversation_spans
|
||||
|
||||
def get_conversation_counter(self, session_id: str) -> int:
|
||||
"""Get the current conversation counter."""
|
||||
with self._lock:
|
||||
return self._conversation_counters.get(session_id, 0)
|
||||
|
||||
def get_session(self, session_id: str) -> Optional[Dict[str, Any]]:
|
||||
"""Get session data."""
|
||||
with self._lock:
|
||||
return self._sessions.get(session_id)
|
||||
|
||||
def update_session(self, session_id: str, updates: Dict[str, Any]) -> None:
|
||||
"""Update session data."""
|
||||
with self._lock:
|
||||
if session_id in self._sessions:
|
||||
self._sessions[session_id].update(updates)
|
||||
|
||||
def end_session(self, session_id: str) -> None:
|
||||
"""End a session."""
|
||||
with self._lock:
|
||||
if session_id in self._sessions:
|
||||
del self._sessions[session_id]
|
||||
if session_id in self._workflow_spans:
|
||||
del self._workflow_spans[session_id]
|
||||
if session_id in self._agent_spans:
|
||||
del self._agent_spans[session_id]
|
||||
if session_id in self._conversation_spans:
|
||||
del self._conversation_spans[session_id]
|
||||
if session_id in self._conversation_counters:
|
||||
del self._conversation_counters[session_id]
|
||||
|
||||
def get_workflow_phase(self, session_id: str) -> str:
|
||||
"""Detect current workflow phase based on state."""
|
||||
with self._lock:
|
||||
session = self._sessions.get(session_id, {})
|
||||
|
||||
if session.get("tools_executed", []):
|
||||
return "executing"
|
||||
elif session.get("step_count", 0) > 0:
|
||||
return "executing"
|
||||
else:
|
||||
return "planning"
|
||||
|
||||
def get_workflow_span(self, session_id: str):
|
||||
"""Get the active workflow span for a session."""
|
||||
with self._lock:
|
||||
return self._workflow_spans.get(session_id)
|
||||
|
||||
def get_agent_span(self, session_id: str):
|
||||
"""Get the active agent span for a session."""
|
||||
with self._lock:
|
||||
return self._agent_spans.get(session_id)
|
||||
|
||||
def get_conversation_span(self, session_id: str):
|
||||
"""Get the active conversation span for a session."""
|
||||
with self._lock:
|
||||
return self._conversation_spans.get(session_id)
|
||||
@@ -0,0 +1,877 @@
|
||||
"""Xpander SDK instrumentation for AgentOps.
|
||||
|
||||
This module provides instrumentation for the Xpander SDK, which uses JSII to convert
|
||||
TypeScript code to Python at runtime. The instrumentation tracks agent sessions,
|
||||
tool executions, and LLM interactions.
|
||||
|
||||
MODIFIED VERSION: Using existing AgentOps utilities where possible while keeping
|
||||
runtime-specific instrumentation logic that cannot be replaced.
|
||||
|
||||
REPLACEMENTS MADE:
|
||||
✅ Span creation: Using tracer.make_span() instead of manual span creation
|
||||
✅ Error handling: Using _finish_span_success/_finish_span_error utilities
|
||||
✅ Attribute management: Using existing SpanAttributeManager
|
||||
✅ Serialization: Using safe_serialize and model_to_dict utilities
|
||||
✅ Attribute setting: Using _update_span utility
|
||||
|
||||
RUNTIME-SPECIFIC LOGIC KEPT (Cannot be replaced):
|
||||
❌ Method wrapping: Runtime method creation requires custom hooks
|
||||
❌ Context persistence: XpanderContext must handle runtime object lifecycle
|
||||
❌ Agent detection: Custom logic for dynamically created agents
|
||||
"""
|
||||
|
||||
import logging
|
||||
import time
|
||||
import json
|
||||
from typing import Any, Optional
|
||||
from opentelemetry.metrics import Meter
|
||||
from opentelemetry.trace import SpanKind as OTelSpanKind
|
||||
from opentelemetry import trace
|
||||
|
||||
# Use existing AgentOps utilities
|
||||
from agentops.instrumentation.common import (
|
||||
CommonInstrumentor,
|
||||
InstrumentorConfig,
|
||||
StandardMetrics,
|
||||
)
|
||||
from agentops.instrumentation.common.span_management import SpanAttributeManager
|
||||
from agentops.instrumentation.common.wrappers import _finish_span_success, _finish_span_error, _update_span
|
||||
from agentops.helpers.serialization import safe_serialize, model_to_dict
|
||||
from agentops.sdk.core import tracer
|
||||
from agentops.instrumentation.agentic.xpander.context import XpanderContext
|
||||
from agentops.semconv import SpanAttributes, SpanKind, ToolAttributes
|
||||
from agentops.semconv.message import MessageAttributes
|
||||
|
||||
# Use existing OpenAI attribute extraction patterns (lazy import to avoid circular imports)
|
||||
# from agentops.instrumentation.providers.openai.attributes.common import (
|
||||
# get_response_attributes,
|
||||
# )
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_instruments = ("xpander-sdk >= 1.0.0",)
|
||||
|
||||
|
||||
# Use existing AgentOps utility instead of custom implementation
|
||||
def safe_set_attribute(span, key: str, value: Any) -> None:
|
||||
"""Set attribute on span using existing AgentOps utility."""
|
||||
try:
|
||||
_update_span(span, {key: value})
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to set attribute {key}: {e}")
|
||||
|
||||
|
||||
class XpanderInstrumentor(CommonInstrumentor):
|
||||
"""Instrumentor for Xpander SDK interactions."""
|
||||
|
||||
def __init__(self, config: Optional[InstrumentorConfig] = None):
|
||||
if config is None:
|
||||
config = InstrumentorConfig(
|
||||
library_name="xpander-sdk", library_version="1.0.0", dependencies=_instruments, metrics_enabled=True
|
||||
)
|
||||
super().__init__(config)
|
||||
self._context = XpanderContext()
|
||||
self._tracer = None
|
||||
# Use existing AgentOps attribute manager
|
||||
self._attribute_manager = SpanAttributeManager("xpander-service", "production")
|
||||
|
||||
def _get_session_id_from_agent(self, agent) -> str:
|
||||
"""Generate consistent session ID from agent."""
|
||||
# First try to get memory_thread_id from agent context if available
|
||||
if hasattr(agent, "memory_thread_id"):
|
||||
return f"session_{agent.memory_thread_id}"
|
||||
|
||||
# Check for execution context
|
||||
if hasattr(agent, "execution") and hasattr(agent.execution, "memory_thread_id"):
|
||||
return f"session_{agent.execution.memory_thread_id}"
|
||||
|
||||
# Fallback to agent-based ID
|
||||
agent_name = getattr(agent, "name", "unknown")
|
||||
agent_id = getattr(agent, "id", "unknown")
|
||||
return f"agent_{agent_name}_{agent_id}"
|
||||
|
||||
def _extract_session_id(self, execution, agent=None) -> str:
|
||||
"""Extract session ID from execution data."""
|
||||
if isinstance(execution, dict):
|
||||
if "memory_thread_id" in execution:
|
||||
return f"session_{execution['memory_thread_id']}"
|
||||
elif "thread_id" in execution:
|
||||
return f"session_{execution['thread_id']}"
|
||||
elif "session_id" in execution:
|
||||
return f"session_{execution['session_id']}"
|
||||
|
||||
# Fallback to agent-based ID if available
|
||||
if agent:
|
||||
return self._get_session_id_from_agent(agent)
|
||||
|
||||
# Last resort fallback
|
||||
return f"session_{int(time.time())}"
|
||||
|
||||
def _extract_tool_name(self, tool_call) -> str:
|
||||
"""Extract tool name from tool call."""
|
||||
# Handle different tool call formats
|
||||
if hasattr(tool_call, "function_name"):
|
||||
return tool_call.function_name
|
||||
elif hasattr(tool_call, "function") and hasattr(tool_call.function, "name"):
|
||||
return tool_call.function.name
|
||||
elif hasattr(tool_call, "name"):
|
||||
return tool_call.name
|
||||
elif isinstance(tool_call, dict):
|
||||
if "function" in tool_call:
|
||||
return tool_call["function"].get("name", "unknown")
|
||||
elif "function_name" in tool_call:
|
||||
return tool_call["function_name"]
|
||||
elif "name" in tool_call:
|
||||
return tool_call["name"]
|
||||
|
||||
# Try to extract from string representation
|
||||
import re
|
||||
|
||||
patterns = [
|
||||
r'function[\'"]\s*:\s*[\'"]([^\'"]+)[\'"]',
|
||||
r'name[\'"]\s*:\s*[\'"]([^\'"]+)[\'"]',
|
||||
r"([a-zA-Z_][a-zA-Z0-9_]*)\.tool",
|
||||
r'function_name[\'"]\s*:\s*[\'"]([^\'"]+)[\'"]',
|
||||
]
|
||||
|
||||
tool_str = str(tool_call)
|
||||
for pattern in patterns:
|
||||
match = re.search(pattern, tool_str, re.IGNORECASE)
|
||||
if match:
|
||||
return match.group(1)
|
||||
|
||||
return "unknown"
|
||||
|
||||
def _extract_tool_params(self, tool_call) -> dict:
|
||||
"""Extract tool parameters from tool call."""
|
||||
# Handle different parameter formats
|
||||
if hasattr(tool_call, "function") and hasattr(tool_call.function, "arguments"):
|
||||
try:
|
||||
args = tool_call.function.arguments
|
||||
if isinstance(args, str):
|
||||
return json.loads(args)
|
||||
elif isinstance(args, dict):
|
||||
return args
|
||||
except (json.JSONDecodeError, AttributeError):
|
||||
pass
|
||||
elif hasattr(tool_call, "arguments"):
|
||||
try:
|
||||
args = tool_call.arguments
|
||||
if isinstance(args, str):
|
||||
return json.loads(args)
|
||||
elif isinstance(args, dict):
|
||||
return args
|
||||
except (json.JSONDecodeError, AttributeError):
|
||||
pass
|
||||
elif isinstance(tool_call, dict):
|
||||
if "function" in tool_call:
|
||||
args = tool_call["function"].get("arguments", "{}")
|
||||
try:
|
||||
return json.loads(args) if isinstance(args, str) else args
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
elif "arguments" in tool_call:
|
||||
args = tool_call["arguments"]
|
||||
try:
|
||||
return json.loads(args) if isinstance(args, str) else args
|
||||
except json.JSONDecodeError:
|
||||
pass
|
||||
|
||||
return {}
|
||||
|
||||
def _extract_llm_data_from_messages(self, messages) -> dict:
|
||||
"""Extract LLM metadata from messages."""
|
||||
data = {}
|
||||
|
||||
if isinstance(messages, dict):
|
||||
# Direct model and usage fields
|
||||
if "model" in messages:
|
||||
data["model"] = messages["model"]
|
||||
if "usage" in messages:
|
||||
data["usage"] = messages["usage"]
|
||||
|
||||
# Check in choices array (OpenAI format)
|
||||
if "choices" in messages and messages["choices"]:
|
||||
choice = messages["choices"][0]
|
||||
if "message" in choice:
|
||||
message = choice["message"]
|
||||
if "model" in message:
|
||||
data["model"] = message["model"]
|
||||
|
||||
elif isinstance(messages, list):
|
||||
# Look for assistant messages with metadata
|
||||
for msg in messages:
|
||||
if isinstance(msg, dict) and msg.get("role") == "assistant":
|
||||
if "model" in msg:
|
||||
data["model"] = msg["model"]
|
||||
if "usage" in msg:
|
||||
data["usage"] = msg["usage"]
|
||||
break
|
||||
|
||||
# Try to extract from any nested structures
|
||||
if not data and hasattr(messages, "__dict__"):
|
||||
msg_dict = messages.__dict__
|
||||
if "model" in msg_dict:
|
||||
data["model"] = msg_dict["model"]
|
||||
if "usage" in msg_dict:
|
||||
data["usage"] = msg_dict["usage"]
|
||||
|
||||
return data
|
||||
|
||||
def _extract_and_set_openai_message_attributes(self, span, messages, result, agent=None):
|
||||
"""Extract and set OpenAI message attributes from messages and response."""
|
||||
try:
|
||||
# Manual extraction since we don't need the OpenAI module for this
|
||||
# Try to get the agent's current message history for prompts
|
||||
agent_messages = []
|
||||
if agent and hasattr(agent, "messages"):
|
||||
agent_messages = getattr(agent, "messages", [])
|
||||
elif agent and hasattr(agent, "conversation_history"):
|
||||
agent_messages = getattr(agent, "conversation_history", [])
|
||||
elif agent and hasattr(agent, "history"):
|
||||
agent_messages = getattr(agent, "history", [])
|
||||
|
||||
# Also try to extract messages from the messages parameter itself
|
||||
if isinstance(messages, list):
|
||||
# If messages is a list of messages, use it directly
|
||||
agent_messages.extend(messages)
|
||||
elif isinstance(messages, dict) and "messages" in messages:
|
||||
# If messages contains a messages key
|
||||
agent_messages.extend(messages.get("messages", []))
|
||||
|
||||
# Set prompt messages (input to LLM)
|
||||
prompt_index = 0
|
||||
for msg in agent_messages[-10:]: # Get last 10 messages to avoid huge context
|
||||
if isinstance(msg, dict):
|
||||
role = msg.get("role", "user")
|
||||
content = msg.get("content", "")
|
||||
|
||||
# Handle different content formats
|
||||
if content and isinstance(content, str) and content.strip():
|
||||
safe_set_attribute(span, MessageAttributes.PROMPT_ROLE.format(i=prompt_index), role)
|
||||
safe_set_attribute(
|
||||
span, MessageAttributes.PROMPT_CONTENT.format(i=prompt_index), content[:2000]
|
||||
)
|
||||
prompt_index += 1
|
||||
elif content and isinstance(content, list):
|
||||
# Handle multi-modal content
|
||||
content_str = str(content)[:2000]
|
||||
safe_set_attribute(span, MessageAttributes.PROMPT_ROLE.format(i=prompt_index), role)
|
||||
safe_set_attribute(span, MessageAttributes.PROMPT_CONTENT.format(i=prompt_index), content_str)
|
||||
prompt_index += 1
|
||||
elif hasattr(msg, "content"):
|
||||
# Handle object with content attribute
|
||||
content = getattr(msg, "content", "")
|
||||
role = getattr(msg, "role", "user")
|
||||
if content and isinstance(content, str) and content.strip():
|
||||
safe_set_attribute(span, MessageAttributes.PROMPT_ROLE.format(i=prompt_index), role)
|
||||
safe_set_attribute(
|
||||
span, MessageAttributes.PROMPT_CONTENT.format(i=prompt_index), str(content)[:2000]
|
||||
)
|
||||
prompt_index += 1
|
||||
|
||||
# Set completion messages (response from LLM)
|
||||
completion_index = 0
|
||||
response_data = result if result else messages
|
||||
|
||||
# Handle different response formats
|
||||
if isinstance(response_data, dict):
|
||||
choices = response_data.get("choices", [])
|
||||
for choice in choices:
|
||||
message = choice.get("message", {})
|
||||
role = message.get("role", "assistant")
|
||||
content = message.get("content", "")
|
||||
|
||||
if content:
|
||||
safe_set_attribute(span, MessageAttributes.COMPLETION_ROLE.format(i=completion_index), role)
|
||||
safe_set_attribute(
|
||||
span, MessageAttributes.COMPLETION_CONTENT.format(i=completion_index), content[:2000]
|
||||
)
|
||||
|
||||
# Handle tool calls in the response
|
||||
tool_calls = message.get("tool_calls", [])
|
||||
for j, tool_call in enumerate(tool_calls):
|
||||
tool_id = tool_call.get("id", "")
|
||||
tool_name = tool_call.get("function", {}).get("name", "")
|
||||
tool_args = tool_call.get("function", {}).get("arguments", "")
|
||||
|
||||
if tool_id:
|
||||
safe_set_attribute(
|
||||
span, MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=completion_index, j=j), tool_id
|
||||
)
|
||||
if tool_name:
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=completion_index, j=j),
|
||||
tool_name,
|
||||
)
|
||||
if tool_args:
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=completion_index, j=j),
|
||||
tool_args[:500],
|
||||
)
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_TYPE.format(i=completion_index, j=j),
|
||||
"function",
|
||||
)
|
||||
|
||||
completion_index += 1
|
||||
elif hasattr(response_data, "choices"):
|
||||
# Handle response object with choices attribute
|
||||
choices = getattr(response_data, "choices", [])
|
||||
for choice in choices:
|
||||
message = getattr(choice, "message", None)
|
||||
if message:
|
||||
role = getattr(message, "role", "assistant")
|
||||
content = getattr(message, "content", "")
|
||||
|
||||
if content:
|
||||
safe_set_attribute(span, MessageAttributes.COMPLETION_ROLE.format(i=completion_index), role)
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_CONTENT.format(i=completion_index),
|
||||
str(content)[:2000],
|
||||
)
|
||||
|
||||
# Handle tool calls
|
||||
tool_calls = getattr(message, "tool_calls", [])
|
||||
for j, tool_call in enumerate(tool_calls):
|
||||
tool_id = getattr(tool_call, "id", "")
|
||||
function = getattr(tool_call, "function", None)
|
||||
if function:
|
||||
tool_name = getattr(function, "name", "")
|
||||
tool_args = getattr(function, "arguments", "")
|
||||
|
||||
if tool_id:
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=completion_index, j=j),
|
||||
tool_id,
|
||||
)
|
||||
if tool_name:
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=completion_index, j=j),
|
||||
tool_name,
|
||||
)
|
||||
if tool_args:
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(
|
||||
i=completion_index, j=j
|
||||
),
|
||||
str(tool_args)[:500],
|
||||
)
|
||||
safe_set_attribute(
|
||||
span,
|
||||
MessageAttributes.COMPLETION_TOOL_CALL_TYPE.format(i=completion_index, j=j),
|
||||
"function",
|
||||
)
|
||||
|
||||
completion_index += 1
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error extracting OpenAI message attributes: {e}")
|
||||
|
||||
def _wrap_init_task(self, original_method):
|
||||
"""Wrap init_task and add_task to create agent span hierarchy."""
|
||||
instrumentor = self
|
||||
|
||||
def wrapper(self, execution=None, input=None, **kwargs):
|
||||
# Normalize parameters - handle both add_task(input=...) and init_task(execution=...)
|
||||
if execution is None and input is not None:
|
||||
# add_task call with input parameter - normalize to execution format
|
||||
if isinstance(input, str):
|
||||
execution = {"input": {"text": input}}
|
||||
else:
|
||||
execution = {"input": input}
|
||||
elif execution is None:
|
||||
# Neither execution nor input provided - create empty execution
|
||||
execution = {}
|
||||
|
||||
# Extract session ID and agent info
|
||||
session_id = instrumentor._extract_session_id(execution)
|
||||
agent_name = getattr(self, "name", "unknown")
|
||||
agent_id = getattr(self, "id", "unknown")
|
||||
|
||||
# Check if session already exists
|
||||
existing_session = instrumentor._context.get_session(session_id)
|
||||
if existing_session:
|
||||
# Session already exists, just continue
|
||||
# Call with original parameters
|
||||
if input is not None:
|
||||
result = original_method(self, input=input, **kwargs)
|
||||
else:
|
||||
result = original_method(self, execution)
|
||||
return result
|
||||
|
||||
# Extract task input
|
||||
task_input = None
|
||||
if isinstance(execution, dict):
|
||||
if "input" in execution:
|
||||
input_data = execution["input"]
|
||||
if isinstance(input_data, dict) and "text" in input_data:
|
||||
task_input = input_data["text"]
|
||||
elif isinstance(input_data, str):
|
||||
task_input = input_data
|
||||
|
||||
# Create top-level conversation/session span - this is the ROOT span
|
||||
conversation_span_attributes = {
|
||||
SpanAttributes.AGENTOPS_ENTITY_NAME: f"Session - {agent_name}",
|
||||
"xpander.span.type": "session",
|
||||
"xpander.session.name": f"Session - {agent_name}",
|
||||
"xpander.agent.name": agent_name,
|
||||
"xpander.agent.id": agent_id,
|
||||
"xpander.session.id": session_id,
|
||||
}
|
||||
session_span, session_ctx, session_token = tracer.make_span(
|
||||
operation_name=f"session.{agent_name}",
|
||||
span_kind=SpanKind.AGENT, # Use AGENT kind for the root session span
|
||||
attributes=conversation_span_attributes,
|
||||
)
|
||||
|
||||
# Set task input on session span
|
||||
if task_input:
|
||||
safe_set_attribute(session_span, SpanAttributes.AGENTOPS_ENTITY_INPUT, task_input[:1000])
|
||||
safe_set_attribute(session_span, "xpander.session.initial_input", task_input[:500])
|
||||
|
||||
# Create workflow span as child of session span (this will be the main execution span)
|
||||
trace.set_span_in_context(session_span)
|
||||
workflow_span_attributes = {
|
||||
"xpander.span.type": "workflow",
|
||||
"xpander.workflow.phase": "planning",
|
||||
"xpander.agent.name": agent_name,
|
||||
"xpander.agent.id": agent_id,
|
||||
"xpander.session.id": session_id,
|
||||
"agent.name": agent_name,
|
||||
"agent.id": agent_id,
|
||||
}
|
||||
workflow_span, workflow_ctx, workflow_token = tracer.make_span(
|
||||
operation_name=f"workflow.{agent_name}",
|
||||
span_kind=SpanKind.WORKFLOW,
|
||||
attributes=workflow_span_attributes,
|
||||
)
|
||||
|
||||
# No separate agent span - workflow span contains all agent info
|
||||
|
||||
# Initialize workflow state with persistent spans
|
||||
agent_info = {
|
||||
"agent_name": agent_name,
|
||||
"agent_id": agent_id,
|
||||
"task_input": task_input,
|
||||
"thread_id": execution.get("memory_thread_id") if isinstance(execution, dict) else None,
|
||||
}
|
||||
instrumentor._context.start_session(session_id, agent_info, workflow_span, None) # No agent span
|
||||
# Store the session span as well
|
||||
instrumentor._context.start_conversation(session_id, session_span)
|
||||
|
||||
try:
|
||||
# Execute original method - don't end agent span here, it will be ended in retrieve_execution_result
|
||||
# Call with original parameters
|
||||
if input is not None:
|
||||
result = original_method(self, input=input, **kwargs)
|
||||
else:
|
||||
result = original_method(self, execution)
|
||||
return result
|
||||
except Exception as e:
|
||||
# Use existing AgentOps error handling utilities
|
||||
_finish_span_error(workflow_span, e)
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
def _wrap_run_tools(self, original_method):
|
||||
"""Wrap run_tools to create execution phase tool spans."""
|
||||
instrumentor = self
|
||||
|
||||
def wrapper(self, tool_calls, payload_extension=None):
|
||||
session_id = instrumentor._get_session_id_from_agent(self)
|
||||
current_session = instrumentor._context.get_session(session_id)
|
||||
|
||||
# Update workflow state
|
||||
step_num = (current_session.get("step_count", 0) + 1) if current_session else 1
|
||||
instrumentor._context.update_session(
|
||||
session_id,
|
||||
{
|
||||
"step_count": step_num,
|
||||
"phase": "executing",
|
||||
"tools_executed": (current_session.get("tools_executed", []) if current_session else [])
|
||||
+ [instrumentor._extract_tool_name(tc) for tc in tool_calls],
|
||||
},
|
||||
)
|
||||
|
||||
# Get current span context (should be the LLM span)
|
||||
current_span = trace.get_current_span()
|
||||
|
||||
# Create execution phase span as child of current LLM span
|
||||
execution_span_context = trace.set_span_in_context(current_span) if current_span else None
|
||||
|
||||
with instrumentor._tracer.start_as_current_span(
|
||||
"xpander.execution",
|
||||
kind=OTelSpanKind.INTERNAL,
|
||||
context=execution_span_context,
|
||||
attributes={
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: SpanKind.TASK,
|
||||
"xpander.span.type": "execution",
|
||||
"xpander.workflow.phase": "executing",
|
||||
"xpander.step.number": step_num,
|
||||
"xpander.step.tool_count": len(tool_calls),
|
||||
"xpander.session.id": session_id,
|
||||
},
|
||||
) as execution_span:
|
||||
# Execute tools and create individual tool spans
|
||||
results = []
|
||||
conversation_finished = False
|
||||
|
||||
for i, tool_call in enumerate(tool_calls):
|
||||
tool_name = instrumentor._extract_tool_name(tool_call)
|
||||
tool_params = instrumentor._extract_tool_params(tool_call)
|
||||
|
||||
# Check if this is the conversation finish tool
|
||||
if tool_name == "xpfinish-agent-execution-finished":
|
||||
conversation_finished = True
|
||||
|
||||
start_time = time.time()
|
||||
|
||||
# Create tool span as child of execution span
|
||||
tool_span_context = trace.set_span_in_context(execution_span)
|
||||
|
||||
with instrumentor._tracer.start_as_current_span(
|
||||
f"tool.{tool_name}",
|
||||
kind=OTelSpanKind.CLIENT,
|
||||
context=tool_span_context,
|
||||
attributes={
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: SpanKind.TOOL,
|
||||
ToolAttributes.TOOL_NAME: tool_name,
|
||||
ToolAttributes.TOOL_PARAMETERS: str(tool_params)[:500],
|
||||
"xpander.span.type": "tool",
|
||||
"xpander.workflow.phase": "executing",
|
||||
"xpander.tool.step": step_num,
|
||||
"xpander.tool.index": i,
|
||||
},
|
||||
) as tool_span:
|
||||
# Execute single tool
|
||||
single_result = original_method(self, [tool_call], payload_extension)
|
||||
results.extend(single_result)
|
||||
|
||||
# Record tool execution details
|
||||
execution_time = time.time() - start_time
|
||||
safe_set_attribute(tool_span, "xpander.tool.execution_time", execution_time)
|
||||
|
||||
# Add tool result if available
|
||||
if single_result:
|
||||
result_summary = f"Executed successfully with {len(single_result)} results"
|
||||
safe_set_attribute(tool_span, "xpander.tool.result_summary", result_summary)
|
||||
|
||||
# Store actual result data using existing AgentOps utilities
|
||||
try:
|
||||
result_content = ""
|
||||
|
||||
for i, result_item in enumerate(single_result):
|
||||
# Handle xpander_sdk.ToolCallResult objects specifically
|
||||
if hasattr(result_item, "__class__") and "ToolCallResult" in str(type(result_item)):
|
||||
# Extract the actual result content from ToolCallResult
|
||||
try:
|
||||
if hasattr(result_item, "result") and result_item.result is not None:
|
||||
actual_result = result_item.result
|
||||
if isinstance(actual_result, str):
|
||||
result_content += actual_result[:1000] + "\n"
|
||||
else:
|
||||
result_content += safe_serialize(actual_result)[:1000] + "\n"
|
||||
elif hasattr(result_item, "data") and result_item.data is not None:
|
||||
result_content += safe_serialize(result_item.data)[:1000] + "\n"
|
||||
else:
|
||||
# Fallback: try to find any content attribute
|
||||
for attr_name in ["content", "output", "value", "response"]:
|
||||
if hasattr(result_item, attr_name):
|
||||
attr_value = getattr(result_item, attr_name)
|
||||
if attr_value is not None:
|
||||
result_content += safe_serialize(attr_value)[:1000] + "\n"
|
||||
break
|
||||
else:
|
||||
# If no content attributes found, indicate this
|
||||
result_content += "ToolCallResult object (no extractable content)\n"
|
||||
except Exception as attr_e:
|
||||
logger.debug(f"Error extracting from ToolCallResult: {attr_e}")
|
||||
result_content += "ToolCallResult object (extraction failed)\n"
|
||||
|
||||
# Handle regular objects and primitives
|
||||
elif isinstance(result_item, (str, int, float, bool)):
|
||||
result_content += str(result_item)[:1000] + "\n"
|
||||
elif hasattr(result_item, "__dict__"):
|
||||
# Convert objects to dict using existing utility
|
||||
result_dict = model_to_dict(result_item)
|
||||
result_content += safe_serialize(result_dict)[:1000] + "\n"
|
||||
else:
|
||||
# Use safe_serialize for consistent conversion
|
||||
result_content += safe_serialize(result_item)[:1000] + "\n"
|
||||
|
||||
if result_content.strip():
|
||||
final_content = result_content.strip()[:2000]
|
||||
safe_set_attribute(tool_span, ToolAttributes.TOOL_RESULT, final_content)
|
||||
else:
|
||||
safe_set_attribute(
|
||||
tool_span, ToolAttributes.TOOL_RESULT, "No extractable content found"
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error setting tool result: {e}")
|
||||
safe_set_attribute(
|
||||
tool_span, ToolAttributes.TOOL_RESULT, f"Error capturing result: {e}"
|
||||
)
|
||||
else:
|
||||
safe_set_attribute(tool_span, "xpander.tool.result_summary", "No results returned")
|
||||
|
||||
# If conversation is finished, mark for session closure
|
||||
if conversation_finished:
|
||||
# Since session span is now the conversation span, we need to close all spans
|
||||
# when the conversation finishes
|
||||
pass # Session closure will be handled in retrieve_execution_result
|
||||
|
||||
return results
|
||||
|
||||
return wrapper
|
||||
|
||||
def _wrap_add_messages(self, original_method):
|
||||
"""Wrap add_messages to create LLM spans with proper parent-child relationship."""
|
||||
instrumentor = self
|
||||
|
||||
def wrapper(self, messages):
|
||||
session_id = instrumentor._get_session_id_from_agent(self)
|
||||
current_session = instrumentor._context.get_session(session_id)
|
||||
current_phase = instrumentor._context.get_workflow_phase(session_id)
|
||||
workflow_span = instrumentor._context.get_workflow_span(session_id)
|
||||
|
||||
# Create LLM span as child of workflow span (not conversation span)
|
||||
# The hierarchy should be: session -> agent/workflow -> LLM -> execution -> tools
|
||||
llm_span_context = trace.set_span_in_context(workflow_span) if workflow_span else None
|
||||
|
||||
# Call original method first to get the actual OpenAI response
|
||||
result = original_method(self, messages)
|
||||
|
||||
# Now create a span that captures the LLM interaction with the actual response data
|
||||
with instrumentor._tracer.start_as_current_span(
|
||||
f"llm.{current_phase}",
|
||||
kind=OTelSpanKind.CLIENT,
|
||||
context=llm_span_context,
|
||||
attributes={
|
||||
SpanAttributes.AGENTOPS_SPAN_KIND: SpanKind.LLM,
|
||||
"xpander.span.type": "llm",
|
||||
"xpander.workflow.phase": current_phase,
|
||||
"xpander.session.id": session_id,
|
||||
},
|
||||
) as llm_span:
|
||||
# Extract and set OpenAI message data from the messages and response
|
||||
instrumentor._extract_and_set_openai_message_attributes(llm_span, messages, result, self)
|
||||
|
||||
# Extract and set LLM metadata from the result if possible
|
||||
llm_data = instrumentor._extract_llm_data_from_messages(result if result else messages)
|
||||
if llm_data:
|
||||
if "model" in llm_data:
|
||||
safe_set_attribute(llm_span, SpanAttributes.LLM_REQUEST_MODEL, llm_data["model"])
|
||||
safe_set_attribute(llm_span, SpanAttributes.LLM_RESPONSE_MODEL, llm_data["model"])
|
||||
|
||||
if "usage" in llm_data:
|
||||
usage = llm_data["usage"]
|
||||
if "prompt_tokens" in usage:
|
||||
safe_set_attribute(llm_span, SpanAttributes.LLM_USAGE_PROMPT_TOKENS, usage["prompt_tokens"])
|
||||
if "completion_tokens" in usage:
|
||||
safe_set_attribute(
|
||||
llm_span, SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, usage["completion_tokens"]
|
||||
)
|
||||
if "total_tokens" in usage:
|
||||
safe_set_attribute(llm_span, SpanAttributes.LLM_USAGE_TOTAL_TOKENS, usage["total_tokens"])
|
||||
# Update workflow state
|
||||
instrumentor._context.update_session(
|
||||
session_id,
|
||||
{
|
||||
"total_tokens": (current_session.get("total_tokens", 0) if current_session else 0)
|
||||
+ usage["total_tokens"]
|
||||
},
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _wrap_is_finished(self, original_method):
|
||||
"""Wrap is_finished to track workflow completion."""
|
||||
instrumentor = self
|
||||
|
||||
def wrapper(self):
|
||||
result = original_method(self)
|
||||
|
||||
if result:
|
||||
session_id = instrumentor._get_session_id_from_agent(self)
|
||||
|
||||
# Update session to finished state
|
||||
instrumentor._context.update_session(session_id, {"phase": "finished", "end_time": time.time()})
|
||||
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _wrap_extract_tool_calls(self, original_method):
|
||||
"""Wrap extract_tool_calls to track tool planning."""
|
||||
|
||||
def wrapper(self, messages):
|
||||
result = original_method(self, messages)
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _wrap_report_execution_metrics(self, original_method):
|
||||
"""Wrap report_execution_metrics to track metrics."""
|
||||
|
||||
def wrapper(self, llm_tokens=None, ai_model=None):
|
||||
result = original_method(self, llm_tokens, ai_model)
|
||||
return result
|
||||
|
||||
return wrapper
|
||||
|
||||
def _wrap_retrieve_execution_result(self, original_method):
|
||||
"""Wrap retrieve_execution_result to finalize agent and workflow spans."""
|
||||
instrumentor = self
|
||||
|
||||
def wrapper(self):
|
||||
session_id = instrumentor._get_session_id_from_agent(self)
|
||||
current_session = instrumentor._context.get_session(session_id)
|
||||
workflow_span = instrumentor._context.get_workflow_span(session_id)
|
||||
session_span = instrumentor._context.get_conversation_span(session_id) # This is now the root session span
|
||||
|
||||
try:
|
||||
# Execute and capture result
|
||||
result = original_method(self)
|
||||
|
||||
# Add workflow summary to the persistent workflow span
|
||||
if workflow_span and current_session:
|
||||
safe_set_attribute(
|
||||
workflow_span, "xpander.workflow.total_steps", current_session.get("step_count", 0)
|
||||
)
|
||||
safe_set_attribute(
|
||||
workflow_span, "xpander.workflow.total_tokens", current_session.get("total_tokens", 0)
|
||||
)
|
||||
safe_set_attribute(
|
||||
workflow_span, "xpander.workflow.tools_used", len(current_session.get("tools_executed", []))
|
||||
)
|
||||
|
||||
# Calculate total execution time
|
||||
start_time = current_session.get("start_time", time.time())
|
||||
execution_time = time.time() - start_time
|
||||
safe_set_attribute(workflow_span, "xpander.workflow.execution_time", execution_time)
|
||||
safe_set_attribute(workflow_span, "xpander.workflow.phase", "completed")
|
||||
|
||||
# Set result details on session and workflow spans
|
||||
if result:
|
||||
result_content = ""
|
||||
if hasattr(result, "result"):
|
||||
result_content = str(result.result)[:1000]
|
||||
|
||||
# Set on session span (root span)
|
||||
if session_span and result_content:
|
||||
safe_set_attribute(session_span, SpanAttributes.AGENTOPS_ENTITY_OUTPUT, result_content)
|
||||
safe_set_attribute(session_span, "xpander.session.final_result", result_content)
|
||||
if hasattr(result, "memory_thread_id"):
|
||||
safe_set_attribute(session_span, "xpander.session.thread_id", result.memory_thread_id)
|
||||
|
||||
if workflow_span:
|
||||
if result_content:
|
||||
safe_set_attribute(workflow_span, "xpander.result.content", result_content)
|
||||
if hasattr(result, "memory_thread_id"):
|
||||
safe_set_attribute(workflow_span, "xpander.result.thread_id", result.memory_thread_id)
|
||||
|
||||
# Add session summary to session span
|
||||
if session_span and current_session:
|
||||
safe_set_attribute(
|
||||
session_span, "xpander.session.total_steps", current_session.get("step_count", 0)
|
||||
)
|
||||
safe_set_attribute(
|
||||
session_span, "xpander.session.total_tokens", current_session.get("total_tokens", 0)
|
||||
)
|
||||
safe_set_attribute(
|
||||
session_span, "xpander.session.tools_used", len(current_session.get("tools_executed", []))
|
||||
)
|
||||
|
||||
start_time = current_session.get("start_time", time.time())
|
||||
execution_time = time.time() - start_time
|
||||
safe_set_attribute(session_span, "xpander.session.execution_time", execution_time)
|
||||
|
||||
# Close all spans - session span should be closed last
|
||||
if workflow_span:
|
||||
_finish_span_success(workflow_span)
|
||||
workflow_span.end()
|
||||
|
||||
if session_span:
|
||||
_finish_span_success(session_span)
|
||||
session_span.end()
|
||||
|
||||
return result
|
||||
|
||||
except Exception as e:
|
||||
# Mark spans as failed and close them in proper order
|
||||
if workflow_span:
|
||||
_finish_span_error(workflow_span, e)
|
||||
workflow_span.end()
|
||||
|
||||
if session_span:
|
||||
_finish_span_error(session_span, e)
|
||||
session_span.end()
|
||||
raise
|
||||
finally:
|
||||
# Clean up session
|
||||
instrumentor._context.end_session(session_id)
|
||||
|
||||
return wrapper
|
||||
|
||||
def _instrument(self, **kwargs):
|
||||
"""Instrument the Xpander SDK."""
|
||||
try:
|
||||
# Import xpander modules
|
||||
from xpander_sdk import Agent
|
||||
|
||||
# Set up tracing using existing AgentOps tracer
|
||||
self._tracer = tracer.get_tracer()
|
||||
# Attribute manager already initialized in __init__
|
||||
|
||||
# Wrap Agent methods
|
||||
Agent.add_task = self._wrap_init_task(Agent.add_task)
|
||||
Agent.init_task = self._wrap_init_task(Agent.init_task) # Also wrap init_task for completeness
|
||||
Agent.run_tools = self._wrap_run_tools(Agent.run_tools)
|
||||
Agent.add_messages = self._wrap_add_messages(Agent.add_messages)
|
||||
Agent.is_finished = self._wrap_is_finished(Agent.is_finished)
|
||||
Agent.extract_tool_calls = self._wrap_extract_tool_calls(Agent.extract_tool_calls)
|
||||
Agent.report_execution_metrics = self._wrap_report_execution_metrics(Agent.report_execution_metrics)
|
||||
Agent.retrieve_execution_result = self._wrap_retrieve_execution_result(Agent.retrieve_execution_result)
|
||||
|
||||
except ImportError:
|
||||
logger.debug("Xpander SDK not available")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to instrument Xpander SDK: {e}")
|
||||
|
||||
def _uninstrument(self, **kwargs):
|
||||
"""Uninstrument the Xpander SDK."""
|
||||
pass
|
||||
|
||||
def _create_metrics(self, meter: Meter) -> StandardMetrics:
|
||||
"""Create metrics for Xpander instrumentation."""
|
||||
return StandardMetrics(
|
||||
requests_active=meter.create_up_down_counter(
|
||||
name="xpander_requests_active",
|
||||
description="Number of active Xpander requests",
|
||||
),
|
||||
requests_duration=meter.create_histogram(
|
||||
name="xpander_requests_duration",
|
||||
description="Duration of Xpander requests",
|
||||
unit="s",
|
||||
),
|
||||
requests_total=meter.create_counter(
|
||||
name="xpander_requests_total",
|
||||
description="Total number of Xpander requests",
|
||||
),
|
||||
requests_error=meter.create_counter(
|
||||
name="xpander_requests_error",
|
||||
description="Number of Xpander request errors",
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,86 @@
|
||||
"""Xpander trace probe for automatic instrumentation activation.
|
||||
|
||||
This module provides automatic instrumentation for Xpander SDK when imported.
|
||||
It should be imported early in the application lifecycle to ensure all
|
||||
Xpander interactions are captured.
|
||||
"""
|
||||
|
||||
import logging
|
||||
from agentops.instrumentation.agentic.xpander.instrumentor import XpanderInstrumentor
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Global instrumentor instance
|
||||
_instrumentor = None
|
||||
|
||||
|
||||
def activate_xpander_instrumentation():
|
||||
"""Activate Xpander instrumentation."""
|
||||
global _instrumentor
|
||||
|
||||
if _instrumentor is None:
|
||||
try:
|
||||
_instrumentor = XpanderInstrumentor()
|
||||
_instrumentor.instrument()
|
||||
logger.info("Xpander instrumentation activated successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to activate Xpander instrumentation: {e}")
|
||||
_instrumentor = None
|
||||
|
||||
return _instrumentor
|
||||
|
||||
|
||||
def deactivate_xpander_instrumentation():
|
||||
"""Deactivate Xpander instrumentation."""
|
||||
global _instrumentor
|
||||
|
||||
if _instrumentor is not None:
|
||||
try:
|
||||
_instrumentor.uninstrument()
|
||||
logger.info("Xpander instrumentation deactivated successfully")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to deactivate Xpander instrumentation: {e}")
|
||||
finally:
|
||||
_instrumentor = None
|
||||
|
||||
|
||||
def get_instrumentor():
|
||||
"""Get the active instrumentor instance."""
|
||||
return _instrumentor
|
||||
|
||||
|
||||
# Stub functions for backward compatibility
|
||||
def wrap_openai_call_for_xpander(openai_call_func, purpose="general"):
|
||||
"""Backward compatibility stub - functionality now handled by auto-instrumentation."""
|
||||
logger.debug(f"wrap_openai_call_for_xpander called with purpose: {purpose}")
|
||||
return openai_call_func
|
||||
|
||||
|
||||
def is_xpander_session_active():
|
||||
"""Check if xpander session is active."""
|
||||
return _instrumentor is not None
|
||||
|
||||
|
||||
def get_active_xpander_session():
|
||||
"""Get active xpander session."""
|
||||
return _instrumentor._context if _instrumentor else None
|
||||
|
||||
|
||||
# Convenience functions for cleaner OpenAI integration
|
||||
def wrap_openai_analysis(openai_call_func):
|
||||
"""Wrap OpenAI calls for analysis/reasoning steps."""
|
||||
return wrap_openai_call_for_xpander(openai_call_func, "analysis")
|
||||
|
||||
|
||||
def wrap_openai_planning(openai_call_func):
|
||||
"""Wrap OpenAI calls for planning steps."""
|
||||
return wrap_openai_call_for_xpander(openai_call_func, "planning")
|
||||
|
||||
|
||||
def wrap_openai_synthesis(openai_call_func):
|
||||
"""Wrap OpenAI calls for synthesis/summary steps."""
|
||||
return wrap_openai_call_for_xpander(openai_call_func, "synthesis")
|
||||
|
||||
|
||||
# Note: Auto-activation is now handled by the main AgentOps instrumentation system
|
||||
# activate_xpander_instrumentation()
|
||||
@@ -0,0 +1,3 @@
|
||||
"""Version information for xpander instrumentation."""
|
||||
|
||||
__version__ = "1.0.0"
|
||||
@@ -0,0 +1,65 @@
|
||||
# AgentOps Instrumentation Common Module
|
||||
|
||||
The `agentops.instrumentation.common` module provides shared utilities for OpenTelemetry instrumentation across different LLM service providers.
|
||||
|
||||
## Core Components
|
||||
|
||||
### Attribute Handler Example
|
||||
|
||||
Attribute handlers extract data from method inputs and outputs:
|
||||
|
||||
```python
|
||||
from typing import Optional, Any, Tuple, Dict
|
||||
from agentops.instrumentation.common.attributes import AttributeMap
|
||||
from agentops.semconv import SpanAttributes
|
||||
|
||||
def my_attribute_handler(args: Optional[Tuple] = None, kwargs: Optional[Dict] = None, return_value: Optional[Any] = None) -> AttributeMap:
|
||||
attributes = {}
|
||||
|
||||
# Extract attributes from kwargs (method inputs)
|
||||
if kwargs:
|
||||
if "model" in kwargs:
|
||||
attributes[SpanAttributes.MODEL_NAME] = kwargs["model"]
|
||||
# ...
|
||||
|
||||
# Extract attributes from return value (method outputs)
|
||||
if return_value:
|
||||
if hasattr(return_value, "model"):
|
||||
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = return_value.model
|
||||
# ...
|
||||
|
||||
return attributes
|
||||
```
|
||||
|
||||
### `WrapConfig` Class
|
||||
|
||||
Config object defining how a method should be wrapped:
|
||||
|
||||
```python
|
||||
from agentops.instrumentation.common.wrappers import WrapConfig
|
||||
from opentelemetry.trace import SpanKind
|
||||
|
||||
config = WrapConfig(
|
||||
trace_name="llm.completion", # Name that will appear in trace spans
|
||||
package="openai.resources", # Path to the module containing the class
|
||||
class_name="Completions", # Name of the class containing the method
|
||||
method_name="create", # Name of the method to wrap
|
||||
handler=my_attribute_handler, # Function that extracts attributes
|
||||
span_kind=SpanKind.CLIENT # Type of span to create
|
||||
)
|
||||
```
|
||||
|
||||
### Wrapping/Unwrapping Methods
|
||||
|
||||
```python
|
||||
from opentelemetry.trace import get_tracer
|
||||
from agentops.instrumentation.common.wrappers import wrap, unwrap
|
||||
|
||||
# Create a tracer and wrap a method
|
||||
tracer = get_tracer("openai", "0.0.0")
|
||||
wrap(config, tracer)
|
||||
|
||||
# Later, unwrap the method
|
||||
unwrap(config)
|
||||
```
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user