同步完整源码 - 2026-05-25

This commit is contained in:
xiaoxue
2026-05-25 01:45:55 +08:00
commit af7cdc0ffd
1581 changed files with 225877 additions and 0 deletions
+1
View File
@@ -0,0 +1 @@
uv.lock binary
+20
View File
@@ -0,0 +1,20 @@
# Code of Conduct
Be kind and be mindful. We're here to accelerate together.
All participants in our project's community are expected to follow this code of conduct, which covers our behavior as contributors, maintainers, and community members.
## Our Pledge
We pledge to make participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
+24
View File
@@ -0,0 +1,24 @@
## Open Source and Proprietary Components
The AgentOps SDK is open source, but the backend and UI are not. We are source available and also support on-prem/VPC hosting upon request.
That said, we love when community members contribute to our open source SDK-- we even mail over swag packs to developers who help out :)
## Contribution Guidelines
We use issues primarily to communicate our thoughts and plans to the community. However, not every issue is suitable for the open-source distribution, as some features may depend on non-open-source parts of our stack.
When looking for issues to work on look for following tags:
- `good first issue`
- `help wanted`
If you have feature requests or bug reports, please open an issue.
## Getting Started
The easiest way to contribute is to start a conversation with us. We highly recommend:
1. Joining our Discord community
2. Sending us a message through our contact form at [agentops.ai/contact](https://agentops.ai/contact)
We look forward to collaborating with you and appreciate your interest in improving AgentOps!
+53
View File
@@ -0,0 +1,53 @@
name: 🐛 Bug Report
description: Noticed something off with Agentops? Let us know!
title: "[Bug]: "
labels: ["bug"]
assignees: []
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to fill out this bug report! Your feedback helps us improve AgentOps.
- type: input
id: contact
attributes:
label: Contact Details
description: How can we get in touch with you if we need more info?
placeholder: ex. email@example.com
validations:
required: false
- type: input
id: package-version
attributes:
label: 📦 Package Version
description: What version of Agentops are you using?
placeholder: ex. 0.3.12
validations:
required: true
- type: input
id: framework-version
attributes:
label: 🎞️ Framework Version
description: Which frameworks are you using? (if applicable)
placeholder: ex. Autogen 0.3.0, CrewAI 0.61.0, Langchain 0.2.1
validations:
required: false
- type: textarea
id: bug-description
attributes:
label: 🔎 Describe the Bug
description: Give a clear and concise description of the bug and how to reproduce it. Include any error messages and screenshots.
placeholder: What went wrong?
validations:
required: true
- type: checkboxes
id: contribution
attributes:
label: 🤝 Contribution
description: Would you be willing to contribute to this update?
options:
- label: Yes, I'd be happy to submit a pull request with these changes.
- label: I need some guidance on how to contribute.
- label: I'd prefer the AgentOps team to handle this update.
validations:
required: true
@@ -0,0 +1,53 @@
name: 📚 Documentation Update
description: Suggest an improvement or addition to our documentation
title: "[Docs]: "
labels: ["documentation"]
assignees: []
body:
- type: markdown
attributes:
value: |
Thank you for helping us improve AgentOps documentation! Your input is valuable in making our docs more helpful and comprehensive.
- type: textarea
id: current-state
attributes:
label: 📘 Current State of Documentation
description: What information is missing or could be improved?
placeholder: Describe the current state of the documentation you'd like to update.
validations:
required: true
- type: textarea
id: suggested-improvement
attributes:
label: 📖 Suggested Improvement
description: Describe your idea for the new/improved documentation.
placeholder: What changes or additions would you like to see?
validations:
required: true
- type: input
id: affected-pages
attributes:
label: 🔗 Affected Documentation Pages
description: If applicable, provide links to the documentation pages that need updating.
placeholder: https://docs.agentops.ai/page-to-update
validations:
required: false
- type: textarea
id: additional-context
attributes:
label: 🔍 Additional Context
description: Any extra information to help us understand the change?
placeholder: Provide any additional details or context for the documentation update.
validations:
required: false
- type: checkboxes
id: contribution
attributes:
label: 🤝 Contribution
description: Would you be willing to contribute to this documentation update?
options:
- label: Yes, I'd be happy to submit a pull request with these changes.
- label: I need some guidance on how to contribute.
- label: I'd prefer the Agentops team to handle this update.
validations:
required: true
@@ -0,0 +1,37 @@
name: 🚀 Feature Request
description: Got an idea to improve AgentOps? We're all ears!
title: "[Feature]: "
labels: ["enhancement"]
assignees: []
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to share your feature idea! Your input helps us improve AgentOps.
- type: textarea
id: feature-description-and-solution
attributes:
label: 💡 Feature Description and Proposed Solution
description: Briefly describe your feature suggestion and how you envision it working. Include any mockups/screenshots if applicable.
placeholder: What's your brilliant idea?
validations:
required: true
- type: textarea
id: problem-related
attributes:
label: 🤔 Related Problem
description: Is your feature request related to a problem you're facing?
placeholder: Describe the problem, if applicable. The more we understand your struggle, the better we can address it.
validations:
required: false
- type: checkboxes
id: contribution
attributes:
label: 🤝 Contribution
description: Would you be willing to contribute to this update?
options:
- label: Yes, I'd be happy to submit a pull request with these changes.
- label: I need some guidance on how to contribute.
- label: I'd prefer the AgentOps team to handle this update.
validations:
required: true
+8
View File
@@ -0,0 +1,8 @@
## 📥 Pull Request
**📘 Description**
_Briefly describe the changes you've made._
**🧪 Testing**
_Describe the tests you performed to validate your changes._
+4
View File
@@ -0,0 +1,4 @@
# Configuration for probot-stale - https://github.com/probot/stale
# Number of days of inactivity before an issue becomes stale
daysUntilStale: 60
@@ -0,0 +1,132 @@
name: Add Notebook Examples to Docs
on:
push:
branches:
- main
paths:
- 'examples/**'
- 'docs/v2/examples/**'
workflow_dispatch:
permissions:
contents: write
pull-requests: write
jobs:
add-notebook-examples-to-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install dependencies
run: pip install jupyter nbconvert
- name: Detect and process notebooks
id: process
run: |
# Determine comparison range
case "${{ github.event_name }}" in
"push")
BASE_SHA="${{ github.event.before }}"
[[ "$BASE_SHA" == "0000000000000000000000000000000000000000" ]] && BASE_SHA="$(git hash-object -t tree /dev/null)"
HEAD_SHA="${{ github.event.after }}"
;;
"pull_request")
BASE_SHA="${{ github.event.pull_request.base.sha }}"
HEAD_SHA="${{ github.event.pull_request.head.sha }}"
;;
"workflow_dispatch")
git fetch origin "${{ github.event.repository.default_branch }}" --depth=50
BASE_SHA="origin/${{ github.event.repository.default_branch }}"
HEAD_SHA="HEAD"
;;
*)
exit 1
;;
esac
# Get changed files and filter in one pass
CHANGED_FILES=$(git diff --name-only "$BASE_SHA" "$HEAD_SHA")
# Use arrays for better performance
declare -A notebooks_set
# Process notebooks directly from changed files
while IFS= read -r file; do
if [[ "$file" =~ ^examples/.*\.ipynb$ ]]; then
notebooks_set["$file"]=1
elif [[ "$file" =~ ^docs/v2/examples/.*\.mdx$ ]] && [[ -f "$file" ]]; then
# Extract source notebook from MDX file
source_notebook=$(grep -oP '(?<=SOURCE_FILE: )[^ ]+' "$file" 2>/dev/null || true)
if [[ -n "$source_notebook" && -f "$source_notebook" ]]; then
notebooks_set["$source_notebook"]=1
fi
fi
done <<< "$CHANGED_FILES"
# Exit if no notebooks found
[[ ${#notebooks_set[@]} -eq 0 ]] && exit 0
# Process each notebook and track results
failed_count=0
processed_notebooks=""
for notebook in "${!notebooks_set[@]}"; do
if python examples/generate_documentation.py "$notebook"; then
processed_notebooks="$processed_notebooks$notebook"$'\n'
else
((failed_count++))
fi
done
[[ $failed_count -gt 0 ]] && exit 1
# Get modified files for PR description
modified_files=$(git status --porcelain | grep -E '^\s*[AM]' | awk '{print $2}' | sort)
# Save outputs for PR
echo "processed_notebooks<<EOF" >> $GITHUB_OUTPUT
echo "$processed_notebooks" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
echo "modified_files<<EOF" >> $GITHUB_OUTPUT
echo "$modified_files" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Create Pull Request
uses: peter-evans/create-pull-request@v7
with:
token: ${{ secrets.GITHUB_TOKEN }}
committer: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
author: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
commit-message: "docs: update examples from notebooks"
title: "docs: update examples from notebooks"
body: |
Updates documentation examples from their source notebooks.
**Triggered by:** @${{ github.actor }}
**Event:** ${{ github.event_name }}
${{ steps.process.outputs.processed_notebooks && format('
## Processed Notebooks
```
{0}
```', steps.process.outputs.processed_notebooks) || '' }}
${{ steps.process.outputs.modified_files && format('
## Modified Files
```
{0}
```', steps.process.outputs.modified_files) || '' }}
branch: update-examples-docs
delete-branch: true
assignees: the-praxs,bboynton97,areibman
reviewers: the-praxs,bboynton97,areibman
+42
View File
@@ -0,0 +1,42 @@
name: Compile llms.txt
on:
push:
branches: [ main ]
paths:
- 'docs/**'
- 'README.md'
- 'CONTRIBUTING.md'
- 'examples/*/README.md'
- 'agentops/*/README.md'
workflow_dispatch:
jobs:
compile-llms-txt:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
token: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11'
- name: Install dependencies
run: |
pip install llms-txt
- name: Compile llms.txt
run: |
cd docs
python compile_llms_txt.py
- name: Commit and push if changed
run: |
git config --local user.email "action@github.com"
git config --local user.name "GitHub Action"
git add llms.txt
git diff --staged --quiet || git commit -m "Auto-update llms.txt from documentation changes"
git push
@@ -0,0 +1,28 @@
name: Delete Bot Comments
on:
issue_comment:
types: [created]
jobs:
delete-bot-comment:
runs-on: ubuntu-latest
if: ${{ github.event.comment.user.login == 'entelligence-ai-pr-reviews' }}
steps:
- name: Delete bot comment
uses: actions/github-script@v6
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const commentId = context.payload.comment.id;
console.log(`Deleting comment ID: ${commentId} from bot: entelligence-ai-pr-reviews`);
await github.rest.issues.deleteComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: commentId
});
console.log('Comment deleted successfully');
+53
View File
@@ -0,0 +1,53 @@
name: Build and Push MCP Docker Image
on:
push:
branches:
- main
tags:
- 'v*'
- 'mcp-v*'
paths:
- 'mcp/**'
- '.github/workflows/docker-mcp-publish.yml'
workflow_dispatch:
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ secrets.DOCKERHUB_USERNAME }}/agentops-mcp
tags: |
type=ref,event=branch
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=raw,value=latest,enable={{is_default_branch}}
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: ./mcp
file: ./mcp/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
@@ -0,0 +1,191 @@
name: Examples Integration Test
# This workflow runs all example scripts to ensure they work correctly
# and that LLM spans are properly tracked in AgentOps using the
# integrated validation functionality.
on:
pull_request:
branches: [ main, develop ]
paths:
- 'examples/**/*.py'
- 'agentops/**'
- '.github/workflows/examples-integration-test.yml'
types: [opened, reopened, ready_for_review]
workflow_dispatch:
env:
PYTHON_VERSION: '3.11'
jobs:
test-examples:
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
example:
# OpenAI examples
- { path: 'examples/openai/openai_example_sync.py', name: 'OpenAI Sync' }
- { path: 'examples/openai/openai_example_async.py', name: 'OpenAI Async' }
- { path: 'examples/openai/multi_tool_orchestration.py', name: 'OpenAI Multi-Tool' }
- { path: 'examples/openai/web_search.py', name: 'OpenAI Web Search' }
- { path: 'examples/openai/o3_responses_example.py', name: 'OpenAI o3 Responses' }
# Anthropic examples
- { path: 'examples/anthropic/anthropic-example-sync.py', name: 'Anthropic Sync' }
- { path: 'examples/anthropic/anthropic-example-async.py', name: 'Anthropic Async' }
- { path: 'examples/anthropic/agentops-anthropic-understanding-tools.py', name: 'Anthropic Tools' }
# LangChain examples
- { path: 'examples/langchain/langchain_examples.py', name: 'LangChain' }
# LiteLLM examples
- { path: 'examples/litellm/litellm_example.py', name: 'LiteLLM' }
# Google Generative AI examples
- { path: 'examples/google_genai/gemini_example.py', name: 'Google Gemini' }
# xAI examples
- { path: 'examples/xai/grok_examples.py', name: 'xAI Grok' }
- { path: 'examples/xai/grok_vision_examples.py', name: 'xAI Grok Vision' }
# CrewAI examples
- { path: 'examples/crewai/job_posting.py', name: 'CrewAI Job Posting' }
- { path: 'examples/crewai/markdown_validator.py', name: 'CrewAI Markdown' }
# AutoGen examples
- { path: 'examples/autogen/AgentChat.py', name: 'AutoGen Agent Chat' }
- { path: 'examples/autogen/MathAgent.py', name: 'AutoGen Math Agent' }
# AG2 examples
- { path: 'examples/ag2/async_human_input.py', name: 'AG2 Async Human Input' }
- { path: 'examples/ag2/tools_wikipedia_search.py', name: 'AG2 Wikipedia Search' }
# Agno examples
- { path: 'examples/agno/agno_async_operations.py', name: 'Agno Async Operations' }
- { path: 'examples/agno/agno_basic_agents.py', name: 'Agno Basic Agents' }
- { path: 'examples/agno/agno_research_team.py', name: 'Agno Research Team' }
- { path: 'examples/agno/agno_tool_integrations.py', name: 'Agno Tool Integrations' }
- { path: 'examples/agno/agno_workflow_setup.py', name: 'Agno Workflow Setup' }
# Google ADK examples
- { path: 'examples/google_adk/human_approval.py', name: 'Google ADK Human Approval' }
- { path: 'examples/google_adk/sse_function_call_example.py', name: 'Google ADK SSE Function Call' }
# LlamaIndex examples
# - { path: 'examples/llamaindex/llamaindex_example.py', name: 'LlamaIndex' }
# Mem0 examples
- { path: 'examples/mem0/mem0_memoryclient_example.py', name: 'Mem0 Memory Client' }
# Watsonx examples
- { path: 'examples/watsonx/watsonx-streaming.py', name: 'Watsonx Streaming' }
- { path: 'examples/watsonx/watsonx-text-chat.py', name: 'Watsonx Text Chat' }
- { path: 'examples/watsonx/watsonx-tokeniation-model.py', name: 'Watsonx Tokenization' }
# LangGraph examples
- { path: 'examples/langgraph/langgraph_example.py', name: 'LangGraph' }
# Smolagents examples
- { path: 'examples/smolagents/multi_smolagents_system.py', name: 'Smolagents Multi System' }
- { path: 'examples/smolagents/text_to_sql.py', name: 'Smolagents Text to SQL' }
# OpenAI Agents examples
- { path: 'examples/openai_agents/agent_guardrails.py', name: 'OpenAI Agents Guardrails' }
- { path: 'examples/openai_agents/agent_patterns.py', name: 'OpenAI Agents Patterns' }
- { path: 'examples/openai_agents/agents_tools.py', name: 'OpenAI Agents Tools' }
- { path: 'examples/openai_agents/customer_service_agent.py', name: 'OpenAI Agents Customer Service' }
# DSPy examples
- { path: 'examples/dspy/dspy_calculator.py', name: 'DSPy ReAct Agent' }
# Haystack examples
- { path: 'examples/haystack/haystack_example.py', name: 'Haystack OpenAI' }
- { path: 'examples/haystack/azure_haystack_example.py', name: 'Haystack Azure Chat' }
# Add more examples as needed
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install AgentOps
run: |
pip install -e .
- name: Install example dependencies
run: |
# Install common dependencies
pip install python-dotenv requests
# Install from requirements.txt in the example's directory
example_dir=$(dirname "${{ matrix.example.path }}")
if [ -f "$example_dir/requirements.txt" ]; then
echo "Installing dependencies from $example_dir/requirements.txt"
pip install -r "$example_dir/requirements.txt"
else
echo "No requirements.txt found in $example_dir"
fi
- name: Run example - ${{ matrix.example.name }}
env:
AGENTOPS_API_KEY: ${{ secrets.AGENTOPS_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
XAI_API_KEY: ${{ secrets.XAI_API_KEY }}
WATSONX_API_KEY: ${{ secrets.WATSONX_API_KEY }}
WATSONX_PROJECT_ID: ${{ secrets.WATSONX_PROJECT_ID }}
WATSONX_URL: ${{ secrets.WATSONX_URL }}
MEM0_API_KEY: ${{ secrets.MEM0_API_KEY }}
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
MISTRAL_API_KEY: ${{ secrets.MISTRAL_API_KEY }}
AI21_API_KEY: ${{ secrets.AI21_API_KEY }}
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
EXA_API_KEY: ${{ secrets.EXA_API_KEY }}
LLAMA_API_KEY: ${{ secrets.LLAMA_API_KEY }}
PERPLEXITY_API_KEY: ${{ secrets.PERPLEXITY_API_KEY }}
REPLICATE_API_TOKEN: ${{ secrets.REPLICATE_API_TOKEN }}
PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
PYTHONPATH: ${{ github.workspace }}
run: |
echo "Running ${{ matrix.example.name }}..."
python "${{ matrix.example.path }}" || exit 1
- name: Check for errors
if: failure()
run: |
echo "Example ${{ matrix.example.name }} failed!"
echo "Path: ${{ matrix.example.path }}"
# Show last 50 lines of any log files
if [ -f agentops.log ]; then
echo "=== AgentOps Log ==="
tail -n 50 agentops.log
fi
summary:
needs: test-examples
runs-on: ubuntu-latest
if: always()
steps:
- name: Summary
run: |
echo "## Examples Integration Test Summary" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "${{ needs.test-examples.result }}" == "success" ]; then
echo "✅ All examples passed!" >> $GITHUB_STEP_SUMMARY
else
echo "❌ Some examples failed. Check the logs above." >> $GITHUB_STEP_SUMMARY
fi
+81
View File
@@ -0,0 +1,81 @@
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries
# To run this workflow manually using GitHub CLI:
# Test build only:
# gh workflow run "Upload Python Package" --ref main --field test_mode=build-only
#
# Test with dry run (no actual publish):
# gh workflow run "Upload Python Package" --ref main --field test_mode=dry-run
#
# Test publish to TestPyPI (requires TEST_PYPI_API_TOKEN):
# gh workflow run "Upload Python Package" --ref main --field test_mode=testpypi
#
# To check workflow status:
# gh run list --workflow "Upload Python Package"
# gh run view <run-id>
name: Upload Python Package
on:
release:
types: [published]
workflow_dispatch:
inputs:
test_mode:
description: 'Test mode (dry-run, testpypi, build-only)'
required: true
default: 'dry-run'
type: choice
options:
- dry-run
- testpypi
- build-only
permissions:
contents: read
id-token: write # Required for trusted publishing
jobs:
deploy:
runs-on: ubuntu-latest
environment:
name: ${{ github.event_name == 'release' && 'pypi' || 'test' }}
url: ${{ github.event_name == 'release' && 'https://pypi.org/p/agentops' || 'https://test.pypi.org/p/agentops' }}
steps:
- uses: actions/checkout@v4
- name: Setup UV
uses: astral-sh/setup-uv@v5
with:
python-version: "3.11" # Specify exact version
cache-dependency-glob: |
**/pyproject.toml
**/requirements*.txt
- name: Build and publish
env:
PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
TEST_PYPI_TOKEN: ${{ secrets.TEST_PYPI_API_TOKEN }}
run: |
uv pip install build
uv build
# Extra safety check
if [[ "${{ github.event_name }}" == "workflow_dispatch" && "${{ github.event.inputs.test_mode }}" == "testpypi" ]]; then
echo "⚠️ Publishing to TestPyPI..."
sleep 5 # Give time to cancel if needed
fi
if [[ "${{ github.event.inputs.test_mode }}" == "dry-run" ]]; then
echo "🔍 Performing dry run..."
uv publish --check-url https://pypi.org/pypi
elif [[ "${{ github.event.inputs.test_mode }}" == "testpypi" ]]; then
uv publish --publish-url https://test.pypi.org/legacy/ --token $TEST_PYPI_TOKEN
elif [[ "${{ github.event.inputs.test_mode }}" == "build-only" ]]; then
echo "✅ Build completed successfully. Skipping publish."
elif [[ "${{ github.event_name }}" == "release" ]]; then
echo "⚠️ Publishing to PyPI in 10 seconds... (Ctrl+C to cancel)"
sleep 10
uv publish --token $PYPI_TOKEN
fi
+109
View File
@@ -0,0 +1,109 @@
# :: Use nektos/act to run this locally
# :: Example:
# :: `act push -j unit-tests --matrix python-version:3.10 --container-architecture linux/amd64`
#
# This workflow runs two separate test suites:
# 1. Unit Tests (python-tests job):
# - Runs across Python 3.9 to 3.13
# - Located in tests/unit directory
# - Coverage report uploaded to Codecov for Python 3.11 only
#
# 2. Integration Tests (integration-tests job):
# - Runs only on Python 3.13
# - Located in tests/integration directory
# - Longer timeout (15 min vs 10 min for unit tests)
# - Separate cache for dependencies
name: Python Tests
on:
workflow_dispatch: {}
push:
branches:
- main
paths:
- 'agentops/**/*.py'
- 'agentops/**/*.ipynb'
- 'tests/**/*.py'
- 'tests/**/*.ipynb'
pull_request:
branches:
- main
paths:
- 'pyproject.toml'
- 'agentops/**/*.py'
- 'agentops/**/*.ipynb'
- 'tests/**/*.py'
- 'tests/**/*.ipynb'
jobs:
unit-tests:
runs-on: ubuntu-latest
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
AGENTOPS_API_KEY: ${{ secrets.AGENTOPS_API_KEY }}
PYTHONUNBUFFERED: "1"
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Setup UV
uses: astral-sh/setup-uv@v5
continue-on-error: true
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: uv-${{ matrix.python-version }}
cache-dependency-glob: "**/pyproject.toml"
- name: Install dependencies
run: |
uv sync --group test --group dev
- name: Run unit tests with coverage
timeout-minutes: 5
run: |
uv run -m pytest tests/unit -v --cov=agentops --cov-report=xml
# Only upload coverage report for python3.11
- name: Upload coverage to Codecov
if: ${{matrix.python-version == '3.11'}}
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: true # Should we?
integration-tests:
runs-on: ubuntu-latest
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
AGENTOPS_API_KEY: ${{ secrets.AGENTOPS_API_KEY }}
PYTHONUNBUFFERED: "1"
steps:
- uses: actions/checkout@v4
- name: Setup UV
uses: astral-sh/setup-uv@v5
continue-on-error: true
with:
python-version: "3.12"
enable-cache: true
cache-suffix: uv-3.12-integration
cache-dependency-glob: "**/pyproject.toml"
- name: Install dependencies
run: |
uv sync --group test --group dev
- name: Run integration tests
timeout-minutes: 5
run: |
uv run pytest tests/integration
+44
View File
@@ -0,0 +1,44 @@
name: 'Check for Stale PRs'
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
permissions:
contents: write
pull-requests: write
jobs:
stale-check:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
with:
# Only process pull requests (not issues)
days-before-stale: 14
days-before-close: 7
# Only target PRs
days-before-issue-stale: -1
days-before-issue-close: -1
# Custom messages
stale-pr-message: |
This pull request has been automatically marked as stale because it has not had any activity in the last 14 days.
If no updates are made within 7 days, this PR will be automatically closed.
close-pr-message: |
This pull request has been automatically closed because it has been stale for 7 days with no activity.
Feel free to reopen this PR if you'd like to continue working on it.
# Labels
stale-pr-label: 'stale'
# Remove stale label on PR update
remove-stale-when-updated: true
# Don't remove any existing labels
labels-to-remove-when-stale: ''
labels-to-remove-when-unstale: ''
+51
View File
@@ -0,0 +1,51 @@
name: Static analysis
# on PR and push to main
on:
push:
branches:
- main
pull_request:
paths:
- '**/*.py'
workflow_dispatch:
permissions:
contents: read
# Limit concurrency by workflow/branch combination.
#
# For pull request builds, pushing additional changes to the
# branch will cancel prior in-progress and pending builds.
#
# For builds triggered on a branch push, additional changes
# will wait for prior builds to complete before starting.
#
# https://docs.github.com/en/actions/using-jobs/using-concurrency
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
pre-commit-checks:
name: Pre-commit checks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: astral-sh/setup-uv@v4
with:
enable-cache: true
cache-dependency-glob: "**/pyproject.toml"
python-version: "3.12.2"
- name: Install packages
run: |
uv sync --group dev
- name: Run pre-commit
run: |
uvx pre-commit run --show-diff-on-failure --color=always --all-files
+183
View File
@@ -0,0 +1,183 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
!app/dashboard/lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
.idea/
# VSCode
.vscode/
# Cursor
.cursorrules
# Benchmarks
.benchmarks/
# MacOS
.DS_Store
# Database
.db
# Time travel
agentops_time_travel.json
.agentops_time_travel.yaml
# Node
node_modules
+9
View File
@@ -0,0 +1,9 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: "v0.12.9"
hooks:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
exclude: ^app/
- id: ruff-format
exclude: ^app/
+390
View File
@@ -0,0 +1,390 @@
# Contributing to AgentOps
Thanks for checking out AgentOps. We're building tools to help developers like you make AI agents that actually work reliably. If you've ever tried to build an agent system, you know the pain - they're a nightmare to debug, impossible to monitor, and when something goes wrong... good luck figuring out why.
We created AgentOps to solve these headaches, and we'd love your help making it even better. Our SDK hooks into all the major Python frameworks (AG2, CrewAI, LangChain) and LLM providers (OpenAI, Anthropic, Cohere, etc.) to give you visibility into what your agents are actually doing.
## How You Can Help
There are tons of ways to contribute, and we genuinely appreciate all of them:
1. **Add More Providers**: Help us support new LLM providers. Each one helps more developers monitor their agents.
2. **Improve Framework Support**: Using a framework we don't support yet? Help us add it!
3. **Make Docs Better**: Found our docs confusing? Help us fix them! Clear documentation makes everyone's life easier.
4. **Share Your Experience**: Using AgentOps? Let us know what's working and what isn't. Your feedback shapes our roadmap.
Even if you're not ready to contribute code, we'd love to hear your thoughts. Drop into our Discord, open an issue, or start a discussion. We're building this for developers like you, so your input matters.
## Table of Contents
- [Getting Started](#getting-started)
- [Development Environment](#development-environment)
- [Testing](#testing)
- [Adding LLM Providers](#adding-llm-providers)
- [Code Style](#code-style)
- [Pull Request Process](#pull-request-process)
- [Documentation](#documentation)
## Getting Started
1. **Fork and Clone**:
First, fork the repository by clicking the 'Fork' button in the top right of the [AgentOps repository](https://github.com/AgentOps-AI/agentops). This creates your own copy of the repository where you can make changes.
Then clone your fork:
```bash
git clone https://github.com/YOUR_USERNAME/agentops.git
cd agentops
```
Add the upstream repository to stay in sync:
```bash
git remote add upstream https://github.com/AgentOps-AI/agentops.git
git fetch upstream
```
Before starting work on a new feature:
```bash
git checkout main
git pull upstream main
git checkout -b feature/your-feature-name
```
2. **Install Dependencies**:
```bash
pip install -e .
```
3. **Set Up Pre-commit Hooks**:
```bash
pre-commit install
```
## Development Environment
1. **Environment Variables**:
Create a `.env` file:
```
AGENTOPS_API_KEY=your_api_key
OPENAI_API_KEY=your_openai_key # For testing
ANTHROPIC_API_KEY=your_anthropic_key # For testing
# Other keys...
```
2. **Virtual Environment**:
We recommend using `poetry` or `venv`:
```bash
python -m venv venv
source venv/bin/activate # Unix
.\venv\Scripts\activate # Windows
```
3. **Pre-commit Setup**:
We use pre-commit hooks to automatically format and lint code. Set them up with:
```bash
pip install pre-commit
pre-commit install
```
That's it! The hooks will run automatically when you commit. To manually check all files:
```bash
pre-commit run --all-files
```
## Testing
We use a comprehensive testing stack to ensure code quality and reliability. Our testing framework includes pytest and several specialized testing tools.
### Testing Dependencies
Install all testing dependencies:
```bash
pip install -e ".[dev]"
```
We use the following testing packages:
- `pytest==7.4.0`: Core testing framework
- `pytest-depends`: Manage test dependencies
- `pytest-asyncio`: Test async code
- `pytest-vcr`: Record and replay HTTP interactions
- `pytest-mock`: Mocking functionality
- `pyfakefs`: Mock filesystem operations
- `requests_mock==1.11.0`: Mock HTTP requests
### Using Tox
We use tox to automate and standardize testing. Tox:
- Creates isolated virtual environments for testing
- Tests against multiple Python versions (3.7-3.12)
- Runs all test suites consistently
- Ensures dependencies are correctly specified
- Verifies the package installs correctly
Run tox:
```bash
tox
```
This will:
1. Create fresh virtual environments
2. Install dependencies
3. Run pytest with our test suite
4. Generate coverage reports
### Running Tests
1. **Run All Tests**:
```bash
tox
```
2. **Run Specific Test File**:
```bash
pytest tests/llms/test_anthropic.py -v
```
3. **Run with Coverage**:
```bash
coverage run -m pytest
coverage report
```
### Writing Tests
1. **Test Structure**:
```python
import pytest
from pytest_mock import MockerFixture
from unittest.mock import Mock, patch
@pytest.mark.asyncio # For async tests
async def test_async_function():
# Test implementation
@pytest.mark.depends(on=['test_prerequisite']) # Declare test dependencies
def test_dependent_function():
# Test implementation
```
2. **Recording HTTP Interactions**:
```python
@pytest.mark.vcr() # Records HTTP interactions
def test_api_call():
response = client.make_request()
assert response.status_code == 200
```
3. **Mocking Filesystem**:
```python
def test_file_operations(fs): # fs fixture provided by pyfakefs
fs.create_file('/fake/file.txt', contents='test')
assert os.path.exists('/fake/file.txt')
```
4. **Mocking HTTP Requests**:
```python
def test_http_client(requests_mock):
requests_mock.get('http://api.example.com', json={'key': 'value'})
response = make_request()
assert response.json()['key'] == 'value'
```
### Testing Best Practices
1. **Test Categories**:
- Unit tests: Test individual components
- Integration tests: Test component interactions
- End-to-end tests: Test complete workflows
- Performance tests: Test response times and resource usage
2. **Fixtures**:
Create reusable test fixtures in `conftest.py`:
```python
@pytest.fixture
def mock_llm_client():
client = Mock()
client.chat.completions.create.return_value = Mock()
return client
```
3. **Test Data**:
- Store test data in `tests/data/`
- Use meaningful test data names
- Document data format and purpose
4. **VCR Cassettes**:
- Store in `tests/cassettes/`
- Sanitize sensitive information
- Update cassettes when API changes
### CI Testing Strategy
We use Jupyter notebooks as integration tests for LLM providers. This approach:
- Tests real-world usage patterns
- Verifies end-to-end functionality
- Ensures examples stay up-to-date
- Tests against actual LLM APIs
1. **Notebook Tests**:
- Located in `examples/` directory
- Each LLM provider has example notebooks
- CI runs notebooks on PR merges to main
- Tests run against multiple Python versions
2. **Test Workflow**:
The `test-notebooks.yml` workflow:
```yaml
name: Test Notebooks
on:
pull_request:
paths:
- "agentops/**"
- "examples/**"
- "tests/**"
```
- Runs on PR merges and manual triggers
- Sets up environment with provider API keys
- Installs AgentOps from main branch
- Executes each notebook
- Excludes specific notebooks that require manual testing
3. **Provider Coverage**:
Each provider should have notebooks demonstrating:
- Basic completion calls
- Streaming responses
- Async operations (if supported)
- Error handling
- Tool usage (if applicable)
4. **Adding Provider Tests**:
- Create notebook in `examples/provider_name/`
- Include all provider functionality
- Add necessary secrets to GitHub Actions
- Update `exclude_notebooks` in workflow if manual testing needed
## Adding LLM Providers
The `agentops/llms/` directory contains provider implementations. Each provider must:
1. **Inherit from BaseProvider**:
```python
@singleton
class NewProvider(BaseProvider):
def __init__(self, client):
super().__init__(client)
self._provider_name = "ProviderName"
```
2. **Implement Required Methods**:
- `handle_response()`: Process LLM responses
- `override()`: Patch the provider's methods
- `undo_override()`: Restore original methods
3. **Handle Events**:
Track:
- Prompts and completions
- Token usage
- Timestamps
- Errors
- Tool usage (if applicable)
4. **Example Implementation Structure**:
```python
def handle_response(self, response, kwargs, init_timestamp, session=None):
llm_event = LLMEvent(init_timestamp=init_timestamp, params=kwargs)
try:
# Process response
llm_event.returns = response.model_dump()
llm_event.prompt = kwargs["messages"]
# ... additional processing
self._safe_record(session, llm_event)
except Exception as e:
self._safe_record(session, ErrorEvent(trigger_event=llm_event, exception=e))
```
## Code Style
1. **Formatting**:
- Use Black for Python code formatting
- Maximum line length: 88 characters
- Use type hints
2. **Documentation**:
- Docstrings for all public methods
- Clear inline comments
- Update relevant documentation
3. **Error Handling**:
- Use specific exception types
- Log errors with meaningful messages
- Include context in error messages
## Pull Request Process
1. **Branch Naming**:
- `feature/description`
- `fix/description`
- `docs/description`
2. **Commit Messages**:
- Clear and descriptive
- Reference issues when applicable
3. **PR Requirements**:
- Pass all tests
- Maintain or improve code coverage
- Include relevant documentation
- Update CHANGELOG.md if applicable
4. **Review Process**:
- At least one approval required
- Address all review comments
- Maintain PR scope
## Documentation
1. **Types of Documentation**:
- API reference
- Integration guides
- Examples
- Troubleshooting guides
2. **Documentation Location**:
- Code documentation in docstrings
- User guides in `docs/`
- Examples in `examples/`
3. **Documentation Style**:
- Clear and concise
- Include code examples
- Explain the why, not just the what
## Getting Help & Community
We encourage active community participation and are here to help!
### Preferred Communication Channels
1. **GitHub Issues & Discussions**:
- Open an [issue](https://github.com/AgentOps-AI/agentops/issues) for:
- Bug reports
- Feature requests
- Documentation improvements
- Start a [discussion](https://github.com/AgentOps-AI/agentops/discussions) for:
- Questions about usage
- Ideas for new features
- Community showcase
- General feedback
2. **Discord Community**:
- Join our [Discord server](https://discord.gg/FagdcwwXRR) for:
- Real-time help
- Community discussions
- Feature announcements
- Sharing your projects
3. **Contact Form**:
- For private inquiries, use our [contact form](https://agentops.ai/contact)
- Please note that public channels are preferred for technical discussions
## License
By contributing to AgentOps, you agree that your contributions will be licensed under the MIT License.
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2023 AgentOps-AI
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+862
View File
@@ -0,0 +1,862 @@
<div align="center">
<a href="https://agentops.ai?ref=gh">
<img src="docs/images/external/logo/github-banner.png" alt="Logo">
</a>
</div>
<div align="center">
<em>Observability and DevTool platform for AI Agents</em>
</div>
<br />
<div align="center">
<a href="https://pepy.tech/project/agentops">
<img src="https://static.pepy.tech/badge/agentops/month" alt="Downloads">
</a>
<a href="https://github.com/agentops-ai/agentops/issues">
<img src="https://img.shields.io/github/commit-activity/m/agentops-ai/agentops" alt="git commit activity">
</a>
<img src="https://img.shields.io/pypi/v/agentops?&color=3670A0" alt="PyPI - Version">
<a href="https://opensource.org/licenses/MIT">
<img src="https://img.shields.io/badge/License-MIT-yellow.svg?&color=3670A0" alt="License: MIT">
</a>
<a href="https://smithery.ai/server/@AgentOps-AI/agentops-mcp">
<img src="https://smithery.ai/badge/@AgentOps-AI/agentops-mcp"/>
</a>
</div>
<p align="center">
<a href="https://twitter.com/agentopsai/">
<img src="https://img.shields.io/twitter/follow/agentopsai?style=social" alt="Twitter" style="height: 20px;">
</a>
<a href="https://discord.gg/FagdcwwXRR">
<img src="https://img.shields.io/badge/discord-7289da.svg?style=flat-square&logo=discord" alt="Discord" style="height: 20px;">
</a>
<a href="https://app.agentops.ai/?ref=gh">
<img src="https://img.shields.io/badge/Dashboard-blue.svg?style=flat-square" alt="Dashboard" style="height: 20px;">
</a>
<a href="https://docs.agentops.ai/introduction">
<img src="https://img.shields.io/badge/Documentation-orange.svg?style=flat-square" alt="Documentation" style="height: 20px;">
</a>
<a href="https://entelligence.ai/AgentOps-AI&agentops">
<img src="https://img.shields.io/badge/Chat%20with%20Docs-green.svg?style=flat-square" alt="Chat with Docs" style="height: 20px;">
</a>
</p>
<div align="center">
<video src="https://github.com/user-attachments/assets/dfb4fa8d-d8c4-4965-9ff6-5b8514c1c22f" width="650" autoplay loop muted></video>
</div>
<br/>
AgentOps helps developers build, evaluate, and monitor AI agents. From prototype to production.
## Open Source
The AgentOps app is open source under the MIT license. Explore the code in our [app directory](https://github.com/AgentOps-AI/agentops/tree/main/app).
## Key Integrations 🔌
<div align="center" style="background-color: white; padding: 20px; border-radius: 10px; margin: 0 auto; max-width: 800px;">
<div style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 30px; margin-bottom: 20px;">
<a href="https://docs.agentops.ai/v2/integrations/openai_agents_python"><img src="docs/images/external/openai/agents-sdk.svg" height="45" alt="OpenAI Agents SDK"></a>
<a href="https://docs.agentops.ai/v1/integrations/crewai"><img src="docs/v1/img/docs-icons/crew-banner.png" height="45" alt="CrewAI"></a>
<a href="https://docs.ag2.ai/docs/ecosystem/agentops"><img src="docs/images/external/ag2/ag2-logo.svg" height="45" alt="AG2 (AutoGen)"></a>
<a href="https://docs.agentops.ai/v1/integrations/microsoft"><img src="docs/images/external/microsoft/microsoft_logo.svg" height="45" alt="Microsoft"></a>
</div>
<div style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 30px; margin-bottom: 20px;">
<a href="https://docs.agentops.ai/v1/integrations/langchain"><img src="docs/images/external/langchain/langchain-logo.svg" height="45" alt="LangChain"></a>
<a href="https://docs.agentops.ai/v1/integrations/camel"><img src="docs/images/external/camel/camel.png" height="45" alt="Camel AI"></a>
<a href="https://docs.llamaindex.ai/en/stable/module_guides/observability/?h=agentops#agentops"><img src="docs/images/external/ollama/ollama-icon.png" height="45" alt="LlamaIndex"></a>
<a href="https://docs.agentops.ai/v1/integrations/cohere"><img src="docs/images/external/cohere/cohere-logo.svg" height="45" alt="Cohere"></a>
</div>
</div>
| | |
| ------------------------------------- | ------------------------------------------------------------- |
| 📊 **Replay Analytics and Debugging** | Step-by-step agent execution graphs |
| 💸 **LLM Cost Management** | Track spend with LLM foundation model providers |
| 🤝 **Framework Integrations** | Native Integrations with CrewAI, AG2 (AutoGen), Agno, LangGraph, & more |
| ⚒️ **Self-Host** | Want to run AgentOps on your own cloud? You're covered |
## Quick Start ⌨️
```bash
pip install agentops
```
#### Session replays in 2 lines of code
Initialize the AgentOps client and automatically get analytics on all your LLM calls.
[Get an API key](https://app.agentops.ai/settings/projects)
```python
import agentops
# Beginning of your program (i.e. main.py, __init__.py)
agentops.init( < INSERT YOUR API KEY HERE >)
...
# End of program
agentops.end_session('Success')
```
All your sessions can be viewed on the [AgentOps dashboard](https://app.agentops.ai?ref=gh)
<br/>
## Self-Hosting
Looking to run the full AgentOps app (Dashboard + API backend) on your machine? Follow the setup guide in `app/README.md`:
- [Run the App and Backend (Dashboard + API)](app/README.md)
<details>
<summary>Agent Debugging</summary>
<a href="https://app.agentops.ai?ref=gh">
<img src="docs/images/external/app_screenshots/session-drilldown-metadata.png" style="width: 90%;" alt="Agent Metadata"/>
</a>
<a href="https://app.agentops.ai?ref=gh">
<img src="docs/images/external/app_screenshots/chat-viewer.png" style="width: 90%;" alt="Chat Viewer"/>
</a>
<a href="https://app.agentops.ai?ref=gh">
<img src="docs/images/external/app_screenshots/session-drilldown-graphs.png" style="width: 90%;" alt="Event Graphs"/>
</a>
</details>
<details>
<summary>Session Replays</summary>
<a href="https://app.agentops.ai?ref=gh">
<img src="docs/images/external/app_screenshots/session-replay.png" style="width: 90%;" alt="Session Replays"/>
</a>
</details>
<details>
<summary>Summary Analytics</summary>
<a href="https://app.agentops.ai?ref=gh">
<img src="docs/images/external/app_screenshots/overview.png" style="width: 90%;" alt="Summary Analytics"/>
</a>
<a href="https://app.agentops.ai?ref=gh">
<img src="docs/images/external/app_screenshots/overview-charts.png" style="width: 90%;" alt="Summary Analytics Charts"/>
</a>
</details>
### First class Developer Experience
Add powerful observability to your agents, tools, and functions with as little code as possible: one line at a time.
<br/>
Refer to our [documentation](http://docs.agentops.ai)
```python
# Create a session span (root for all other spans)
from agentops.sdk.decorators import session
@session
def my_workflow():
# Your session code here
return result
```
```python
# Create an agent span for tracking agent operations
from agentops.sdk.decorators import agent
@agent
class MyAgent:
def __init__(self, name):
self.name = name
# Agent methods here
```
```python
# Create operation/task spans for tracking specific operations
from agentops.sdk.decorators import operation, task
@operation # or @task
def process_data(data):
# Process the data
return result
```
```python
# Create workflow spans for tracking multi-operation workflows
from agentops.sdk.decorators import workflow
@workflow
def my_workflow(data):
# Workflow implementation
return result
```
```python
# Nest decorators for proper span hierarchy
from agentops.sdk.decorators import session, agent, operation
@agent
class MyAgent:
@operation
def nested_operation(self, message):
return f"Processed: {message}"
@operation
def main_operation(self):
result = self.nested_operation("test message")
return result
@session
def my_session():
agent = MyAgent()
return agent.main_operation()
```
All decorators support:
- Input/Output Recording
- Exception Handling
- Async/await functions
- Generator functions
- Custom attributes and names
## Integrations 🦾
### OpenAI Agents SDK 🖇️
Build multi-agent systems with tools, handoffs, and guardrails. AgentOps natively integrates with the OpenAI Agents SDKs for both Python and TypeScript.
#### Python
```bash
pip install openai-agents
```
- [Python integration guide](https://docs.agentops.ai/v2/integrations/openai_agents_python)
- [OpenAI Agents Python documentation](https://openai.github.io/openai-agents-python/)
#### TypeScript
```bash
npm install agentops @openai/agents
```
- [TypeScript integration guide](https://docs.agentops.ai/v2/integrations/openai_agents_js)
- [OpenAI Agents JS documentation](https://openai.github.io/openai-agents-js)
### CrewAI 🛶
Build Crew agents with observability in just 2 lines of code. Simply set an `AGENTOPS_API_KEY` in your environment, and your crews will get automatic monitoring on the AgentOps dashboard.
```bash
pip install 'crewai[agentops]'
```
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/crewai)
- [Official CrewAI documentation](https://docs.crewai.com/how-to/AgentOps-Observability)
### AG2 🤖
With only two lines of code, add full observability and monitoring to AG2 (formerly AutoGen) agents. Set an `AGENTOPS_API_KEY` in your environment and call `agentops.init()`
- [AG2 Observability Example](https://github.com/ag2ai/ag2/blob/main/notebook/agentchat_agentops.ipynb)
- [AG2 - AgentOps Documentation](https://docs.ag2.ai/latest/docs/ecosystem/agentops/)
### Camel AI 🐪
Track and analyze CAMEL agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.
- [Camel AI](https://www.camel-ai.org/) - Advanced agent communication framework
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/camel)
- [Official Camel AI documentation](https://docs.camel-ai.org/cookbooks/agents_tracking.html)
<details>
<summary>Installation</summary>
```bash
pip install "camel-ai[all]==0.2.11"
pip install agentops
```
```python
import os
import agentops
from camel.agents import ChatAgent
from camel.messages import BaseMessage
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType
# Initialize AgentOps
agentops.init(os.getenv("AGENTOPS_API_KEY"), tags=["CAMEL Example"])
# Import toolkits after AgentOps init for tracking
from camel.toolkits import SearchToolkit
# Set up the agent with search tools
sys_msg = BaseMessage.make_assistant_message(
role_name='Tools calling operator',
content='You are a helpful assistant'
)
# Configure tools and model
tools = [*SearchToolkit().get_tools()]
model = ModelFactory.create(
model_platform=ModelPlatformType.OPENAI,
model_type=ModelType.GPT_4O_MINI,
)
# Create and run the agent
camel_agent = ChatAgent(
system_message=sys_msg,
model=model,
tools=tools,
)
response = camel_agent.step("What is AgentOps?")
print(response)
agentops.end_session("Success")
```
Check out our [Camel integration guide](https://docs.agentops.ai/v1/integrations/camel) for more examples including multi-agent scenarios.
</details>
### Langchain 🦜🔗
AgentOps works seamlessly with applications built using Langchain. To use the handler, install Langchain as an optional dependency:
<details>
<summary>Installation</summary>
```shell
pip install agentops[langchain]
```
To use the handler, import and set
```python
import os
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent, AgentType
from agentops.integration.callbacks.langchain import LangchainCallbackHandler
AGENTOPS_API_KEY = os.environ['AGENTOPS_API_KEY']
handler = LangchainCallbackHandler(api_key=AGENTOPS_API_KEY, tags=['Langchain Example'])
llm = ChatOpenAI(openai_api_key=OPENAI_API_KEY,
callbacks=[handler],
model='gpt-3.5-turbo')
agent = initialize_agent(tools,
llm,
agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION,
verbose=True,
callbacks=[handler], # You must pass in a callback handler to record your agent
handle_parsing_errors=True)
```
Check out the [Langchain Examples Notebook](./examples/langchain/langchain_examples.ipynb) for more details including Async handlers.
</details>
### Cohere ⌨️
First class support for Cohere(>=5.4.0). This is a living integration, should you need any added functionality please message us on Discord!
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/cohere)
- [Official Cohere documentation](https://docs.cohere.com/reference/about)
<details>
<summary>Installation</summary>
```bash
pip install cohere
```
```python python
import cohere
import agentops
# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
co = cohere.Client()
chat = co.chat(
message="Is it pronounced ceaux-hear or co-hehray?"
)
print(chat)
agentops.end_session('Success')
```
```python python
import cohere
import agentops
# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
co = cohere.Client()
stream = co.chat_stream(
message="Write me a haiku about the synergies between Cohere and AgentOps"
)
for event in stream:
if event.event_type == "text-generation":
print(event.text, end='')
agentops.end_session('Success')
```
</details>
### Anthropic ﹨
Track agents built with the Anthropic Python SDK (>=0.32.0).
- [AgentOps integration guide](https://docs.agentops.ai/v1/integrations/anthropic)
- [Official Anthropic documentation](https://docs.anthropic.com/en/docs/welcome)
<details>
<summary>Installation</summary>
```bash
pip install anthropic
```
```python python
import anthropic
import agentops
# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
client = anthropic.Anthropic(
# This is the default and can be omitted
api_key=os.environ.get("ANTHROPIC_API_KEY"),
)
message = client.messages.create(
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Tell me a cool fact about AgentOps",
}
],
model="claude-3-opus-20240229",
)
print(message.content)
agentops.end_session('Success')
```
Streaming
```python python
import anthropic
import agentops
# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
client = anthropic.Anthropic(
# This is the default and can be omitted
api_key=os.environ.get("ANTHROPIC_API_KEY"),
)
stream = client.messages.create(
max_tokens=1024,
model="claude-3-opus-20240229",
messages=[
{
"role": "user",
"content": "Tell me something cool about streaming agents",
}
],
stream=True,
)
response = ""
for event in stream:
if event.type == "content_block_delta":
response += event.delta.text
elif event.type == "message_stop":
print("\n")
print(response)
print("\n")
```
Async
```python python
import asyncio
from anthropic import AsyncAnthropic
client = AsyncAnthropic(
# This is the default and can be omitted
api_key=os.environ.get("ANTHROPIC_API_KEY"),
)
async def main() -> None:
message = await client.messages.create(
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Tell me something interesting about async agents",
}
],
model="claude-3-opus-20240229",
)
print(message.content)
await main()
```
</details>
### Mistral 〽️
Track agents built with the Mistral Python SDK (>=0.32.0).
- [AgentOps integration example](./examples/mistral//mistral_example.ipynb)
- [Official Mistral documentation](https://docs.mistral.ai)
<details>
<summary>Installation</summary>
```bash
pip install mistralai
```
Sync
```python python
from mistralai import Mistral
import agentops
# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
client = Mistral(
# This is the default and can be omitted
api_key=os.environ.get("MISTRAL_API_KEY"),
)
message = client.chat.complete(
messages=[
{
"role": "user",
"content": "Tell me a cool fact about AgentOps",
}
],
model="open-mistral-nemo",
)
print(message.choices[0].message.content)
agentops.end_session('Success')
```
Streaming
```python python
from mistralai import Mistral
import agentops
# Beginning of program's code (i.e. main.py, __init__.py)
agentops.init(<INSERT YOUR API KEY HERE>)
client = Mistral(
# This is the default and can be omitted
api_key=os.environ.get("MISTRAL_API_KEY"),
)
message = client.chat.stream(
messages=[
{
"role": "user",
"content": "Tell me something cool about streaming agents",
}
],
model="open-mistral-nemo",
)
response = ""
for event in message:
if event.data.choices[0].finish_reason == "stop":
print("\n")
print(response)
print("\n")
else:
response += event.text
agentops.end_session('Success')
```
Async
```python python
import asyncio
from mistralai import Mistral
client = Mistral(
# This is the default and can be omitted
api_key=os.environ.get("MISTRAL_API_KEY"),
)
async def main() -> None:
message = await client.chat.complete_async(
messages=[
{
"role": "user",
"content": "Tell me something interesting about async agents",
}
],
model="open-mistral-nemo",
)
print(message.choices[0].message.content)
await main()
```
Async Streaming
```python python
import asyncio
from mistralai import Mistral
client = Mistral(
# This is the default and can be omitted
api_key=os.environ.get("MISTRAL_API_KEY"),
)
async def main() -> None:
message = await client.chat.stream_async(
messages=[
{
"role": "user",
"content": "Tell me something interesting about async streaming agents",
}
],
model="open-mistral-nemo",
)
response = ""
async for event in message:
if event.data.choices[0].finish_reason == "stop":
print("\n")
print(response)
print("\n")
else:
response += event.text
await main()
```
</details>
### CamelAI ﹨
Track agents built with the CamelAI Python SDK (>=0.32.0).
- [CamelAI integration guide](https://docs.camel-ai.org/cookbooks/agents_tracking.html#)
- [Official CamelAI documentation](https://docs.camel-ai.org/index.html)
<details>
<summary>Installation</summary>
```bash
pip install camel-ai[all]
pip install agentops
```
```python python
#Import Dependencies
import agentops
import os
from getpass import getpass
from dotenv import load_dotenv
#Set Keys
load_dotenv()
openai_api_key = os.getenv("OPENAI_API_KEY") or "<your openai key here>"
agentops_api_key = os.getenv("AGENTOPS_API_KEY") or "<your agentops key here>"
```
</details>
[You can find usage examples here!](examples/camelai_examples/README.md).
### LiteLLM 🚅
AgentOps provides support for LiteLLM(>=1.3.1), allowing you to call 100+ LLMs using the same Input/Output Format.
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/litellm)
- [Official LiteLLM documentation](https://docs.litellm.ai/docs/providers)
<details>
<summary>Installation</summary>
```bash
pip install litellm
```
```python python
# Do not use LiteLLM like this
# from litellm import completion
# ...
# response = completion(model="claude-3", messages=messages)
# Use LiteLLM like this
import litellm
...
response = litellm.completion(model="claude-3", messages=messages)
# or
response = await litellm.acompletion(model="claude-3", messages=messages)
```
</details>
### LlamaIndex 🦙
AgentOps works seamlessly with applications built using LlamaIndex, a framework for building context-augmented generative AI applications with LLMs.
<details>
<summary>Installation</summary>
```shell
pip install llama-index-instrumentation-agentops
```
To use the handler, import and set
```python
from llama_index.core import set_global_handler
# NOTE: Feel free to set your AgentOps environment variables (e.g., 'AGENTOPS_API_KEY')
# as outlined in the AgentOps documentation, or pass the equivalent keyword arguments
# anticipated by AgentOps' AOClient as **eval_params in set_global_handler.
set_global_handler("agentops")
```
Check out the [LlamaIndex docs](https://docs.llamaindex.ai/en/stable/module_guides/observability/?h=agentops#agentops) for more details.
</details>
### Llama Stack 🦙🥞
AgentOps provides support for Llama Stack Python Client(>=0.0.53), allowing you to monitor your Agentic applications.
- [AgentOps integration example 1](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-fdddf65549f3714f8f007ce7dfd1cde720329fe54155d54389dd50fbd81813cb)
- [AgentOps integration example 2](https://github.com/AgentOps-AI/agentops/pull/530/files/65a5ab4fdcf310326f191d4b870d4f553591e3ea#diff-6688ff4fb7ab1ce7b1cc9b8362ca27264a3060c16737fb1d850305787a6e3699)
- [Official Llama Stack Python Client](https://github.com/meta-llama/llama-stack-client-python)
### SwarmZero AI 🐝
Track and analyze SwarmZero agents with full observability. Set an `AGENTOPS_API_KEY` in your environment and initialize AgentOps to get started.
- [SwarmZero](https://swarmzero.ai) - Advanced multi-agent framework
- [AgentOps integration example](https://docs.agentops.ai/v1/integrations/swarmzero)
- [SwarmZero AI integration example](https://docs.swarmzero.ai/examples/ai-agents/build-and-monitor-a-web-search-agent)
- [SwarmZero AI - AgentOps documentation](https://docs.swarmzero.ai/sdk/observability/agentops)
- [Official SwarmZero Python SDK](https://github.com/swarmzero/swarmzero)
<details>
<summary>Installation</summary>
```bash
pip install swarmzero
pip install agentops
```
```python
from dotenv import load_dotenv
load_dotenv()
import agentops
agentops.init(<INSERT YOUR API KEY HERE>)
from swarmzero import Agent, Swarm
# ...
```
</details>
## Evaluations Roadmap 🧭
| Platform | Dashboard | Evals |
| ---------------------------------------------------------------------------- | ------------------------------------------ | -------------------------------------- |
| ✅ Python SDK | ✅ Multi-session and Cross-session metrics | ✅ Custom eval metrics |
| 🚧 Evaluation builder API | ✅ Custom event tag tracking | 🔜 Agent scorecards |
| 🚧 [Javascript/Typescript SDK (Alpha)](https://github.com/AgentOps-AI/agentops-node) | ✅ Session replays | 🔜 Evaluation playground + leaderboard |
## Debugging Roadmap 🧭
| Performance testing | Environments | LLM Testing | Reasoning and execution testing |
| ----------------------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------- | ------------------------------------------------- |
| ✅ Event latency analysis | 🔜 Non-stationary environment testing | 🔜 LLM non-deterministic function detection | 🚧 Infinite loops and recursive thought detection |
| ✅ Agent workflow execution pricing | 🔜 Multi-modal environments | 🚧 Token limit overflow flags | 🔜 Faulty reasoning detection |
| 🚧 Success validators (external) | 🔜 Execution containers | 🔜 Context limit overflow flags | 🔜 Generative code validators |
| 🔜 Agent controllers/skill tests | ✅ Honeypot and prompt injection detection ([PromptArmor](https://promptarmor.com)) | ✅ API bill tracking | 🔜 Error breakpoint analysis |
| 🔜 Information context constraint testing | 🔜 Anti-agent roadblocks (i.e. Captchas) | 🔜 CI/CD integration checks | |
| 🔜 Regression testing | ✅ Multi-agent framework visualization | | |
### Why AgentOps? 🤔
Without the right tools, AI agents are slow, expensive, and unreliable. Our mission is to bring your agent from prototype to production. Here's why AgentOps stands out:
- **Comprehensive Observability**: Track your AI agents' performance, user interactions, and API usage.
- **Real-Time Monitoring**: Get instant insights with session replays, metrics, and live monitoring tools.
- **Cost Control**: Monitor and manage your spend on LLM and API calls.
- **Failure Detection**: Quickly identify and respond to agent failures and multi-agent interaction issues.
- **Tool Usage Statistics**: Understand how your agents utilize external tools with detailed analytics.
- **Session-Wide Metrics**: Gain a holistic view of your agents' sessions with comprehensive statistics.
AgentOps is designed to make agent observability, testing, and monitoring easy.
## Star History
Check out our growth in the community:
<img src="https://api.star-history.com/svg?repos=AgentOps-AI/agentops&type=Date" style="max-width: 500px" width="50%" alt="Logo">
## Popular projects using AgentOps
| Repository | Stars |
| :-------- | -----: |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/2707039?s=40&v=4" width="20" height="20" alt=""> &nbsp; [geekan](https://github.com/geekan) / [MetaGPT](https://github.com/geekan/MetaGPT) | 42787 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/130722866?s=40&v=4" width="20" height="20" alt=""> &nbsp; [run-llama](https://github.com/run-llama) / [llama_index](https://github.com/run-llama/llama_index) | 34446 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/170677839?s=40&v=4" width="20" height="20" alt=""> &nbsp; [crewAIInc](https://github.com/crewAIInc) / [crewAI](https://github.com/crewAIInc/crewAI) | 18287 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/134388954?s=40&v=4" width="20" height="20" alt=""> &nbsp; [camel-ai](https://github.com/camel-ai) / [camel](https://github.com/camel-ai/camel) | 5166 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/152537519?s=40&v=4" width="20" height="20" alt=""> &nbsp; [superagent-ai](https://github.com/superagent-ai) / [superagent](https://github.com/superagent-ai/superagent) | 5050 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/30197649?s=40&v=4" width="20" height="20" alt=""> &nbsp; [iyaja](https://github.com/iyaja) / [llama-fs](https://github.com/iyaja/llama-fs) | 4713 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/188122941?s=40&v=4" width="20" height="20" alt=""> &nbsp; [ag2ai](https://github.com/ag2ai) / [ag2](https://github.com/ag2ai/ag2) | 4240 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/162546372?s=40&v=4" width="20" height="20" alt=""> &nbsp; [BasedHardware](https://github.com/BasedHardware) / [Omi](https://github.com/BasedHardware/Omi) | 2723 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/454862?s=40&v=4" width="20" height="20" alt=""> &nbsp; [MervinPraison](https://github.com/MervinPraison) / [PraisonAI](https://github.com/MervinPraison/PraisonAI) | 2007 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/140554352?s=40&v=4" width="20" height="20" alt=""> &nbsp; [AgentOps-AI](https://github.com/AgentOps-AI) / [Jaiqu](https://github.com/AgentOps-AI/Jaiqu) | 272 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/173542722?s=48&v=4" width="20" height="20" alt=""> &nbsp; [swarmzero](https://github.com/swarmzero) / [swarmzero](https://github.com/swarmzero/swarmzero) | 195 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/3074263?s=40&v=4" width="20" height="20" alt=""> &nbsp; [strnad](https://github.com/strnad) / [CrewAI-Studio](https://github.com/strnad/CrewAI-Studio) | 134 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/18406448?s=40&v=4" width="20" height="20" alt=""> &nbsp; [alejandro-ao](https://github.com/alejandro-ao) / [exa-crewai](https://github.com/alejandro-ao/exa-crewai) | 55 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/64493665?s=40&v=4" width="20" height="20" alt=""> &nbsp; [tonykipkemboi](https://github.com/tonykipkemboi) / [youtube_yapper_trapper](https://github.com/tonykipkemboi/youtube_yapper_trapper) | 47 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/17598928?s=40&v=4" width="20" height="20" alt=""> &nbsp; [sethcoast](https://github.com/sethcoast) / [cover-letter-builder](https://github.com/sethcoast/cover-letter-builder) | 27 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/109994880?s=40&v=4" width="20" height="20" alt=""> &nbsp; [bhancockio](https://github.com/bhancockio) / [chatgpt4o-analysis](https://github.com/bhancockio/chatgpt4o-analysis) | 19 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/14105911?s=40&v=4" width="20" height="20" alt=""> &nbsp; [breakstring](https://github.com/breakstring) / [Agentic_Story_Book_Workflow](https://github.com/breakstring/Agentic_Story_Book_Workflow) | 14 |
|<img class="avatar mr-2" src="https://avatars.githubusercontent.com/u/124134656?s=40&v=4" width="20" height="20" alt=""> &nbsp; [MULTI-ON](https://github.com/MULTI-ON) / [multion-python](https://github.com/MULTI-ON/multion-python) | 13 |
_Generated using [github-dependents-info](https://github.com/nvuillam/github-dependents-info), by [Nicolas Vuillamy](https://github.com/nvuillam)_
+5
View File
@@ -0,0 +1,5 @@
#### Disclaimer on Error Reporting
_By implementing this SDK, you acknowledge and agree that the SDK may automatically send error reports to our servers. These reports are strictly limited to information relevant for identifying and resolving issues, such as error messages, stack traces (including file paths that may include your system username), and the SDK version. Our primary aim in collecting this data is to enhance the developer experience by identifying, diagnosing, and fixing issues more efficiently. No personal information or sensitive data will be transmitted as part of these reports without your explicit consent._
_This collection is in compliance with the California Consumer Privacy Act (CCPA). We commit to not selling your Personal Information and ensuring its confidentiality and security. California residents have the right to know about, access, and delete their Personal Information collected by us. For details on how to exercise these rights, please refer to our [Privacy Policy](https://docs.google.com/document/d/1SjWwa8nhHFLXa0ytSHv5F7osfNDclA6GU9-RvTevGDs/edit?usp=sharing). Our policy also includes information on how to opt-out of the sale of Personal Information, though we do not engage in this practice._
+488
View File
@@ -0,0 +1,488 @@
# For backwards compatibility
from agentops.legacy import (
start_session,
end_session,
track_agent,
track_tool,
end_all_sessions,
Session,
ToolEvent,
ErrorEvent,
ActionEvent,
LLMEvent,
) # type: ignore
# Import all required modules at the top
from opentelemetry.trace import get_current_span
from agentops.semconv import (
AgentAttributes,
ToolAttributes,
WorkflowAttributes,
CoreAttributes,
SpanKind,
SpanAttributes,
)
import json
from typing import List, Optional, Union, Dict, Any
from agentops.client import Client
from agentops.sdk.core import TraceContext, tracer
from agentops.sdk.decorators import trace, session, agent, task, workflow, operation, tool, guardrail, track_endpoint
from agentops.enums import TraceState, SUCCESS, ERROR, UNSET
from opentelemetry.trace.status import StatusCode
from agentops.logging.config import logger
from agentops.helpers.deprecation import deprecated, warn_deprecated_param
import threading
# Import validation functions
from agentops.validation import validate_trace_spans, print_validation_summary, ValidationError
# Thread-safe client management
_client_lock = threading.Lock()
_client = None
def get_client() -> Client:
"""Get the singleton client instance in a thread-safe manner"""
global _client
# Double-checked locking pattern for thread safety
if _client is None:
with _client_lock:
if _client is None:
_client = Client()
return _client
@deprecated("Automatically tracked in v4.")
def record(event):
"""
Legacy function to record an event. This is kept for backward compatibility.
In the current version, this simply sets the end_timestamp on the event.
Args:
event: The event to record
"""
from agentops.helpers.time import get_ISO_time
# TODO: Manual timestamp assignment is a temporary fix; should use proper event lifecycle
if event and hasattr(event, "end_timestamp"):
event.end_timestamp = get_ISO_time()
return event
def init(
api_key: Optional[str] = None,
endpoint: Optional[str] = None,
app_url: Optional[str] = None,
max_wait_time: Optional[int] = None,
max_queue_size: Optional[int] = None,
tags: Optional[List[str]] = None,
default_tags: Optional[List[str]] = None,
trace_name: Optional[str] = None,
instrument_llm_calls: Optional[bool] = None,
auto_start_session: Optional[bool] = None,
auto_init: Optional[bool] = None,
skip_auto_end_session: Optional[bool] = None,
env_data_opt_out: Optional[bool] = None,
log_level: Optional[Union[str, int]] = None,
fail_safe: Optional[bool] = None,
log_session_replay_url: Optional[bool] = None,
exporter_endpoint: Optional[str] = None,
**kwargs,
):
"""
Initializes the AgentOps SDK.
Args:
api_key (str, optional): API Key for AgentOps services. If none is provided, key will
be read from the AGENTOPS_API_KEY environment variable.
endpoint (str, optional): The endpoint for the AgentOps service. If none is provided, key will
be read from the AGENTOPS_API_ENDPOINT environment variable. Defaults to 'https://api.agentops.ai'.
app_url (str, optional): The dashboard URL for the AgentOps app. If none is provided, key will
be read from the AGENTOPS_APP_URL environment variable. Defaults to 'https://app.agentops.ai'.
max_wait_time (int, optional): The maximum time to wait in milliseconds before flushing the queue.
Defaults to 5,000 (5 seconds)
max_queue_size (int, optional): The maximum size of the event queue. Defaults to 512.
tags (List[str], optional): [Deprecated] Use `default_tags` instead.
default_tags (List[str], optional): Default tags for the sessions that can be used for grouping or sorting later (e.g. ["GPT-4"]).
trace_name (str, optional): Name for the default trace/session. If none is provided, defaults to "default".
instrument_llm_calls (bool): Whether to instrument LLM calls and emit LLMEvents.
auto_start_session (bool): Whether to start a session automatically when the client is created.
auto_init (bool): Whether to automatically initialize the client on import. Defaults to True.
skip_auto_end_session (optional, bool): Don't automatically end session based on your framework's decision-making
(i.e. Crew determining when tasks are complete and ending the session)
env_data_opt_out (bool): Whether to opt out of collecting environment data.
log_level (str, int): The log level to use for the client. Defaults to 'CRITICAL'.
fail_safe (bool): Whether to suppress errors and continue execution when possible.
log_session_replay_url (bool): Whether to log session replay URLs to the console. Defaults to True.
exporter_endpoint (str, optional): Endpoint for the exporter. If none is provided, key will
be read from the AGENTOPS_EXPORTER_ENDPOINT environment variable.
**kwargs: Additional configuration parameters to be passed to the client.
"""
global _client
# Check for deprecated parameters and emit warnings
if tags is not None:
warn_deprecated_param("tags", "default_tags")
# Merge tags and default_tags if both are provided
merged_tags = None
if tags and default_tags:
merged_tags = list(set(tags + default_tags))
elif tags:
merged_tags = tags
elif default_tags:
merged_tags = default_tags
# Check if in a Jupyter Notebook (manual start/end_trace())
try:
get_ipython().__class__.__name__ == "ZMQInteractiveShell" # type: ignore
auto_start_session = False
except NameError:
pass
# Prepare initialization arguments
init_kwargs = {
"api_key": api_key,
"endpoint": endpoint,
"app_url": app_url,
"max_wait_time": max_wait_time,
"max_queue_size": max_queue_size,
"default_tags": merged_tags,
"trace_name": trace_name,
"instrument_llm_calls": instrument_llm_calls,
"auto_start_session": auto_start_session,
"auto_init": auto_init,
"skip_auto_end_session": skip_auto_end_session,
"env_data_opt_out": env_data_opt_out,
"log_level": log_level,
"fail_safe": fail_safe,
"log_session_replay_url": log_session_replay_url,
"exporter_endpoint": exporter_endpoint,
**kwargs,
}
# Get the current client instance (creates new one if needed)
client = get_client()
# Initialize the client directly
return client.init(**init_kwargs)
def configure(**kwargs):
"""Update client configuration
Args:
**kwargs: Configuration parameters. Supported parameters include:
- api_key: API Key for AgentOps services
- endpoint: The endpoint for the AgentOps service
- app_url: The dashboard URL for the AgentOps app
- max_wait_time: Maximum time to wait in milliseconds before flushing the queue
- max_queue_size: Maximum size of the event queue
- default_tags: Default tags for the sessions
- instrument_llm_calls: Whether to instrument LLM calls
- auto_start_session: Whether to start a session automatically
- skip_auto_end_session: Don't automatically end session
- env_data_opt_out: Whether to opt out of collecting environment data
- log_level: The log level to use for the client
- fail_safe: Whether to suppress errors and continue execution
- exporter: Custom span exporter for OpenTelemetry trace data
- processor: Custom span processor for OpenTelemetry trace data
- exporter_endpoint: Endpoint for the exporter
"""
global _client
# List of valid parameters that can be passed to configure
valid_params = {
"api_key",
"endpoint",
"app_url",
"max_wait_time",
"max_queue_size",
"default_tags",
"instrument_llm_calls",
"auto_start_session",
"skip_auto_end_session",
"env_data_opt_out",
"log_level",
"fail_safe",
"exporter",
"processor",
"exporter_endpoint",
}
# Check for invalid parameters
invalid_params = set(kwargs.keys()) - valid_params
if invalid_params:
logger.warning(f"Invalid configuration parameters: {invalid_params}")
client = get_client()
client.configure(**kwargs)
def start_trace(
trace_name: str = "session", tags: Optional[Union[Dict[str, Any], List[str]]] = None
) -> Optional[TraceContext]:
"""
Starts a new trace (root span) and returns its context.
This allows for multiple concurrent, user-managed traces.
Args:
trace_name: Name for the trace (e.g., "session", "my_custom_task").
tags: Optional tags to attach to the trace span (list of strings or dict).
Returns:
A TraceContext object containing the span and context token, or None if SDK not initialized.
"""
if not tracer.initialized:
# Optionally, attempt to initialize the client if not already, or log a more severe warning.
# For now, align with legacy start_session that would try to init.
# However, explicit init is preferred before starting traces.
logger.warning("AgentOps SDK not initialized. Attempting to initialize with defaults before starting trace.")
try:
init() # Attempt to initialize with environment variables / defaults
if not tracer.initialized:
logger.error("SDK initialization failed. Cannot start trace.")
return None
except Exception as e:
logger.error(f"SDK auto-initialization failed during start_trace: {e}. Cannot start trace.")
return None
return tracer.start_trace(trace_name=trace_name, tags=tags)
def end_trace(
trace_context: Optional[TraceContext] = None, end_state: Union[TraceState, StatusCode, str] = TraceState.SUCCESS
) -> None:
"""
Ends a trace (its root span) and finalizes it.
If no trace_context is provided, ends all active session spans.
Args:
trace_context: The TraceContext object returned by start_trace. If None, ends all active traces.
end_state: The final state of the trace (e.g., "Success", "Indeterminate", "Error").
"""
if not tracer.initialized:
logger.warning("AgentOps SDK not initialized. Cannot end trace.")
return
tracer.end_trace(trace_context=trace_context, end_state=end_state)
def update_trace_metadata(metadata: Dict[str, Any], prefix: str = "trace.metadata") -> bool:
"""
Update metadata on the current running trace.
Args:
metadata: Dictionary of key-value pairs to set as trace metadata.
Values must be strings, numbers, booleans, or lists of these types.
Lists are converted to JSON string representation.
Keys can be either custom keys or semantic convention aliases.
prefix: Prefix for metadata attributes (default: "trace.metadata").
Ignored for semantic convention attributes.
Returns:
bool: True if metadata was successfully updated, False otherwise.
"""
if not tracer.initialized:
logger.warning("AgentOps SDK not initialized. Cannot update trace metadata.")
return False
# Build semantic convention mappings dynamically
def build_semconv_mappings():
"""Build mappings from user-friendly keys to semantic convention attributes."""
mappings = {}
# Helper function to extract attribute name from semantic convention
def extract_key_from_attr(attr_value: str) -> str:
parts = attr_value.split(".")
if len(parts) >= 2:
# Handle special cases
if parts[0] == "error":
# error.type -> error_type
return "_".join(parts)
else:
# Default: entity.attribute -> entity_attribute
return "_".join(parts)
return attr_value
# Process each semantic convention class
for cls in [AgentAttributes, ToolAttributes, WorkflowAttributes, CoreAttributes, SpanAttributes]:
for attr_name, attr_value in cls.__dict__.items():
if not attr_name.startswith("_") and isinstance(attr_value, str):
# Skip gen_ai attributes
if attr_value.startswith("gen_ai."):
continue
# Generate user-friendly key
user_key = extract_key_from_attr(attr_value)
mappings[user_key] = attr_value
# Add some additional convenience mappings
if attr_value == CoreAttributes.TAGS:
mappings["tags"] = attr_value
return mappings
# Build mappings if using semantic conventions
SEMCONV_MAPPINGS = build_semconv_mappings()
# Collect all valid semantic convention attributes
VALID_SEMCONV_ATTRS = set()
for cls in [AgentAttributes, ToolAttributes, WorkflowAttributes, CoreAttributes, SpanAttributes]:
for key, value in cls.__dict__.items():
if not key.startswith("_") and isinstance(value, str):
# Include all attributes except gen_ai ones
if not value.startswith("gen_ai."):
VALID_SEMCONV_ATTRS.add(value)
# Find the current trace span
span = None
# Get the current span from OpenTelemetry context
current_span = get_current_span()
# Check if the current span is valid and recording
if current_span and hasattr(current_span, "is_recording") and current_span.is_recording():
# Check if this is a trace/session span or a child span
span_name = getattr(current_span, "name", "")
# If it's a session/trace span, use it directly
if span_name.endswith(f".{SpanKind.SESSION}"):
span = current_span
else:
# It's a child span, try to find the root trace span
# Get all active traces
active_traces = tracer.get_active_traces()
if active_traces:
# Find the trace that contains the current span
current_trace_id = current_span.get_span_context().trace_id
for trace_id_str, trace_ctx in active_traces.items():
try:
# Convert hex string back to int for comparison
trace_id = int(trace_id_str, 16)
if trace_id == current_trace_id:
span = trace_ctx.span
break
except (ValueError, AttributeError):
continue
# If we couldn't find the parent trace, use the current span
if not span:
span = current_span
else:
# No active traces, use the current span
span = current_span
# If no current span or it's not recording, check active traces
if not span:
active_traces = tracer.get_active_traces()
if active_traces:
# Get the most recently created trace (last in the dict)
trace_context = list(active_traces.values())[-1]
span = trace_context.span
logger.debug("Using most recent active trace for metadata update")
else:
logger.warning("No active trace found. Cannot update metadata.")
return False
# Ensure the span is recording before updating
if not span or (hasattr(span, "is_recording") and not span.is_recording()):
logger.warning("Span is not recording. Cannot update metadata.")
return False
# Update the span attributes with the metadata
try:
updated_count = 0
for key, value in metadata.items():
# Validate the value type
if value is None:
continue
# Convert lists to JSON string representation for OpenTelemetry compatibility
if isinstance(value, list):
# Ensure all list items are valid types
if all(isinstance(item, (str, int, float, bool)) for item in value):
value = json.dumps(value)
else:
logger.warning(f"Skipping metadata key '{key}': list contains invalid types")
continue
elif not isinstance(value, (str, int, float, bool)):
logger.warning(f"Skipping metadata key '{key}': value type {type(value)} not supported")
continue
# Determine the attribute key
attribute_key = key
# Check if key is already a valid semantic convention attribute
if key in VALID_SEMCONV_ATTRS:
# Key is already a valid semantic convention, use as-is
attribute_key = key
elif key in SEMCONV_MAPPINGS:
# It's a user-friendly key, map it to semantic convention
attribute_key = SEMCONV_MAPPINGS[key]
logger.debug(f"Mapped '{key}' to semantic convention '{attribute_key}'")
else:
# Not a semantic convention, use with prefix
attribute_key = f"{prefix}.{key}"
# Set the attribute
span.set_attribute(attribute_key, value)
updated_count += 1
if updated_count > 0:
logger.debug(f"Successfully updated {updated_count} metadata attributes on trace")
return True
else:
logger.warning("No valid metadata attributes were updated")
return False
except Exception as e:
logger.error(f"Error updating trace metadata: {e}")
return False
__all__ = [
# Legacy exports
"start_session",
"end_session",
"track_agent",
"track_tool",
"end_all_sessions",
"Session",
"ToolEvent",
"ErrorEvent",
"ActionEvent",
"LLMEvent",
# Modern exports
"init",
"start_trace",
"end_trace",
"update_trace_metadata",
"Client",
"get_client",
# Decorators
"trace",
"session",
"agent",
"task",
"workflow",
"operation",
"tool",
"guardrail",
"track_endpoint",
# Enums
"TraceState",
"SUCCESS",
"ERROR",
"UNSET",
# Validation
"validate_trace_spans",
"print_validation_summary",
"ValidationError",
]
+5
View File
@@ -0,0 +1,5 @@
from agentops.client.client import Client
from agentops.client.api import ApiClient
__all__ = ["Client", "ApiClient"]
+71
View File
@@ -0,0 +1,71 @@
"""
API client for the AgentOps API.
This module provides the client for the AgentOps API.
"""
from typing import Dict, Type, TypeVar, cast
from agentops.client.api.base import BaseApiClient
from agentops.client.api.types import AuthTokenResponse
from agentops.client.api.versions.v3 import V3Client
from agentops.client.api.versions.v4 import V4Client
# Define a type variable for client classes
T = TypeVar("T", bound=BaseApiClient)
__all__ = ["ApiClient", "BaseApiClient", "AuthTokenResponse"]
class ApiClient:
"""
Master API client that contains all version-specific clients.
This client provides a unified interface for accessing different API versions.
It lazily initializes version-specific clients when they are first accessed.
"""
def __init__(self, endpoint: str = "https://api.agentops.ai"):
"""
Initialize the master API client.
Args:
endpoint: The base URL for the API
"""
self.endpoint = endpoint
self._clients: Dict[str, BaseApiClient] = {}
@property
def v3(self) -> V3Client:
"""
Get the V3 API client.
Returns:
The V3 API client
"""
return self._get_client("v3", V3Client)
@property
def v4(self) -> V4Client:
"""
Get the V4 API client.
Returns:
The V4 API client
"""
return self._get_client("v4", V4Client)
def _get_client(self, version: str, client_class: Type[T]) -> T:
"""
Get or create a version-specific client.
Args:
version: The API version
client_class: The client class to instantiate
Returns:
The version-specific client
"""
if version not in self._clients:
self._clients[version] = client_class(self.endpoint)
return cast(T, self._clients[version])
+160
View File
@@ -0,0 +1,160 @@
"""
Base API client classes for making HTTP requests.
This module provides the foundation for all API clients in the AgentOps SDK.
"""
from typing import Any, Dict, Optional, Protocol
import requests
from agentops.client.http.http_client import HttpClient
from agentops.helpers.version import get_agentops_version
class TokenFetcher(Protocol):
"""Protocol for token fetching functions"""
def __call__(self, api_key: str) -> str: ...
class BaseApiClient:
"""
Base class for API communication with async HTTP methods.
This class provides the core HTTP functionality without authentication.
All HTTP methods are asynchronous.
"""
def __init__(self, endpoint: str):
"""
Initialize the base API client.
Args:
endpoint: The base URL for the API
"""
self.endpoint = endpoint
self.http_client = HttpClient()
self.last_response: Optional[requests.Response] = None
def prepare_headers(self, custom_headers: Optional[Dict[str, str]] = None) -> Dict[str, str]:
"""
Prepare headers for API requests.
Args:
custom_headers: Additional headers to include
Returns:
Headers dictionary with standard headers and any custom headers
"""
headers = {
"Content-Type": "application/json",
"Connection": "keep-alive",
"Keep-Alive": "timeout=10, max=1000",
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
}
if custom_headers:
headers.update(custom_headers)
return headers
def _get_full_url(self, path: str) -> str:
"""
Get the full URL for a path.
Args:
path: The API endpoint path
Returns:
The full URL
"""
return f"{self.endpoint}{path}"
async def async_request(
self,
method: str,
path: str,
data: Optional[Dict[str, Any]] = None,
headers: Optional[Dict[str, str]] = None,
timeout: int = 30,
) -> Optional[Dict[str, Any]]:
"""
Make a generic async HTTP request
Args:
method: HTTP method (e.g., 'get', 'post', 'put', 'delete')
path: API endpoint path
data: Request payload (for POST, PUT methods)
headers: Request headers
timeout: Request timeout in seconds
Returns:
JSON response as dictionary, or None if request failed
Raises:
Exception: If the request fails
"""
url = self._get_full_url(path)
try:
response_data = await self.http_client.async_request(
method=method, url=url, data=data, headers=headers, timeout=timeout
)
return response_data
except Exception as e:
raise Exception(f"{method.upper()} request failed: {str(e)}") from e
async def post(self, path: str, data: Dict[str, Any], headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
"""
Make async POST request
Args:
path: API endpoint path
data: Request payload
headers: Request headers
Returns:
JSON response as dictionary, or None if request failed
"""
return await self.async_request("post", path, data=data, headers=headers)
async def get(self, path: str, headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
"""
Make async GET request
Args:
path: API endpoint path
headers: Request headers
Returns:
JSON response as dictionary, or None if request failed
"""
return await self.async_request("get", path, headers=headers)
async def put(self, path: str, data: Dict[str, Any], headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
"""
Make async PUT request
Args:
path: API endpoint path
data: Request payload
headers: Request headers
Returns:
JSON response as dictionary, or None if request failed
"""
return await self.async_request("put", path, data=data, headers=headers)
async def delete(self, path: str, headers: Dict[str, str]) -> Optional[Dict[str, Any]]:
"""
Make async DELETE request
Args:
path: API endpoint path
headers: Request headers
Returns:
JSON response as dictionary, or None if request failed
"""
return await self.async_request("delete", path, headers=headers)
+21
View File
@@ -0,0 +1,21 @@
"""
Common types used across API client modules.
This module contains type definitions used by multiple API client modules.
"""
from typing import TypedDict
class AuthTokenResponse(TypedDict):
"""Response from the auth/token endpoint"""
token: str
project_id: str
class UploadedObjectResponse(TypedDict):
"""Response from the v4/objects/upload endpoint"""
url: str
size: int
+10
View File
@@ -0,0 +1,10 @@
"""
API client versions package.
This package contains client implementations for different API versions.
"""
from agentops.client.api.versions.v3 import V3Client
from agentops.client.api.versions.v4 import V4Client
__all__ = ["V3Client", "V4Client"]
+69
View File
@@ -0,0 +1,69 @@
"""
V3 API client for the AgentOps API.
This module provides the client for the V3 version of the AgentOps API.
"""
from agentops.client.api.base import BaseApiClient
from agentops.client.api.types import AuthTokenResponse
from agentops.client.http.http_client import HttpClient
from agentops.logging import logger
from termcolor import colored
class V3Client(BaseApiClient):
"""Client for the AgentOps V3 API"""
def __init__(self, endpoint: str):
"""
Initialize the V3 API client.
Args:
endpoint: The base URL for the API
"""
# Set up with V3-specific auth endpoint
super().__init__(endpoint)
async def fetch_auth_token(self, api_key: str) -> AuthTokenResponse:
"""
Asynchronously fetch authentication token.
Args:
api_key: The API key to authenticate with
Returns:
AuthTokenResponse containing token and project information, or None if failed
"""
try:
path = "/v3/auth/token"
data = {"api_key": api_key}
headers = self.prepare_headers()
# Build full URL
url = self._get_full_url(path)
# Make async request
response_data = await HttpClient.async_request(
method="POST", url=url, data=data, headers=headers, timeout=30
)
token = response_data.get("token")
if not token:
logger.warning("Authentication failed: Perhaps an invalid API key?")
return None
# Check project premium status
if response_data.get("project_prem_status") != "pro":
logger.info(
colored(
"\x1b[34mYou're on the agentops free plan 🤔\x1b[0m",
"blue",
)
)
return response_data
except Exception:
return None
# Add V3-specific API methods here
+145
View File
@@ -0,0 +1,145 @@
"""
V4 API client for the AgentOps API.
This module provides the client for the V4 version of the AgentOps API.
"""
from typing import Optional, Union, Dict, Any
import requests
from agentops.client.api.base import BaseApiClient
from agentops.client.http.http_client import HttpClient
from agentops.exceptions import ApiServerException
from agentops.helpers.version import get_agentops_version
class V4Client(BaseApiClient):
"""Client for the AgentOps V4 API"""
def __init__(self, endpoint: str):
"""Initialize the V4 API client."""
super().__init__(endpoint)
self.auth_token: Optional[str] = None
def set_auth_token(self, token: str):
"""
Set the authentication token for API requests.
Args:
token: The authentication token to set
"""
self.auth_token = token
def prepare_headers(self, custom_headers: Optional[Dict[str, str]] = None) -> Dict[str, str]:
"""
Prepare headers for API requests.
Args:
custom_headers: Additional headers to include
Returns:
Headers dictionary with standard headers and any custom headers
"""
headers = {
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
}
# Only add Authorization header if we have a token
if self.auth_token:
headers["Authorization"] = f"Bearer {self.auth_token}"
if custom_headers:
headers.update(custom_headers)
return headers
def post(self, path: str, body: Union[str, bytes], headers: Optional[Dict[str, str]] = None) -> requests.Response:
"""
Make a POST request to the V4 API.
Args:
path: The API path to POST to
body: The request body (string or bytes)
headers: Optional headers to include
Returns:
The response object
"""
url = self._get_full_url(path)
request_headers = headers or self.prepare_headers()
return HttpClient.get_session().post(url, json={"body": body}, headers=request_headers, timeout=30)
def upload_object(self, body: Union[str, bytes]) -> Dict[str, Any]:
"""
Upload an object to the V4 API.
Args:
body: The object body to upload
Returns:
Dictionary containing upload response data
Raises:
ApiServerException: If the upload fails
"""
try:
# Convert bytes to string for consistency with test expectations
if isinstance(body, bytes):
body = body.decode("utf-8")
response = self.post("/v4/objects/upload/", body, self.prepare_headers())
if response.status_code != 200:
error_msg = f"Upload failed: {response.status_code}"
try:
error_data = response.json()
if "error" in error_data:
error_msg = error_data["error"]
except:
pass
raise ApiServerException(error_msg)
try:
return response.json()
except Exception as e:
raise ApiServerException(f"Failed to process upload response: {str(e)}")
except requests.exceptions.RequestException as e:
raise ApiServerException(f"Failed to upload object: {e}")
def upload_logfile(self, body: Union[str, bytes], trace_id: str) -> Dict[str, Any]:
"""
Upload a logfile to the V4 API.
Args:
body: The logfile content to upload
trace_id: The trace ID associated with the logfile
Returns:
Dictionary containing upload response data
Raises:
ApiServerException: If the upload fails
"""
try:
# Convert bytes to string for consistency with test expectations
if isinstance(body, bytes):
body = body.decode("utf-8")
headers = {**self.prepare_headers(), "Trace-Id": str(trace_id)}
response = self.post("/v4/logs/upload/", body, headers)
if response.status_code != 200:
error_msg = f"Upload failed: {response.status_code}"
try:
error_data = response.json()
if "error" in error_data:
error_msg = error_data["error"]
except:
pass
raise ApiServerException(error_msg)
try:
return response.json()
except Exception as e:
raise ApiServerException(f"Failed to process upload response: {str(e)}")
except requests.exceptions.RequestException as e:
raise ApiServerException(f"Failed to upload logfile: {e}")
+280
View File
@@ -0,0 +1,280 @@
import atexit
import asyncio
import threading
from typing import Optional, Any
from agentops.client.api import ApiClient
from agentops.config import Config
from agentops.instrumentation import instrument_all
from agentops.logging import logger
from agentops.logging.config import configure_logging, intercept_opentelemetry_logging
from agentops.sdk.core import TraceContext, tracer
from agentops.legacy import Session
# Global variables to hold the client's auto-started trace and its legacy session wrapper
_client_init_trace_context: Optional[TraceContext] = None
_client_legacy_session_for_init_trace: Optional[Session] = None
# Single atexit handler registered flag
_atexit_registered = False
def _end_init_trace_atexit():
"""Global atexit handler to end the client's auto-initialized trace during shutdown."""
global _client_init_trace_context, _client_legacy_session_for_init_trace
if _client_init_trace_context is not None:
logger.debug("Auto-ending client's init trace during shutdown.")
try:
# Use global tracer to end the trace directly
if tracer.initialized and _client_init_trace_context.span.is_recording():
tracer.end_trace(_client_init_trace_context, end_state="Shutdown")
except Exception as e:
logger.warning(f"Error ending client's init trace during shutdown: {e}")
finally:
_client_init_trace_context = None
_client_legacy_session_for_init_trace = None # Clear its legacy wrapper too
class Client:
"""Singleton client for AgentOps service"""
config: Config
_initialized: bool
_init_trace_context: Optional[TraceContext] = None # Stores the context of the auto-started trace
_legacy_session_for_init_trace: Optional[Session] = (
None # Stores the legacy Session wrapper for the auto-started trace
)
__instance = None # Class variable for singleton pattern
api: ApiClient
_auth_token: Optional[str] = None
_project_id: Optional[str] = None
_auth_lock = threading.Lock()
_auth_task: Optional[asyncio.Task] = None
def __new__(cls, *args: Any, **kwargs: Any) -> "Client":
if cls.__instance is None:
cls.__instance = super(Client, cls).__new__(cls)
# Initialize instance variables that should only be set once per instance
cls.__instance._init_trace_context = None
cls.__instance._legacy_session_for_init_trace = None
cls.__instance._auth_token = None
cls.__instance._project_id = None
cls.__instance._auth_lock = threading.Lock()
cls.__instance._auth_task = None
return cls.__instance
def __init__(self):
# Initialization of attributes like config, _initialized should happen here if they are instance-specific
# and not shared via __new__ for a true singleton that can be re-configured.
# However, the current pattern re-initializes config in init().
if (
not hasattr(self, "_initialized") or not self._initialized
): # Ensure init logic runs only once per actual initialization intent
self.config = Config() # Initialize config here for the instance
self._initialized = False
# self._init_trace_context = None # Already done in __new__
# self._legacy_session_for_init_trace = None # Already done in __new__
def get_current_jwt(self) -> Optional[str]:
"""Get the current JWT token."""
with self._auth_lock:
return self._auth_token
def _set_auth_data(self, token: str, project_id: str):
"""Set authentication data thread-safely."""
with self._auth_lock:
self._auth_token = token
self._project_id = project_id
# Update the HTTP client's project ID
from agentops.client.http.http_client import HttpClient
HttpClient.set_project_id(project_id)
async def _fetch_auth_async(self, api_key: str) -> Optional[dict]:
"""Asynchronously fetch authentication token."""
try:
response = await self.api.v3.fetch_auth_token(api_key)
if response:
self._set_auth_data(response["token"], response["project_id"])
# Update V4 client with token
self.api.v4.set_auth_token(response["token"])
# Update tracer config with real project ID
tracing_config = {"project_id": response["project_id"]}
tracer.update_config(tracing_config)
logger.debug("Successfully fetched authentication token asynchronously")
return response
else:
logger.debug("Authentication failed - will continue without authentication")
return None
except Exception:
return None
def _start_auth_task(self, api_key: str):
"""Start the async authentication task."""
if self._auth_task and not self._auth_task.done():
return # Task already running
try:
loop = asyncio.get_event_loop()
if loop.is_running():
# Use existing event loop
self._auth_task = loop.create_task(self._fetch_auth_async(api_key))
else:
# Create new event loop in background thread
def run_async_auth():
asyncio.run(self._fetch_auth_async(api_key))
import threading
auth_thread = threading.Thread(target=run_async_auth, daemon=True)
auth_thread.start()
except RuntimeError:
# Create new event loop in background thread
def run_async_auth():
asyncio.run(self._fetch_auth_async(api_key))
import threading
auth_thread = threading.Thread(target=run_async_auth, daemon=True)
auth_thread.start()
def init(self, **kwargs: Any) -> None: # Return type updated to None
# Recreate the Config object to parse environment variables at the time of initialization
# This allows re-init with new env vars if needed, though true singletons usually init once.
self.config = Config()
self.configure(**kwargs)
# Only treat as re-initialization if a different non-None API key is explicitly provided
provided_api_key = kwargs.get("api_key")
if self.initialized and provided_api_key is not None and provided_api_key != self.config.api_key:
logger.warning("AgentOps Client being re-initialized with a different API key. This is unusual.")
# Reset initialization status to allow re-init with new key/config
self._initialized = False
if self._init_trace_context and self._init_trace_context.span.is_recording():
logger.warning("Ending previously auto-started trace due to re-initialization.")
tracer.end_trace(self._init_trace_context, "Reinitialized")
self._init_trace_context = None
self._legacy_session_for_init_trace = None
if self.initialized:
logger.debug("AgentOps Client already initialized.")
# If auto_start_session was true, return the existing legacy session wrapper
if self.config.auto_start_session:
return self._legacy_session_for_init_trace
return None # If not auto-starting, and already initialized, return None
if not self.config.api_key:
logger.warning(
"No API key provided. AgentOps will initialize but authentication will fail. "
"Set AGENTOPS_API_KEY environment variable or pass api_key parameter."
)
# Continue without API key - spans will be created but exports will fail gracefully
configure_logging(self.config)
intercept_opentelemetry_logging()
self.api = ApiClient(self.config.endpoint)
# Initialize tracer with JWT provider for dynamic updates
tracing_config = self.config.dict()
tracing_config["project_id"] = "temporary" # Will be updated when auth completes
# Create JWT provider function for dynamic updates
def jwt_provider():
return self.get_current_jwt()
# Initialize tracer with JWT provider
tracer.initialize_from_config(tracing_config, jwt_provider=jwt_provider)
if self.config.instrument_llm_calls:
instrument_all()
# Start authentication task only if we have an API key
if self.config.api_key:
self._start_auth_task(self.config.api_key)
else:
logger.debug("No API key available - skipping authentication task")
global _atexit_registered
if not _atexit_registered:
atexit.register(_end_init_trace_atexit) # Register new atexit handler
_atexit_registered = True
# Auto-start trace if configured
if self.config.auto_start_session:
if self._init_trace_context is None or not self._init_trace_context.span.is_recording():
logger.debug("Auto-starting init trace.")
trace_name = self.config.trace_name or "default"
self._init_trace_context = tracer.start_trace(
trace_name=trace_name,
tags=list(self.config.default_tags) if self.config.default_tags else None,
is_init_trace=True,
)
if self._init_trace_context:
self._legacy_session_for_init_trace = Session(self._init_trace_context)
# For backward compatibility, also update the global references in legacy and client modules
# These globals are what old code might have been using via agentops.legacy.get_session() or similar indirect access.
global _client_init_trace_context, _client_legacy_session_for_init_trace
_client_init_trace_context = self._init_trace_context
_client_legacy_session_for_init_trace = self._legacy_session_for_init_trace
# Update legacy module's _current_session and _current_trace_context
# This is tricky; direct access to another module's globals is not ideal.
# Prefer explicit calls if possible, but for maximum BC:
try:
import agentops.legacy
agentops.legacy._current_session = self._legacy_session_for_init_trace
agentops.legacy._current_trace_context = self._init_trace_context
except ImportError:
pass # Should not happen
else:
logger.error("Failed to start the auto-init trace.")
# Even if auto-start fails, core services up to the tracer might be initialized.
# Set self.initialized to True if tracer is up, but return None.
self._initialized = tracer.initialized
return None # Failed to start trace
self._initialized = True # Successfully initialized and auto-trace started (if configured)
# For backward compatibility, return the legacy session wrapper when auto_start_session=True
return self._legacy_session_for_init_trace
else:
logger.debug("Auto-start session is disabled. No init trace started by client.")
self._initialized = True # Successfully initialized, just no auto-trace
return None # No auto-session, so return None
def configure(self, **kwargs: Any) -> None:
"""Update client configuration"""
self.config.configure(**kwargs)
@property
def initialized(self) -> bool:
return self._initialized
@initialized.setter
def initialized(self, value: bool) -> None:
if self._initialized and self._initialized != value:
# Allow re-setting to False if we are intentionally re-initializing
# This logic is now partly in init() to handle re-init cases
pass
self._initialized = value
# ------------------------------------------------------------
# Remove the old __instance = None at the end of the class definition if it's a repeat
# __instance = None # This was a class variable, should be defined once
# Make _init_trace_context and _legacy_session_for_init_trace accessible
# to the atexit handler if it becomes a static/class method or needs access
# For now, the atexit handler is global and uses global vars copied from these.
# Deprecate and remove the old global _active_session from this module.
# Consumers should use agentops.start_trace() or rely on the auto-init trace.
# For a transition, the auto-init trace's legacy wrapper is set to legacy module's globals.
+87
View File
@@ -0,0 +1,87 @@
# AgentOps HTTP Client Architecture
This directory contains the HTTP client architecture for the AgentOps SDK. The architecture follows a clean separation of concerns design principle.
## Components
### HttpClient
The `HttpClient` class provides low-level HTTP functionality:
- Connection pooling
- Retry logic
- Basic HTTP methods (GET, POST, PUT, DELETE)
### AuthManager
The `AuthManager` class handles authentication concerns:
- Token acquisition and storage
- Token refresh logic
- Authentication header preparation
- Thread-safe token operations
### HTTP Adapters
#### BaseHTTPAdapter
- Enhanced connection pooling and retry logic
- Used by the `HttpClient` for basic HTTP operations
#### AuthenticatedHttpAdapter
- Extends `BaseHTTPAdapter` with authentication capabilities
- Automatically adds authentication headers to requests
- Handles token refresh when authentication fails
- Can be mounted to any requests.Session
## Design Principles
1. **Separation of Concerns**
- HTTP concerns are isolated from authentication concerns
- Each component has a single responsibility
2. **Composition over Inheritance**
- Components use composition rather than inheritance
- `ApiClient` composes `HttpClient` and `AuthManager`
3. **Clear Interfaces**
- Each component has a well-defined interface
- Implementation details are hidden
4. **Dependency Flow**
- Dependencies flow in one direction
- Lower-level components (HTTP, Auth) don't depend on higher-level components
## Usage
### Basic API Client Usage
The HTTP client architecture is used by the `ApiClient` class, which provides a high-level interface for making API calls. Specific API versions (like `V3Client`) extend the `ApiClient` to provide version-specific functionality.
```python
# Example usage
from agentops.client.v3_client import V3Client
client = V3Client(endpoint="https://api.agentops.ai")
response = client.authenticated_request(
method="get",
path="/v3/some/endpoint",
api_key="your-api-key"
)
```
### Using with External Libraries
The architecture also supports integration with external libraries that need authenticated HTTP sessions:
```python
# Example with OpenTelemetry exporter
from agentops.client.v3_client import V3Client
from agentops.client.exporters import AuthenticatedOTLPExporter
client = V3Client(endpoint="https://api.agentops.ai")
session = client.create_authenticated_session(api_key="your-api-key")
exporter = AuthenticatedOTLPExporter(
endpoint="https://api.agentops.ai/v3/traces",
api_client=client,
api_key="your-api-key"
)
```
View File
+116
View File
@@ -0,0 +1,116 @@
from typing import Optional
from requests.adapters import HTTPAdapter
from urllib3.util import Retry
# from agentops.client.auth_manager import AuthManager
class BaseHTTPAdapter(HTTPAdapter):
"""Base HTTP adapter with enhanced connection pooling and retry logic"""
def __init__(
self,
pool_connections: int = 15,
pool_maxsize: int = 256,
max_retries: Optional[Retry] = None,
):
"""
Initialize the base HTTP adapter.
Args:
pool_connections: Number of connection pools to cache
pool_maxsize: Maximum number of connections to save in the pool
max_retries: Retry configuration for failed requests
"""
if max_retries is None:
max_retries = Retry(
total=3,
backoff_factor=0.1,
status_forcelist=[500, 502, 503, 504],
)
super().__init__(pool_connections=pool_connections, pool_maxsize=pool_maxsize, max_retries=max_retries)
# class AuthenticatedHttpAdapter(BaseHTTPAdapter):
# """HTTP adapter with automatic JWT authentication and refresh"""
#
# def __init__(
# self,
# auth_manager: AuthManager,
# api_key: str,
# token_fetcher: Callable[[str], Union[str, AuthTokenResponse]],
# pool_connections: int = 15,
# pool_maxsize: int = 256,
# max_retries: Optional[Retry] = None,
# ):
# """
# Initialize the authenticated HTTP adapter.
#
# Args:
# auth_manager: The authentication manager to use
# api_key: The API key to authenticate with
# token_fetcher: Function to fetch a new token if needed
# pool_connections: Number of connection pools to cache
# pool_maxsize: Maximum number of connections to save in the pool
# max_retries: Retry configuration for failed requests
# """
# self.auth_manager = auth_manager
# self.api_key = api_key
# self.token_fetcher = token_fetcher
#
# super().__init__(
# pool_connections=pool_connections,
# pool_maxsize=pool_maxsize,
# max_retries=max_retries
# )
#
# def add_headers(self, request, **kwargs):
# """Add authentication headers to the request"""
# # Get fresh auth headers from the auth manager
# self.auth_manager.maybe_fetch(self.api_key, self.token_fetcher)
# auth_headers = self.auth_manager.prepare_auth_headers(self.api_key)
#
# # Update request headers
# for key, value in auth_headers.items():
# request.headers[key] = value
#
# return request
#
# def send(self, request, **kwargs):
# """Send the request with authentication retry logic"""
# # Ensure allow_redirects is set to False
# kwargs["allow_redirects"] = False
#
# # Add auth headers to initial request
# request = self.add_headers(request, **kwargs)
#
# # Make the initial request
# response = super().send(request, **kwargs)
#
# # If we get a 401/403, check if it's due to token expiration
# if self.auth_manager.is_token_expired_response(response):
# logger.debug("Token expired, attempting to refresh")
# try:
# # Force token refresh
# self.auth_manager.clear_token()
# self.auth_manager.maybe_fetch(self.api_key, self.token_fetcher)
#
# # Update request with new token
# request = self.add_headers(request, **kwargs)
#
# # Retry the request
# logger.debug("Retrying request with new token")
# response = super().send(request, **kwargs)
# except AgentOpsApiJwtExpiredException as e:
# # Authentication failed
# logger.warning(f"Failed to refresh authentication token: {e}")
# except ApiServerException as e:
# # Server error during token refresh
# logger.error(f"Server error during token refresh: {e}")
# except Exception as e:
# # Unexpected error during token refresh
# logger.error(f"Unexpected error during token refresh: {e}")
#
# return response
+169
View File
@@ -0,0 +1,169 @@
from typing import Dict, Optional
import threading
import requests
from agentops.client.http.http_adapter import BaseHTTPAdapter
from agentops.logging import logger
from agentops.helpers.version import get_agentops_version
# Import aiohttp for async requests
try:
import aiohttp
AIOHTTP_AVAILABLE = True
except ImportError:
AIOHTTP_AVAILABLE = False
# Don't log warning here, only when actually trying to use async functionality
class HttpClient:
"""HTTP client with async-first design and optional sync fallback for log uploads"""
_session: Optional[requests.Session] = None
_async_session: Optional[aiohttp.ClientSession] = None
_project_id: Optional[str] = None
_session_lock = threading.Lock()
@classmethod
def get_project_id(cls) -> Optional[str]:
"""Get the stored project ID"""
return cls._project_id
@classmethod
def set_project_id(cls, project_id: str) -> None:
"""Set the project ID"""
cls._project_id = project_id
@classmethod
def get_session(cls) -> requests.Session:
"""
Get or create the global session with optimized connection pooling.
Note: This method is deprecated. Use async_request() instead.
Only kept for log upload module compatibility.
"""
if cls._session is None:
with cls._session_lock:
if cls._session is None: # Double-check locking
cls._session = requests.Session()
# Configure connection pooling
adapter = BaseHTTPAdapter()
# Mount adapter for both HTTP and HTTPS
cls._session.mount("http://", adapter)
cls._session.mount("https://", adapter)
# Set default headers
cls._session.headers.update(
{
"Connection": "keep-alive",
"Keep-Alive": "timeout=10, max=1000",
"Content-Type": "application/json",
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
}
)
logger.debug(f"Agentops version: agentops-python/{get_agentops_version() or 'unknown'}")
return cls._session
@classmethod
async def get_async_session(cls) -> Optional[aiohttp.ClientSession]:
"""Get or create the global async session with optimized connection pooling"""
if not AIOHTTP_AVAILABLE:
logger.warning("aiohttp not available, cannot create async session")
return None
# Always create a new session if the current one is None or closed
if cls._async_session is None or cls._async_session.closed:
# Close the old session if it exists but is closed
if cls._async_session is not None and cls._async_session.closed:
cls._async_session = None
# Create connector with connection pooling
connector = aiohttp.TCPConnector(
limit=100, # Total connection pool size
limit_per_host=30, # Per-host connection limit
ttl_dns_cache=300, # DNS cache TTL
use_dns_cache=True,
enable_cleanup_closed=True,
)
# Create session with default headers
headers = {
"Content-Type": "application/json",
"User-Agent": f"agentops-python/{get_agentops_version() or 'unknown'}",
}
cls._async_session = aiohttp.ClientSession(
connector=connector, headers=headers, timeout=aiohttp.ClientTimeout(total=30)
)
return cls._async_session
@classmethod
async def close_async_session(cls):
"""Close the async session"""
if cls._async_session and not cls._async_session.closed:
await cls._async_session.close()
cls._async_session = None
@classmethod
async def async_request(
cls,
method: str,
url: str,
data: Optional[Dict] = None,
headers: Optional[Dict] = None,
timeout: int = 30,
) -> Optional[Dict]:
"""
Make an async HTTP request and return JSON response
Args:
method: HTTP method (e.g., 'get', 'post', 'put', 'delete')
url: Full URL for the request
data: Request payload (for POST, PUT methods)
headers: Request headers
timeout: Request timeout in seconds
Returns:
JSON response as dictionary, or None if request failed
"""
if not AIOHTTP_AVAILABLE:
logger.warning("aiohttp not available, cannot make async request")
return None
try:
session = await cls.get_async_session()
if not session:
return None
logger.debug(f"Making async {method} request to {url}")
# Prepare request parameters
kwargs = {"timeout": aiohttp.ClientTimeout(total=timeout), "headers": headers or {}}
if data and method.lower() in ["post", "put", "patch"]:
kwargs["json"] = data
# Make the request
async with session.request(method.upper(), url, **kwargs) as response:
logger.debug(f"Async request response status: {response.status}")
# Check if response is successful
if response.status >= 400:
return None
# Parse JSON response
try:
response_data = await response.json()
logger.debug(
f"Async request successful, response keys: {list(response_data.keys()) if response_data else 'None'}"
)
return response_data
except Exception:
return None
except Exception:
return None
+274
View File
@@ -0,0 +1,274 @@
import json
import logging
import os
import sys
from dataclasses import dataclass, field
from typing import List, Optional, Set, TypedDict, Union
from uuid import UUID
from opentelemetry.sdk.trace import SpanProcessor
from opentelemetry.sdk.trace.export import SpanExporter
from agentops.helpers.env import get_env_bool, get_env_int, get_env_list
from agentops.helpers.serialization import AgentOpsJSONEncoder
class ConfigDict(TypedDict):
api_key: Optional[str]
endpoint: Optional[str]
app_url: Optional[str]
max_wait_time: Optional[int]
export_flush_interval: Optional[int]
max_queue_size: Optional[int]
default_tags: Optional[List[str]]
trace_name: Optional[str]
instrument_llm_calls: Optional[bool]
auto_start_session: Optional[bool]
auto_init: Optional[bool]
skip_auto_end_session: Optional[bool]
env_data_opt_out: Optional[bool]
log_level: Optional[Union[str, int]]
fail_safe: Optional[bool]
prefetch_jwt_token: Optional[bool]
log_session_replay_url: Optional[bool]
@dataclass
class Config:
api_key: Optional[str] = field(
default_factory=lambda: os.getenv("AGENTOPS_API_KEY"),
metadata={"description": "API key for authentication with AgentOps services"},
)
endpoint: str = field(
default_factory=lambda: os.getenv("AGENTOPS_API_ENDPOINT", "https://api.agentops.ai"),
metadata={"description": "Base URL for the AgentOps API"},
)
app_url: str = field(
default_factory=lambda: os.getenv("AGENTOPS_APP_URL", "https://app.agentops.ai"),
metadata={"description": "Dashboard URL for the AgentOps application"},
)
max_wait_time: int = field(
default_factory=lambda: get_env_int("AGENTOPS_MAX_WAIT_TIME", 5000),
metadata={"description": "Maximum time in milliseconds to wait for API responses"},
)
export_flush_interval: int = field(
default_factory=lambda: get_env_int("AGENTOPS_EXPORT_FLUSH_INTERVAL", 1000),
metadata={"description": "Time interval in milliseconds between automatic exports of telemetry data"},
)
max_queue_size: int = field(
default_factory=lambda: get_env_int("AGENTOPS_MAX_QUEUE_SIZE", 512),
metadata={"description": "Maximum number of events to queue before forcing a flush"},
)
default_tags: Set[str] = field(
default_factory=lambda: get_env_list("AGENTOPS_DEFAULT_TAGS"),
metadata={"description": "Default tags to apply to all sessions"},
)
trace_name: Optional[str] = field(
default_factory=lambda: os.getenv("AGENTOPS_TRACE_NAME"),
metadata={"description": "Default name for the trace/session"},
)
instrument_llm_calls: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_INSTRUMENT_LLM_CALLS", True),
metadata={"description": "Whether to automatically instrument and track LLM API calls"},
)
auto_start_session: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_AUTO_START_SESSION", True),
metadata={"description": "Whether to automatically start a session when initializing"},
)
auto_init: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_AUTO_INIT", True),
metadata={"description": "Whether to automatically initialize the client on import"},
)
skip_auto_end_session: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_SKIP_AUTO_END_SESSION", False),
metadata={"description": "Whether to skip automatically ending sessions on program exit"},
)
env_data_opt_out: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_ENV_DATA_OPT_OUT", False),
metadata={"description": "Whether to opt out of collecting environment data"},
)
log_level: Union[str, int] = field(
default_factory=lambda: os.getenv("AGENTOPS_LOG_LEVEL", "INFO"),
metadata={"description": "Logging level for AgentOps logs"},
)
fail_safe: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_FAIL_SAFE", False),
metadata={"description": "Whether to suppress errors and continue execution when possible"},
)
prefetch_jwt_token: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_PREFETCH_JWT_TOKEN", True),
metadata={"description": "Whether to prefetch JWT token during initialization"},
)
log_session_replay_url: bool = field(
default_factory=lambda: get_env_bool("AGENTOPS_LOG_SESSION_REPLAY_URL", True),
metadata={"description": "Whether to log session replay URLs to the console"},
)
exporter_endpoint: Optional[str] = field(
default_factory=lambda: os.getenv("AGENTOPS_EXPORTER_ENDPOINT", "https://otlp.agentops.ai/v1/traces"),
metadata={
"description": "Endpoint for the span exporter. When not provided, the default AgentOps endpoint will be used."
},
)
exporter: Optional[SpanExporter] = field(
default_factory=lambda: None, metadata={"description": "Custom span exporter for OpenTelemetry trace data"}
)
processor: Optional[SpanProcessor] = field(
default_factory=lambda: None, metadata={"description": "Custom span processor for OpenTelemetry trace data"}
)
def configure(
self,
api_key: Optional[str] = None,
endpoint: Optional[str] = None,
app_url: Optional[str] = None,
max_wait_time: Optional[int] = None,
export_flush_interval: Optional[int] = None,
max_queue_size: Optional[int] = None,
default_tags: Optional[List[str]] = None,
trace_name: Optional[str] = None,
instrument_llm_calls: Optional[bool] = None,
auto_start_session: Optional[bool] = None,
auto_init: Optional[bool] = None,
skip_auto_end_session: Optional[bool] = None,
env_data_opt_out: Optional[bool] = None,
log_level: Optional[Union[str, int]] = None,
fail_safe: Optional[bool] = None,
prefetch_jwt_token: Optional[bool] = None,
log_session_replay_url: Optional[bool] = None,
exporter: Optional[SpanExporter] = None,
processor: Optional[SpanProcessor] = None,
exporter_endpoint: Optional[str] = None,
):
"""Configure settings from kwargs, validating where necessary"""
if api_key is not None:
self.api_key = api_key
if not TESTING: # Allow setting dummy keys in tests
try:
UUID(api_key)
except ValueError:
# Log warning but don't throw exception - let async auth handle it
from agentops.logging import logger
logger.warning(
f"API key format appears invalid: {api_key[:8]}... "
f"Authentication may fail. Find your API key at {self.endpoint}/settings/projects"
)
# Continue with the invalid key - async auth will handle the failure gracefully
if endpoint is not None:
self.endpoint = endpoint
if app_url is not None:
self.app_url = app_url
if max_wait_time is not None:
self.max_wait_time = max_wait_time
if export_flush_interval is not None:
self.export_flush_interval = export_flush_interval
if max_queue_size is not None:
self.max_queue_size = max_queue_size
if default_tags is not None:
self.default_tags = set(default_tags)
if trace_name is not None:
self.trace_name = trace_name
if instrument_llm_calls is not None:
self.instrument_llm_calls = instrument_llm_calls
if auto_start_session is not None:
self.auto_start_session = auto_start_session
if auto_init is not None:
self.auto_init = auto_init
if skip_auto_end_session is not None:
self.skip_auto_end_session = skip_auto_end_session
if env_data_opt_out is not None:
self.env_data_opt_out = env_data_opt_out
if log_level is not None:
if isinstance(log_level, str):
log_level_str = log_level.upper()
if hasattr(logging, log_level_str):
self.log_level = getattr(logging, log_level_str)
else:
self.log_level = logging.INFO
else:
self.log_level = log_level
if fail_safe is not None:
self.fail_safe = fail_safe
if prefetch_jwt_token is not None:
self.prefetch_jwt_token = prefetch_jwt_token
if log_session_replay_url is not None:
self.log_session_replay_url = log_session_replay_url
if exporter is not None:
self.exporter = exporter
if processor is not None:
self.processor = processor
if exporter_endpoint is not None:
self.exporter_endpoint = exporter_endpoint
# else:
# self.exporter_endpoint = self.endpoint
def dict(self):
"""Return a dictionary representation of the config"""
return {
"api_key": self.api_key,
"endpoint": self.endpoint,
"app_url": self.app_url,
"max_wait_time": self.max_wait_time,
"export_flush_interval": self.export_flush_interval,
"max_queue_size": self.max_queue_size,
"default_tags": self.default_tags,
"trace_name": self.trace_name,
"instrument_llm_calls": self.instrument_llm_calls,
"auto_start_session": self.auto_start_session,
"auto_init": self.auto_init,
"skip_auto_end_session": self.skip_auto_end_session,
"env_data_opt_out": self.env_data_opt_out,
"log_level": self.log_level,
"fail_safe": self.fail_safe,
"prefetch_jwt_token": self.prefetch_jwt_token,
"log_session_replay_url": self.log_session_replay_url,
"exporter": self.exporter,
"processor": self.processor,
"exporter_endpoint": self.exporter_endpoint,
}
def json(self):
"""Return a JSON representation of the config"""
return json.dumps(self.dict(), cls=AgentOpsJSONEncoder)
# checks if pytest is imported
TESTING = "pytest" in sys.modules
+36
View File
@@ -0,0 +1,36 @@
"""
AgentOps enums for user-friendly API.
This module provides simple enums that users can import from agentops
without needing to know about OpenTelemetry internals.
"""
from enum import Enum
from opentelemetry.trace.status import StatusCode
class TraceState(Enum):
"""
Enum for trace end states.
This provides a user-friendly interface that maps to OpenTelemetry StatusCode internally.
Users can simply use agentops.TraceState.SUCCESS instead of importing OpenTelemetry.
"""
SUCCESS = StatusCode.OK
ERROR = StatusCode.ERROR
UNSET = StatusCode.UNSET
def __str__(self) -> str:
"""Return the name for string representation."""
return self.name
def to_status_code(self) -> StatusCode:
"""Convert to OpenTelemetry StatusCode."""
return self.value
# For backward compatibility, also provide these as module-level constants
SUCCESS = TraceState.SUCCESS
ERROR = TraceState.ERROR
UNSET = TraceState.UNSET
+38
View File
@@ -0,0 +1,38 @@
class MultiSessionException(Exception):
def __init__(self, message):
super().__init__(message)
class NoSessionException(Exception):
def __init__(self, message="No session found"):
super().__init__(message)
class NoApiKeyException(Exception):
def __init__(
self,
message="Could not initialize AgentOps client - API Key is missing."
+ "\n\t Find your API key at https://app.agentops.ai/settings/projects",
):
super().__init__(message)
class InvalidApiKeyException(Exception):
def __init__(self, api_key, endpoint):
message = f"API Key is invalid: {{{api_key}}}.\n\t Find your API key at {endpoint}/settings/projects"
super().__init__(message)
class ApiServerException(Exception):
def __init__(self, message):
super().__init__(message)
class AgentOpsClientNotInitializedException(RuntimeError):
def __init__(self, message="AgentOps client must be initialized before using this feature"):
super().__init__(message)
class AgentOpsApiJwtExpiredException(Exception):
def __init__(self, message="JWT token has expired"):
super().__init__(message)
+44
View File
@@ -0,0 +1,44 @@
from agentops.helpers.time import get_ISO_time
from agentops.helpers.serialization import (
AgentOpsJSONEncoder,
serialize_uuid,
safe_serialize,
is_jsonable,
filter_unjsonable,
)
from agentops.helpers.system import (
get_host_env,
get_sdk_details,
get_os_details,
get_cpu_details,
get_ram_details,
get_disk_details,
get_installed_packages,
get_current_directory,
get_virtual_env,
)
from agentops.helpers.version import get_agentops_version, check_agentops_update
from agentops.helpers.env import get_env_bool, get_env_int, get_env_list
__all__ = [
"get_ISO_time",
"AgentOpsJSONEncoder",
"serialize_uuid",
"safe_serialize",
"is_jsonable",
"filter_unjsonable",
"get_host_env",
"get_sdk_details",
"get_os_details",
"get_cpu_details",
"get_ram_details",
"get_disk_details",
"get_installed_packages",
"get_current_directory",
"get_virtual_env",
"get_agentops_version",
"check_agentops_update",
"get_env_bool",
"get_env_int",
"get_env_list",
]
+54
View File
@@ -0,0 +1,54 @@
"""
Helpers for interacting with the AgentOps dashboard.
"""
from typing import Union, Optional
from termcolor import colored
from opentelemetry.sdk.trace import Span, ReadableSpan
from agentops.logging import logger
def get_trace_url(span: Union[Span, ReadableSpan]) -> str:
"""
Generate a trace URL for a direct link to the session on the AgentOps dashboard.
Args:
span: The span to generate the URL for.
Returns:
The session URL.
"""
trace_id: Union[int, str] = span.context.trace_id
# Convert trace_id to hex string if it's not already
# We don't add dashes to this to format it as a UUID since the dashboard doesn't either
if isinstance(trace_id, int):
trace_id = format(trace_id, "032x")
# Get the app_url from the config - import here to avoid circular imports
from agentops import get_client
app_url = get_client().config.app_url
return f"{app_url}/sessions?trace_id={trace_id}"
def log_trace_url(span: Union[Span, ReadableSpan], title: Optional[str] = None) -> None:
"""
Log the trace URL for the AgentOps dashboard.
Args:
span: The span to log the URL for.
title: Optional title for the trace.
"""
from agentops import get_client
try:
client = get_client()
if not client.config.log_session_replay_url:
return
except Exception:
return
session_url = get_trace_url(span)
logger.info(colored(f"\x1b[34mSession Replay for {title} trace: {session_url}\x1b[0m", "blue"))
+50
View File
@@ -0,0 +1,50 @@
"""
Deprecation utilities for AgentOps SDK.
"""
import functools
from typing import Set, Callable, Any
from agentops.logging import logger
# Track which deprecation warnings have been shown to avoid spam
_shown_warnings: Set[str] = set()
def deprecated(message: str):
"""
Decorator to mark functions as deprecated.
Args:
message: Deprecation message to show
"""
def decorator(func: Callable) -> Callable:
@functools.wraps(func)
def wrapper(*args: Any, **kwargs: Any) -> Any:
warning_key = f"{func.__module__}.{func.__name__}"
if warning_key not in _shown_warnings:
logger.warning(f"{func.__name__}() is deprecated and will be removed in v4 in the future. {message}")
_shown_warnings.add(warning_key)
return func(*args, **kwargs)
return wrapper
return decorator
def warn_deprecated_param(param_name: str, replacement: str = None):
"""
Warn about deprecated parameter usage.
Args:
param_name: Name of the deprecated parameter
replacement: Suggested replacement parameter
"""
warning_key = f"param.{param_name}"
if warning_key not in _shown_warnings:
if replacement:
message = f"Parameter '{param_name}' is deprecated and will be removed in v4 in the future. Use '{replacement}' instead."
else:
message = f"Parameter '{param_name}' is deprecated and will be removed in v4 in the future."
logger.warning(message)
_shown_warnings.add(warning_key)
+52
View File
@@ -0,0 +1,52 @@
"""Environment variable helper functions"""
import os
from typing import List, Optional, Set
def get_env_bool(key: str, default: bool) -> bool:
"""Get boolean from environment variable
Args:
key: Environment variable name
default: Default value if not set
Returns:
bool: Parsed boolean value
"""
val = os.getenv(key)
if val is None:
return default
return val.lower() in ("true", "1", "t", "yes")
def get_env_int(key: str, default: int) -> int:
"""Get integer from environment variable
Args:
key: Environment variable name
default: Default value if not set
Returns:
int: Parsed integer value
"""
try:
return int(os.getenv(key, default))
except (TypeError, ValueError):
return default
def get_env_list(key: str, default: Optional[List[str]] = None) -> Set[str]:
"""Get comma-separated list from environment variable
Args:
key: Environment variable name
default: Default list if not set
Returns:
Set[str]: Set of parsed values
"""
val = os.getenv(key)
if val is None:
return set(default or [])
return set(val.split(","))
+137
View File
@@ -0,0 +1,137 @@
"""Serialization helpers for AgentOps"""
import json
from datetime import datetime
from decimal import Decimal
from enum import Enum
from typing import Any
from uuid import UUID
from agentops.logging import logger
def is_jsonable(x):
try:
json.dumps(x)
return True
except (TypeError, OverflowError):
return False
def filter_unjsonable(d: dict) -> dict:
def filter_dict(obj):
if isinstance(obj, dict):
return {
k: (
filter_dict(v)
if isinstance(v, (dict, list)) or is_jsonable(v)
else str(v)
if isinstance(v, UUID)
else ""
)
for k, v in obj.items()
}
elif isinstance(obj, list):
return [
(
filter_dict(x)
if isinstance(x, (dict, list)) or is_jsonable(x)
else str(x)
if isinstance(x, UUID)
else ""
)
for x in obj
]
else:
return obj if is_jsonable(obj) or isinstance(obj, UUID) else ""
return filter_dict(d)
class AgentOpsJSONEncoder(json.JSONEncoder):
"""Custom JSON encoder for AgentOps types"""
def default(self, obj: Any) -> Any:
if isinstance(obj, UUID):
return str(obj)
if isinstance(obj, datetime):
return obj.isoformat()
if isinstance(obj, Decimal):
return str(obj)
if isinstance(obj, set):
return list(obj)
if hasattr(obj, "to_json"):
return obj.to_json()
if isinstance(obj, Enum):
return obj.value
return str(obj)
def serialize_uuid(obj: UUID) -> str:
"""Serialize UUID to string"""
return str(obj)
def model_to_dict(obj: Any) -> dict:
"""Convert a model object to a dictionary safely.
Handles various model types including:
- Pydantic models (model_dump/dict methods)
- Dictionary-like objects
- API response objects with parse method
- Objects with __dict__ attribute
Args:
obj: The model object to convert to dictionary
Returns:
Dictionary representation of the object, or empty dict if conversion fails
"""
if obj is None:
return {}
if isinstance(obj, dict):
return obj
if hasattr(obj, "model_dump"): # Pydantic v2
return obj.model_dump()
elif hasattr(obj, "dict"): # Pydantic v1
return obj.dict()
# TODO this is causing recursion on nested objects.
# elif hasattr(obj, "parse"): # Raw API response
# return model_to_dict(obj.parse())
else:
# Try to use __dict__ as fallback
try:
return obj.__dict__
except:
return {}
def safe_serialize(obj: Any) -> Any:
"""Safely serialize an object to JSON-compatible format
This function handles complex objects by:
1. Returning strings untouched (even if they contain JSON)
2. Converting models to dictionaries
3. Using custom JSON encoder to handle special types
4. Falling back to string representation only when necessary
Args:
obj: The object to serialize
Returns:
If obj is a string, returns the original string untouched.
Otherwise, returns a JSON string representation of the object.
"""
# Return strings untouched
if isinstance(obj, str):
return obj
# Convert any model objects to dictionaries
if hasattr(obj, "model_dump") or hasattr(obj, "dict") or hasattr(obj, "parse"):
obj = model_to_dict(obj)
try:
return json.dumps(obj, cls=AgentOpsJSONEncoder)
except (TypeError, ValueError) as e:
logger.warning(f"Failed to serialize object: {e}")
return str(obj)
+178
View File
@@ -0,0 +1,178 @@
import importlib.metadata
import os
import platform
import socket
import sys
import psutil # type: ignore
from agentops.logging import logger
from agentops.helpers.version import get_agentops_version
def get_imported_libraries():
"""
Get the top-level imported libraries in the current script.
Returns:
list: List of imported libraries
"""
user_libs = []
builtin_modules = {
"builtins",
"sys",
"os",
"_thread",
"abc",
"io",
"re",
"types",
"collections",
"enum",
"math",
"datetime",
"time",
"warnings",
}
try:
main_module = sys.modules.get("__main__")
if main_module and hasattr(main_module, "__dict__"):
for name, obj in main_module.__dict__.items():
if isinstance(obj, type(sys)) and hasattr(obj, "__name__"):
mod_name = obj.__name__.split(".")[0]
if mod_name and not mod_name.startswith("_") and mod_name not in builtin_modules:
user_libs.append(mod_name)
except Exception as e:
logger.debug(f"Error getting imports: {e}")
return user_libs
def get_sdk_details():
try:
return {
"AgentOps SDK Version": get_agentops_version(),
"Python Version": platform.python_version(),
"System Packages": get_sys_packages(),
}
except:
return {}
def get_sys_packages():
sys_packages = {}
for module in sys.modules:
try:
version = importlib.metadata.version(module)
sys_packages[module] = version
except importlib.metadata.PackageNotFoundError:
# Skip built-in modules and those without package metadata
continue
return sys_packages
def get_installed_packages():
try:
return {
# TODO: add to opt out
"Installed Packages": {
dist.metadata.get("Name"): dist.metadata.get("Version") for dist in importlib.metadata.distributions()
}
}
except:
return {}
def get_current_directory():
try:
return {"Project Working Directory": os.getcwd()}
except:
return {}
def get_virtual_env():
try:
return {"Virtual Environment": os.environ.get("VIRTUAL_ENV", None)}
except:
return {}
def get_os_details():
try:
return {
"Hostname": socket.gethostname(),
"OS": platform.system(),
"OS Version": platform.version(),
"OS Release": platform.release(),
}
except:
return {}
def get_cpu_details():
try:
return {
"Physical cores": psutil.cpu_count(logical=False),
"Total cores": psutil.cpu_count(logical=True),
# "Max Frequency": f"{psutil.cpu_freq().max:.2f}Mhz", # Fails right now
"CPU Usage": f"{psutil.cpu_percent()}%",
}
except:
return {}
def get_ram_details():
try:
ram_info = psutil.virtual_memory()
return {
"Total": f"{ram_info.total / (1024**3):.2f} GB",
"Available": f"{ram_info.available / (1024**3):.2f} GB",
"Used": f"{ram_info.used / (1024**3):.2f} GB",
"Percentage": f"{ram_info.percent}%",
}
except:
return {}
def get_disk_details():
partitions = psutil.disk_partitions()
disk_info = {}
for partition in partitions:
try:
usage = psutil.disk_usage(partition.mountpoint)
disk_info[partition.device] = {
"Mountpoint": partition.mountpoint,
"Total": f"{usage.total / (1024**3):.2f} GB",
"Used": f"{usage.used / (1024**3):.2f} GB",
"Free": f"{usage.free / (1024**3):.2f} GB",
"Percentage": f"{usage.percent}%",
}
except OSError as inaccessible:
# Skip inaccessible partitions, such as removable drives with no media
logger.debug("Mountpoint %s inaccessible: %s", partition.mountpoint, inaccessible)
return disk_info
def get_host_env(opt_out: bool = False):
if opt_out:
return {
"SDK": get_sdk_details(),
"OS": get_os_details(),
"Project Working Directory": get_current_directory(),
"Virtual Environment": get_virtual_env(),
}
else:
return {
"SDK": get_sdk_details(),
"OS": get_os_details(),
"CPU": get_cpu_details(),
"RAM": get_ram_details(),
"Disk": get_disk_details(),
"Installed Packages": get_installed_packages(),
"Project Working Directory": get_current_directory(),
"Virtual Environment": get_virtual_env(),
}
+11
View File
@@ -0,0 +1,11 @@
from datetime import datetime, timezone
def get_ISO_time():
"""
Get the current UTC time in ISO 8601 format with milliseconds precision in UTC timezone.
Returns:
str: The current UTC time as a string in ISO 8601 format.
"""
return datetime.now(timezone.utc).isoformat()
+36
View File
@@ -0,0 +1,36 @@
from importlib.metadata import PackageNotFoundError, version
import requests
from agentops.logging import logger
def get_agentops_version():
try:
pkg_version = version("agentops")
return pkg_version
except Exception as e:
logger.warning("Error reading package version: %s", e)
return None
def check_agentops_update():
try:
response = requests.get("https://pypi.org/pypi/agentops/json")
if response.status_code == 200:
json_data = response.json()
latest_version = json_data["info"]["version"]
try:
current_version = version("agentops")
except PackageNotFoundError:
return None
if not latest_version == current_version:
logger.warning(
" WARNING: agentops is out of date. Please update with the command: 'pip install --upgrade agentops'"
)
except Exception as e:
logger.debug(f"Failed to check for updates: {e}")
return None
+133
View File
@@ -0,0 +1,133 @@
# OpenTelemetry Implementation Notes
This document outlines best practices and implementation details for OpenTelemetry in AgentOps instrumentations.
## Key Concepts
### Context Propagation
OpenTelemetry relies on proper context propagation to maintain parent-child relationships between spans. This is essential for:
- Creating accurate trace waterfalls in visualizations
- Ensuring all spans from the same logical operation share a trace ID
- Allowing proper querying and filtering of related operations
### Core Patterns
When implementing instrumentations that need to maintain context across different execution contexts:
1. **Store span contexts in dictionaries:**
```python
# Use weakref dictionaries to avoid memory leaks
self._span_contexts = weakref.WeakKeyDictionary()
self._trace_root_contexts = weakref.WeakKeyDictionary()
```
2. **Create spans with explicit parent contexts:**
```python
parent_context = self._get_parent_context(trace_obj)
with trace.start_as_current_span(
name=span_name,
context=parent_context,
kind=trace.SpanKind.CLIENT,
attributes=attributes,
) as span:
# Span operations here
# Store the span's context for future reference
context = trace.set_span_in_context(span)
self._span_contexts[span_obj] = context
```
3. **Implement helper methods to retrieve appropriate parent contexts:**
```python
def _get_parent_context(self, trace_obj):
# Try to get the trace's root context if it exists
if trace_obj in self._trace_root_contexts:
return self._trace_root_contexts[trace_obj]
# Otherwise, use the current context
return context_api.context.get_current()
```
4. **Debug trace continuity:**
```python
current_span = trace.get_current_span()
span_context = current_span.get_span_context()
trace_id = format_trace_id(span_context.trace_id)
logging.debug(f"Current span trace ID: {trace_id}")
```
## Common Pitfalls
1. **Naming conflicts:** Avoid using `trace` as a parameter name when you're also importing the OpenTelemetry `trace` module
```python
# Bad
def on_trace_start(self, trace):
# This will cause conflicts with the imported trace module
# Good
def on_trace_start(self, trace_obj):
# No conflicts with OpenTelemetry's trace module
```
2. **Missing parent contexts:** Always explicitly provide parent contexts when available, don't rely on current context alone
3. **Memory leaks:** Use `weakref.WeakKeyDictionary()` for storing spans to allow garbage collection
4. **Lost context:** When calling async or callback functions, be sure to preserve and pass the context
## Testing Context Propagation
To verify proper context propagation:
1. Enable debug logging for trace IDs
2. Run a simple end-to-end test that generates multiple spans
3. Verify all spans share the same trace ID
4. Check that parent-child relationships are correctly established
```python
# Example debug logging
logging.debug(f"Span {span.name} has trace ID: {format_trace_id(span.get_span_context().trace_id)}")
```
## Timestamp Handling in OpenTelemetry
When working with OpenTelemetry spans and timestamps:
1. **Automatic Timestamp Tracking:** OpenTelemetry automatically tracks timestamps for spans. When a span is created with `tracer.start_span()` or `tracer.start_as_current_span()`, the start time is captured automatically. When `span.end()` is called, the end time is recorded.
2. **No Manual Timestamp Setting Required:** The standard instrumentation pattern does not require manually setting timestamp attributes on spans. Instead, OpenTelemetry handles this internally through the SpanProcessor and Exporter classes.
3. **Timestamp Representation:** In the OpenTelemetry data model, timestamps are stored as nanoseconds since the Unix epoch (January 1, 1970).
4. **Serialization Responsibility:** The serialization of timestamps from OTel spans to output formats like JSON is handled by the Exporter components. If timestamps aren't appearing correctly in output APIs, the issue is likely in the API exporter, not in the span creation code.
5. **Debugging Timestamps:** To debug timestamp issues, verify that spans are properly starting and ending, rather than manually setting timestamp attributes:
```python
# Good pattern - timestamps handled by OpenTelemetry automatically
with tracer.start_as_current_span("my_operation") as span:
# Do work
pass # span.end() is called automatically
```
Note: If timestamps are missing in API output (e.g., empty "start_time" fields), focus on fixes in the exporter and serialization layer, not by manually tracking timestamps in instrumentation code.
## Attributes in OpenTelemetry
When working with span attributes in OpenTelemetry:
1. **Root Attributes Node:** The root `attributes` object in the API output JSON should always be empty. This is by design. All attribute data should be stored in the `span_attributes` object.
2. **Span Attributes:** The `span_attributes` object is where all user-defined and semantic attribute data should be stored. This allows for a structured, hierarchical representation of attributes.
3. **Structure Difference:** While the root `attributes` appears as an empty object in the API output, this is normal and expected. Do not attempt to populate this object directly or duplicate data from `span_attributes` into it.
4. **Setting Attributes:** Always set span attributes using the semantic conventions defined in the `agentops/semconv` module:
```python
from agentops.semconv import agent
# Good pattern - using semantic conventions
span.set_attribute(agent.AGENT_NAME, "My Agent")
```
+167
View File
@@ -0,0 +1,167 @@
# AgentOps Instrumentation
This package provides OpenTelemetry instrumentation for various LLM providers and related services.
## Available Instrumentors
- **OpenAI** (`v0.27.0+` and `v1.0.0+`)
- **Anthropic** (`v0.7.0+`)
- **Google GenAI** (`v0.1.0+`)
- **IBM WatsonX AI** (`v0.1.0+`)
- **CrewAI** (`v0.56.0+`)
- **AG2/AutoGen** (`v0.3.2+`)
- **Google ADK** (`v0.1.0+`)
- **Agno** (`v0.0.1+`)
- **Mem0** (`v0.1.0+`)
- **smolagents** (`v0.1.0+`)
## Common Module Usage
The `agentops.instrumentation.common` module provides shared utilities for creating instrumentations:
### Base Instrumentor
Use `CommonInstrumentor` for creating new instrumentations:
```python
from agentops.instrumentation.common import CommonInstrumentor, InstrumentorConfig, WrapConfig
class MyInstrumentor(CommonInstrumentor):
def __init__(self):
config = InstrumentorConfig(
library_name="my-library",
library_version="1.0.0",
wrapped_methods=[
WrapConfig(
trace_name="my.method",
package="my_library.module",
class_name="MyClass",
method_name="my_method",
handler=my_attribute_handler
)
],
dependencies=["my-library >= 1.0.0"]
)
super().__init__(config)
```
### Attribute Handlers
Create attribute handlers to extract data from method calls:
```python
from agentops.instrumentation.common import AttributeMap
def my_attribute_handler(args=None, kwargs=None, return_value=None) -> AttributeMap:
attributes = {}
if kwargs and "model" in kwargs:
attributes["llm.request.model"] = kwargs["model"]
if return_value and hasattr(return_value, "usage"):
attributes["llm.usage.total_tokens"] = return_value.usage.total_tokens
return attributes
```
### Span Management
Use the span management utilities for consistent span creation:
```python
from agentops.instrumentation.common import create_span, SpanAttributeManager
# Create an attribute manager
attr_manager = SpanAttributeManager(service_name="my-service")
# Use the create_span context manager
with create_span(
tracer,
"my.operation",
attributes={"my.attribute": "value"},
attribute_manager=attr_manager
) as span:
# Your operation code here
pass
```
### Token Counting
Use the token counting utilities for consistent token usage extraction:
```python
from agentops.instrumentation.common import TokenUsageExtractor, set_token_usage_attributes
# Extract token usage from a response
usage = TokenUsageExtractor.extract_from_response(response)
# Set token usage attributes on a span
set_token_usage_attributes(span, response)
```
### Streaming Support
Use streaming utilities for handling streaming responses:
```python
from agentops.instrumentation.common import create_stream_wrapper_factory, StreamingResponseHandler
# Create a stream wrapper factory
wrapper = create_stream_wrapper_factory(
tracer,
"my.stream",
extract_chunk_content=StreamingResponseHandler.extract_generic_chunk_content,
initial_attributes={"stream.type": "text"}
)
# Apply to streaming methods
wrap_function_wrapper("my_module", "stream_method", wrapper)
```
### Metrics
Use standard metrics for consistency across instrumentations:
```python
from agentops.instrumentation.common import StandardMetrics, MetricsRecorder
# Create standard metrics
metrics = StandardMetrics.create_standard_metrics(meter)
# Use the metrics recorder
recorder = MetricsRecorder(metrics)
recorder.record_token_usage(prompt_tokens=100, completion_tokens=50)
recorder.record_duration(1.5)
```
## Creating a New Instrumentor
1. Create a new directory under `agentops/instrumentation/` for your provider
2. Create an `__init__.py` file with version information
3. Create an `instrumentor.py` file extending `CommonInstrumentor`
4. Create attribute handlers in an `attributes/` subdirectory
5. Add your instrumentor to the main `__init__.py` configuration
Example structure:
```
agentops/instrumentation/
├── my_provider/
│ ├── __init__.py
│ ├── instrumentor.py
│ └── attributes/
│ ├── __init__.py
│ └── handlers.py
```
## Best Practices
1. **Use Common Utilities**: Leverage the common module for consistency
2. **Follow Semantic Conventions**: Use attributes from `agentops.semconv`
3. **Handle Errors Gracefully**: Wrap operations in try-except blocks
4. **Support Async**: Provide both sync and async method wrapping
5. **Document Attributes**: Comment on what attributes are captured
6. **Test Thoroughly**: Write unit tests for your instrumentor
## Examples
See the `examples/` directory for usage examples of each instrumentor.
+610
View File
@@ -0,0 +1,610 @@
"""
AgentOps Instrumentation Module
This module provides automatic instrumentation for various LLM providers and agentic libraries.
It works by monitoring Python imports and automatically instrumenting packages as they are imported.
Key Features:
- Automatic detection and instrumentation of LLM providers (OpenAI, Anthropic, etc.)
- Support for agentic libraries (CrewAI, AutoGen, etc.)
- Version-aware instrumentation (only activates for supported versions)
- Smart handling of provider vs agentic library conflicts
- Non-intrusive monitoring using Python's import system
"""
from typing import Optional, Set, TypedDict
try:
from typing import NotRequired
except ImportError:
from typing_extensions import NotRequired
from types import ModuleType
from dataclasses import dataclass
import importlib
import sys
from packaging.version import Version, parse
import builtins
# Add os and site for path checking
import os
import site
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor # type: ignore
from agentops.logging import logger
from agentops.sdk.core import tracer
from agentops.instrumentation.common import get_library_version
# Define the structure for instrumentor configurations
class InstrumentorConfig(TypedDict):
module_name: str
class_name: str
min_version: str
package_name: NotRequired[str] # Optional: actual pip package name if different from module
# Configuration for supported LLM providers
PROVIDERS: dict[str, InstrumentorConfig] = {
"openai": {
"module_name": "agentops.instrumentation.providers.openai",
"class_name": "OpenaiInstrumentor",
"min_version": "1.0.0",
},
"anthropic": {
"module_name": "agentops.instrumentation.providers.anthropic",
"class_name": "AnthropicInstrumentor",
"min_version": "0.32.0",
},
"ibm_watsonx_ai": {
"module_name": "agentops.instrumentation.providers.ibm_watsonx_ai",
"class_name": "WatsonxInstrumentor",
"min_version": "0.1.0",
},
"google.genai": {
"module_name": "agentops.instrumentation.providers.google_genai",
"class_name": "GoogleGenaiInstrumentor",
"min_version": "0.1.0",
"package_name": "google-genai", # Actual pip package name
},
"mem0": {
"module_name": "agentops.instrumentation.providers.mem0",
"class_name": "Mem0Instrumentor",
"min_version": "0.1.0",
"package_name": "mem0ai",
},
}
# Configuration for supported agentic libraries
AGENTIC_LIBRARIES: dict[str, InstrumentorConfig] = {
"crewai": {
"module_name": "agentops.instrumentation.agentic.crewai",
"class_name": "CrewaiInstrumentor",
"min_version": "0.56.0",
},
"autogen": {
"module_name": "agentops.instrumentation.agentic.ag2",
"class_name": "AG2Instrumentor",
"min_version": "0.3.2",
},
"agents": {
"module_name": "agentops.instrumentation.agentic.openai_agents",
"class_name": "OpenAIAgentsInstrumentor",
"min_version": "0.0.1",
},
"google.adk": {
"module_name": "agentops.instrumentation.agentic.google_adk",
"class_name": "GooogleAdkInstrumentor",
"min_version": "0.1.0",
},
"agno": {
"module_name": "agentops.instrumentation.agentic.agno",
"class_name": "AgnoInstrumentor",
"min_version": "1.5.8",
},
"smolagents": {
"module_name": "agentops.instrumentation.agentic.smolagents",
"class_name": "SmolagentsInstrumentor",
"min_version": "1.0.0",
},
"langgraph": {
"module_name": "agentops.instrumentation.agentic.langgraph",
"class_name": "LanggraphInstrumentor",
"min_version": "0.2.0",
},
"xpander_sdk": {
"module_name": "agentops.instrumentation.agentic.xpander",
"class_name": "XpanderInstrumentor",
"min_version": "1.0.0",
"package_name": "xpander-sdk",
},
"haystack": {
"module_name": "agentops.instrumentation.agentic.haystack",
"class_name": "HaystackInstrumentor",
"min_version": "2.0.0",
"package_name": "haystack-ai",
},
}
# Combine all target packages for monitoring
TARGET_PACKAGES = set(PROVIDERS.keys()) | set(AGENTIC_LIBRARIES.keys())
# Create a single instance of the manager
# _manager = InstrumentationManager() # Removed
# Module-level state variables
_active_instrumentors: list[BaseInstrumentor] = []
_original_builtins_import = builtins.__import__ # Store original import
_instrumenting_packages: Set[str] = set()
_has_agentic_library: bool = False
# New helper function to check module origin
def _is_installed_package(module_obj: ModuleType, package_name_key: str) -> bool:
"""
Determines if the given module object corresponds to an installed site-package
rather than a local module, especially when names might collide.
`package_name_key` is the key from TARGET_PACKAGES (e.g., 'agents', 'google.adk').
"""
if not hasattr(module_obj, "__file__") or not module_obj.__file__:
logger.debug(
f"_is_installed_package: Module '{package_name_key}' has no __file__, assuming it might be an SDK namespace package. Returning True."
)
return True
module_path = os.path.normcase(os.path.realpath(os.path.abspath(module_obj.__file__)))
# Priority 1: Check if it's in any site-packages directory.
site_packages_dirs = site.getsitepackages()
if isinstance(site_packages_dirs, str):
site_packages_dirs = [site_packages_dirs]
if hasattr(site, "USER_SITE") and site.USER_SITE and os.path.exists(site.USER_SITE):
site_packages_dirs.append(site.USER_SITE)
normalized_site_packages_dirs = [
os.path.normcase(os.path.realpath(p)) for p in site_packages_dirs if p and os.path.exists(p)
]
for sp_dir in normalized_site_packages_dirs:
if module_path.startswith(sp_dir):
logger.debug(
f"_is_installed_package: Module '{package_name_key}' is a library, instrumenting '{package_name_key}'."
)
return True
# Priority 2: If not in site-packages, it's highly likely a local module or not an SDK we target.
logger.debug(f"_is_installed_package: Module '{package_name_key}' is a local module, skipping instrumentation.")
return False
def _is_package_instrumented(package_name: str) -> bool:
"""Check if a package is already instrumented by looking at active instrumentors."""
# Handle package.module names by converting dots to underscores for comparison
normalized_target_name = package_name.replace(".", "_").lower()
for instrumentor in _active_instrumentors:
# Check based on the key it was registered with
if (
hasattr(instrumentor, "_agentops_instrumented_package_key")
and instrumentor._agentops_instrumented_package_key == package_name
):
return True
# Fallback to class name check (existing logic, less precise)
# We use split('.')[-1] for cases like 'google.genai' to match GenAIInstrumentor
instrumentor_class_name_prefix = instrumentor.__class__.__name__.lower().replace("instrumentor", "")
target_base_name = package_name.split(".")[-1].lower()
normalized_class_name_match = (
normalized_target_name.startswith(instrumentor_class_name_prefix)
or target_base_name == instrumentor_class_name_prefix
)
if normalized_class_name_match:
# This fallback can be noisy, let's make it more specific or rely on the key above more
# For now, if the key matches or this broad name match works, consider instrumented.
# This helps if _agentops_instrumented_package_key was somehow not set.
return True
return False
def _uninstrument_providers():
"""Uninstrument all provider instrumentors while keeping agentic libraries active."""
global _active_instrumentors
new_active_instrumentors = []
uninstrumented_any = False
for instrumentor in _active_instrumentors:
instrumented_key = getattr(instrumentor, "_agentops_instrumented_package_key", None)
if instrumented_key and instrumented_key in PROVIDERS:
try:
instrumentor.uninstrument()
logger.debug(
f"AgentOps: Uninstrumented provider: {instrumentor.__class__.__name__} (for package '{instrumented_key}') due to agentic library activation."
)
uninstrumented_any = True
except Exception as e:
logger.error(f"Error uninstrumenting provider {instrumentor.__class__.__name__}: {e}")
else:
# Keep non-provider instrumentors or those without our key (shouldn't happen for managed ones)
new_active_instrumentors.append(instrumentor)
if uninstrumented_any or not new_active_instrumentors and _active_instrumentors:
logger.debug(
f"_uninstrument_providers: Processed. Previous active: {len(_active_instrumentors)}, New active after filtering providers: {len(new_active_instrumentors)}"
)
_active_instrumentors = new_active_instrumentors
def _should_instrument_package(package_name: str) -> bool:
"""
Determine if a package should be instrumented based on current state.
Handles special cases for agentic libraries and providers.
"""
global _has_agentic_library
# If already instrumented by AgentOps (using our refined check), skip.
if _is_package_instrumented(package_name):
logger.debug(f"_should_instrument_package: '{package_name}' already instrumented by AgentOps. Skipping.")
return False
is_target_agentic = package_name in AGENTIC_LIBRARIES
is_target_provider = package_name in PROVIDERS
if not is_target_agentic and not is_target_provider:
logger.debug(
f"_should_instrument_package: '{package_name}' is not a targeted provider or agentic library. Skipping."
)
return False
if _has_agentic_library:
# An agentic library is already active.
if is_target_agentic:
logger.debug(
f"AgentOps: An agentic library is active. Skipping instrumentation for subsequent agentic library '{package_name}'."
)
return False
if is_target_provider:
logger.debug(
f"AgentOps: An agentic library is active. Skipping instrumentation for provider '{package_name}'."
)
return False
else:
# No agentic library is active yet.
if is_target_agentic:
logger.debug(
f"AgentOps: '{package_name}' is the first-targeted agentic library. Will uninstrument providers if any are/become active."
)
_uninstrument_providers()
return True
if is_target_provider:
logger.debug(
f"_should_instrument_package: '{package_name}' is a provider, no agentic library active. Allowing."
)
return True
logger.debug(
f"_should_instrument_package: Defaulting to False for '{package_name}' (state: _has_agentic_library={_has_agentic_library})"
)
return False
def _perform_instrumentation(package_name: str):
"""Helper function to perform instrumentation for a given package."""
global _instrumenting_packages, _active_instrumentors, _has_agentic_library
if not _should_instrument_package(package_name):
return
# Get the appropriate configuration for the package
# Ensure package_name is a key in either PROVIDERS or AGENTIC_LIBRARIES
if package_name not in PROVIDERS and package_name not in AGENTIC_LIBRARIES:
logger.debug(
f"_perform_instrumentation: Package '{package_name}' not found in PROVIDERS or AGENTIC_LIBRARIES. Skipping."
)
return
config = PROVIDERS.get(package_name) or AGENTIC_LIBRARIES.get(package_name)
loader = InstrumentorLoader(**config)
# instrument_one already checks loader.should_activate
instrumentor_instance = instrument_one(loader)
if instrumentor_instance is not None:
# Check if it was *actually* instrumented by instrument_one by seeing if the instrument method was called successfully.
# This relies on instrument_one returning None if its internal .instrument() call failed (if we revert that, this needs adjustment)
# For now, assuming instrument_one returns instance only on full success.
# User request was to return instrumentor even if .instrument() fails. So, we check if _agentops_instrumented_package_key was set by us.
# Let's assume instrument_one might return an instance whose .instrument() failed.
# The key is set before _active_instrumentors.append, so if it's already there and matches, it means it's a re-attempt on the same package.
# The _is_package_instrumented check at the start of _should_instrument_package should prevent most re-entry for the same package_name.
# Store the package key this instrumentor is for, to aid _is_package_instrumented
instrumentor_instance._agentops_instrumented_package_key = package_name
# Add to active_instrumentors only if it's not a duplicate in terms of package_key being instrumented
# This is a safeguard, _is_package_instrumented should catch this earlier.
is_newly_added = True
for existing_inst in _active_instrumentors:
if (
hasattr(existing_inst, "_agentops_instrumented_package_key")
and existing_inst._agentops_instrumented_package_key == package_name
):
is_newly_added = False
logger.debug(
f"_perform_instrumentation: Instrumentor for '{package_name}' already in _active_instrumentors. Not adding again."
)
break
if is_newly_added:
_active_instrumentors.append(instrumentor_instance)
# If this was an agentic library AND it's newly effectively instrumented.
if (
package_name in AGENTIC_LIBRARIES and not _has_agentic_library
): # Check _has_agentic_library to ensure this is the *first* one.
# _uninstrument_providers() was already called in _should_instrument_package for the first agentic library.
_has_agentic_library = True
# Special case: If mem0 is instrumented, also instrument concurrent.futures
if (package_name == "mem0" or package_name == "autogen") and is_newly_added:
try:
# Check if concurrent.futures module is available
# Create config for concurrent.futures instrumentor
concurrent_config = InstrumentorConfig(
module_name="agentops.instrumentation.utilities.concurrent_futures",
class_name="ConcurrentFuturesInstrumentor",
min_version="3.7.0", # Python 3.7+ (concurrent.futures is stdlib)
package_name="python", # Special case for stdlib modules
)
# Create and instrument concurrent.futures
concurrent_loader = InstrumentorLoader(**concurrent_config)
concurrent_instrumentor = instrument_one(concurrent_loader)
if concurrent_instrumentor is not None:
concurrent_instrumentor._agentops_instrumented_package_key = "concurrent.futures"
_active_instrumentors.append(concurrent_instrumentor)
logger.debug("AgentOps: Instrumented concurrent.futures as a dependency of mem0.")
except Exception as e:
logger.debug(f"Could not instrument concurrent.futures for mem0: {e}")
else:
logger.debug(
f"_perform_instrumentation: instrument_one for '{package_name}' returned None. Not added to active instrumentors."
)
def _import_monitor(name: str, globals_dict=None, locals_dict=None, fromlist=(), level=0):
"""
Monitor imports and instrument packages as they are imported.
This replaces the built-in import function to intercept package imports.
"""
global _instrumenting_packages, _has_agentic_library
# If an agentic library is already instrumented, skip all further instrumentation
if _has_agentic_library:
return _original_builtins_import(name, globals_dict, locals_dict, fromlist, level)
# First, do the actual import
module = _original_builtins_import(name, globals_dict, locals_dict, fromlist, level)
# Check for exact matches first (handles package.module like google.adk)
packages_to_check = set()
# Check the imported module itself
if name in TARGET_PACKAGES:
packages_to_check.add(name)
else:
# Check if any target package is a prefix of the import name
for target in TARGET_PACKAGES:
if name.startswith(target + ".") or name == target:
packages_to_check.add(target)
# For "from X import Y" style imports, also check submodules
if fromlist:
for item in fromlist:
# Construct potential full name, e.g., "google.adk" from name="google", item="adk"
# Or if name="os", item="path", full_name="os.path"
# If the original name itself is a multi-part name like "a.b", and item is "c", then "a.b.c"
# This logic needs to correctly identify the root package if 'name' is already a sub-package.
# The existing TARGET_PACKAGES check is simpler: it checks against pre-defined full names.
# Check full name if item forms part of a target package name
full_item_name_candidate = f"{name}.{item}"
if full_item_name_candidate in TARGET_PACKAGES:
packages_to_check.add(full_item_name_candidate)
else: # Fallback to checking if 'name' itself is a target
for target in TARGET_PACKAGES:
if name == target or name.startswith(target + "."):
packages_to_check.add(target) # Check the base target if a submodule is imported from it.
# Instrument all matching packages
for package_to_check in packages_to_check:
if package_to_check not in _instrumenting_packages and not _is_package_instrumented(package_to_check):
target_module_obj = sys.modules.get(package_to_check)
if target_module_obj:
is_sdk = _is_installed_package(target_module_obj, package_to_check)
if not is_sdk:
logger.debug(
f"AgentOps: Target '{package_to_check}' appears to be a local module/directory. Skipping AgentOps SDK instrumentation for it."
)
continue
else:
logger.debug(
f"_import_monitor: No module object found in sys.modules for '{package_to_check}', proceeding with SDK instrumentation attempt."
)
_instrumenting_packages.add(package_to_check)
try:
_perform_instrumentation(package_to_check)
# If we just instrumented an agentic library, stop
if _has_agentic_library:
break
except Exception as e:
logger.error(f"Error instrumenting {package_to_check}: {str(e)}")
finally:
_instrumenting_packages.discard(package_to_check)
return module
@dataclass
class InstrumentorLoader:
"""
Represents a dynamically-loadable instrumentor.
Handles version checking and instantiation of instrumentors.
"""
module_name: str
class_name: str
min_version: str
package_name: Optional[str] = None # Optional: actual pip package name
@property
def module(self) -> ModuleType:
"""Get the instrumentor module."""
return importlib.import_module(self.module_name)
@property
def should_activate(self) -> bool:
"""Check if the package is available and meets version requirements."""
try:
# Special case for stdlib modules (like concurrent.futures)
if self.package_name == "python":
import sys
python_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
return Version(python_version) >= parse(self.min_version)
# Use explicit package_name if provided, otherwise derive from module_name
if self.package_name:
provider_name = self.package_name
else:
provider_name = self.module_name.split(".")[-1]
# Use common version utility
module_version = get_library_version(provider_name)
return module_version != "unknown" and Version(module_version) >= parse(self.min_version)
except Exception:
return False
def get_instance(self) -> BaseInstrumentor:
"""Create and return a new instance of the instrumentor."""
return getattr(self.module, self.class_name)()
def instrument_one(loader: InstrumentorLoader) -> Optional[BaseInstrumentor]:
"""
Instrument a single package using the provided loader.
Returns the instrumentor instance if successful, None otherwise.
"""
if not loader.should_activate:
# This log is important for users to know why something wasn't instrumented.
logger.debug(
f"AgentOps: Package '{loader.package_name or loader.module_name}' not found or version is less than minimum required ('{loader.min_version}'). Skipping instrumentation."
)
return None
instrumentor = loader.get_instance()
try:
# Use the provider directly from the global tracer instance
instrumentor.instrument(tracer_provider=tracer.provider)
logger.debug(
f"AgentOps: Successfully instrumented '{loader.class_name}' for package '{loader.package_name or loader.module_name}'."
)
except Exception as e:
logger.error(
f"Failed to instrument {loader.class_name} for {loader.package_name or loader.module_name}: {e}",
exc_info=True,
)
return instrumentor
def instrument_all():
"""Start monitoring and instrumenting packages if not already started."""
# Check if active_instrumentors is empty, as a proxy for not started.
if not _active_instrumentors:
builtins.__import__ = _import_monitor
global _instrumenting_packages, _has_agentic_library
# If an agentic library is already instrumented, don't instrument anything else
if _has_agentic_library:
return
for name in list(sys.modules.keys()):
# Stop if an agentic library gets instrumented during the loop
if _has_agentic_library:
break
module = sys.modules.get(name)
if not isinstance(module, ModuleType):
continue
# Check for exact matches first (handles package.module like google.adk)
package_to_check = None
if name in TARGET_PACKAGES:
package_to_check = name
else:
# Check if any target package is a prefix of the module name
for target in TARGET_PACKAGES:
if name.startswith(target + ".") or name == target:
package_to_check = target
break
if (
package_to_check
and package_to_check not in _instrumenting_packages
and not _is_package_instrumented(package_to_check)
):
target_module_obj = sys.modules.get(package_to_check)
if target_module_obj:
is_sdk = _is_installed_package(target_module_obj, package_to_check)
if not is_sdk:
continue
else:
logger.debug(
f"instrument_all: No module object found for '{package_to_check}' in sys.modules during startup scan. Proceeding cautiously."
)
_instrumenting_packages.add(package_to_check)
try:
_perform_instrumentation(package_to_check)
except Exception as e:
logger.error(f"Error instrumenting {package_to_check}: {str(e)}")
finally:
_instrumenting_packages.discard(package_to_check)
def uninstrument_all():
"""Stop monitoring and uninstrument all packages."""
global _active_instrumentors, _has_agentic_library
builtins.__import__ = _original_builtins_import
for instrumentor in _active_instrumentors:
instrumentor.uninstrument()
logger.debug(f"Uninstrumented {instrumentor.__class__.__name__}")
_active_instrumentors = []
_has_agentic_library = False
def get_active_libraries() -> set[str]:
"""
Get all actively used libraries in the current execution context.
Returns a set of package names that are currently imported and being monitored.
"""
active_libs = set()
for name, module in sys.modules.items():
if not isinstance(module, ModuleType):
continue
# Check for exact matches first
if name in TARGET_PACKAGES:
active_libs.add(name)
else:
# Check if any target package is a prefix of the module name
for target in TARGET_PACKAGES:
if name.startswith(target + ".") or name == target:
active_libs.add(target)
break
return active_libs
@@ -0,0 +1,18 @@
"""AG2 Instrumentation for AgentOps
This module provides instrumentation for AG2 (AutoGen), adding telemetry to track agent
interactions, conversation flows, and tool usage while focusing on summary-level data rather
than individual message exchanges.
"""
from agentops.instrumentation.common import LibraryInfo
# Library information
_library_info = LibraryInfo(name="ag2")
LIBRARY_NAME = _library_info.name
LIBRARY_VERSION = _library_info.version
# Import after defining constants to avoid circular imports
from agentops.instrumentation.agentic.ag2.instrumentor import AG2Instrumentor # noqa: E402
__all__ = ["AG2Instrumentor", "LIBRARY_NAME", "LIBRARY_VERSION"]
@@ -0,0 +1,922 @@
"""AG2 (AutoGen) Instrumentation Module
This module provides the main instrumentor class and wrapping functions for AG2 (AutoGen).
It focuses on collecting summary-level telemetry rather than individual message events.
"""
import json
from typing import Dict, Any
from wrapt import wrap_function_wrapper
from opentelemetry.trace import SpanKind
from opentelemetry.metrics import Meter
from opentelemetry.instrumentation.utils import unwrap as otel_unwrap
import contextvars
import threading
from opentelemetry import context as otel_context
from agentops.logging import logger
from agentops.instrumentation.common import (
CommonInstrumentor,
InstrumentorConfig,
StandardMetrics,
create_span,
SpanAttributeManager,
)
from agentops.instrumentation.agentic.ag2 import LIBRARY_NAME, LIBRARY_VERSION
from agentops.semconv.message import MessageAttributes
from agentops.semconv.span_attributes import SpanAttributes
from agentops.semconv.span_kinds import AgentOpsSpanKindValues
from agentops.semconv.agent import AgentAttributes
from agentops.semconv.workflow import WorkflowAttributes
from agentops.semconv.tool import ToolAttributes
class AG2Instrumentor(CommonInstrumentor):
"""Instrumentor for AG2 (AutoGen)
This instrumentor captures high-level events from AG2's agent interactions,
focusing on summaries rather than individual messages, and providing detailed
tool usage information.
"""
def __init__(self):
config = InstrumentorConfig(
library_name=LIBRARY_NAME,
library_version=LIBRARY_VERSION,
wrapped_methods=[], # We'll use custom wrapping
metrics_enabled=True,
dependencies=["ag2 >= 0.3.2"],
)
super().__init__(config)
self._attribute_manager = None
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
"""Create metrics for AG2 instrumentation."""
return StandardMetrics.create_standard_metrics(meter)
def _initialize(self, **kwargs):
"""Initialize attribute manager and AG2-specific concurrent.futures instrumentation."""
self._attribute_manager = SpanAttributeManager(service_name="agentops", deployment_environment="production")
def _custom_wrap(self, **kwargs):
"""Perform custom wrapping for AG2 methods."""
methods_to_wrap = [
("autogen.agentchat.conversable_agent", "ConversableAgent.__init__", self._agent_init_wrapper),
("autogen.agentchat.conversable_agent", "ConversableAgent.run", self._agent_run_wrapper_with_context),
("autogen.agentchat.conversable_agent", "ConversableAgent.initiate_chat", self._initiate_chat_wrapper),
(
"autogen.agentchat.conversable_agent",
"ConversableAgent.a_initiate_chat",
self._async_initiate_chat_wrapper,
),
(
"autogen.agentchat.conversable_agent",
"ConversableAgent._generate_oai_reply_from_client",
self._generate_oai_reply_from_client_wrapper,
),
("autogen.agentchat.conversable_agent", "ConversableAgent.receive", self._receive_wrapper),
("autogen.agentchat.conversable_agent", "ConversableAgent.a_receive", self._async_receive_wrapper),
("autogen.agentchat.groupchat", "GroupChatManager.run_chat", self._group_chat_run_wrapper),
("autogen.agentchat.groupchat", "GroupChatManager.a_run_chat", self._async_group_chat_run_wrapper),
(
"autogen.agentchat.conversable_agent",
"ConversableAgent.execute_function",
lambda tracer: self._tool_execution_wrapper(tracer, "function"),
),
(
"autogen.agentchat.conversable_agent",
"ConversableAgent.run_code",
lambda tracer: self._tool_execution_wrapper(tracer, "code"),
),
("autogen.agentchat.groupchat", "GroupChat.select_speaker", self._group_chat_select_speaker_wrapper),
]
for module, method, wrapper_factory in methods_to_wrap:
try:
wrap_function_wrapper(module, method, wrapper_factory(self._tracer))
except (AttributeError, ModuleNotFoundError) as e:
logger.debug(f"Failed to wrap {method}: {e}")
def _custom_unwrap(self, **kwargs):
"""Remove instrumentation from AG2."""
# Unwrap all instrumented methods
methods_to_unwrap = [
("autogen.agentchat.conversable_agent", "ConversableAgent.__init__"),
("autogen.agentchat.conversable_agent", "ConversableAgent.run"),
("autogen.agentchat.conversable_agent", "ConversableAgent.initiate_chat"),
("autogen.agentchat.conversable_agent", "ConversableAgent.a_initiate_chat"),
("autogen.agentchat.conversable_agent", "ConversableAgent._generate_oai_reply_from_client"),
("autogen.agentchat.conversable_agent", "ConversableAgent.receive"),
("autogen.agentchat.conversable_agent", "ConversableAgent.a_receive"),
("autogen.agentchat.groupchat", "GroupChatManager.run_chat"),
("autogen.agentchat.groupchat", "GroupChatManager.a_run_chat"),
("autogen.agentchat.conversable_agent", "ConversableAgent.execute_function"),
("autogen.agentchat.conversable_agent", "ConversableAgent.run_code"),
("autogen.agentchat.groupchat", "GroupChat.select_speaker"),
]
try:
for module, method in methods_to_unwrap:
otel_unwrap(module, method)
logger.debug("Successfully uninstrumented AG2")
except Exception as e:
logger.debug(f"Failed to unwrap AG2 methods: {e}")
def _set_llm_config_attributes(self, span, llm_config):
if not isinstance(llm_config, dict):
return
if "model" in llm_config:
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, llm_config["model"])
for param, attr in [
("temperature", SpanAttributes.LLM_REQUEST_TEMPERATURE),
("top_p", SpanAttributes.LLM_REQUEST_TOP_P),
("frequency_penalty", SpanAttributes.LLM_REQUEST_FREQUENCY_PENALTY),
("presence_penalty", SpanAttributes.LLM_REQUEST_PRESENCE_PENALTY),
]:
if param in llm_config and llm_config[param] is not None:
span.set_attribute(attr, llm_config[param])
def _agent_init_wrapper(self, tracer):
"""Wrapper for capturing agent initialization."""
def wrapper(wrapped, instance, args, kwargs):
try:
name = kwargs.get("name", "unnamed_agent")
llm_config = kwargs.get("llm_config", {})
result = wrapped(*args, **kwargs)
model = "unknown"
if isinstance(llm_config, dict) and llm_config:
model = llm_config.get("model", "unknown")
instance._agentops_metadata = {"name": name, "type": "ConversableAgent", "model": model}
return result
except Exception:
return wrapped(*args, **kwargs)
return wrapper
def _generate_oai_reply_from_client_wrapper(self, tracer):
"""Wrapper for capturing _generate_oai_reply_from_client method calls with token metrics."""
def wrapper(wrapped, instance, args, kwargs):
agent_name = getattr(instance, "name", "unnamed_agent")
# Get model name from llm_client for span naming
llm_client = args[0] if args else kwargs.get("llm_client")
# Extract model from _config_list
model_name = "unknown"
if hasattr(llm_client, "_config_list") and llm_client._config_list:
if isinstance(llm_client._config_list, list) and len(llm_client._config_list) > 0:
config = llm_client._config_list[0]
if isinstance(config, dict) and "model" in config:
model_name = config["model"]
span_name = f"{model_name}.llm"
with create_span(
tracer, span_name, kind=SpanKind.CLIENT, attribute_manager=self._attribute_manager
) as span:
# Set span kind for actual LLM client call
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.LLM.value)
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "llm")
span.set_attribute("llm.client_call", "true")
# Get messages from args
messages = args[1] if len(args) > 1 else kwargs.get("messages", [])
# Extract input from messages and set gen_ai.prompt
if messages and isinstance(messages, list) and len(messages) > 0:
# Set gen_ai.prompt array with full conversation history
prompt_index = 0
for msg in messages:
if isinstance(msg, dict) and msg.get("role") in ["user", "assistant", "system"]:
role = msg.get("role")
content = msg.get("content", "")
if content and role:
span.set_attribute(
f"{SpanAttributes.LLM_PROMPTS}.{prompt_index}.content", self._safe_str(content)
)
span.set_attribute(f"{SpanAttributes.LLM_PROMPTS}.{prompt_index}.role", role)
prompt_index += 1
# Set entity input to the latest user message (what triggered this LLM call)
latest_user_message = None
for msg in messages:
if isinstance(msg, dict) and msg.get("role") == "user":
content = msg.get("content", "")
if content:
latest_user_message = content
if latest_user_message:
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, self._safe_str(latest_user_message))
# Call the wrapped method - this is where the actual LLM call happens
result = wrapped(*args, **kwargs)
# Set the output and gen_ai.completion
if result:
if isinstance(result, dict):
content = result.get("content", "")
if content:
content_str = self._safe_str(content)
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, content_str)
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.content", content_str)
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.role", "assistant")
# If model information is in the result
if "model" in result:
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, result["model"])
elif isinstance(result, str):
# Handle string result (which is what AG2 returns)
result_str = self._safe_str(result)
# Set entity output
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, result_str)
# Set gen_ai.completion with full content
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.content", result_str)
span.set_attribute(f"{SpanAttributes.LLM_COMPLETIONS}.0.role", "assistant")
# Try to get token metrics from the client's usage tracking
try:
# The OpenAIWrapper tracks usage in actual_usage_summary and total_usage_summary
if hasattr(llm_client, "actual_usage_summary") and llm_client.actual_usage_summary:
# Get the latest usage
for model, usage in llm_client.actual_usage_summary.items():
if model != "total_cost" and isinstance(usage, dict):
prompt_tokens = usage.get("prompt_tokens", 0)
completion_tokens = usage.get("completion_tokens", 0)
total_tokens = usage.get("total_tokens", 0)
cost = usage.get("cost", 0.0)
# Set token usage metrics
if prompt_tokens > 0:
span.set_attribute(SpanAttributes.LLM_USAGE_PROMPT_TOKENS, str(prompt_tokens))
if completion_tokens > 0:
span.set_attribute(
SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, str(completion_tokens)
)
if total_tokens > 0:
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, str(total_tokens))
if cost > 0:
span.set_attribute(SpanAttributes.LLM_USAGE_TOOL_COST, str(cost))
# Set request/response model
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model)
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, model)
span.set_attribute(SpanAttributes.LLM_SYSTEM, "ag2")
break # Use the first model's metrics
except Exception as e:
logger.debug(f"[AG2 DEBUG] Could not extract token metrics: {e}")
return result
return wrapper
def _initiate_chat_wrapper(self, tracer):
"""Wrapper for capturing individual chat initiation as a parent span."""
def wrapper(wrapped, instance, args, kwargs):
recipient_agent = args[0] if args else None
if not recipient_agent:
return wrapped(*args, **kwargs)
# Get agent names for span identification
initiator_name = getattr(instance, "name", "unnamed_initiator")
recipient_name = getattr(recipient_agent, "name", "unnamed_agent")
span_name = f"ag2.chat.{initiator_name}_to_{recipient_name}.workflow"
with create_span(
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
) as span:
# Set span kind as agent for proper categorization
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
span.set_attribute(AgentAttributes.FROM_AGENT, initiator_name)
span.set_attribute(AgentAttributes.TO_AGENT, recipient_name)
span.set_attribute("agent.type", "individual")
span.set_attribute("agent.initiator", initiator_name)
span.set_attribute("agent.recipient", recipient_name)
# Set agentops entity attributes
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
# Extract initial message
initial_message = kwargs.get("message", "")
if initial_message:
initial_message = self._safe_str(initial_message)
span.set_attribute("agent.initial_message", initial_message)
# Set entity input
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, initial_message)
# Extract system messages and put them in agent attributes
initiator_system_msg = getattr(instance, "system_message", "")
if initiator_system_msg:
initiator_system_msg = self._safe_str(initiator_system_msg)
span.set_attribute("agent.initiator_system_message", initiator_system_msg)
recipient_system_msg = getattr(recipient_agent, "system_message", "")
if recipient_system_msg:
recipient_system_msg = self._safe_str(recipient_system_msg)
span.set_attribute("agent.system_instruction", recipient_system_msg)
# Also set in gen_ai for compatibility
span.set_attribute(SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION, recipient_system_msg)
# Extract LLM config and set gen_ai attributes
recipient_llm_config = getattr(recipient_agent, "llm_config", {})
if isinstance(recipient_llm_config, dict) and recipient_llm_config:
model = recipient_llm_config.get("model", "unknown")
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model)
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, model)
span.set_attribute(SpanAttributes.LLM_SYSTEM, "ag2")
# Also set LLM config attributes
self._set_llm_config_attributes(span, recipient_llm_config)
result = wrapped(*args, **kwargs)
# Extract chat history after completion
self._extract_chat_history(span, instance, recipient_agent)
return result
return wrapper
def _async_initiate_chat_wrapper(self, tracer):
"""Wrapper for capturing async individual chat initiation as a parent span."""
async def wrapper(wrapped, instance, args, kwargs):
recipient_agent = args[0] if args else None
if not recipient_agent:
return await wrapped(*args, **kwargs)
# Get agent names for span identification
initiator_name = getattr(instance, "name", "unnamed_initiator")
recipient_name = getattr(recipient_agent, "name", "unnamed_agent")
span_name = f"ag2.chat.{initiator_name}_to_{recipient_name}.workflow"
with create_span(
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
) as span:
# Set span kind as agent for proper categorization
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
span.set_attribute(AgentAttributes.FROM_AGENT, initiator_name)
span.set_attribute(AgentAttributes.TO_AGENT, recipient_name)
span.set_attribute("agent.type", "individual_async")
span.set_attribute("agent.initiator", initiator_name)
span.set_attribute("agent.recipient", recipient_name)
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
# Extract initial message
initial_message = kwargs.get("message", "")
if initial_message:
initial_message = self._safe_str(initial_message)
span.set_attribute("agent.initial_message", initial_message)
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, initial_message)
# Extract system messages
recipient_system_msg = getattr(recipient_agent, "system_message", "")
if recipient_system_msg:
recipient_system_msg = self._safe_str(recipient_system_msg)
span.set_attribute("agent.system_instruction", recipient_system_msg)
span.set_attribute(SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION, recipient_system_msg)
# Extract LLM config
recipient_llm_config = getattr(recipient_agent, "llm_config", {})
if isinstance(recipient_llm_config, dict) and recipient_llm_config:
self._set_llm_config_attributes(span, recipient_llm_config)
result = await wrapped(*args, **kwargs)
# Extract chat history after completion
self._extract_chat_history(span, instance, recipient_agent)
return result
return wrapper
def _receive_wrapper(self, tracer):
"""Wrapper for capturing message receive events."""
def wrapper(wrapped, instance, args, kwargs):
agent_name = getattr(instance, "name", "unnamed_agent")
span_name = f"ag2.agent.{agent_name}.receive"
with create_span(
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
) as span:
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
# Get message and sender
message = args[0] if args else kwargs.get("message", "")
sender = args[1] if len(args) > 1 else kwargs.get("sender")
if sender:
sender_name = getattr(sender, "name", "unknown")
span.set_attribute("agent.sender", sender_name)
if message:
if isinstance(message, dict):
content = message.get("content", "")
if content:
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, self._safe_str(content))
elif isinstance(message, str):
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, self._safe_str(message))
result = wrapped(*args, **kwargs)
return result
return wrapper
def _async_receive_wrapper(self, tracer):
"""Wrapper for async capturing message reception."""
async def wrapper(wrapped, instance, args, kwargs):
agent_name = getattr(instance, "name", "unnamed_agent")
span_name = f"ag2.agent.{agent_name}.async_receive"
with create_span(
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
) as span:
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.AGENT.value)
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
# Get message from the first argument
message = args[0] if args else None
# Enhanced message processing
if message:
if isinstance(message, dict):
# Dict message format
sender_name = message.get("name", "unknown")
content = self._extract_message_content(message)
role = message.get("role", "user")
# Set sender and message attributes
span.set_attribute("agent.sender", sender_name)
span.set_attribute("message.role", role)
if content:
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, content)
span.set_attribute("message.content", content)
elif isinstance(message, str):
# String message format
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, message)
span.set_attribute("message.content", message)
# Get sender from the second argument if available
sender = args[1] if len(args) > 1 else None
if sender and hasattr(sender, "name"):
span.set_attribute("agent.sender_name", sender.name)
return await wrapped(*args, **kwargs)
return wrapper
def _async_group_chat_run_wrapper(self, tracer):
"""Wrapper for capturing async group chat execution."""
async def wrapper(wrapped, instance, args, kwargs):
with create_span(
tracer,
"ag2.groupchat.run.task.async",
kind=SpanKind.INTERNAL,
attribute_manager=self._attribute_manager,
) as span:
group_chat = getattr(instance, "groupchat", None)
agents = getattr(group_chat, "agents", []) if group_chat else []
agent_names = [getattr(agent, "name", f"agent_{i}") for i, agent in enumerate(agents)]
span.set_attribute(AgentAttributes.AGENT_ROLE, "GroupChatManager")
span.set_attribute(AgentAttributes.AGENT_NAME, getattr(instance, "name", "unnamed_manager"))
span.set_attribute("groupchat.agents", ", ".join(agent_names))
span.set_attribute("groupchat.agent_count", len(agents))
# Capture input message if available
message = kwargs.get("message", "")
if message:
content_to_set = self._extract_message_content(message)
span.set_attribute("groupchat.input_message", content_to_set)
result = await wrapped(*args, **kwargs)
self._capture_group_chat_summary(span, instance, result)
return result
return wrapper
def _agent_run_wrapper_with_context(self, tracer):
"""Wrapper for capturing agent run with context propagation and proper span lifecycle."""
def wrapper(wrapped, instance, args, kwargs):
agent_name = getattr(instance, "name", "unnamed_agent")
agent_type = getattr(instance, "_agentops_metadata", {}).get("type", "ConversableAgent")
span_name = f"ag2.agent.{agent_name}.run.workflow"
with create_span(
tracer, span_name, kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
) as span:
model = getattr(instance, "_agentops_metadata", {}).get("model", "unknown")
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
span.set_attribute(AgentAttributes.AGENT_ROLE, agent_type)
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model)
llm_config = getattr(instance, "llm_config", None)
self._set_llm_config_attributes(span, llm_config)
# Capture input message if available
message = kwargs.get("message", "")
if message:
content_to_set = self._extract_message_content(message)
span.set_attribute("agent.run.input_message", content_to_set)
# Capture BOTH contextvars and OpenTelemetry context
ctx = contextvars.copy_context()
current_otel_context = otel_context.get_current()
# Thread tracking for proper span lifecycle
active_threads = []
# Store the original Thread.__init__ and start methods
original_thread_init = threading.Thread.__init__
original_thread_start = threading.Thread.start
def context_aware_init(self, group=None, target=None, name=None, args=(), kwargs=None, *, daemon=None):
"""Modified Thread.__init__ that wraps the target to run in both captured contexts."""
if kwargs is None:
kwargs = {}
if target and callable(target):
original_target = target
def wrapped_target(*target_args, **target_kwargs):
# Run in both contextvars AND OpenTelemetry context
def run_with_otel_context():
# Attach the OpenTelemetry context in the thread
token = otel_context.attach(current_otel_context)
try:
return original_target(*target_args, **target_kwargs)
finally:
otel_context.detach(token)
# Run with contextvars context
return ctx.run(run_with_otel_context)
target = wrapped_target
# Keep original daemon setting but ensure conversations don't run indefinitely
# If daemon was not explicitly set, default to False (AG2's normal behavior)
if daemon is None:
daemon = False
original_thread_init(
self, group=group, target=target, name=name, args=args, kwargs=kwargs, daemon=daemon
)
def context_aware_start(self):
"""Modified Thread.start that tracks the thread."""
active_threads.append(self)
return original_thread_start(self)
# Temporarily patch Thread.__init__ and start just for this run() call
threading.Thread.__init__ = context_aware_init
threading.Thread.start = context_aware_start
try:
response = wrapped(*args, **kwargs)
except Exception as e:
logger.error(f"[AG2 DEBUG] Error in agent.run execution: {e}")
raise
finally:
# Always restore the original Thread methods
try:
threading.Thread.__init__ = original_thread_init
threading.Thread.start = original_thread_start
except Exception as e:
logger.error(f"[AG2 DEBUG] Error restoring Thread methods: {e}")
# Force restore
threading.Thread.__init__ = (
threading.Thread.__init__.__wrapped__
if hasattr(threading.Thread.__init__, "__wrapped__")
else original_thread_init
)
threading.Thread.start = (
threading.Thread.start.__wrapped__
if hasattr(threading.Thread.start, "__wrapped__")
else original_thread_start
)
# Try to get final results from response if available
try:
if hasattr(response, "get_chat_results"):
chat_results = response.get_chat_results()
if chat_results:
self._capture_conversation_summary(span, instance, chat_results)
elif hasattr(response, "chat_history"):
self._capture_conversation_summary(span, instance, response)
elif hasattr(response, "get") and callable(response.get):
model_info = response.get("model", "")
if model_info:
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, str(model_info))
except Exception as e:
logger.debug(f"[AG2 DEBUG] Could not extract final results: {e}")
span.set_attribute(WorkflowAttributes.WORKFLOW_STEP_STATUS, "completed")
return response
return wrapper
def _group_chat_run_wrapper(self, tracer):
"""Wrapper for capturing group chat execution."""
def wrapper(wrapped, instance, args, kwargs):
with create_span(
tracer, "ag2.groupchat.run.task", kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
) as span:
group_chat = getattr(instance, "groupchat", None)
agents = getattr(group_chat, "agents", []) if group_chat else []
agent_names = [getattr(agent, "name", f"agent_{i}") for i, agent in enumerate(agents)]
span.set_attribute(AgentAttributes.AGENT_ROLE, "GroupChatManager")
span.set_attribute(AgentAttributes.AGENT_NAME, getattr(instance, "name", "unnamed_manager"))
span.set_attribute("groupchat.agents", ", ".join(agent_names))
span.set_attribute("groupchat.agent_count", len(agents))
# Capture input message if available
message = kwargs.get("message", "")
if message:
content_to_set = self._extract_message_content(message)
span.set_attribute("groupchat.input_message", content_to_set)
result = wrapped(*args, **kwargs)
self._capture_group_chat_summary(span, instance, result)
return result
return wrapper
def _tool_execution_wrapper(self, tracer, tool_type):
"""Wrapper for capturing tool execution."""
def wrapper(wrapped, instance, args, kwargs):
span_name = f"ag2.tool.{tool_type}.tool_usage"
with create_span(
tracer, span_name, kind=SpanKind.CLIENT, attribute_manager=self._attribute_manager
) as span:
# Set span kind and type as tool for proper categorization
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, AgentOpsSpanKindValues.TOOL.value)
agent_name = getattr(instance, "name", "unnamed_agent")
span.set_attribute(AgentAttributes.AGENT_NAME, agent_name)
span.set_attribute(ToolAttributes.TOOL_NAME, tool_type)
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "tool")
if tool_type == "function" and args:
func_call = args[0]
if isinstance(func_call, dict):
span.set_attribute(
MessageAttributes.TOOL_CALL_NAME.format(i=0), func_call.get("name", "unknown")
)
if "arguments" in func_call:
try:
span.set_attribute(
MessageAttributes.TOOL_CALL_ARGUMENTS.format(i=0),
json.dumps(func_call["arguments"]),
)
except:
pass
elif tool_type == "code" and args:
code = args[0]
if isinstance(code, str):
span.set_attribute("tool.code.size", len(code))
span.set_attribute("tool.code.language", kwargs.get("lang", "unknown"))
result = wrapped(*args, **kwargs)
self._process_tool_result(span, result, tool_type)
return result
return wrapper
def _group_chat_select_speaker_wrapper(self, tracer):
"""Wrapper for capturing which agent is selected to speak in a group chat."""
def wrapper(wrapped, instance, args, kwargs):
previous_speaker_name = "unknown"
messages = getattr(instance, "messages", [])
if messages and len(messages) > 0:
previous_speaker_name = messages[-1].get("name", "unknown")
selected_speaker = wrapped(*args, **kwargs)
if not selected_speaker:
return selected_speaker
current_speaker_name = getattr(selected_speaker, "name", "unnamed")
with create_span(
tracer, "ag2.handoff", kind=SpanKind.INTERNAL, attribute_manager=self._attribute_manager
) as span:
span.set_attribute(AgentAttributes.FROM_AGENT, previous_speaker_name)
span.set_attribute(AgentAttributes.TO_AGENT, current_speaker_name)
span.set_attribute(AgentAttributes.AGENT_NAME, current_speaker_name)
span.set_attribute(AgentAttributes.AGENT_ROLE, selected_speaker.__class__.__name__)
system_message = getattr(selected_speaker, "system_message", "")
if system_message:
system_message = self._safe_str(system_message)
span.set_attribute(SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION, system_message)
self._set_llm_config_attributes(span, getattr(selected_speaker, "llm_config", None))
if messages:
for msg in reversed(messages):
if msg.get("name") == current_speaker_name:
if "metadata" in msg and isinstance(msg["metadata"], dict):
meta = msg["metadata"]
if "model" in meta:
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, meta["model"])
break
span.set_attribute("groupchat.role", "participant")
return selected_speaker
return wrapper
# Helper methods
def _safe_str(self, value):
"""Safely convert value to string."""
if value is None:
return ""
return str(value) if not isinstance(value, str) else value
def _extract_message_content(self, message):
"""Extract content from various message formats."""
if isinstance(message, dict):
content = message.get("content", "")
return self._safe_str(content)
elif isinstance(message, str):
return message
else:
return str(message)
def _extract_chat_history(self, span, initiator, recipient):
"""Extract chat history information."""
try:
# Get recipient chat history
recipient_chat_history = getattr(recipient, "chat_history", [])
if recipient_chat_history:
message_count = len(recipient_chat_history)
span.set_attribute("conversation.message_count", message_count)
# Record sample of conversation messages
if message_count > 0:
first_msg = recipient_chat_history[0]
last_msg = recipient_chat_history[-1]
self._set_message_attributes(span, first_msg, 0, "prompt")
self._set_message_attributes(span, last_msg, 0, "completion")
# Check for tool usage
span.set_attribute("chat.used_tools", "tool_calls" in last_msg)
# Capture metadata
if "metadata" in last_msg and isinstance(last_msg["metadata"], dict):
meta = last_msg["metadata"]
if "model" in meta:
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, meta["model"])
except Exception as e:
logger.debug(f"Could not extract chat history: {e}")
def _set_message_attributes(self, span, message, index, prefix):
"""Set message attributes on span."""
if isinstance(message, dict):
role = message.get("role", "unknown")
content = message.get("content", "")
name = message.get("name", "unknown")
span.set_attribute(f"messaging.{prefix}.role.{index}", role)
content = self._safe_str(content)
span.set_attribute(f"messaging.{prefix}.content.{index}", content)
span.set_attribute(f"messaging.{prefix}.speaker.{index}", name)
def _process_tool_result(self, span, result, tool_type):
"""Process and set tool execution result attributes."""
if tool_type == "function" and isinstance(result, tuple) and len(result) > 0:
success = result[0] if isinstance(result[0], bool) else False
span.set_attribute(ToolAttributes.TOOL_STATUS, "success" if success else "error")
if len(result) > 1 and isinstance(result[1], dict):
try:
span.set_attribute(ToolAttributes.TOOL_RESULT, json.dumps(result[1]))
except:
pass
if tool_type == "code" and isinstance(result, tuple) and len(result) >= 3:
exit_code = result[0]
span.set_attribute("exit_code", exit_code)
span.set_attribute(ToolAttributes.TOOL_STATUS, "success" if exit_code == 0 else "error")
if len(result) > 1 and result[1]:
stdout = self._safe_str(result[1])
span.set_attribute("tool.code.stdout", stdout)
if len(result) > 2 and result[2]:
stderr = self._safe_str(result[2])
span.set_attribute("tool.code.stderr", stderr)
def _capture_conversation_summary(self, span, agent, response):
"""Extract and record conversation summary data."""
if not hasattr(response, "chat_history"):
return
try:
chat_history = getattr(response, "chat_history", [])
message_count = len(chat_history)
user_messages = sum(1 for msg in chat_history if msg.get("role") == "user")
assistant_messages = sum(1 for msg in chat_history if msg.get("role") == "assistant")
span.set_attribute("conversation.message_count", message_count)
span.set_attribute("conversation.user_messages", user_messages)
span.set_attribute("conversation.assistant_messages", assistant_messages)
# Set prompts and completions
span.set_attribute(SpanAttributes.LLM_PROMPTS, user_messages)
span.set_attribute(SpanAttributes.LLM_COMPLETIONS, assistant_messages)
if message_count > 0:
for i, msg in enumerate(chat_history[: min(2, message_count)]):
self._set_message_attributes(span, msg, i, "prompt")
if message_count > 2:
self._set_message_attributes(span, chat_history[-1], 0, "completion")
except Exception as e:
logger.error(f"[AG2 DEBUG] Error capturing conversation summary: {e}")
def _capture_group_chat_summary(self, span, manager, result):
"""Extract and record group chat summary data."""
try:
messages = getattr(manager.groupchat, "messages", [])
message_count = len(messages)
agent_message_counts = {}
for message in messages:
agent_name = message.get("name", "unknown")
if agent_name not in agent_message_counts:
agent_message_counts[agent_name] = 0
agent_message_counts[agent_name] += 1
span.set_attribute("conversation.message_count", message_count)
for agent_name, count in agent_message_counts.items():
span.set_attribute(f"conversation.agent_messages.{agent_name}", count)
if hasattr(manager.groupchat, "speaker_selection_method"):
span.set_attribute(
"groupchat.speaker_selection_method", str(manager.groupchat.speaker_selection_method)
)
if message_count > 0:
for i, msg in enumerate(messages[: min(2, message_count)]):
role = msg.get("role", "unknown")
content = msg.get("content", "")
name = msg.get("name", "unknown")
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), role)
content = self._safe_str(content)
span.set_attribute(MessageAttributes.PROMPT_CONTENT.format(i=i), content)
span.set_attribute(MessageAttributes.PROMPT_SPEAKER.format(i=i), name)
if message_count > 2:
last_msg = messages[-1]
role = last_msg.get("role", "unknown")
content = last_msg.get("content", "")
name = last_msg.get("name", "unknown")
span.set_attribute(MessageAttributes.COMPLETION_ROLE.format(i=0), role)
content = self._safe_str(content)
span.set_attribute(MessageAttributes.COMPLETION_CONTENT.format(i=0), content)
span.set_attribute(MessageAttributes.COMPLETION_SPEAKER.format(i=0), name)
if "metadata" in last_msg and isinstance(last_msg["metadata"], dict):
meta = last_msg["metadata"]
if "model" in meta:
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, meta["model"])
except Exception as e:
logger.error(f"Error capturing group chat summary: {e}")
@@ -0,0 +1,19 @@
"""Agno Agent instrumentation package."""
import logging
from agentops.instrumentation.common import LibraryInfo
from .instrumentor import AgnoInstrumentor
logger = logging.getLogger(__name__)
# Library information
_library_info = LibraryInfo(name="agno", default_version="1.5.8")
LIBRARY_NAME = _library_info.name
LIBRARY_VERSION = _library_info.version
__all__ = [
"AgnoInstrumentor",
"LIBRARY_NAME",
"LIBRARY_VERSION",
]
@@ -0,0 +1,20 @@
"""Agno instrumentation attribute handlers."""
from .agent import get_agent_run_attributes
from .metrics import get_metrics_attributes
from .team import get_team_run_attributes
from .tool import get_tool_execution_attributes
from .workflow import get_workflow_run_attributes, get_workflow_session_attributes, get_workflow_cache_attributes
from .storage import get_storage_read_attributes, get_storage_write_attributes
__all__ = [
"get_agent_run_attributes",
"get_metrics_attributes",
"get_team_run_attributes",
"get_tool_execution_attributes",
"get_workflow_run_attributes",
"get_workflow_session_attributes",
"get_workflow_cache_attributes",
"get_storage_read_attributes",
"get_storage_write_attributes",
]
@@ -0,0 +1,250 @@
"""Agno Agent run attributes handler."""
from typing import Optional, Tuple, Dict, Any
from agentops.instrumentation.common.attributes import AttributeMap
from agentops.semconv import SpanAttributes, AgentAttributes, ToolAttributes
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
import json
def get_agent_run_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> AttributeMap:
"""Extract span attributes for Agent.run/arun calls.
Args:
args: Positional arguments passed to the run method (self, message, ...)
kwargs: Keyword arguments passed to the run method
return_value: The return value from the run method (RunResponse)
Returns:
A dictionary of span attributes to be set on the agent span
"""
attributes: AttributeMap = {}
# Initialize variables to avoid UnboundLocalError
agent_name = None
# Base attributes
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.AGENT
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
# AgentOps entity attributes
attributes[SpanAttributes.AGENTOPS_ENTITY_NAME] = "agent"
# Extract agent information from args[0] (self)
if args and len(args) >= 1:
agent = args[0]
# Core agent identification - set directly at root level
if hasattr(agent, "agent_id") and agent.agent_id:
agent_id = str(agent.agent_id)
attributes[AgentAttributes.AGENT_ID] = agent_id
if hasattr(agent, "name") and agent.name:
agent_name = str(agent.name)
attributes[AgentAttributes.AGENT_NAME] = agent_name
if hasattr(agent, "role") and agent.role:
agent_role = str(agent.role)
attributes[AgentAttributes.AGENT_ROLE] = agent_role
# Check if agent is part of a team
if hasattr(agent, "_team") and agent._team:
team = agent._team
if hasattr(team, "name") and team.name:
attributes["agent.parent_team"] = str(team.name)
attributes["agent.parent_team_display"] = f"Under {team.name}"
if hasattr(team, "team_id") and team.team_id:
attributes["agent.parent_team_id"] = str(team.team_id)
# Model information -
if hasattr(agent, "model") and agent.model:
model = agent.model
if hasattr(model, "id"):
model_id = str(model.id)
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_id
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = model_id
if hasattr(model, "provider"):
model_provider = str(model.provider)
attributes["agent.model_provider"] = model_provider
# Agent configuration details - set directly at root level
if hasattr(agent, "description") and agent.description:
attributes["agent.description"] = str(agent.description)
if hasattr(agent, "goal") and agent.goal:
attributes["agent.goal"] = str(agent.goal)
if hasattr(agent, "instructions") and agent.instructions:
if isinstance(agent.instructions, list):
attributes["agent.instruction"] = " | ".join(str(i) for i in agent.instructions)
else:
attributes["agent.instruction"] = str(agent.instructions)
if hasattr(agent, "expected_output") and agent.expected_output:
attributes["agent.expected_output"] = str(agent.expected_output)
if hasattr(agent, "markdown"):
attributes["agent.markdown"] = str(agent.markdown)
if hasattr(agent, "reasoning"):
attributes[AgentAttributes.AGENT_REASONING] = str(agent.reasoning)
if hasattr(agent, "stream"):
attributes["agent.stream"] = str(agent.stream)
if hasattr(agent, "show_tool_calls"):
attributes["agent.show_tool_calls"] = str(agent.show_tool_calls)
if hasattr(agent, "tool_call_limit") and agent.tool_call_limit:
attributes["agent.tool_call_limit"] = str(agent.tool_call_limit)
# Tools information
if hasattr(agent, "tools") and agent.tools:
# Set tool count based on actual number of tools
attributes["agent.tools_count"] = str(len(agent.tools))
# Collect all tool names for the AGENT_TOOLS attribute
tool_names = []
if len(agent.tools) == 1:
# Single tool - set directly at root level
tool = agent.tools[0]
tool_name = None
if hasattr(tool, "name"):
tool_name = str(tool.name)
attributes[ToolAttributes.TOOL_NAME] = tool_name
elif hasattr(tool, "__name__"):
tool_name = str(tool.__name__)
attributes[ToolAttributes.TOOL_NAME] = tool_name
elif callable(tool):
tool_name = getattr(tool, "__name__", "unknown_tool")
attributes[ToolAttributes.TOOL_NAME] = tool_name
if tool_name:
tool_names.append(tool_name)
if hasattr(tool, "description") and tool.description:
attributes[ToolAttributes.TOOL_DESCRIPTION] = str(tool.description)
elif hasattr(tool, "__doc__") and tool.__doc__:
# Fallback to docstring if no description attribute
attributes[ToolAttributes.TOOL_DESCRIPTION] = str(tool.__doc__).strip()
else:
# Multiple tools - use indexed format
for i, tool in enumerate(agent.tools):
tool_name = None
if hasattr(tool, "name"):
tool_name = str(tool.name)
attributes[f"tool.{i}.name"] = tool_name
elif hasattr(tool, "__name__"):
tool_name = str(tool.__name__)
attributes[f"tool.{i}.name"] = tool_name
elif callable(tool):
tool_name = getattr(tool, "__name__", "unknown_tool")
attributes[f"tool.{i}.name"] = tool_name
if tool_name:
tool_names.append(tool_name)
if hasattr(tool, "description") and tool.description:
attributes[f"tool.{i}.description"] = str(tool.description)
elif hasattr(tool, "__doc__") and tool.__doc__:
# Fallback to docstring if no description attribute
attributes[f"tool.{i}.description"] = str(tool.__doc__).strip()
# Set the AGENT_TOOLS attribute with all tool names
if tool_names:
attributes[AgentAttributes.AGENT_TOOLS] = json.dumps(tool_names)
if hasattr(agent, "knowledge") and agent.knowledge:
knowledge_type = type(agent.knowledge).__name__
attributes["agent.knowledge_type"] = knowledge_type
if hasattr(agent, "storage") and agent.storage:
storage_type = type(agent.storage).__name__
attributes["agent.storage_type"] = storage_type
# Session information
if hasattr(agent, "session_id") and agent.session_id:
session_id = str(agent.session_id)
attributes["agent.session_id"] = session_id
if hasattr(agent, "user_id") and agent.user_id:
user_id = str(agent.user_id)
attributes["agent.user_id"] = user_id
# Output key if present
if hasattr(agent, "output_key") and agent.output_key:
attributes["agent.output_key"] = str(agent.output_key)
# Extract run input information
if args and len(args) >= 2:
message = args[1] # The message argument
if message:
message_str = str(message)
attributes["agent.input"] = message_str
# AgentOps entity input
attributes[SpanAttributes.AGENTOPS_ENTITY_INPUT] = message_str
# Extract kwargs information
if kwargs:
if kwargs.get("stream") is not None:
attributes[SpanAttributes.LLM_REQUEST_STREAMING] = str(kwargs["stream"])
if kwargs.get("session_id"):
attributes["agent.run_session_id"] = str(kwargs["session_id"])
if kwargs.get("user_id"):
attributes["agent.run_user_id"] = str(kwargs["user_id"])
# Extract return value information
if return_value:
if hasattr(return_value, "run_id") and return_value.run_id:
run_id = str(return_value.run_id)
attributes["agent.run_id"] = run_id
if hasattr(return_value, "content") and return_value.content:
content = str(return_value.content)
attributes["agent.output"] = content
if hasattr(return_value, "event") and return_value.event:
event = str(return_value.event)
attributes["agent.event"] = event
# Tool executions from the response
if hasattr(return_value, "tools") and return_value.tools:
# Track the number of tool executions
attributes["agent.tool_executions_count"] = str(len(return_value.tools))
for i, tool_exec in enumerate(return_value.tools): # No limit - show all tools
if hasattr(tool_exec, "tool_name") and tool_exec.tool_name:
attributes[f"tool.{i}.name"] = str(tool_exec.tool_name)
if hasattr(tool_exec, "tool_args") and tool_exec.tool_args:
try:
args_str = json.dumps(tool_exec.tool_args)
attributes[f"tool.{i}.parameters"] = args_str
except:
attributes[f"tool.{i}.parameters"] = str(tool_exec.tool_args)
if hasattr(tool_exec, "result") and tool_exec.result:
result_str = str(tool_exec.result)
attributes[f"tool.{i}.result"] = result_str
if hasattr(tool_exec, "tool_call_error") and tool_exec.tool_call_error:
attributes[f"tool.{i}.error"] = str(tool_exec.tool_call_error)
attributes[f"tool.{i}.status"] = "success" # Default to success
# Add display name for better UI visualization
if agent_name:
# Check if we have parent team info
parent_team = attributes.get("agent.parent_team")
if parent_team:
attributes["agent.display_name"] = f"{agent_name} (Agent under {parent_team})"
else:
attributes["agent.display_name"] = f"{agent_name}"
return attributes
@@ -0,0 +1,214 @@
"""Agno Agent session metrics attributes handler."""
from typing import Optional, Tuple, Dict, Any
from agentops.instrumentation.common.attributes import AttributeMap
from agentops.semconv import SpanAttributes
def get_metrics_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> AttributeMap:
"""Extract span attributes for Agent._set_session_metrics calls.
Args:
args: Positional arguments passed to the _set_session_metrics method (self, run_messages)
kwargs: Keyword arguments passed to the _set_session_metrics method
return_value: The return value from the _set_session_metrics method
Returns:
A dictionary of span attributes to be set on the metrics span
"""
attributes: AttributeMap = {}
# Base attributes
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = "llm"
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
attributes[SpanAttributes.AGENTOPS_ENTITY_NAME] = "LLM"
# Initialize usage tracking variables
usage_data = {}
# Initialize counters for indexed messages
prompt_count = 0
completion_count = 0
# Extract agent and run_messages from args (self, run_messages)
if args and len(args) >= 2:
agent = args[0] # self (Agent instance)
run_messages = args[1] # RunMessages object
# Add agent display name for LLM calls
if hasattr(agent, "name") and agent.name:
attributes["agno.llm.display_name"] = f"{agent.name} → LLM"
# Model information - get additional request parameters if available
if hasattr(agent, "model") and agent.model:
model = agent.model
# Set model ID first
if hasattr(model, "id"):
attributes[SpanAttributes.LLM_REQUEST_MODEL] = str(model.id)
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = str(model.id)
# Additional model parameters
if hasattr(model, "temperature") and model.temperature is not None:
attributes[SpanAttributes.LLM_REQUEST_TEMPERATURE] = str(model.temperature)
if hasattr(model, "max_tokens") and model.max_tokens is not None:
attributes[SpanAttributes.LLM_REQUEST_MAX_TOKENS] = str(model.max_tokens)
if hasattr(model, "top_p") and model.top_p is not None:
attributes[SpanAttributes.LLM_REQUEST_TOP_P] = str(model.top_p)
if hasattr(model, "provider"):
attributes["agno.model.provider"] = str(model.provider)
# Add model class name for better identification (with null check)
if hasattr(model, "__class__") and hasattr(model.__class__, "__name__"):
model_class = model.__class__.__name__
attributes["agno.model.class"] = model_class
if hasattr(run_messages, "messages") and run_messages.messages:
messages = run_messages.messages
# Initialize token tracking
total_prompt_tokens = 0
total_completion_tokens = 0
total_output_tokens = 0
total_input_tokens = 0
total_tokens = 0
total_time = 0.0
# Process messages to create individual indexed gen_ai.prompt.{i} and gen_ai.completion.{i} attributes
for i, msg in enumerate(messages):
# Extract message content for prompts/completions
if hasattr(msg, "role") and hasattr(msg, "content"):
# Only process messages with actual content
if msg.content is not None and str(msg.content).strip() != "" and str(msg.content) != "None":
content = str(msg.content)
# No truncation - keep full content for observability
if msg.role == "user":
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.role"] = "user"
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.content"] = content
prompt_count += 1
elif msg.role == "assistant":
attributes[f"{SpanAttributes.LLM_COMPLETIONS}.{completion_count}.role"] = "assistant"
attributes[f"{SpanAttributes.LLM_COMPLETIONS}.{completion_count}.content"] = content
completion_count += 1
elif msg.role == "system":
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.role"] = "system"
attributes[f"{SpanAttributes.LLM_PROMPTS}.{prompt_count}.content"] = content
prompt_count += 1
# Extract token metrics from message
if hasattr(msg, "metrics") and msg.metrics:
metrics = msg.metrics
# Handle different token metric patterns
if hasattr(metrics, "prompt_tokens") and metrics.prompt_tokens > 0:
total_prompt_tokens += metrics.prompt_tokens
if hasattr(metrics, "completion_tokens") and metrics.completion_tokens > 0:
total_completion_tokens += metrics.completion_tokens
if hasattr(metrics, "total_tokens") and metrics.total_tokens > 0:
total_tokens += metrics.total_tokens
# For messages that only have output_tokens
if hasattr(metrics, "output_tokens") and metrics.output_tokens > 0:
total_output_tokens += metrics.output_tokens
if hasattr(metrics, "input_tokens") and metrics.input_tokens > 0:
total_input_tokens += metrics.input_tokens
if hasattr(metrics, "time") and metrics.time:
total_time += metrics.time
# Token metrics from agent session metrics
if hasattr(agent, "session_metrics") and agent.session_metrics:
session_metrics = agent.session_metrics
# Try to get model name from session metrics if not already set
if SpanAttributes.LLM_REQUEST_MODEL not in attributes:
if hasattr(session_metrics, "model") and session_metrics.model:
model_id = str(session_metrics.model)
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_id
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = model_id
# Only set token variables if the attributes actually exist
session_prompt_tokens = None
session_completion_tokens = None
session_output_tokens = None
session_input_tokens = None
session_total_tokens = None
if hasattr(session_metrics, "prompt_tokens"):
session_prompt_tokens = session_metrics.prompt_tokens
if hasattr(session_metrics, "completion_tokens"):
session_completion_tokens = session_metrics.completion_tokens
if hasattr(session_metrics, "output_tokens"):
session_output_tokens = session_metrics.output_tokens
if hasattr(session_metrics, "input_tokens"):
session_input_tokens = session_metrics.input_tokens
if hasattr(session_metrics, "total_tokens"):
session_total_tokens = session_metrics.total_tokens
# For Anthropic, output_tokens represents completion tokens
if session_output_tokens is not None and session_output_tokens > 0:
if session_completion_tokens is None or session_completion_tokens == 0:
session_completion_tokens = session_output_tokens
# For some providers, input_tokens represents prompt tokens
if session_input_tokens is not None and session_input_tokens > 0:
if session_prompt_tokens is None or session_prompt_tokens == 0:
session_prompt_tokens = session_input_tokens
# Only set token attributes if we have actual values
if session_total_tokens is not None and session_total_tokens > 0:
usage_data["total_tokens"] = session_total_tokens
# Set breakdown if available
if session_prompt_tokens is not None and session_prompt_tokens > 0:
usage_data["prompt_tokens"] = session_prompt_tokens
if session_completion_tokens is not None and session_completion_tokens > 0:
usage_data["completion_tokens"] = session_completion_tokens
# Additional token types from session metrics - only set if present
if hasattr(session_metrics, "cached_tokens") and session_metrics.cached_tokens > 0:
usage_data["cache_read_input_tokens"] = session_metrics.cached_tokens
if hasattr(session_metrics, "reasoning_tokens") and session_metrics.reasoning_tokens > 0:
usage_data["reasoning_tokens"] = session_metrics.reasoning_tokens
# If we don't have token data from session metrics, try message aggregation
if "total_tokens" not in usage_data:
# Set aggregated token usage from messages
if total_prompt_tokens > 0 or total_input_tokens > 0:
usage_data["prompt_tokens"] = total_prompt_tokens or total_input_tokens
if total_completion_tokens > 0 or total_output_tokens > 0:
usage_data["completion_tokens"] = total_completion_tokens or total_output_tokens
if total_tokens > 0:
usage_data["total_tokens"] = total_tokens
# Extract user message info if available
if hasattr(run_messages, "user_message") and run_messages.user_message:
user_msg = run_messages.user_message
if hasattr(user_msg, "content"):
content = str(user_msg.content)
attributes["agno.metrics.user_input"] = content
# Set individual LLM usage attributes only for values we actually have
if "prompt_tokens" in usage_data:
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = usage_data["prompt_tokens"]
if "completion_tokens" in usage_data:
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = usage_data["completion_tokens"]
if "total_tokens" in usage_data:
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = usage_data["total_tokens"]
if "cache_read_input_tokens" in usage_data:
attributes[SpanAttributes.LLM_USAGE_CACHE_READ_INPUT_TOKENS] = usage_data["cache_read_input_tokens"]
if "reasoning_tokens" in usage_data:
attributes[SpanAttributes.LLM_USAGE_REASONING_TOKENS] = usage_data["reasoning_tokens"]
# But only if we have any usage data
if usage_data:
for key, value in usage_data.items():
attributes[f"gen_ai.usage.{key}"] = value
return attributes
@@ -0,0 +1,158 @@
"""Storage operation attribute handlers for Agno workflow instrumentation."""
import json
from typing import Any, Dict, Optional, Tuple
from opentelemetry.util.types import AttributeValue
from agentops.semconv.span_attributes import SpanAttributes
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
from agentops.instrumentation.common.attributes import get_common_attributes
def get_storage_read_attributes(
args: Tuple[Any, ...] = (),
kwargs: Optional[Dict[str, Any]] = None,
return_value: Optional[Any] = None,
) -> Dict[str, AttributeValue]:
"""Extract attributes from storage read operations.
Args:
args: Positional arguments passed to read_from_storage
kwargs: Keyword arguments passed to read_from_storage
return_value: Return value from read_from_storage (the cached data or None)
Returns:
Dictionary of OpenTelemetry attributes for storage read operations
"""
attributes = get_common_attributes()
kwargs = kwargs or {}
# Mark this as a storage operation within workflow context
attributes["storage.operation"] = "read"
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.WORKFLOW
if args and len(args) > 0:
workflow = args[0]
# Get workflow information
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
attributes["storage.workflow_id"] = str(workflow.workflow_id)
if hasattr(workflow, "session_id") and workflow.session_id:
attributes["storage.session_id"] = str(workflow.session_id)
# Get storage type
if hasattr(workflow, "storage") and workflow.storage:
storage_type = type(workflow.storage).__name__
attributes["storage.backend"] = storage_type
# Get session state info for context
if hasattr(workflow, "session_state") and isinstance(workflow.session_state, dict):
# Get all cache keys
cache_keys = list(workflow.session_state.keys())
attributes["storage.cache_size"] = len(cache_keys)
if cache_keys:
attributes["storage.cache_keys"] = json.dumps(cache_keys)
# Analyze the return value to determine cache hit/miss
if return_value is not None:
# Cache hit
attributes["storage.cache_hit"] = True
attributes["storage.result"] = "hit"
# Get data type and size
data_type = type(return_value).__name__
attributes["storage.data_type"] = data_type
# For dict/list, show structure
if isinstance(return_value, dict):
attributes["storage.data_keys"] = json.dumps(list(return_value.keys()))
attributes["storage.data_size"] = len(return_value)
elif isinstance(return_value, (list, tuple)):
attributes["storage.data_size"] = len(return_value)
elif isinstance(return_value, str):
attributes["storage.data_size"] = len(return_value)
# Show full string data without truncation
attributes["storage.data_preview"] = return_value
else:
# Cache miss
attributes["storage.cache_hit"] = False
attributes["storage.result"] = "miss"
return attributes
def get_storage_write_attributes(
args: Tuple[Any, ...] = (),
kwargs: Optional[Dict[str, Any]] = None,
return_value: Optional[Any] = None,
) -> Dict[str, AttributeValue]:
"""Extract attributes from storage write operations.
Args:
args: Positional arguments passed to write_to_storage
kwargs: Keyword arguments passed to write_to_storage
return_value: Return value from write_to_storage (usually None or success indicator)
Returns:
Dictionary of OpenTelemetry attributes for storage write operations
"""
attributes = get_common_attributes()
kwargs = kwargs or {}
# Mark this as a storage operation within workflow context
attributes["storage.operation"] = "write"
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.WORKFLOW
if args and len(args) > 0:
workflow = args[0]
# Get workflow information
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
attributes["storage.workflow_id"] = str(workflow.workflow_id)
if hasattr(workflow, "session_id") and workflow.session_id:
attributes["storage.session_id"] = str(workflow.session_id)
# Get storage type
if hasattr(workflow, "storage") and workflow.storage:
storage_type = type(workflow.storage).__name__
attributes["storage.backend"] = storage_type
# Get session state info to see what's being written
if hasattr(workflow, "session_state") and isinstance(workflow.session_state, dict):
# Get cache state after write
cache_keys = list(workflow.session_state.keys())
attributes["storage.cache_size"] = len(cache_keys)
if cache_keys:
attributes["storage.cache_keys"] = json.dumps(cache_keys)
# Try to identify what was written (the newest/changed data)
# This is a heuristic - in practice you might need to track state changes
if cache_keys:
# Show the last key as likely the one just written
last_key = cache_keys[-1]
attributes["storage.written_key"] = last_key
# Get value preview
value = workflow.session_state.get(last_key)
if value is not None:
value_type = type(value).__name__
attributes["storage.written_value_type"] = value_type
if isinstance(value, str):
if len(value) > 100:
attributes["storage.written_value_preview"] = value[:100] + "..."
else:
attributes["storage.written_value_preview"] = value
attributes["storage.written_value_size"] = len(value)
elif isinstance(value, (dict, list)):
attributes["storage.written_value_size"] = len(value)
attributes["storage.written_value_preview"] = f"{value_type} with {len(value)} items"
# Check write result
if return_value is not None:
attributes["storage.write_success"] = True
else:
# Most storage writes return None on success, so this is normal
attributes["storage.write_success"] = True
return attributes
@@ -0,0 +1,195 @@
"""Agno Team run attributes handler."""
from typing import Optional, Tuple, Dict, Any
import json
from agentops.instrumentation.common.attributes import AttributeMap
from agentops.semconv import SpanAttributes, WorkflowAttributes
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
def get_team_run_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> AttributeMap:
"""Extract span attributes for Team method calls (both internal and public).
Args:
args: Positional arguments passed to the Team method
kwargs: Keyword arguments passed to the Team method
return_value: The return value from the Team method
Returns:
A dictionary of span attributes to be set on the workflow span
"""
attributes: AttributeMap = {}
# Base attributes
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.WORKFLOW
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
attributes[WorkflowAttributes.WORKFLOW_TYPE] = "team_run"
# Extract team information from instance
if args and len(args) > 0:
team = args[0] # self (Team instance)
# Team identification
if hasattr(team, "name") and team.name:
attributes["team.name"] = str(team.name)
attributes["team.display_name"] = f"{team.name} (Team)"
if hasattr(team, "team_id") and team.team_id:
attributes["team.team_id"] = str(team.team_id)
if hasattr(team, "mode") and team.mode:
attributes["team.mode"] = str(team.mode)
if hasattr(team, "members") and team.members:
attributes["team.members_count"] = str(len(team.members))
# Add detailed member information
member_agents = []
for i, member in enumerate(team.members):
member_info = {}
if hasattr(member, "name") and member.name:
member_info["name"] = str(member.name)
if hasattr(member, "agent_id") and member.agent_id:
member_info["id"] = str(member.agent_id)
if hasattr(member, "role") and member.role:
member_info["role"] = str(member.role)
if hasattr(member, "model") and member.model:
if hasattr(member.model, "id"):
member_info["model"] = str(member.model.id)
# Add member info to list
if member_info:
member_agents.append(member_info)
# Also add individual member attributes
for key, value in member_info.items():
attributes[f"team.member.{i}.{key}"] = value
# Add aggregated member list
if member_agents:
try:
attributes["team.members"] = json.dumps(member_agents)
# Also add a simple list of member names
member_names = [m.get("name", "Unknown") for m in member_agents]
attributes["team.member_names"] = ", ".join(member_names)
except:
attributes["team.members"] = str(member_agents)
# Process input arguments - handle both internal and public method signatures
if args and len(args) >= 2:
input_arg = args[1]
# Check if it's internal method (has run_messages) or public method (direct message)
if hasattr(input_arg, "messages"):
# Internal method: args[1] is run_messages
run_messages = input_arg
# Get the user message for workflow input
user_messages = [msg for msg in run_messages.messages if hasattr(msg, "role") and msg.role == "user"]
if user_messages:
last_user_msg = user_messages[-1]
if hasattr(last_user_msg, "content"):
attributes[WorkflowAttributes.WORKFLOW_INPUT] = str(last_user_msg.content)
attributes[WorkflowAttributes.WORKFLOW_INPUT_TYPE] = "message"
# Count total messages
attributes["team.messages_count"] = str(len(run_messages.messages))
else:
# Public method: args[1] is the message directly
message = input_arg
if message is not None:
if isinstance(message, str):
message_content = message
elif hasattr(message, "content"):
message_content = str(message.content)
elif hasattr(message, "get_content_string"):
message_content = message.get_content_string()
else:
message_content = str(message)
attributes[WorkflowAttributes.WORKFLOW_INPUT] = message_content
attributes[WorkflowAttributes.WORKFLOW_INPUT_TYPE] = "message"
# Process keyword arguments
if kwargs:
if kwargs.get("user_id"):
attributes[SpanAttributes.LLM_USER] = kwargs["user_id"]
if kwargs.get("session_id"):
attributes["team.session_id"] = kwargs["session_id"]
if kwargs.get("response_format"):
attributes["team.response_format"] = str(type(kwargs["response_format"]).__name__)
if kwargs.get("stream"):
attributes["team.streaming"] = str(kwargs["stream"])
if kwargs.get("stream_intermediate_steps"):
attributes["team.stream_intermediate_steps"] = str(kwargs["stream_intermediate_steps"])
if kwargs.get("retries"):
attributes["team.retries"] = str(kwargs["retries"])
# Media attachments
if kwargs.get("audio"):
attributes["team.has_audio"] = "true"
if kwargs.get("images"):
attributes["team.has_images"] = "true"
if kwargs.get("videos"):
attributes["team.has_videos"] = "true"
if kwargs.get("files"):
attributes["team.has_files"] = "true"
if kwargs.get("knowledge_filters"):
attributes["team.has_knowledge_filters"] = "true"
# Process return value (TeamRunResponse or Iterator)
if return_value:
# Handle both single response and iterator
if hasattr(return_value, "__iter__") and not isinstance(return_value, str):
# It's an iterator for streaming
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = "team_run_response_stream"
attributes["team.is_streaming"] = "true"
else:
# Non-streaming response
if hasattr(return_value, "content"):
# It's a TeamRunResponse with content
content = str(return_value.content)
attributes[WorkflowAttributes.WORKFLOW_OUTPUT] = content
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = "team_run_response"
else:
# Unknown return type or response without content
output = str(return_value)
attributes[WorkflowAttributes.WORKFLOW_OUTPUT] = output
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = type(return_value).__name__
# Set additional team response attributes (for both streaming and non-streaming)
if hasattr(return_value, "run_id"):
attributes["team.run_id"] = str(return_value.run_id)
if hasattr(return_value, "session_id"):
attributes["team.response_session_id"] = str(return_value.session_id)
if hasattr(return_value, "team_id"):
attributes["team.response_team_id"] = str(return_value.team_id)
if hasattr(return_value, "model"):
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = str(return_value.model)
if hasattr(return_value, "model_provider"):
attributes["team.model_provider"] = str(return_value.model_provider)
if hasattr(return_value, "event"):
attributes["team.event"] = str(return_value.event)
# Team-specific attributes
if hasattr(return_value, "content_type"):
attributes["team.response_content_type"] = str(return_value.content_type)
return attributes
# Keep the public function as an alias for backward compatibility
get_team_public_run_attributes = get_team_run_attributes
@@ -0,0 +1,210 @@
"""Agno tool execution attributes handler."""
import json
from typing import Optional, Tuple, Dict, Any
import time
from agentops.instrumentation.common.attributes import AttributeMap
from agentops.semconv import SpanAttributes
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
from agentops.semconv.tool import ToolAttributes
def get_tool_execution_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> AttributeMap:
"""Extract span attributes for tool execution calls (FunctionCall.execute/aexecute).
Args:
args: Positional arguments passed to the execute method (self)
kwargs: Keyword arguments passed to the execute method
return_value: The return value from the execute method (FunctionExecutionResult)
Returns:
A dictionary of span attributes to be set on the tool execution span
"""
attributes: AttributeMap = {}
# Base attributes
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKind.TOOL
attributes[SpanAttributes.LLM_SYSTEM] = "agno"
# AgentOps entity attributes
attributes[SpanAttributes.AGENTOPS_ENTITY_NAME] = "tool"
# Process the FunctionCall object (self in execute method)
if args and len(args) > 0:
function_call = args[0]
# Add detailed function call information
attributes["tool.function_call_type"] = str(type(function_call).__name__)
# Extract tool information
if hasattr(function_call, "function") and function_call.function:
function = function_call.function
# Get function name and add display name
if hasattr(function, "__name__"):
func_name = function.__name__
attributes["tool.function_name"] = func_name
attributes["tool.display_name"] = f"{func_name} (Tool)"
tool_name = getattr(function, "name", "unknown_tool")
# Set span attributes for the tool execution span
attributes[ToolAttributes.TOOL_NAME] = tool_name
attributes["tool.function_name"] = tool_name
# Function details and context
if hasattr(function, "description"):
description = getattr(function, "description", "")
if description:
attributes[ToolAttributes.TOOL_DESCRIPTION] = description
attributes["tool.function_description"] = description
# Function source information
if hasattr(function, "entrypoint") and function.entrypoint:
entrypoint = function.entrypoint
if hasattr(entrypoint, "__module__"):
attributes["tool.function_module"] = str(entrypoint.__module__)
if hasattr(entrypoint, "__name__"):
attributes["tool.function_method"] = str(entrypoint.__name__)
if hasattr(entrypoint, "__qualname__"):
attributes["tool.function_qualname"] = str(entrypoint.__qualname__)
# Tool capabilities
if hasattr(function, "requires_confirmation"):
attributes["tool.requires_confirmation"] = str(function.requires_confirmation)
if hasattr(function, "show_result"):
attributes["tool.show_result"] = str(function.show_result)
if hasattr(function, "stop_after_tool_call"):
attributes["tool.stop_after_tool_call"] = str(function.stop_after_tool_call)
# Extract tool arguments with better formatting
if hasattr(function_call, "arguments") and function_call.arguments:
try:
if isinstance(function_call.arguments, str):
args_dict = json.loads(function_call.arguments)
else:
args_dict = function_call.arguments
# Format arguments nicely
formatted_args = []
for key, value in args_dict.items():
value_str = str(value)
formatted_args.append(f"{key}={value_str}")
attributes[ToolAttributes.TOOL_PARAMETERS] = json.dumps(args_dict)
attributes["tool.formatted_args"] = ", ".join(formatted_args)
attributes["tool.args_count"] = str(len(args_dict))
except Exception as e:
attributes[ToolAttributes.TOOL_PARAMETERS] = str(function_call.arguments)
attributes["tool.args_parse_error"] = str(e)
# Extract call ID and metadata
if hasattr(function_call, "tool_call_id"):
attributes["tool.call_id"] = str(function_call.tool_call_id)
# Check for any agent context
if hasattr(function_call, "_agent") and function_call._agent:
agent = function_call._agent
if hasattr(agent, "name"):
attributes["tool.calling_agent_name"] = str(agent.name)
if hasattr(agent, "agent_id"):
attributes["tool.calling_agent_id"] = str(agent.agent_id)
# Process return value
if return_value is not None:
# Add timing information
attributes["tool.execution_timestamp"] = str(int(time.time() * 1000))
# Determine execution status and result information
if hasattr(return_value, "value"):
# FunctionExecutionResult with value
result_value = return_value.value
attributes["tool.execution_status"] = "success"
else:
# Direct return value
result_value = return_value
attributes["tool.execution_status"] = "success"
# Process result value
if result_value is not None:
result_type = type(result_value).__name__
attributes["tool.execution_result_status"] = str(result_type)
# Handle FunctionExecutionResult objects specifically
if hasattr(result_value, "status") and hasattr(result_value, "result"):
# This looks like a FunctionExecutionResult
status = getattr(result_value, "status", "unknown")
actual_result = getattr(result_value, "result", None)
error = getattr(result_value, "error", None)
attributes["tool.execution_result_status"] = str(status)
attributes[ToolAttributes.TOOL_STATUS] = str(status)
if error:
attributes["tool.execution_error"] = str(error)
attributes["tool.error"] = str(error)
if actual_result is not None:
actual_result_type = type(actual_result).__name__
attributes["tool.actual_result_type"] = actual_result_type
# Enhanced generator handling
if hasattr(actual_result, "__iter__") and hasattr(actual_result, "__next__"):
attributes["tool.result_is_generator"] = "true"
# Try to get more meaningful information about the generator
generator_info = []
# Get function name from the generator
if hasattr(actual_result, "gi_code"):
func_name = actual_result.gi_code.co_name
attributes["tool.generator_function"] = func_name
generator_info.append(f"function={func_name}")
if generator_info:
result_str = f"Generator<{actual_result_type}>({', '.join(generator_info)})"
else:
result_str = f"Generator<{actual_result_type}> - {str(actual_result)}"
else:
# Regular result
result_str = str(actual_result)
else:
result_str = str(status)
else:
# Not a FunctionExecutionResult, handle as direct result
if hasattr(result_value, "__iter__") and hasattr(result_value, "__next__"):
# It's a generator
attributes["tool.result_is_generator"] = "true"
if hasattr(result_value, "gi_code"):
func_name = result_value.gi_code.co_name
attributes["tool.generator_function"] = func_name
result_str = f"Generator<{result_type}> function={func_name} - {str(result_value)}"
else:
result_str = f"Generator<{result_type}> - {str(result_value)}"
else:
# Regular result
result_str = str(result_value)
else:
result_str = "None"
# Set the main result attribute
attributes[ToolAttributes.TOOL_RESULT] = result_str
# Add additional analysis attributes
attributes["tool.result_length"] = str(len(result_str))
# Set final execution status
if not attributes.get(ToolAttributes.TOOL_STATUS):
attributes[ToolAttributes.TOOL_STATUS] = "success"
# Add execution summary for debugging
tool_name = attributes.get(ToolAttributes.TOOL_NAME, "unknown")
call_type = attributes.get("tool.transfer_type", "unknown")
attributes["tool.execution_summary"] = f"Tool '{tool_name}' executed with type '{call_type}'"
return attributes
@@ -0,0 +1,254 @@
"""Workflow attribute extraction for agno workflow instrumentation."""
from typing import Any, Dict, Optional, Tuple
from opentelemetry.util.types import AttributeValue
import json
from agentops.semconv.instrumentation import InstrumentationAttributes
from agentops.semconv.span_kinds import SpanKind as AgentOpsSpanKind
from agentops.semconv.workflow import WorkflowAttributes
from agentops.instrumentation.common.attributes import get_common_attributes
def get_workflow_run_attributes(
args: Tuple[Any, ...] = (),
kwargs: Optional[Dict[str, Any]] = None,
return_value: Optional[Any] = None,
) -> Dict[str, AttributeValue]:
"""Extract attributes from workflow run operations.
Args:
args: Positional arguments passed to the workflow run method
kwargs: Keyword arguments passed to the workflow run method
return_value: Return value from the workflow run method
Returns:
Dictionary of OpenTelemetry attributes for workflow runs
"""
attributes = get_common_attributes()
kwargs = kwargs or {}
if args and len(args) > 0:
workflow = args[0]
# Core workflow attributes
if hasattr(workflow, "name") and workflow.name:
attributes[WorkflowAttributes.WORKFLOW_NAME] = str(workflow.name)
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
attributes[WorkflowAttributes.WORKFLOW_ID] = str(workflow.workflow_id)
if hasattr(workflow, "description") and workflow.description:
attributes[WorkflowAttributes.WORKFLOW_DESCRIPTION] = str(workflow.description)
if hasattr(workflow, "app_id") and workflow.app_id:
attributes[WorkflowAttributes.WORKFLOW_APP_ID] = str(workflow.app_id)
# Set workflow type
attributes[WorkflowAttributes.WORKFLOW_TYPE] = "agno_workflow"
# Session and user attributes
if hasattr(workflow, "session_id") and workflow.session_id:
attributes[WorkflowAttributes.WORKFLOW_SESSION_ID] = str(workflow.session_id)
if hasattr(workflow, "session_name") and workflow.session_name:
attributes[WorkflowAttributes.WORKFLOW_SESSION_NAME] = str(workflow.session_name)
if hasattr(workflow, "user_id") and workflow.user_id:
attributes[WorkflowAttributes.WORKFLOW_USER_ID] = str(workflow.user_id)
# Run-specific attributes
if hasattr(workflow, "run_id") and workflow.run_id:
attributes[WorkflowAttributes.WORKFLOW_RUN_ID] = str(workflow.run_id)
# Configuration attributes
if hasattr(workflow, "debug_mode"):
attributes[WorkflowAttributes.WORKFLOW_DEBUG_MODE] = bool(workflow.debug_mode)
if hasattr(workflow, "monitoring"):
attributes[WorkflowAttributes.WORKFLOW_MONITORING] = bool(workflow.monitoring)
if hasattr(workflow, "telemetry"):
attributes[WorkflowAttributes.WORKFLOW_TELEMETRY] = bool(workflow.telemetry)
# Memory and storage attributes
if hasattr(workflow, "memory") and workflow.memory:
memory_type = type(workflow.memory).__name__
attributes[WorkflowAttributes.WORKFLOW_MEMORY_TYPE] = memory_type
if hasattr(workflow, "storage") and workflow.storage:
storage_type = type(workflow.storage).__name__
attributes[WorkflowAttributes.WORKFLOW_STORAGE_TYPE] = storage_type
# Input parameters from kwargs
if kwargs:
# Store workflow input
attributes[WorkflowAttributes.WORKFLOW_INPUT] = str(kwargs)
# Count and types of input parameters
attributes[WorkflowAttributes.WORKFLOW_INPUT_PARAMETER_COUNT] = len(kwargs)
param_types = list(set(type(v).__name__ for v in kwargs.values()))
if param_types:
attributes[WorkflowAttributes.WORKFLOW_INPUT_TYPE] = str(param_types)
# Store specific input keys (without values for privacy)
input_keys = list(kwargs.keys())
if input_keys:
attributes[WorkflowAttributes.WORKFLOW_INPUT_PARAMETER_KEYS] = str(input_keys)
# Workflow method parameters if available
if hasattr(workflow, "_run_parameters") and workflow._run_parameters:
param_count = len(workflow._run_parameters)
attributes[WorkflowAttributes.WORKFLOW_METHOD_PARAMETER_COUNT] = param_count
if hasattr(workflow, "_run_return_type") and workflow._run_return_type:
attributes[WorkflowAttributes.WORKFLOW_METHOD_RETURN_TYPE] = str(workflow._run_return_type)
# Process return value attributes
if return_value is not None:
return_type = type(return_value).__name__
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TYPE] = return_type
# Handle RunResponse objects
if hasattr(return_value, "content"):
# Store workflow output
if return_value.content:
attributes[WorkflowAttributes.WORKFLOW_OUTPUT] = str(return_value.content)
if hasattr(return_value, "content_type"):
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_CONTENT_TYPE] = str(return_value.content_type)
if hasattr(return_value, "event"):
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_EVENT] = str(return_value.event)
if hasattr(return_value, "model"):
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_MODEL] = (
str(return_value.model) if return_value.model else ""
)
if hasattr(return_value, "model_provider"):
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_MODEL_PROVIDER] = (
str(return_value.model_provider) if return_value.model_provider else ""
)
# Count various response components
if hasattr(return_value, "messages") and return_value.messages:
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_MESSAGE_COUNT] = len(return_value.messages)
if hasattr(return_value, "tools") and return_value.tools:
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_TOOL_COUNT] = len(return_value.tools)
if hasattr(return_value, "images") and return_value.images:
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_IMAGE_COUNT] = len(return_value.images)
if hasattr(return_value, "videos") and return_value.videos:
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_VIDEO_COUNT] = len(return_value.videos)
if hasattr(return_value, "audio") and return_value.audio:
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_AUDIO_COUNT] = len(return_value.audio)
# Handle generators/iterators
elif hasattr(return_value, "__iter__") and not isinstance(return_value, (str, bytes)):
attributes[WorkflowAttributes.WORKFLOW_OUTPUT_IS_STREAMING] = True
# Set span kind - AgentOpsSpanKind.WORKFLOW is already a string
attributes[InstrumentationAttributes.INSTRUMENTATION_TYPE] = AgentOpsSpanKind.WORKFLOW
return attributes
def get_workflow_session_attributes(
args: Tuple[Any, ...] = (),
kwargs: Optional[Dict[str, Any]] = None,
return_value: Optional[Any] = None,
) -> Dict[str, AttributeValue]:
"""Extract attributes from workflow session operations.
Args:
args: Positional arguments passed to the session method
kwargs: Keyword arguments passed to the session method
return_value: Return value from the session method
Returns:
Dictionary of OpenTelemetry attributes for workflow sessions
"""
attributes = get_common_attributes()
kwargs = kwargs or {}
if args and len(args) > 0:
workflow = args[0]
# Session attributes
if hasattr(workflow, "session_id") and workflow.session_id:
attributes[WorkflowAttributes.WORKFLOW_SESSION_ID] = str(workflow.session_id)
if hasattr(workflow, "session_name") and workflow.session_name:
attributes[WorkflowAttributes.WORKFLOW_SESSION_NAME] = str(workflow.session_name)
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
attributes[WorkflowAttributes.WORKFLOW_SESSION_WORKFLOW_ID] = str(workflow.workflow_id)
if hasattr(workflow, "user_id") and workflow.user_id:
attributes[WorkflowAttributes.WORKFLOW_SESSION_USER_ID] = str(workflow.user_id)
# Session state attributes
if hasattr(workflow, "session_state") and workflow.session_state:
if isinstance(workflow.session_state, dict):
attributes[WorkflowAttributes.WORKFLOW_SESSION_STATE_KEYS] = str(list(workflow.session_state.keys()))
attributes[WorkflowAttributes.WORKFLOW_SESSION_STATE_SIZE] = len(workflow.session_state)
# Storage attributes
if hasattr(workflow, "storage") and workflow.storage:
storage_type = type(workflow.storage).__name__
attributes[WorkflowAttributes.WORKFLOW_SESSION_STORAGE_TYPE] = storage_type
# Process session return value if it's a WorkflowSession
if return_value is not None and hasattr(return_value, "session_id"):
attributes[WorkflowAttributes.WORKFLOW_SESSION_RETURNED_SESSION_ID] = str(return_value.session_id)
if hasattr(return_value, "created_at") and return_value.created_at:
attributes[WorkflowAttributes.WORKFLOW_SESSION_CREATED_AT] = int(return_value.created_at)
if hasattr(return_value, "updated_at") and return_value.updated_at:
attributes[WorkflowAttributes.WORKFLOW_SESSION_UPDATED_AT] = int(return_value.updated_at)
# Set span kind - AgentOpsSpanKind.WORKFLOW is already a string
attributes[InstrumentationAttributes.INSTRUMENTATION_TYPE] = AgentOpsSpanKind.WORKFLOW
return attributes
def get_workflow_cache_attributes(
args: Tuple[Any, ...] = (),
kwargs: Optional[Dict[str, Any]] = None,
return_value: Optional[Any] = None,
) -> Dict[str, AttributeValue]:
"""Extract attributes from workflow cache operations.
Args:
args: Positional arguments passed to the cache method
kwargs: Keyword arguments passed to the cache method
return_value: Return value from the cache method
Returns:
Dictionary of OpenTelemetry attributes for cache operations
"""
attributes = get_common_attributes()
kwargs = kwargs or {}
if args and len(args) > 0:
workflow = args[0]
# Get workflow information
if hasattr(workflow, "workflow_id") and workflow.workflow_id:
attributes["cache.workflow_id"] = str(workflow.workflow_id)
if hasattr(workflow, "session_id") and workflow.session_id:
attributes["cache.session_id"] = str(workflow.session_id)
# Get cache state
if hasattr(workflow, "session_state") and isinstance(workflow.session_state, dict):
attributes["cache.size"] = len(workflow.session_state)
attributes["cache.keys"] = json.dumps(list(workflow.session_state.keys()))
# Determine cache operation type and result
if len(args) > 1:
cache_key = str(args[1])
attributes["cache.key"] = cache_key
if return_value is not None:
attributes["cache.hit"] = True
attributes["cache.result"] = "hit"
# Add value info
if isinstance(return_value, str):
attributes["cache.value_size"] = len(return_value)
if len(return_value) <= 100:
attributes["cache.value"] = return_value
else:
attributes["cache.value_preview"] = return_value[:100] + "..."
else:
attributes["cache.hit"] = False
attributes["cache.result"] = "miss"
return attributes
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2023 openllmetry
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
@@ -0,0 +1,10 @@
This package contains code derived from the OpenLLMetry project, which is licensed under the Apache License, Version 2.0.
Original repository: https://github.com/traceloop/openllmetry
Copyright notice from the original project:
Copyright (c) Traceloop (https://traceloop.com)
The Apache 2.0 license can be found in the LICENSE file in this directory.
This code has been modified and adapted for use in the AgentOps project.
@@ -0,0 +1,6 @@
"""OpenTelemetry CrewAI instrumentation"""
from agentops.instrumentation.agentic.crewai.version import __version__
from agentops.instrumentation.agentic.crewai.instrumentation import CrewaiInstrumentor
__all__ = ["CrewaiInstrumentor", "__version__"]
@@ -0,0 +1,335 @@
"""OpenTelemetry instrumentation for CrewAI."""
import json
import logging
from typing import Any
from opentelemetry.trace import Span
from agentops.semconv.span_attributes import SpanAttributes
from agentops.semconv.agent import AgentAttributes
from agentops.semconv.message import MessageAttributes
# Initialize logger for logging potential issues and operations
logger = logging.getLogger(__name__)
def _parse_tools(tools):
"""Parse tools into a JSON string with name and description."""
result = []
for tool in tools:
res = {}
if hasattr(tool, "name") and tool.name is not None:
res["name"] = tool.name
if hasattr(tool, "description") and tool.description is not None:
res["description"] = tool.description
if res:
result.append(res)
return result
def set_span_attribute(span: Span, key: str, value: Any) -> None:
"""Set a single attribute on a span."""
if value is not None and value != "":
if hasattr(value, "__str__"):
value = str(value)
span.set_attribute(key, value)
class CrewAISpanAttributes:
"""Manages span attributes for CrewAI instrumentation."""
def __init__(self, span: Span, instance, skip_agent_processing=False) -> None:
self.span = span
self.instance = instance
self.skip_agent_processing = skip_agent_processing
self.process_instance()
def process_instance(self):
"""Process the instance based on its type."""
instance_type = self.instance.__class__.__name__
self._set_attribute(SpanAttributes.LLM_SYSTEM, "crewai")
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, instance_type)
method_mapping = {
"Crew": self._process_crew,
"Agent": self._process_agent,
"Task": self._process_task,
"LLM": self._process_llm,
}
method = method_mapping.get(instance_type)
if method:
method()
def _process_crew(self):
"""Process a Crew instance."""
crew_id = getattr(self.instance, "id", "")
self._set_attribute("crewai.crew.id", str(crew_id))
self._set_attribute("crewai.crew.type", "crewai.crew")
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "workflow")
logger.debug(f"CrewAI: Processing crew with id {crew_id}")
for key, value in self.instance.__dict__.items():
if value is None:
continue
if key == "tasks":
if isinstance(value, list):
self._set_attribute("crewai.crew.max_turns", str(len(value)))
logger.debug(f"CrewAI: Found {len(value)} tasks")
elif key == "agents":
if isinstance(value, list):
logger.debug(f"CrewAI: Found {len(value)} agents in crew")
if not self.skip_agent_processing:
self._parse_agents(value)
elif key == "llms":
self._parse_llms(value)
elif key == "result":
self._set_attribute("crewai.crew.final_output", str(value))
self._set_attribute("crewai.crew.output", str(value))
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, str(value))
else:
self._set_attribute(f"crewai.crew.{key}", str(value))
def _process_agent(self):
"""Process an Agent instance."""
agent = {}
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "agent")
for key, value in self.instance.__dict__.items():
if key == "tools":
parsed_tools = _parse_tools(value)
for i, tool in enumerate(parsed_tools):
tool_prefix = f"crewai.agent.tool.{i}."
for tool_key, tool_value in tool.items():
self._set_attribute(f"{tool_prefix}{tool_key}", str(tool_value))
agent[key] = json.dumps(parsed_tools)
if value is None:
continue
if key != "tools":
agent[key] = str(value)
self._set_attribute(AgentAttributes.AGENT_ID, agent.get("id", ""))
self._set_attribute(AgentAttributes.AGENT_ROLE, agent.get("role", ""))
self._set_attribute(AgentAttributes.AGENT_NAME, agent.get("name", ""))
self._set_attribute(AgentAttributes.AGENT_TOOLS, agent.get("tools", ""))
if "reasoning" in agent:
self._set_attribute(AgentAttributes.AGENT_REASONING, agent.get("reasoning", ""))
if "goal" in agent:
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, agent.get("goal", ""))
self._set_attribute("crewai.agent.goal", agent.get("goal", ""))
self._set_attribute("crewai.agent.backstory", agent.get("backstory", ""))
self._set_attribute("crewai.agent.cache", agent.get("cache", ""))
self._set_attribute("crewai.agent.allow_delegation", agent.get("allow_delegation", ""))
self._set_attribute("crewai.agent.allow_code_execution", agent.get("allow_code_execution", ""))
self._set_attribute("crewai.agent.max_retry_limit", agent.get("max_retry_limit", ""))
if hasattr(self.instance, "llm") and self.instance.llm is not None:
model_name = (
getattr(self.instance.llm, "model", None) or getattr(self.instance.llm, "model_name", None) or ""
)
temp = getattr(self.instance.llm, "temperature", None)
max_tokens = getattr(self.instance.llm, "max_tokens", None)
top_p = getattr(self.instance.llm, "top_p", None)
self._set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model_name)
if temp is not None:
self._set_attribute(SpanAttributes.LLM_REQUEST_TEMPERATURE, str(temp))
if max_tokens is not None:
self._set_attribute(SpanAttributes.LLM_REQUEST_MAX_TOKENS, str(max_tokens))
if top_p is not None:
self._set_attribute(SpanAttributes.LLM_REQUEST_TOP_P, str(top_p))
self._set_attribute("crewai.agent.llm", str(model_name))
self._set_attribute(AgentAttributes.AGENT_MODELS, str(model_name))
def _process_task(self):
"""Process a Task instance."""
task = {}
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "workflow.step")
for key, value in self.instance.__dict__.items():
if value is None:
continue
if key == "tools":
parsed_tools = _parse_tools(value)
for i, tool in enumerate(parsed_tools):
tool_prefix = f"crewai.task.tool.{i}."
for tool_key, tool_value in tool.items():
self._set_attribute(f"{tool_prefix}{tool_key}", str(tool_value))
task[key] = json.dumps(parsed_tools)
elif key == "agent":
task[key] = value.role if value else None
if value:
agent_id = getattr(value, "id", "")
self._set_attribute(AgentAttributes.FROM_AGENT, str(agent_id))
else:
task[key] = str(value)
self._set_attribute("crewai.task.name", task.get("description", ""))
self._set_attribute("crewai.task.type", "task")
self._set_attribute("crewai.task.input", task.get("context", ""))
self._set_attribute("crewai.task.expected_output", task.get("expected_output", ""))
if "description" in task:
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_INPUT, task.get("description", ""))
if "output" in task:
self._set_attribute(SpanAttributes.AGENTOPS_ENTITY_OUTPUT, task.get("output", ""))
self._set_attribute("crewai.task.output", task.get("output", ""))
if "id" in task:
self._set_attribute("crewai.task.id", str(task.get("id", "")))
if "status" in task:
self._set_attribute("crewai.task.status", task.get("status", ""))
self._set_attribute("crewai.task.agent", task.get("agent", ""))
self._set_attribute("crewai.task.human_input", task.get("human_input", ""))
self._set_attribute("crewai.task.processed_by_agents", str(task.get("processed_by_agents", "")))
if "tools" in task and task["tools"]:
try:
tools = json.loads(task["tools"])
for i, tool in enumerate(tools):
self._set_attribute(MessageAttributes.TOOL_CALL_NAME.format(i=i), tool.get("name", ""))
self._set_attribute(
MessageAttributes.TOOL_CALL_DESCRIPTION.format(i=i), tool.get("description", "")
)
except (json.JSONDecodeError, TypeError):
logger.warning(f"Failed to parse tools for task: {task.get('id', 'unknown')}")
def _process_llm(self):
"""Process an LLM instance."""
llm = {}
self._set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "llm")
for key, value in self.instance.__dict__.items():
if value is None:
continue
llm[key] = str(value)
model_name = llm.get("model_name", "") or llm.get("model", "")
self._set_attribute(SpanAttributes.LLM_REQUEST_MODEL, model_name)
self._set_attribute(SpanAttributes.LLM_REQUEST_TEMPERATURE, llm.get("temperature", ""))
self._set_attribute(SpanAttributes.LLM_REQUEST_MAX_TOKENS, llm.get("max_tokens", ""))
self._set_attribute(SpanAttributes.LLM_REQUEST_TOP_P, llm.get("top_p", ""))
if "frequency_penalty" in llm:
self._set_attribute(SpanAttributes.LLM_REQUEST_FREQUENCY_PENALTY, llm.get("frequency_penalty", ""))
if "presence_penalty" in llm:
self._set_attribute(SpanAttributes.LLM_REQUEST_PRESENCE_PENALTY, llm.get("presence_penalty", ""))
if "streaming" in llm:
self._set_attribute(SpanAttributes.LLM_REQUEST_STREAMING, llm.get("streaming", ""))
if "api_key" in llm:
self._set_attribute("gen_ai.request.api_key_present", "true")
if "base_url" in llm:
self._set_attribute(SpanAttributes.LLM_OPENAI_API_BASE, llm.get("base_url", ""))
if "api_version" in llm:
self._set_attribute(SpanAttributes.LLM_OPENAI_API_VERSION, llm.get("api_version", ""))
def _parse_agents(self, agents):
"""Parse agents into a list of dictionaries."""
if not agents:
logger.debug("CrewAI: No agents to parse")
return
agent_count = len(agents)
logger.debug(f"CrewAI: Parsing {agent_count} agents")
# Pre-process all agents to collect their data first
agent_data_list = []
for idx, agent in enumerate(agents):
if agent is None:
logger.debug(f"CrewAI: Agent at index {idx} is None, skipping")
agent_data_list.append(None)
continue
logger.debug(f"CrewAI: Processing agent at index {idx}")
try:
agent_data = self._extract_agent_data(agent)
agent_data_list.append(agent_data)
except Exception as e:
logger.error(f"CrewAI: Error extracting data for agent at index {idx}: {str(e)}")
agent_data_list.append(None)
# Now set all attributes at once for each agent
for idx, agent_data in enumerate(agent_data_list):
if agent_data is None:
continue
for key, value in agent_data.items():
if key == "tools" and isinstance(value, list):
for tool_idx, tool in enumerate(value):
for tool_key, tool_value in tool.items():
self._set_attribute(f"crewai.agents.{idx}.tools.{tool_idx}.{tool_key}", str(tool_value))
else:
self._set_attribute(f"crewai.agents.{idx}.{key}", value)
def _parse_llms(self, llms):
"""Parse LLMs into a list of dictionaries."""
for idx, llm in enumerate(llms):
if llm is not None:
model_name = getattr(llm, "model", None) or getattr(llm, "model_name", None) or ""
llm_data = {
"model": model_name,
"temperature": llm.temperature,
"max_tokens": llm.max_tokens,
"max_completion_tokens": llm.max_completion_tokens,
"top_p": llm.top_p,
"n": llm.n,
"seed": llm.seed,
"base_url": llm.base_url,
"api_version": llm.api_version,
}
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_MODEL}.{idx}", model_name)
if hasattr(llm, "temperature"):
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_TEMPERATURE}.{idx}", str(llm.temperature))
if hasattr(llm, "max_tokens"):
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_MAX_TOKENS}.{idx}", str(llm.max_tokens))
if hasattr(llm, "top_p"):
self._set_attribute(f"{SpanAttributes.LLM_REQUEST_TOP_P}.{idx}", str(llm.top_p))
for key, value in llm_data.items():
if value is not None:
self._set_attribute(f"crewai.llms.{idx}.{key}", str(value))
def _extract_agent_data(self, agent):
"""Extract data from an agent."""
model = getattr(agent.llm, "model", None) or getattr(agent.llm, "model_name", None) or ""
tools_list = []
if hasattr(agent, "tools") and agent.tools:
tools_list = _parse_tools(agent.tools)
return {
"id": str(agent.id),
"role": agent.role,
"goal": agent.goal,
"backstory": agent.backstory,
"cache": agent.cache,
"config": agent.config,
"verbose": agent.verbose,
"allow_delegation": agent.allow_delegation,
"tools": tools_list,
"max_iter": agent.max_iter,
"llm": str(model),
}
def _set_attribute(self, key, value):
"""Set an attribute on the span."""
if value is not None and value != "":
set_span_attribute(self.span, key, value)
@@ -0,0 +1,535 @@
import os
import time
import logging
from typing import Dict, Any
from contextlib import contextmanager
from opentelemetry.trace import SpanKind, get_current_span
from opentelemetry.metrics import Meter
from opentelemetry.instrumentation.utils import unwrap
from agentops.instrumentation.common import (
CommonInstrumentor,
InstrumentorConfig,
StandardMetrics,
create_wrapper_factory,
create_span,
SpanAttributeManager,
safe_set_attribute,
set_token_usage_attributes,
TokenUsageExtractor,
)
from agentops.instrumentation.agentic.crewai.version import __version__
from agentops.semconv import SpanAttributes, AgentOpsSpanKindValues, ToolAttributes, MessageAttributes
from agentops.semconv.core import CoreAttributes
from agentops.instrumentation.agentic.crewai.crewai_span_attributes import CrewAISpanAttributes, set_span_attribute
from agentops import get_client
# Initialize logger
logger = logging.getLogger(__name__)
_instruments = ("crewai >= 0.70.0",)
# Global context to store tool executions by parent span ID
_tool_executions_by_agent = {}
@contextmanager
def store_tool_execution():
"""Context manager to store tool execution details for later attachment to agent spans."""
parent_span = get_current_span()
parent_span_id = getattr(parent_span.get_span_context(), "span_id", None)
if parent_span_id:
if parent_span_id not in _tool_executions_by_agent:
_tool_executions_by_agent[parent_span_id] = []
tool_details = {}
try:
yield tool_details
if tool_details:
_tool_executions_by_agent[parent_span_id].append(tool_details)
finally:
pass
def attach_tool_executions_to_agent_span(span):
"""Attach stored tool executions to the agent span."""
span_id = getattr(span.get_span_context(), "span_id", None)
if span_id and span_id in _tool_executions_by_agent:
for idx, tool_execution in enumerate(_tool_executions_by_agent[span_id]):
for key, value in tool_execution.items():
if value is not None:
span.set_attribute(f"crewai.agent.tool_execution.{idx}.{key}", str(value))
del _tool_executions_by_agent[span_id]
class CrewaiInstrumentor(CommonInstrumentor):
"""Instrumentor for CrewAI framework."""
def __init__(self):
config = InstrumentorConfig(
library_name="crewai",
library_version=__version__,
wrapped_methods=[], # We'll use custom wrapping for CrewAI
metrics_enabled=is_metrics_enabled(),
dependencies=_instruments,
)
super().__init__(config)
self._attribute_manager = None
def _initialize(self, **kwargs):
"""Initialize attribute manager."""
application_name = kwargs.get("application_name", "default_application")
environment = kwargs.get("environment", "default_environment")
self._attribute_manager = SpanAttributeManager(
service_name=application_name, deployment_environment=environment
)
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
"""Create metrics for CrewAI instrumentation."""
return StandardMetrics.create_standard_metrics(meter)
def _custom_wrap(self, **kwargs):
"""Perform custom wrapping for CrewAI methods."""
from wrapt import wrap_function_wrapper
# Get attribute manager for all wrappers
attr_manager = self._attribute_manager
# Define wrappers using the new create_wrapper_factory
wrap_function_wrapper(
"crewai.crew",
"Crew.kickoff",
create_wrapper_factory(wrap_kickoff_impl, self._metrics, attr_manager)(self._tracer),
)
wrap_function_wrapper(
"crewai.agent",
"Agent.execute_task",
create_wrapper_factory(wrap_agent_execute_task_impl, self._metrics, attr_manager)(self._tracer),
)
wrap_function_wrapper(
"crewai.task",
"Task.execute_sync",
create_wrapper_factory(wrap_task_execute_impl, self._metrics, attr_manager)(self._tracer),
)
wrap_function_wrapper(
"crewai.llm",
"LLM.call",
create_wrapper_factory(wrap_llm_call_impl, self._metrics, attr_manager)(self._tracer),
)
wrap_function_wrapper(
"crewai.utilities.tool_utils",
"execute_tool_and_check_finality",
create_wrapper_factory(wrap_tool_execution_impl, self._metrics, attr_manager)(self._tracer),
)
wrap_function_wrapper(
"crewai.tools.tool_usage",
"ToolUsage.use",
create_wrapper_factory(wrap_tool_usage_impl, self._metrics, attr_manager)(self._tracer),
)
def _custom_unwrap(self, **kwargs):
"""Perform custom unwrapping for CrewAI methods."""
unwrap("crewai.crew", "Crew.kickoff")
unwrap("crewai.agent", "Agent.execute_task")
unwrap("crewai.task", "Task.execute_sync")
unwrap("crewai.llm", "LLM.call")
unwrap("crewai.utilities.tool_utils", "execute_tool_and_check_finality")
unwrap("crewai.tools.tool_usage", "ToolUsage.use")
# Implementation functions for wrappers
def wrap_kickoff_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
"""Implementation of kickoff wrapper."""
logger.debug(
f"CrewAI: Starting workflow instrumentation for Crew with {len(getattr(instance, 'agents', []))} agents"
)
config = get_client().config
attributes = {
SpanAttributes.LLM_SYSTEM: "crewai",
}
if config.default_tags and len(config.default_tags) > 0:
tag_list = list(config.default_tags)
attributes[CoreAttributes.TAGS] = tag_list
# Use trace_name from config if available, otherwise default to "crewai.workflow"
span_name = config.trace_name if config.trace_name else "crewai.workflow"
with create_span(
tracer, span_name, kind=SpanKind.INTERNAL, attributes=attributes, attribute_manager=attr_manager
) as span:
logger.debug("CrewAI: Processing crew instance attributes")
# First set general crew attributes but skip agent processing
crew_attrs = CrewAISpanAttributes(span=span, instance=instance, skip_agent_processing=True)
# Prioritize agent processing before task execution
if hasattr(instance, "agents") and instance.agents:
logger.debug(f"CrewAI: Explicitly processing {len(instance.agents)} agents before task execution")
crew_attrs._parse_agents(instance.agents)
logger.debug("CrewAI: Executing wrapped crew kickoff function")
result = wrapped(*args, **kwargs)
if result:
class_name = instance.__class__.__name__
span.set_attribute(f"crewai.{class_name.lower()}.result", str(result))
if class_name == "Crew":
_process_crew_result(span, instance, result)
# Set token usage using common utilities
set_token_usage_attributes(span, result)
_calculate_efficiency_metrics(span, result)
return result
def _process_crew_result(span, instance, result):
"""Process crew execution result."""
if hasattr(result, "usage_metrics"):
span.set_attribute("crewai.crew.usage_metrics", str(getattr(result, "usage_metrics")))
if hasattr(result, "tasks_output") and result.tasks_output:
span.set_attribute("crewai.crew.tasks_output", str(result.tasks_output))
try:
task_details_by_description = _build_task_details_map(instance)
_process_task_outputs(span, result.tasks_output, task_details_by_description)
except Exception as ex:
logger.warning(f"Failed to parse task outputs: {ex}")
def _build_task_details_map(instance):
"""Build a map of task descriptions to task details."""
task_details_by_description = {}
if hasattr(instance, "tasks"):
for task in instance.tasks:
if task is not None:
agent_id = ""
agent_role = ""
if hasattr(task, "agent") and task.agent:
agent_id = str(getattr(task.agent, "id", ""))
agent_role = getattr(task.agent, "role", "")
tools = []
if hasattr(task, "tools") and task.tools:
for tool in task.tools:
tool_info = {}
if hasattr(tool, "name"):
tool_info["name"] = tool.name
if hasattr(tool, "description"):
tool_info["description"] = tool.description
if tool_info:
tools.append(tool_info)
task_details_by_description[task.description] = {
"agent_id": agent_id,
"agent_role": agent_role,
"async_execution": getattr(task, "async_execution", False),
"human_input": getattr(task, "human_input", False),
"output_file": getattr(task, "output_file", ""),
"tools": tools,
}
return task_details_by_description
def _process_task_outputs(span, tasks_output, task_details_by_description):
"""Process task outputs and set attributes."""
for idx, task_output in enumerate(tasks_output):
task_prefix = f"crewai.crew.tasks.{idx}"
task_attrs = {
"description": getattr(task_output, "description", ""),
"name": getattr(task_output, "name", ""),
"expected_output": getattr(task_output, "expected_output", ""),
"summary": getattr(task_output, "summary", ""),
"raw": getattr(task_output, "raw", ""),
"agent": getattr(task_output, "agent", ""),
"output_format": str(getattr(task_output, "output_format", "")),
}
for attr_name, attr_value in task_attrs.items():
if attr_value:
safe_set_attribute(span, f"{task_prefix}.{attr_name}", attr_value, max_length=1000)
span.set_attribute(f"{task_prefix}.status", "completed")
span.set_attribute(f"{task_prefix}.id", str(idx))
description = task_attrs.get("description", "")
if description and description in task_details_by_description:
details = task_details_by_description[description]
span.set_attribute(f"{task_prefix}.agent_id", details["agent_id"])
span.set_attribute(f"{task_prefix}.async_execution", str(details["async_execution"]))
span.set_attribute(f"{task_prefix}.human_input", str(details["human_input"]))
if details["output_file"]:
span.set_attribute(f"{task_prefix}.output_file", details["output_file"])
for tool_idx, tool in enumerate(details["tools"]):
for tool_key, tool_value in tool.items():
span.set_attribute(f"{task_prefix}.tools.{tool_idx}.{tool_key}", str(tool_value))
def _calculate_efficiency_metrics(span, result):
"""Calculate and set efficiency metrics."""
if hasattr(result, "token_usage"):
try:
usage = TokenUsageExtractor.extract_from_response(result)
# Calculate efficiency
if usage.prompt_tokens and usage.completion_tokens and usage.prompt_tokens > 0:
efficiency = usage.completion_tokens / usage.prompt_tokens
span.set_attribute("crewai.crew.token_efficiency", f"{efficiency:.4f}")
# Calculate cache efficiency
if usage.cached_prompt_tokens and usage.prompt_tokens and usage.prompt_tokens > 0:
cache_ratio = usage.cached_prompt_tokens / usage.prompt_tokens
span.set_attribute("crewai.crew.cache_efficiency", f"{cache_ratio:.4f}")
except Exception as ex:
logger.warning(f"Failed to calculate efficiency metrics: {ex}")
def wrap_agent_execute_task_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
"""Implementation of agent execute task wrapper."""
agent_name = instance.role if hasattr(instance, "role") else "agent"
with create_span(
tracer,
f"{agent_name}.agent",
kind=SpanKind.CLIENT,
attributes={
SpanAttributes.AGENTOPS_SPAN_KIND: AgentOpsSpanKindValues.AGENT.value,
},
attribute_manager=attr_manager,
) as span:
CrewAISpanAttributes(span=span, instance=instance)
result = wrapped(*args, **kwargs)
attach_tool_executions_to_agent_span(span)
# Record token metrics if available
if metrics.get("token_histogram") and hasattr(instance, "_token_process"):
token_process = instance._token_process.get_summary()
if hasattr(token_process, "prompt_tokens"):
metrics["token_histogram"].record(
token_process.prompt_tokens,
attributes={
SpanAttributes.LLM_SYSTEM: "crewai",
SpanAttributes.LLM_TOKEN_TYPE: "input",
SpanAttributes.LLM_RESPONSE_MODEL: str(instance.llm.model),
},
)
if hasattr(token_process, "completion_tokens"):
metrics["token_histogram"].record(
token_process.completion_tokens,
attributes={
SpanAttributes.LLM_SYSTEM: "crewai",
SpanAttributes.LLM_TOKEN_TYPE: "output",
SpanAttributes.LLM_RESPONSE_MODEL: str(instance.llm.model),
},
)
if hasattr(instance, "llm") and hasattr(instance.llm, "model"):
set_span_attribute(span, SpanAttributes.LLM_REQUEST_MODEL, str(instance.llm.model))
set_span_attribute(span, SpanAttributes.LLM_RESPONSE_MODEL, str(instance.llm.model))
return result
def wrap_task_execute_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
"""Implementation of task execute wrapper."""
task_name = instance.description if hasattr(instance, "description") else "task"
config = get_client().config
attributes = {
SpanAttributes.AGENTOPS_SPAN_KIND: AgentOpsSpanKindValues.TASK.value,
}
if config.default_tags and len(config.default_tags) > 0:
tag_list = list(config.default_tags)
attributes[CoreAttributes.TAGS] = tag_list
with create_span(
tracer, f"{task_name}.task", kind=SpanKind.CLIENT, attributes=attributes, attribute_manager=attr_manager
) as span:
CrewAISpanAttributes(span=span, instance=instance)
result = wrapped(*args, **kwargs)
set_span_attribute(span, SpanAttributes.AGENTOPS_ENTITY_OUTPUT, str(result))
return result
def wrap_llm_call_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
"""Implementation of LLM call wrapper."""
llm = instance.model if hasattr(instance, "model") else "llm"
start_time = time.time()
with create_span(tracer, f"{llm}.llm", kind=SpanKind.CLIENT, attribute_manager=attr_manager) as span:
CrewAISpanAttributes(span=span, instance=instance)
result = wrapped(*args, **kwargs)
# Set prompt attributes from args
if args and isinstance(args[0], list):
for i, message in enumerate(args[0]):
if isinstance(message, dict):
if "role" in message:
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), message["role"])
if "content" in message:
span.set_attribute(MessageAttributes.PROMPT_CONTENT.format(i=i), message["content"])
# Set completion attributes from result
if result:
span.set_attribute(MessageAttributes.COMPLETION_CONTENT.format(i=0), str(result))
span.set_attribute(MessageAttributes.COMPLETION_ROLE.format(i=0), "assistant")
# Set token usage attributes from callbacks
if "callbacks" in kwargs and kwargs["callbacks"] and hasattr(kwargs["callbacks"][0], "token_cost_process"):
token_process = kwargs["callbacks"][0].token_cost_process
if hasattr(token_process, "completion_tokens"):
span.set_attribute(SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, token_process.completion_tokens)
if hasattr(token_process, "prompt_tokens"):
span.set_attribute(SpanAttributes.LLM_USAGE_PROMPT_TOKENS, token_process.prompt_tokens)
if hasattr(token_process, "total_tokens"):
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, token_process.total_tokens)
# Record duration metric
if metrics.get("duration_histogram"):
duration = time.time() - start_time
metrics["duration_histogram"].record(
duration,
attributes={
SpanAttributes.LLM_SYSTEM: "crewai",
SpanAttributes.LLM_RESPONSE_MODEL: str(instance.model),
},
)
return result
def wrap_tool_execution_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
"""Implementation of tool execution wrapper."""
agent_action = args[0] if args else None
tools = args[1] if len(args) > 1 else []
if not agent_action:
return wrapped(*args, **kwargs)
tool_name = getattr(agent_action, "tool", "unknown_tool")
tool_input = getattr(agent_action, "tool_input", "")
with store_tool_execution() as tool_details:
tool_details["name"] = tool_name
tool_details["parameters"] = str(tool_input)
matching_tool = next((tool for tool in tools if hasattr(tool, "name") and tool.name == tool_name), None)
if matching_tool and hasattr(matching_tool, "description"):
tool_details["description"] = str(matching_tool.description)
start_time = time.time()
with create_span(
tracer,
f"{tool_name}.tool",
kind=SpanKind.CLIENT,
attributes={
SpanAttributes.AGENTOPS_SPAN_KIND: "tool",
ToolAttributes.TOOL_NAME: tool_name,
ToolAttributes.TOOL_PARAMETERS: str(tool_input),
},
attribute_manager=attr_manager,
) as span:
if matching_tool and hasattr(matching_tool, "description"):
span.set_attribute(ToolAttributes.TOOL_DESCRIPTION, str(matching_tool.description))
result = wrapped(*args, **kwargs)
# Record duration metric
if metrics.get("duration_histogram"):
duration = time.time() - start_time
metrics["duration_histogram"].record(
duration,
attributes={
SpanAttributes.LLM_SYSTEM: "crewai",
ToolAttributes.TOOL_NAME: tool_name,
},
)
if hasattr(result, "result"):
tool_result = str(result.result)
span.set_attribute(ToolAttributes.TOOL_RESULT, tool_result)
tool_details["result"] = tool_result
tool_status = "success" if not hasattr(result, "error") or not result.error else "error"
span.set_attribute(ToolAttributes.TOOL_STATUS, tool_status)
tool_details["status"] = tool_status
if hasattr(result, "error") and result.error:
tool_details["error"] = str(result.error)
duration = time.time() - start_time
tool_details["duration"] = f"{duration:.3f}"
return result
def wrap_tool_usage_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
"""Implementation of tool usage wrapper."""
calling = args[0] if args else None
if not calling:
return wrapped(*args, **kwargs)
tool_name = getattr(calling, "tool_name", "unknown_tool")
with store_tool_execution() as tool_details:
tool_details["name"] = tool_name
if hasattr(calling, "arguments") and calling.arguments:
tool_details["parameters"] = str(calling.arguments)
with create_span(
tracer,
f"{tool_name}.tool_usage",
kind=SpanKind.INTERNAL,
attributes={
SpanAttributes.AGENTOPS_SPAN_KIND: "tool.usage",
ToolAttributes.TOOL_NAME: tool_name,
},
attribute_manager=attr_manager,
) as span:
if hasattr(calling, "arguments") and calling.arguments:
span.set_attribute(ToolAttributes.TOOL_PARAMETERS, str(calling.arguments))
result = wrapped(*args, **kwargs)
tool_result = str(result)
span.set_attribute(ToolAttributes.TOOL_RESULT, tool_result)
tool_details["result"] = tool_result
tool_status = "success"
span.set_attribute(ToolAttributes.TOOL_STATUS, tool_status)
tool_details["status"] = tool_status
return result
def is_metrics_enabled() -> bool:
return (os.getenv("AGENTOPS_METRICS_ENABLED") or "true").lower() == "true"
@@ -0,0 +1 @@
__version__ = "0.36.0"
@@ -0,0 +1,19 @@
"""Google ADK Instrumentation for AgentOps
This module provides instrumentation for Google's Agent Development Kit (ADK),
capturing agent execution, LLM calls, tool calls, and other ADK-specific events.
"""
from agentops.instrumentation.common import LibraryInfo
# Library information
_library_info = LibraryInfo(
name="agentops.instrumentation.google_adk", package_name="google-adk", default_version="0.0.0"
)
LIBRARY_NAME = _library_info.name
LIBRARY_VERSION = _library_info.version
from agentops.instrumentation.agentic.google_adk.instrumentor import GooogleAdkInstrumentor # noqa: E402
from agentops.instrumentation.agentic.google_adk import patch # noqa: E402
__all__ = ["LIBRARY_NAME", "LIBRARY_VERSION", "GooogleAdkInstrumentor", "patch"]
@@ -0,0 +1,68 @@
"""Google ADK Instrumentation for AgentOps
This module provides instrumentation for Google's Agent Development Kit (ADK).
It uses a patching approach to:
1. Disable ADK's built-in telemetry to prevent duplicate spans
2. Create AgentOps spans that mirror ADK's telemetry structure
3. Extract and properly index LLM messages and tool calls
"""
from typing import Dict, Any
from agentops.logging import logger
from opentelemetry.metrics import Meter
from agentops.instrumentation.common import CommonInstrumentor, StandardMetrics, InstrumentorConfig
from agentops.instrumentation.agentic.google_adk.patch import patch_adk, unpatch_adk
# Library info for tracer/meter
LIBRARY_NAME = "agentops.instrumentation.google_adk"
LIBRARY_VERSION = "0.1.0"
class GooogleAdkInstrumentor(CommonInstrumentor):
"""An instrumentor for Google Agent Development Kit (ADK).
This instrumentor patches Google ADK to:
- Prevent ADK from creating its own telemetry spans
- Create AgentOps spans for agent runs, LLM calls, and tool calls
- Properly extract and index message content and tool interactions
"""
def __init__(self):
"""Initialize the Google ADK instrumentor."""
# Create instrumentor config
config = InstrumentorConfig(
library_name=LIBRARY_NAME,
library_version=LIBRARY_VERSION,
wrapped_methods=[], # We use patching instead of wrapping
metrics_enabled=True,
dependencies=["google-adk >= 0.1.0"],
)
super().__init__(config)
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
"""Create metrics for the instrumentor.
Returns a dictionary of metric name to metric instance.
"""
# Create standard metrics for LLM operations
return StandardMetrics.create_standard_metrics(meter)
def _custom_wrap(self, **kwargs):
"""Apply custom patching for Google ADK.
This is called after normal wrapping, but we use it for patching
since we don't have normal wrapped methods.
"""
# Apply patches with our tracer
patch_adk(self._tracer)
logger.info("Google ADK instrumentation enabled")
def _custom_unwrap(self, **kwargs):
"""Remove custom patching from Google ADK.
This method removes all patches and restores ADK's original behavior.
"""
unpatch_adk()
logger.info("Google ADK instrumentation disabled")
@@ -0,0 +1,767 @@
"""Patch functions for Google ADK instrumentation.
This module patches key methods in Google ADK to:
1. Prevent ADK from creating its own spans
2. Create AgentOps spans that mirror ADK's telemetry
3. Extract and set proper attributes on spans
"""
import json
import wrapt
from typing import Any
from opentelemetry import trace as opentelemetry_api_trace
from opentelemetry.trace import SpanKind as SpanKind
from agentops.logging import logger
from agentops.semconv import SpanAttributes, ToolAttributes, MessageAttributes, AgentAttributes
_wrapped_methods = []
class NoOpSpan:
"""A no-op span that does nothing."""
def __init__(self, *args, **kwargs):
pass
def __enter__(self):
return self
def __exit__(self, *args):
pass
def set_attribute(self, *args, **kwargs):
pass
def set_attributes(self, *args, **kwargs):
pass
def add_event(self, *args, **kwargs):
pass
def set_status(self, *args, **kwargs):
pass
def update_name(self, *args, **kwargs):
pass
def is_recording(self):
return False
def end(self, *args, **kwargs):
pass
def record_exception(self, *args, **kwargs):
pass
class NoOpTracer:
"""A tracer that creates no-op spans to prevent ADK from creating real spans."""
def start_as_current_span(self, *args, **kwargs):
"""Return a no-op context manager."""
return NoOpSpan()
def start_span(self, *args, **kwargs):
"""Return a no-op span."""
return NoOpSpan()
def use_span(self, *args, **kwargs):
"""Return a no-op context manager."""
return NoOpSpan()
def _build_llm_request_for_trace(llm_request) -> dict:
"""Build a dictionary representation of the LLM request for tracing."""
from google.genai import types
result = {
"model": llm_request.model,
"config": llm_request.config.model_dump(exclude_none=True, exclude="response_schema"),
"contents": [],
}
for content in llm_request.contents:
parts = [part for part in content.parts if not hasattr(part, "inline_data") or not part.inline_data]
result["contents"].append(types.Content(role=content.role, parts=parts).model_dump(exclude_none=True))
return result
def _extract_messages_from_contents(contents: list) -> dict:
"""Extract messages from LLM contents for proper indexing."""
attributes = {}
for i, content in enumerate(contents):
# Get role and normalize it
raw_role = content.get("role", "user")
# Hardcode role mapping for consistency
if raw_role == "model":
role = "assistant"
elif raw_role == "user":
role = "user"
elif raw_role == "system":
role = "system"
else:
role = raw_role # Keep original if not recognized
parts = content.get("parts", [])
# Set role
attributes[MessageAttributes.PROMPT_ROLE.format(i=i)] = role
# Extract content from parts
text_parts = []
for part in parts:
if "text" in part and part.get("text") is not None:
text_parts.append(str(part["text"]))
elif "function_call" in part:
# Function calls in prompts are typically from the model's previous responses
func_call = part["function_call"]
# Store as a generic attribute since MessageAttributes doesn't have prompt tool calls
attributes[f"gen_ai.prompt.{i}.function_call.name"] = func_call.get("name", "")
attributes[f"gen_ai.prompt.{i}.function_call.args"] = json.dumps(func_call.get("args", {}))
if "id" in func_call:
attributes[f"gen_ai.prompt.{i}.function_call.id"] = func_call["id"]
elif "function_response" in part:
# Function responses are typically user messages with tool results
func_resp = part["function_response"]
attributes[f"gen_ai.prompt.{i}.function_response.name"] = func_resp.get("name", "")
attributes[f"gen_ai.prompt.{i}.function_response.result"] = json.dumps(func_resp.get("response", {}))
if "id" in func_resp:
attributes[f"gen_ai.prompt.{i}.function_response.id"] = func_resp["id"]
# Combine text parts
if text_parts:
attributes[MessageAttributes.PROMPT_CONTENT.format(i=i)] = "\n".join(text_parts)
return attributes
def _extract_llm_attributes(llm_request_dict: dict, llm_response: Any) -> dict:
"""Extract attributes from LLM request and response."""
attributes = {}
# Model
if "model" in llm_request_dict:
attributes[SpanAttributes.LLM_REQUEST_MODEL] = llm_request_dict["model"]
# Config
if "config" in llm_request_dict:
config = llm_request_dict["config"]
# System instruction - commented out, now handled as a system role message
# if "system_instruction" in config:
# attributes[SpanAttributes.LLM_REQUEST_SYSTEM_INSTRUCTION] = config["system_instruction"]
# Temperature
if "temperature" in config:
attributes[SpanAttributes.LLM_REQUEST_TEMPERATURE] = config["temperature"]
# Max output tokens
if "max_output_tokens" in config:
attributes[SpanAttributes.LLM_REQUEST_MAX_TOKENS] = config["max_output_tokens"]
# Top P
if "top_p" in config:
attributes[SpanAttributes.LLM_REQUEST_TOP_P] = config["top_p"]
# Top K
if "top_k" in config:
attributes[SpanAttributes.LLM_REQUEST_TOP_K] = config["top_k"]
# Candidate count
if "candidate_count" in config:
attributes[SpanAttributes.LLM_REQUEST_CANDIDATE_COUNT] = config["candidate_count"]
# Stop sequences
if "stop_sequences" in config:
attributes[SpanAttributes.LLM_REQUEST_STOP_SEQUENCES] = json.dumps(config["stop_sequences"])
# Response MIME type
if "response_mime_type" in config:
attributes["gen_ai.request.response_mime_type"] = config["response_mime_type"]
# Tools/Functions
if "tools" in config:
# Extract tool definitions
for i, tool in enumerate(config["tools"]):
if "function_declarations" in tool:
for j, func in enumerate(tool["function_declarations"]):
attributes[f"gen_ai.request.tools.{j}.name"] = func.get("name", "")
attributes[f"gen_ai.request.tools.{j}.description"] = func.get("description", "")
# Messages - handle system instruction and regular contents
message_index = 0
# First, add system instruction as a system role message if present
# TODO: This is not Chat Completions format but doing this for frontend rendering consistency
if "config" in llm_request_dict and "system_instruction" in llm_request_dict["config"]:
system_instruction = llm_request_dict["config"]["system_instruction"]
attributes[MessageAttributes.PROMPT_ROLE.format(i=message_index)] = "system"
attributes[MessageAttributes.PROMPT_CONTENT.format(i=message_index)] = system_instruction
message_index += 1
# Then add regular contents with proper indexing
if "contents" in llm_request_dict:
for content in llm_request_dict["contents"]:
# Get role and normalize it
raw_role = content.get("role", "user")
# Hardcode role mapping for consistency
if raw_role == "model":
role = "assistant"
elif raw_role == "user":
role = "user"
elif raw_role == "system":
role = "system"
else:
role = raw_role # Keep original if not recognized
parts = content.get("parts", [])
# Set role
attributes[MessageAttributes.PROMPT_ROLE.format(i=message_index)] = role
# Extract content from parts
text_parts = []
for part in parts:
if "text" in part and part.get("text") is not None:
text_parts.append(str(part["text"]))
elif "function_call" in part:
# Function calls in prompts are typically from the model's previous responses
func_call = part["function_call"]
# Store as a generic attribute since MessageAttributes doesn't have prompt tool calls
attributes[f"gen_ai.prompt.{message_index}.function_call.name"] = func_call.get("name", "")
attributes[f"gen_ai.prompt.{message_index}.function_call.args"] = json.dumps(
func_call.get("args", {})
)
if "id" in func_call:
attributes[f"gen_ai.prompt.{message_index}.function_call.id"] = func_call["id"]
elif "function_response" in part:
# Function responses are typically user messages with tool results
func_resp = part["function_response"]
attributes[f"gen_ai.prompt.{message_index}.function_response.name"] = func_resp.get("name", "")
attributes[f"gen_ai.prompt.{message_index}.function_response.result"] = json.dumps(
func_resp.get("response", {})
)
if "id" in func_resp:
attributes[f"gen_ai.prompt.{message_index}.function_response.id"] = func_resp["id"]
# Combine text parts
if text_parts:
attributes[MessageAttributes.PROMPT_CONTENT.format(i=message_index)] = "\n".join(text_parts)
message_index += 1
# Response
if llm_response:
try:
response_dict = json.loads(llm_response) if isinstance(llm_response, str) else llm_response
# Response model
if "model" in response_dict:
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = response_dict["model"]
# Usage metadata
if "usage_metadata" in response_dict:
usage = response_dict["usage_metadata"]
if "prompt_token_count" in usage:
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = usage["prompt_token_count"]
if "candidates_token_count" in usage:
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = usage["candidates_token_count"]
if "total_token_count" in usage:
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = usage["total_token_count"]
# Additional token details if available
if "prompt_tokens_details" in usage:
for detail in usage["prompt_tokens_details"]:
if "modality" in detail and "token_count" in detail:
attributes[f"gen_ai.usage.prompt_tokens.{detail['modality'].lower()}"] = detail[
"token_count"
]
if "candidates_tokens_details" in usage:
for detail in usage["candidates_tokens_details"]:
if "modality" in detail and "token_count" in detail:
attributes[f"gen_ai.usage.completion_tokens.{detail['modality'].lower()}"] = detail[
"token_count"
]
# Response content
if "content" in response_dict and "parts" in response_dict["content"]:
parts = response_dict["content"]["parts"]
# Set completion role and content - hardcode role as 'assistant' for consistency
attributes[MessageAttributes.COMPLETION_ROLE.format(i=0)] = "assistant"
text_parts = []
tool_call_index = 0
for part in parts:
if "text" in part and part.get("text") is not None:
text_parts.append(str(part["text"]))
elif "function_call" in part:
# This is a function call in the response
func_call = part["function_call"]
attributes[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=tool_call_index)] = (
func_call.get("name", "")
)
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=tool_call_index)] = (
json.dumps(func_call.get("args", {}))
)
if "id" in func_call:
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=tool_call_index)] = (
func_call["id"]
)
tool_call_index += 1
if text_parts:
attributes[MessageAttributes.COMPLETION_CONTENT.format(i=0)] = "\n".join(text_parts)
# Finish reason
if "finish_reason" in response_dict:
attributes[SpanAttributes.LLM_RESPONSE_FINISH_REASON] = response_dict["finish_reason"]
# Response ID
if "id" in response_dict:
attributes[SpanAttributes.LLM_RESPONSE_ID] = response_dict["id"]
except Exception as e:
logger.debug(f"Failed to extract response attributes: {e}")
return attributes
# Wrapper for Runner.run_async - REMOVED per user request
# We just pass through without creating a span
def _runner_run_async_wrapper(agentops_tracer):
def actual_decorator(wrapped, instance, args, kwargs):
async def new_function():
# Just pass through without creating a span
async_gen = wrapped(*args, **kwargs)
async for item in async_gen:
yield item
return new_function()
return actual_decorator
def extract_agent_attributes(instance):
attributes = {}
# Use AgentAttributes from semconv
attributes[AgentAttributes.AGENT_NAME] = instance.name
if hasattr(instance, "description"):
attributes["agent.description"] = instance.description
if hasattr(instance, "model"):
attributes["agent.model"] = instance.model
if hasattr(instance, "instruction"):
attributes["agent.instruction"] = instance.instruction
if hasattr(instance, "tools"):
for tool in instance.tools:
if hasattr(tool, "name"):
attributes[ToolAttributes.TOOL_NAME] = tool.name
if hasattr(tool, "description"):
attributes[ToolAttributes.TOOL_DESCRIPTION] = tool.description
if hasattr(instance, "output_key"):
attributes["agent.output_key"] = instance.output_key
# Subagents
if hasattr(instance, "sub_agents"):
# recursively extract attributes from subagents but add a prefix to the keys, also with indexing, because we can have multiple subagents, also subagent can have subagents, So have to index them even if they are not in the same level
for i, sub_agent in enumerate(instance.sub_agents):
sub_agent_attributes = extract_agent_attributes(sub_agent)
for key, value in sub_agent_attributes.items():
attributes[f"agent.sub_agents.{i}.{key}"] = value
return attributes
# Wrapper for BaseAgent.run_async
def _base_agent_run_async_wrapper(agentops_tracer):
def actual_decorator(wrapped, instance, args, kwargs):
async def new_function():
agent_name = instance.name if hasattr(instance, "name") else "unknown"
span_name = f"adk.agent.{agent_name}"
with agentops_tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "agent")
span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "agent")
span.set_attributes(extract_agent_attributes(instance))
# # Extract invocation context if available
if len(args) > 0 and hasattr(args[0], "invocation_id"):
span.set_attribute("adk.invocation_id", args[0].invocation_id)
async_gen = wrapped(*args, **kwargs)
async for item in async_gen:
yield item
return new_function()
return actual_decorator
# Wrapper for BaseLlmFlow._call_llm_async
def _base_llm_flow_call_llm_async_wrapper(agentops_tracer):
def actual_decorator(wrapped, instance, args, kwargs):
async def new_function():
# Extract model info and llm_request if available
model_name = "unknown"
llm_request = None
if len(args) > 1:
llm_request = args[1]
if hasattr(llm_request, "model"):
model_name = llm_request.model
span_name = f"adk.llm.{model_name}"
with agentops_tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "request")
span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "request")
# Extract and set attributes from llm_request before the call
if llm_request:
llm_request_dict = _build_llm_request_for_trace(llm_request)
# Only extract request attributes here, response will be set later by _finalize_model_response_event
llm_attrs = _extract_llm_attributes(llm_request_dict, None)
for key, value in llm_attrs.items():
span.set_attribute(key, value)
# Note: The actual LLM response attributes will be set by
# _finalize_model_response_event_wrapper when ADK finalizes the response
async_gen = wrapped(*args, **kwargs)
async for item in async_gen:
yield item
return new_function()
return actual_decorator
# Wrapper for ADK telemetry functions - these add attributes to current span
def _adk_trace_tool_call_wrapper(agentops_tracer):
@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
# Call original to preserve ADK behavior
result = wrapped(*args, **kwargs)
tool_args = args[0] if args else kwargs.get("args")
current_span = opentelemetry_api_trace.get_current_span()
if current_span.is_recording() and tool_args is not None:
current_span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
current_span.set_attribute("gcp.vertex.agent.tool_call_args", json.dumps(tool_args))
return result
return wrapper
def _adk_trace_tool_response_wrapper(agentops_tracer):
@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
# Call original to preserve ADK behavior
result = wrapped(*args, **kwargs)
invocation_context = args[0] if len(args) > 0 else kwargs.get("invocation_context")
event_id = args[1] if len(args) > 1 else kwargs.get("event_id")
function_response_event = args[2] if len(args) > 2 else kwargs.get("function_response_event")
current_span = opentelemetry_api_trace.get_current_span()
if current_span.is_recording():
current_span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
if invocation_context:
current_span.set_attribute("gcp.vertex.agent.invocation_id", invocation_context.invocation_id)
if event_id:
current_span.set_attribute("gcp.vertex.agent.event_id", event_id)
if function_response_event:
current_span.set_attribute(
"gcp.vertex.agent.tool_response", function_response_event.model_dump_json(exclude_none=True)
)
current_span.set_attribute("gcp.vertex.agent.llm_request", "{}")
current_span.set_attribute("gcp.vertex.agent.llm_response", "{}")
return result
return wrapper
def _adk_trace_call_llm_wrapper(agentops_tracer):
@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
# Call the original first to ensure ADK's behavior is preserved
result = wrapped(*args, **kwargs)
invocation_context = args[0] if len(args) > 0 else kwargs.get("invocation_context")
event_id = args[1] if len(args) > 1 else kwargs.get("event_id")
llm_request = args[2] if len(args) > 2 else kwargs.get("llm_request")
llm_response = args[3] if len(args) > 3 else kwargs.get("llm_response")
current_span = opentelemetry_api_trace.get_current_span()
if current_span.is_recording():
current_span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
if llm_request:
current_span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, llm_request.model)
if invocation_context:
current_span.set_attribute("gcp.vertex.agent.invocation_id", invocation_context.invocation_id)
current_span.set_attribute("gcp.vertex.agent.session_id", invocation_context.session.id)
if event_id:
current_span.set_attribute("gcp.vertex.agent.event_id", event_id)
if llm_request:
llm_request_dict = _build_llm_request_for_trace(llm_request)
current_span.set_attribute("gcp.vertex.agent.llm_request", json.dumps(llm_request_dict))
# Extract and set all attributes including usage
llm_response_json = None
if llm_response:
llm_response_json = llm_response.model_dump_json(exclude_none=True)
current_span.set_attribute("gcp.vertex.agent.llm_response", llm_response_json)
llm_attrs = _extract_llm_attributes(llm_request_dict, llm_response_json)
for key, value in llm_attrs.items():
current_span.set_attribute(key, value)
return result
return wrapper
def _adk_trace_send_data_wrapper(agentops_tracer):
@wrapt.decorator
def wrapper(wrapped, instance, args, kwargs):
# Call original to preserve ADK behavior
result = wrapped(*args, **kwargs)
invocation_context = args[0] if len(args) > 0 else kwargs.get("invocation_context")
event_id = args[1] if len(args) > 1 else kwargs.get("event_id")
data = args[2] if len(args) > 2 else kwargs.get("data")
current_span = opentelemetry_api_trace.get_current_span()
if current_span.is_recording():
if invocation_context:
current_span.set_attribute("gcp.vertex.agent.invocation_id", invocation_context.invocation_id)
if event_id:
current_span.set_attribute("gcp.vertex.agent.event_id", event_id)
if data:
from google.genai import types
current_span.set_attribute(
"gcp.vertex.agent.data",
json.dumps(
[
types.Content(role=content.role, parts=content.parts).model_dump(exclude_none=True)
for content in data
]
),
)
return result
return wrapper
# Wrapper for _finalize_model_response_event to capture response attributes
def _finalize_model_response_event_wrapper(agentops_tracer):
def actual_decorator(wrapped, instance, args, kwargs):
# Call the original method
result = wrapped(*args, **kwargs)
# Extract llm_request and llm_response from args
llm_request = args[0] if len(args) > 0 else kwargs.get("llm_request")
llm_response = args[1] if len(args) > 1 else kwargs.get("llm_response")
# Get the current span and set response attributes
current_span = opentelemetry_api_trace.get_current_span()
if current_span.is_recording() and llm_request and llm_response:
span_name = getattr(current_span, "name", "")
if "adk.llm" in span_name:
# Build request dict
llm_request_dict = _build_llm_request_for_trace(llm_request)
# Extract response attributes
llm_response_json = llm_response.model_dump_json(exclude_none=True)
llm_attrs = _extract_llm_attributes(llm_request_dict, llm_response_json)
# Only set response-related attributes (request attrs already set)
for key, value in llm_attrs.items():
if "usage" in key or "completion" in key or "response" in key:
current_span.set_attribute(key, value)
return result
return actual_decorator
# Wrapper for tool execution that creates a single merged span
def _call_tool_async_wrapper(agentops_tracer):
"""Wrapper that creates a single span for tool call and response."""
def actual_decorator(wrapped, instance, args, kwargs):
async def new_function():
# Extract tool info from args
tool = args[0] if args else kwargs.get("tool")
tool_args = args[1] if len(args) > 1 else kwargs.get("args", {})
tool_context = args[2] if len(args) > 2 else kwargs.get("tool_context")
tool_name = getattr(tool, "name", "unknown_tool")
span_name = f"adk.tool.{tool_name}"
with agentops_tracer.start_as_current_span(span_name, kind=SpanKind.CLIENT) as span:
span.set_attribute(SpanAttributes.AGENTOPS_SPAN_KIND, "tool")
span.set_attribute(SpanAttributes.LLM_SYSTEM, "gcp.vertex.agent")
span.set_attribute(SpanAttributes.AGENTOPS_ENTITY_NAME, "tool")
# Set tool call attributes
span.set_attribute(ToolAttributes.TOOL_NAME, tool_name)
if hasattr(tool, "description"):
span.set_attribute(ToolAttributes.TOOL_DESCRIPTION, tool.description)
if hasattr(tool, "is_long_running"):
span.set_attribute("tool.is_long_running", tool.is_long_running)
span.set_attribute(ToolAttributes.TOOL_PARAMETERS, json.dumps(tool_args))
if tool_context and hasattr(tool_context, "function_call_id"):
span.set_attribute("tool.call_id", tool_context.function_call_id)
if tool_context and hasattr(tool_context, "invocation_context"):
span.set_attribute("adk.invocation_id", tool_context.invocation_context.invocation_id)
# Execute the tool
result = await wrapped(*args, **kwargs)
# Set tool response attributes
if result:
if isinstance(result, dict):
span.set_attribute(ToolAttributes.TOOL_RESULT, json.dumps(result))
else:
span.set_attribute(ToolAttributes.TOOL_RESULT, str(result))
return result
return new_function()
return actual_decorator
def _patch(module_name: str, object_name: str, method_name: str, wrapper_function, agentops_tracer):
"""Helper to apply a patch and keep track of it."""
try:
module = __import__(module_name, fromlist=[object_name])
obj = getattr(module, object_name)
wrapt.wrap_function_wrapper(obj, method_name, wrapper_function(agentops_tracer))
_wrapped_methods.append((obj, method_name))
logger.debug(f"Successfully wrapped {module_name}.{object_name}.{method_name}")
except Exception as e:
logger.warning(f"Could not wrap {module_name}.{object_name}.{method_name}: {e}")
def _patch_module_function(module_name: str, function_name: str, wrapper_function, agentops_tracer):
"""Helper to patch module-level functions."""
try:
module = __import__(module_name, fromlist=[function_name])
wrapt.wrap_function_wrapper(module, function_name, wrapper_function(agentops_tracer))
_wrapped_methods.append((module, function_name))
logger.debug(f"Successfully wrapped {module_name}.{function_name}")
except Exception as e:
logger.warning(f"Could not wrap {module_name}.{function_name}: {e}")
def patch_adk(agentops_tracer):
"""Apply all patches to Google ADK modules."""
logger.debug("Applying Google ADK patches for AgentOps instrumentation")
# First, disable ADK's own tracer by replacing it with our NoOpTracer
noop_tracer = NoOpTracer()
try:
import google.adk.telemetry as adk_telemetry
# Replace the tracer with our no-op version
adk_telemetry.tracer = noop_tracer
logger.debug("Replaced ADK's tracer with NoOpTracer")
except Exception as e:
logger.warning(f"Failed to replace ADK tracer: {e}")
# Also replace the tracer in all modules that have already imported it
modules_to_patch = [
"google.adk.runners",
"google.adk.agents.base_agent",
"google.adk.flows.llm_flows.base_llm_flow",
"google.adk.flows.llm_flows.functions",
]
import sys
for module_name in modules_to_patch:
if module_name in sys.modules:
try:
module = sys.modules[module_name]
if hasattr(module, "tracer"):
module.tracer = noop_tracer
logger.debug(f"Replaced tracer in {module_name}")
except Exception as e:
logger.warning(f"Failed to replace tracer in {module_name}: {e}")
# Patch methods that create top-level AgentOps spans
# Skip runner patching - we don't want adk.runner spans
_patch("google.adk.agents.base_agent", "BaseAgent", "run_async", _base_agent_run_async_wrapper, agentops_tracer)
# Patch ADK's telemetry functions to add attributes to AgentOps spans
_patch_module_function("google.adk.telemetry", "trace_tool_call", _adk_trace_tool_call_wrapper, agentops_tracer)
_patch_module_function(
"google.adk.telemetry", "trace_tool_response", _adk_trace_tool_response_wrapper, agentops_tracer
)
_patch_module_function("google.adk.telemetry", "trace_call_llm", _adk_trace_call_llm_wrapper, agentops_tracer)
_patch_module_function("google.adk.telemetry", "trace_send_data", _adk_trace_send_data_wrapper, agentops_tracer)
# Patch method that creates nested spans
_patch(
"google.adk.flows.llm_flows.base_llm_flow",
"BaseLlmFlow",
"_call_llm_async",
_base_llm_flow_call_llm_async_wrapper,
agentops_tracer,
)
# Also patch _finalize_model_response_event to capture response attributes
_patch(
"google.adk.flows.llm_flows.base_llm_flow",
"BaseLlmFlow",
"_finalize_model_response_event",
_finalize_model_response_event_wrapper,
agentops_tracer,
)
# Patch tool execution to create merged tool spans
_patch_module_function(
"google.adk.flows.llm_flows.functions", "__call_tool_async", _call_tool_async_wrapper, agentops_tracer
)
logger.info("Google ADK patching complete")
def unpatch_adk():
"""Remove all patches from Google ADK modules."""
logger.debug("Removing Google ADK patches")
# Restore ADK's tracer
try:
import google.adk.telemetry as adk_telemetry
from opentelemetry import trace
adk_telemetry.tracer = trace.get_tracer("gcp.vertex.agent")
logger.debug("Restored ADK's built-in tracer")
except Exception as e:
logger.warning(f"Failed to restore ADK tracer: {e}")
# Unwrap all methods
for obj, method_name in _wrapped_methods:
try:
if hasattr(getattr(obj, method_name), "__wrapped__"):
original = getattr(obj, method_name).__wrapped__
setattr(obj, method_name, original)
logger.debug(f"Successfully unwrapped {obj}.{method_name}")
except Exception as e:
logger.warning(f"Failed to unwrap {obj}.{method_name}: {e}")
_wrapped_methods.clear()
logger.info("Google ADK unpatching complete")
@@ -0,0 +1 @@
from .instrumentor import HaystackInstrumentor # noqa: F401
@@ -0,0 +1,186 @@
from typing import Any, Dict
from opentelemetry.trace import SpanKind
from opentelemetry.instrumentation.utils import unwrap
from wrapt import wrap_function_wrapper
from agentops.instrumentation.common import (
CommonInstrumentor,
InstrumentorConfig,
StandardMetrics,
create_wrapper_factory,
create_span,
SpanAttributeManager,
)
from agentops.semconv import SpanAttributes
_instruments = ("haystack-ai >= 2.0.0",)
class HaystackInstrumentor(CommonInstrumentor):
def __init__(self):
config = InstrumentorConfig(
library_name="haystack",
library_version="2",
wrapped_methods=[],
metrics_enabled=False,
dependencies=_instruments,
)
super().__init__(config)
self._attribute_manager = None
def _initialize(self, **kwargs):
application_name = kwargs.get("application_name", "default_application")
environment = kwargs.get("environment", "default_environment")
self._attribute_manager = SpanAttributeManager(
service_name=application_name, deployment_environment=environment
)
def _create_metrics(self, meter) -> Dict[str, Any]:
return StandardMetrics.create_standard_metrics(meter)
def _custom_wrap(self, **kwargs):
attr_manager = self._attribute_manager
wrap_function_wrapper(
"haystack.components.generators.openai",
"OpenAIGenerator.run",
create_wrapper_factory(_wrap_haystack_run_impl, self._metrics, attr_manager)(self._tracer),
)
wrap_function_wrapper(
"haystack.components.generators.chat",
"AzureOpenAIChatGenerator.run",
create_wrapper_factory(_wrap_haystack_run_impl, self._metrics, attr_manager)(self._tracer),
)
try:
wrap_function_wrapper(
"haystack.components.generators.openai",
"OpenAIGenerator.stream",
create_wrapper_factory(_wrap_haystack_stream_impl, self._metrics, attr_manager)(self._tracer),
)
except Exception:
pass
try:
wrap_function_wrapper(
"haystack.components.generators.chat",
"AzureOpenAIChatGenerator.stream",
create_wrapper_factory(_wrap_haystack_stream_impl, self._metrics, attr_manager)(self._tracer),
)
except Exception:
pass
def _custom_unwrap(self, **kwargs):
unwrap("haystack.components.generators.openai", "OpenAIGenerator.run")
unwrap("haystack.components.generators.chat", "AzureOpenAIChatGenerator.run")
try:
unwrap("haystack.components.generators.openai", "OpenAIGenerator.stream")
except Exception:
pass
try:
unwrap("haystack.components.generators.chat", "AzureOpenAIChatGenerator.stream")
except Exception:
pass
def _first_non_empty_text(value):
if isinstance(value, list) and value:
return _first_non_empty_text(value[0])
if isinstance(value, dict):
if "content" in value:
return str(value["content"])
if "text" in value:
return str(value["text"])
if "replies" in value and value["replies"]:
return str(value["replies"][0])
if value is None:
return None
return str(value)
def _extract_prompt(args, kwargs):
if "prompt" in kwargs:
return kwargs.get("prompt")
if "messages" in kwargs:
return kwargs.get("messages")
if args:
return args[0]
return None
def _get_model_name(instance):
for attr in ("model", "model_name", "deployment_name", "deployment"):
if hasattr(instance, attr):
val = getattr(instance, attr)
if val:
return str(val)
return None
def _wrap_haystack_run_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
model = _get_model_name(instance)
with create_span(
tracer,
"haystack.generator.run",
kind=SpanKind.CLIENT,
attributes={
SpanAttributes.LLM_SYSTEM: "haystack",
"gen_ai.model": model,
SpanAttributes.LLM_REQUEST_STREAMING: False,
},
attribute_manager=attr_manager,
) as span:
prompt = _extract_prompt(args, kwargs)
prompt_text = _first_non_empty_text(prompt)
if prompt_text:
span.set_attribute("gen_ai.prompt.0.content", prompt_text[:500])
result = wrapped(*args, **kwargs)
reply_text = None
if isinstance(result, dict):
reply_text = _first_non_empty_text(result.get("replies"))
if not reply_text:
reply_text = _first_non_empty_text(result)
else:
reply_text = _first_non_empty_text(result)
if reply_text:
span.set_attribute("gen_ai.response.0.content", str(reply_text)[:500])
return result
def _wrap_haystack_stream_impl(tracer, metrics, attr_manager, wrapped, instance, args, kwargs):
model = _get_model_name(instance)
with create_span(
tracer,
"haystack.generator.stream",
kind=SpanKind.CLIENT,
attributes={
SpanAttributes.LLM_SYSTEM: "haystack",
"gen_ai.model": model,
SpanAttributes.LLM_REQUEST_STREAMING: True,
},
attribute_manager=attr_manager,
) as span:
prompt = _extract_prompt(args, kwargs)
prompt_text = _first_non_empty_text(prompt)
if prompt_text:
span.set_attribute("gen_ai.prompt.0.content", prompt_text[:500])
out = wrapped(*args, **kwargs)
try:
chunk_count = 0
for chunk in out:
chunk_count += 1
last_text = _first_non_empty_text(chunk)
if last_text:
span.set_attribute("gen_ai.response.0.content", str(last_text)[:500])
yield chunk
span.set_attribute("gen_ai.response.chunk_count", chunk_count)
except TypeError:
return out
@@ -0,0 +1,3 @@
from agentops.instrumentation.agentic.langgraph.instrumentation import LanggraphInstrumentor
__all__ = ["LanggraphInstrumentor"]
@@ -0,0 +1,54 @@
from typing import Dict, Any
import json
def ensure_no_none_values(attributes: Dict[str, Any]) -> Dict[str, Any]:
return {k: v for k, v in attributes.items() if v is not None}
def set_graph_attributes(span: Any, graph_nodes: list = None, graph_edges: list = None) -> None:
if graph_nodes:
span.set_attribute("langgraph.graph.nodes", json.dumps(graph_nodes))
span.set_attribute("langgraph.graph.node_count", len(graph_nodes))
for i, node in enumerate(graph_nodes):
span.set_attribute(f"langgraph.node.{i}.name", node)
span.set_attribute(f"langgraph.node.{i}.type", "unknown")
if graph_edges:
span.set_attribute("langgraph.graph.edges", json.dumps(graph_edges))
span.set_attribute("langgraph.graph.edge_count", len(graph_edges))
for i, edge in enumerate(graph_edges):
parts = edge.split("->")
if len(parts) == 2:
span.set_attribute(f"langgraph.edge.{i}.source", parts[0])
span.set_attribute(f"langgraph.edge.{i}.target", parts[1])
def extract_messages_from_input(input_data: Any) -> list:
if isinstance(input_data, dict) and "messages" in input_data:
return input_data["messages"]
return []
def extract_messages_from_output(output_data: Any) -> list:
if isinstance(output_data, dict) and "messages" in output_data:
return output_data["messages"]
return []
def get_message_content(message: Any) -> str:
if hasattr(message, "content"):
return str(message.content)
return ""
def get_message_role(message: Any) -> str:
if hasattr(message, "role"):
return message.role
elif hasattr(message, "type"):
return message.type
elif hasattr(message, "__class__"):
return message.__class__.__name__.replace("Message", "").lower()
return "unknown"
@@ -0,0 +1,598 @@
from functools import wraps
from typing import Any, Callable, Collection, Dict, Optional, Tuple
import json
import inspect
from opentelemetry import trace
from opentelemetry.trace import SpanKind, Status, StatusCode, get_tracer
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor
from opentelemetry.instrumentation.utils import unwrap
from wrapt import wrap_function_wrapper
from agentops.semconv import (
SpanAttributes,
WorkflowAttributes,
MessageAttributes,
)
from .attributes import (
ensure_no_none_values,
set_graph_attributes,
extract_messages_from_input,
get_message_content,
get_message_role,
)
import sys
if "typing_extensions" not in sys.modules:
from unittest import mock
sys.modules["typing_extensions"] = mock.MagicMock()
class LanggraphInstrumentor(BaseInstrumentor):
def __init__(self, config: Optional[Dict[str, Any]] = None):
super().__init__()
self.config = config or {}
self._tracer = None
def instrumentation_dependencies(self) -> Collection[str]:
return ["langgraph >= 0.0.1"]
def _instrument(self, **kwargs):
tracer_provider = kwargs.get("tracer_provider")
self._tracer = get_tracer("agentops.instrumentation.agentic.langgraph", "0.1.0", tracer_provider)
# Initialize context variable for tracking graph executions
import contextvars
self._current_graph_execution = contextvars.ContextVar("current_graph_execution", default=None)
wrap_function_wrapper("langgraph.graph.state", "StateGraph.__init__", self._wrap_state_graph_init)
wrap_function_wrapper("langgraph.graph.state", "StateGraph.compile", self._wrap_state_graph_compile)
wrap_function_wrapper("langgraph.pregel", "Pregel.invoke", self._wrap_invoke)
wrap_function_wrapper("langgraph.pregel", "Pregel.stream", self._wrap_stream)
wrap_function_wrapper("langgraph.graph.state", "StateGraph.add_node", self._wrap_add_node)
def _uninstrument(self, **kwargs):
unwrap("langgraph.graph.state", "StateGraph.__init__")
unwrap("langgraph.graph.state", "StateGraph.compile")
unwrap("langgraph.pregel", "Pregel.invoke")
unwrap("langgraph.pregel", "Pregel.stream")
unwrap("langgraph.graph.state", "StateGraph.add_node")
def _wrap_state_graph_init(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
if not self._tracer:
return wrapped(*args, **kwargs)
with self._tracer.start_as_current_span("langgraph.graph.initialize", kind=SpanKind.INTERNAL) as span:
span.set_attributes(
ensure_no_none_values(
{
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
WorkflowAttributes.WORKFLOW_TYPE: "graph_initialization",
SpanAttributes.AGENTOPS_ENTITY_NAME: "Graph Initialization",
}
)
)
try:
result = wrapped(*args, **kwargs)
instance._langgraph_instrumented = True
instance._langgraph_nodes = []
instance._langgraph_edges = []
return result
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
def _wrap_state_graph_compile(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
if not self._tracer:
return wrapped(*args, **kwargs)
with self._tracer.start_as_current_span("langgraph.graph.compile", kind=SpanKind.INTERNAL) as span:
span.set_attributes(
ensure_no_none_values(
{
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
WorkflowAttributes.WORKFLOW_TYPE: "graph_compilation",
SpanAttributes.AGENTOPS_ENTITY_NAME: "Graph Compilation",
SpanAttributes.LLM_SYSTEM: "langgraph",
}
)
)
try:
result = wrapped(*args, **kwargs)
nodes = []
edges = []
if hasattr(instance, "nodes"):
nodes = list(instance.nodes.keys()) if hasattr(instance.nodes, "keys") else []
if hasattr(instance, "edges") and hasattr(instance.edges, "items"):
for source, targets in instance.edges.items():
if isinstance(targets, dict):
for target in targets.values():
edges.append(f"{source}->{target}")
elif isinstance(targets, list):
for target in targets:
edges.append(f"{source}->{target}")
set_graph_attributes(span, nodes, edges)
return result
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
def _wrap_invoke(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
if not self._tracer:
return wrapped(*args, **kwargs)
current_span = trace.get_current_span()
if current_span and current_span.name == "langgraph.workflow.execute":
return wrapped(*args, **kwargs)
with self._tracer.start_as_current_span("langgraph.workflow.execute", kind=SpanKind.INTERNAL) as span:
span.set_attributes(
ensure_no_none_values(
{
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
WorkflowAttributes.WORKFLOW_TYPE: "langgraph_invoke",
SpanAttributes.AGENTOPS_ENTITY_NAME: "Workflow Execution",
SpanAttributes.LLM_REQUEST_STREAMING: False,
"langgraph.execution.mode": "invoke",
}
)
)
execution_state = {"executed_nodes": [], "message_count": 0, "final_response": None}
# Set the current execution state in context
token = self._current_graph_execution.set(execution_state)
try:
input_data = args[0] if args else kwargs.get("input", {})
messages = extract_messages_from_input(input_data)
if messages:
execution_state["message_count"] = len(messages)
for i, msg in enumerate(messages[:3]):
content = get_message_content(msg)
role = get_message_role(msg)
if content:
span.set_attribute(f"gen_ai.prompt.{i}.content", content[:500])
span.set_attribute(f"gen_ai.prompt.{i}.role", role)
result = wrapped(*args, **kwargs)
# Extract execution information from result
if isinstance(result, dict):
# Check for messages in result
if "messages" in result:
output_messages = result["messages"]
if isinstance(output_messages, list):
# Count all messages in the result
total_messages = len([msg for msg in output_messages if hasattr(msg, "content")])
execution_state["message_count"] = total_messages
if output_messages:
# Find the last non-tool message
for msg in reversed(output_messages):
if hasattr(msg, "content") and not hasattr(msg, "tool_call_id"):
content = get_message_content(msg)
if content:
execution_state["final_response"] = content
span.set_attribute("gen_ai.response.0.content", content[:500])
break
# Capture final execution state before returning
final_executed_nodes = list(execution_state["executed_nodes"]) # Copy the list
final_node_count = len(final_executed_nodes)
final_message_count = execution_state["message_count"]
final_response = execution_state["final_response"]
span.set_status(Status(StatusCode.OK))
span.set_attributes(
ensure_no_none_values(
{
"langgraph.graph.executed_nodes": json.dumps(final_executed_nodes),
"langgraph.graph.node_execution_count": final_node_count,
"langgraph.graph.message_count": final_message_count,
"langgraph.graph.final_response": final_response,
"langgraph.graph.status": "success",
}
)
)
return result
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
finally:
# Reset the context
self._current_graph_execution.reset(token)
def _wrap_stream(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
if not self._tracer:
return wrapped(*args, **kwargs)
current_span = trace.get_current_span()
if current_span and current_span.name == "langgraph.workflow.execute":
return wrapped(*args, **kwargs)
span = self._tracer.start_span("langgraph.workflow.execute", kind=SpanKind.INTERNAL)
span.set_attributes(
ensure_no_none_values(
{
SpanAttributes.AGENTOPS_SPAN_KIND: "workflow",
WorkflowAttributes.WORKFLOW_TYPE: "langgraph_stream",
SpanAttributes.AGENTOPS_ENTITY_NAME: "Workflow Stream",
SpanAttributes.LLM_REQUEST_STREAMING: True,
"langgraph.execution.mode": "stream",
}
)
)
execution_state = {"executed_nodes": [], "message_count": 0, "chunk_count": 0, "final_response": None}
# Set the current execution state in context
token = self._current_graph_execution.set(execution_state)
try:
# Extract input messages
input_data = args[0] if args else kwargs.get("input", {})
messages = extract_messages_from_input(input_data)
if messages:
execution_state["message_count"] = len(messages)
for i, msg in enumerate(messages[:3]):
content = get_message_content(msg)
role = get_message_role(msg)
if content:
span.set_attribute(f"gen_ai.prompt.{i}.content", content[:500])
span.set_attribute(f"gen_ai.prompt.{i}.role", role)
stream_gen = wrapped(*args, **kwargs)
def stream_wrapper():
try:
for chunk in stream_gen:
execution_state["chunk_count"] += 1
if isinstance(chunk, dict):
# Debug: print chunk structure
# print(f"DEBUG: Chunk keys: {list(chunk.keys())}")
for key in chunk:
# Track node executions (excluding special keys)
if (
key not in ["__start__", "__end__", "__interrupt__"]
and key not in execution_state["executed_nodes"]
):
execution_state["executed_nodes"].append(key)
# Track messages in the chunk value
chunk_value = chunk[key]
if isinstance(chunk_value, dict):
# Check for messages in the chunk value
if "messages" in chunk_value:
msg_list = chunk_value["messages"]
if isinstance(msg_list, list):
execution_state["message_count"] += len(msg_list)
for msg in msg_list:
content = get_message_content(msg)
if content:
execution_state["final_response"] = content
elif key == "messages" and isinstance(chunk_value, list):
# Sometimes messages might be directly in the chunk
execution_state["message_count"] += len(chunk_value)
for msg in chunk_value:
content = get_message_content(msg)
if content:
execution_state["final_response"] = content
yield chunk
# Capture final execution state before ending
final_executed_nodes = list(execution_state["executed_nodes"])
final_node_count = len(final_executed_nodes)
final_message_count = execution_state["message_count"]
final_chunk_count = execution_state["chunk_count"]
final_response = execution_state["final_response"]
span.set_status(Status(StatusCode.OK))
span.set_attributes(
ensure_no_none_values(
{
"langgraph.graph.executed_nodes": json.dumps(final_executed_nodes),
"langgraph.graph.node_execution_count": final_node_count,
"langgraph.graph.message_count": final_message_count,
"langgraph.graph.total_chunks": final_chunk_count,
"langgraph.graph.final_response": final_response,
"langgraph.graph.status": "success",
}
)
)
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
finally:
span.end()
return stream_wrapper()
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
span.end()
raise
finally:
# Reset the context
self._current_graph_execution.reset(token)
def _wrap_add_node(self, wrapped: Callable, instance: Any, args: Tuple, kwargs: Dict) -> Any:
if not self._tracer:
return wrapped(*args, **kwargs)
# Get node name and function
if args:
key = args[0]
action = args[1] if len(args) > 1 else kwargs.get("action")
else:
key = kwargs.get("key")
action = kwargs.get("action")
if not action:
return wrapped(*args, **kwargs)
# Create wrapped node function that instruments LLM calls
def create_wrapped_node(original_func):
if inspect.iscoroutinefunction(original_func):
@wraps(original_func)
async def wrapped_node_async(state):
# Track node execution in parent graph span
self._track_node_execution(key)
# Check if this node contains an LLM call
is_llm_node = self._detect_llm_node(original_func)
if is_llm_node:
with self._tracer.start_as_current_span("langgraph.node.execute", kind=SpanKind.CLIENT) as span:
span.set_attributes(
ensure_no_none_values(
{
SpanAttributes.AGENTOPS_SPAN_KIND: "llm",
SpanAttributes.AGENTOPS_ENTITY_NAME: f"Node: {key}",
SpanAttributes.LLM_SYSTEM: "langgraph",
"langgraph.node.name": key,
}
)
)
try:
# Call the original function
result = await original_func(state)
# Extract LLM information from the result
self._extract_llm_info_from_result(span, state, result)
span.set_status(Status(StatusCode.OK))
return result
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
else:
# Non-LLM node, just execute normally
return await original_func(state)
else:
@wraps(original_func)
def wrapped_node_sync(state):
# Track node execution in parent graph span
self._track_node_execution(key)
# Check if this node contains an LLM call
is_llm_node = self._detect_llm_node(original_func)
if is_llm_node:
with self._tracer.start_as_current_span("langgraph.node.execute", kind=SpanKind.CLIENT) as span:
span.set_attributes(
ensure_no_none_values(
{
SpanAttributes.AGENTOPS_SPAN_KIND: "llm",
SpanAttributes.AGENTOPS_ENTITY_NAME: f"Node: {key}",
SpanAttributes.LLM_SYSTEM: "langgraph",
}
)
)
try:
# Call the original function
result = original_func(state)
# Extract LLM information from the result
self._extract_llm_info_from_result(span, state, result)
span.set_status(Status(StatusCode.OK))
return result
except Exception as e:
span.record_exception(e)
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
else:
# Non-LLM node, just execute normally
return original_func(state)
return wrapped_node_sync
return wrapped_node_async if inspect.iscoroutinefunction(original_func) else wrapped_node_sync
# Wrap the action function
wrapped_action = create_wrapped_node(action)
# Call the original add_node with the wrapped action
if args and len(args) > 1:
new_args = (args[0], wrapped_action) + args[2:]
return wrapped(*new_args, **kwargs)
else:
kwargs["action"] = wrapped_action
return wrapped(*args, **kwargs)
def _track_node_execution(self, node_name: str) -> None:
"""Track node execution in the active graph span."""
# Use context variable to track the current execution
if hasattr(self, "_current_graph_execution"):
execution_state = self._current_graph_execution.get()
if execution_state and node_name not in execution_state["executed_nodes"]:
execution_state["executed_nodes"].append(node_name)
def _detect_llm_node(self, func: Callable) -> bool:
"""Detect if a node function contains LLM calls."""
try:
# Get the source code of the function
source = inspect.getsource(func)
# Check for common LLM patterns
llm_patterns = [
"ChatOpenAI",
"ChatAnthropic",
"ChatGoogleGenerativeAI",
".invoke(",
".ainvoke(",
".stream(",
".astream(",
"llm.",
"model.",
"chat.",
]
for pattern in llm_patterns:
if pattern in source:
return True
# Check if function has 'llm' or 'model' in its local variables
if hasattr(func, "__code__"):
local_vars = func.__code__.co_varnames
if any(var in ["llm", "model", "chat"] for var in local_vars):
return True
except Exception:
# If we can't inspect the source, assume it might be an LLM node
pass
return False
def _extract_llm_info_from_result(self, span: Any, state: Dict, result: Any) -> None:
"""Extract LLM information from the node execution result."""
try:
# Extract messages from state
if isinstance(state, dict) and "messages" in state:
messages = state["messages"]
# Set input messages
for i, msg in enumerate(messages[-5:]): # Last 5 messages as context
if hasattr(msg, "content"):
span.set_attribute(MessageAttributes.PROMPT_CONTENT.format(i=i), str(msg.content)[:1000])
if hasattr(msg, "role"):
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), msg.role)
elif hasattr(msg, "type"):
span.set_attribute(MessageAttributes.PROMPT_ROLE.format(i=i), msg.type)
# Extract messages from result
if isinstance(result, dict) and "messages" in result:
output_messages = result["messages"]
if output_messages:
last_msg = output_messages[-1] if isinstance(output_messages, list) else output_messages
# Extract model information from message if available
if hasattr(last_msg, "response_metadata"):
metadata = last_msg.response_metadata
if isinstance(metadata, dict):
if "model_name" in metadata:
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, metadata["model_name"])
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, metadata["model_name"])
elif "model" in metadata:
span.set_attribute(SpanAttributes.LLM_REQUEST_MODEL, metadata["model"])
span.set_attribute(SpanAttributes.LLM_RESPONSE_MODEL, metadata["model"])
# Token usage
if "token_usage" in metadata:
usage = metadata["token_usage"]
if isinstance(usage, dict):
if "prompt_tokens" in usage:
span.set_attribute(
SpanAttributes.LLM_USAGE_PROMPT_TOKENS, usage["prompt_tokens"]
)
if "completion_tokens" in usage:
span.set_attribute(
SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, usage["completion_tokens"]
)
if "total_tokens" in usage:
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, usage["total_tokens"])
# Response ID
if "id" in metadata and metadata["id"] is not None:
span.set_attribute(SpanAttributes.LLM_RESPONSE_ID, metadata["id"])
# Finish reason
if "finish_reason" in metadata:
span.set_attribute(
MessageAttributes.COMPLETION_FINISH_REASON.format(i=0), metadata["finish_reason"]
)
# Content
if hasattr(last_msg, "content"):
span.set_attribute(
MessageAttributes.COMPLETION_CONTENT.format(i=0), str(last_msg.content)[:1000]
)
if hasattr(last_msg, "role"):
span.set_attribute(MessageAttributes.COMPLETION_ROLE.format(i=0), last_msg.role)
# Check for tool calls
if hasattr(last_msg, "tool_calls") and last_msg.tool_calls:
for j, tool_call in enumerate(last_msg.tool_calls[:5]):
if hasattr(tool_call, "name"):
span.set_attribute(
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=j), tool_call.name
)
if hasattr(tool_call, "args"):
span.set_attribute(
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=j),
json.dumps(tool_call.args)[:500],
)
if hasattr(tool_call, "id"):
span.set_attribute(
MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=j), tool_call.id
)
# Additional attributes from message
if hasattr(last_msg, "id") and last_msg.id is not None:
span.set_attribute(SpanAttributes.LLM_RESPONSE_ID, last_msg.id)
# Usage information might be on the message itself
if hasattr(last_msg, "usage_metadata"):
usage = last_msg.usage_metadata
if hasattr(usage, "input_tokens"):
span.set_attribute(SpanAttributes.LLM_USAGE_PROMPT_TOKENS, usage.input_tokens)
if hasattr(usage, "output_tokens"):
span.set_attribute(SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, usage.output_tokens)
if hasattr(usage, "total_tokens"):
span.set_attribute(SpanAttributes.LLM_USAGE_TOTAL_TOKENS, usage.total_tokens)
except Exception:
# Don't fail the span if we can't extract info
pass
@@ -0,0 +1 @@
__version__ = "0.1.0"
@@ -0,0 +1,156 @@
# OpenAI Agents SDK Instrumentation
This module provides automatic instrumentation for the OpenAI Agents SDK, adding telemetry that follows OpenTelemetry semantic conventions for Generative AI systems.
## Architecture Overview
The OpenAI Agents SDK instrumentor works by:
1. Intercepting the Agents SDK's trace processor interface to capture Agent, Function, Generation, and other span types
2. Monkey-patching the Agents SDK `Runner` class to capture the full execution lifecycle, including streaming operations
3. Converting all captured data to OpenTelemetry spans and metrics following semantic conventions
The instrumentation is organized into several key components:
1. **Instrumentor (`instrumentor.py`)**: The entry point that patches the Agents SDK and configures trace capture
2. **Processor (`processor.py`)**: Receives events from the SDK and prepares them for export
3. **Exporter (`exporter.py`)**: Converts SDK spans to OpenTelemetry spans and exports them
4. **Attributes Module (`attributes/`)**: Specialized modules for extracting and formatting span attributes
## Attribute Processing Modules
The attribute modules extract and format OpenTelemetry-compatible attributes from span data:
- **Common (`attributes/common.py`)**: Core attribute extraction functions for all span types and utility functions
- **Completion (`attributes/completion.py`)**: Handles different completion content formats (Chat Completions API, Response API, Agents SDK)
- **Model (`attributes/model.py`)**: Extracts model information and parameters
- **Tokens (`attributes/tokens.py`)**: Processes token usage data and metrics
- **Response (`attributes/response.py`)**: Handles interpretation of Response API objects
Each getter function in these modules is focused on a single responsibility and does not modify global state. Functions are designed to be composable, allowing different attribute types to be combined as needed in the exporter.
## Span Types
The instrumentor captures the following span types:
- **Trace**: The root span representing an entire agent workflow execution
- Created using `get_base_trace_attributes()` to initialize with standard fields
- Captures workflow name, trace ID, and workflow-level metadata
- **Agent**: Represents an agent's execution lifecycle
- Processed using `get_agent_span_attributes()` with `AGENT_SPAN_ATTRIBUTES` mapping
- Uses `SpanKind.CONSUMER` to indicate an agent receiving a request
- Captures agent name, input, output, tools, and other metadata
- **Function**: Represents a tool/function call
- Processed using `get_function_span_attributes()` with `FUNCTION_SPAN_ATTRIBUTES` mapping
- Uses `SpanKind.CLIENT` to indicate an outbound call to a function
- Captures function name, input arguments, output results, and from_agent information
- **Generation**: Captures details of model generation
- Processed using `get_generation_span_attributes()` with `GENERATION_SPAN_ATTRIBUTES` mapping
- Uses `SpanKind.CLIENT` to indicate an outbound call to an LLM
- Captures model name, configuration, usage statistics, and response content
- **Response**: Lightweight span for tracking model response data
- Processed using `get_response_span_attributes()` with `RESPONSE_SPAN_ATTRIBUTES` mapping
- Extracts response content and metadata from different API formats
- **Handoff**: Represents control transfer between agents
- Processed using `get_handoff_span_attributes()` with `HANDOFF_SPAN_ATTRIBUTES` mapping
- Tracks from_agent and to_agent information
## Span Lifecycle Management
The exporter (`exporter.py`) handles the full span lifecycle:
1. **Start Events**:
- Create spans but DO NOT END them
- Store span references in tracking dictionaries
- Use OpenTelemetry's start_span to control when spans end
- Leave status as UNSET to indicate in-progress
2. **End Events**:
- Look up existing span by ID in tracking dictionaries
- If found and not ended:
- Update span with all final attributes
- Set status to OK or ERROR based on task outcome
- End the span manually
- If not found or already ended:
- Create a new complete span with all data
- End it immediately
3. **Error Handling**:
- Check if spans are already ended before attempting updates
- Provide informative log messages about span lifecycle
- Properly clean up tracking resources
This approach is essential because:
- Agents SDK sends separate start and end events for each task
- We need to maintain a single span for the entire task lifecycle to get accurate timing
- Final data (outputs, token usage, etc.) is only available at the end event
- We want to avoid creating duplicate spans for the same task
- Spans must be properly created and ended to avoid leaks
The span lifecycle management ensures spans have:
- Accurate start and end times (preserving the actual task duration)
- Complete attribute data from both start and end events
- Proper status reflecting task completion
- All final outputs, errors, and metrics
- Clean resource management with no memory leaks
## Key Design Patterns
### Semantic Conventions
All attribute names follow the OpenTelemetry semantic conventions defined in `agentops.semconv`:
```python
# Using constants from semconv module
attributes[CoreAttributes.TRACE_ID] = trace_id
attributes[WorkflowAttributes.WORKFLOW_NAME] = trace.name
attributes[SpanAttributes.LLM_SYSTEM] = "openai"
attributes[MessageAttributes.COMPLETION_CONTENT.format(i=0)] = content
```
### Target → Source Attribute Mapping
We use a consistent pattern for attribute extraction with typed mapping dictionaries:
```python
# Attribute mapping example
AGENT_SPAN_ATTRIBUTES: AttributeMap = {
# target_attribute: source_attribute
AgentAttributes.AGENT_NAME: "name",
WorkflowAttributes.WORKFLOW_INPUT: "input",
WorkflowAttributes.FINAL_OUTPUT: "output",
# ...
}
```
### Structured Attribute Handling
- Always use MessageAttributes semantic conventions for content and tool calls
- For chat completions, use MessageAttributes.COMPLETION_CONTENT.format(i=0)
- For tool calls, use MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=0), etc.
- Never try to combine or aggregate contents into a single attribute
- Each message component should have its own properly formatted attribute
- This ensures proper display in OpenTelemetry backends and dashboards
### Serialization Rules
1. We do not serialize data structures arbitrarily; everything has a semantic convention
2. Span attributes should use semantic conventions and avoid complex serialized structures
3. Keep all string data in its original form - do not parse JSON within strings
4. If a function has JSON attributes for its arguments, do not parse that JSON - keep as string
5. If a completion or response body text/content contains JSON, keep it as a string
7. Function arguments and tool call arguments should remain in their raw string form
### Critical Notes for Attribute Handling
- NEVER manually set the root completion attributes (`SpanAttributes.LLM_COMPLETIONS` or "gen_ai.completion")
- Let OpenTelemetry backend derive these values from the detailed attributes
- Setting root completion attributes creates duplication and inconsistency
- Tests should verify attribute existence using MessageAttributes constants
- Do not check for the presence of SpanAttributes.LLM_COMPLETIONS
- Verify individual content/tool attributes instead of root attributes
@@ -0,0 +1,145 @@
# OpenAI Agents Spans and Traces
This document describes the span types, naming conventions, and attribute patterns used by the AgentOps instrumentation for the OpenAI Agents SDK.
## Span Types and Classes
The instrumentation works with these specific span data classes:
1. **AgentSpanData**: Represents a single agent's operation
- Has attributes for name, input, output, tools, and handoffs
- Processed by `get_agent_span_attributes()` using `AGENT_SPAN_ATTRIBUTES` mapping
2. **FunctionSpanData**: Represents tool or function calls
- Has attributes for name, input, output, and from_agent
- Processed by `get_function_span_attributes()` using `FUNCTION_SPAN_ATTRIBUTES` mapping
3. **GenerationSpanData**: Represents LLM model invocations
- Has attributes for model, input, output, tools, and from_agent
- Processed by `get_generation_span_attributes()` using `GENERATION_SPAN_ATTRIBUTES` mapping
4. **HandoffSpanData**: Represents agent-to-agent handoffs
- Has attributes for from_agent and to_agent
- Processed by `get_handoff_span_attributes()` using `HANDOFF_SPAN_ATTRIBUTES` mapping
5. **ResponseSpanData**: Represents model response data
- Has attributes for input and response
- Processed by `get_response_span_attributes()` using `RESPONSE_SPAN_ATTRIBUTES` mapping
## Span Naming Conventions
Spans are named according to these conventions:
1. **Trace Spans**: `agents.trace.{workflow_name}`
- Represents the entire agent workflow
- Named after the workflow or trace name
2. **Agent Spans**: `agents.agent`
- Represents a single agent's operation
- Uses `SpanKind.CONSUMER`
3. **Function Spans**: `agents.function`
- Represents tool or function calls
- Uses `SpanKind.CLIENT`
4. **Generation Spans**: `agents.generation`
- Represents LLM model invocations
- Uses `SpanKind.CLIENT`
5. **Handoff Spans**: `agents.handoff`
- Represents agent-to-agent handoffs
- Uses `SpanKind.INTERNAL`
6. **Response Spans**: `agents.response`
- Represents model response data
- Uses `SpanKind.CLIENT`
## Span Hierarchy
The spans follow a parent-child relationship that reflects the execution flow:
```
agents.trace.{workflow_name}
└── agents.agent
├── agents.generation
├── agents.function
├── agents.response
└── agents.handoff
```
## Semantic Conventions and Attributes
Each span type has attributes following OpenTelemetry semantic conventions:
### Common Attributes (All Spans)
- `trace.id`: OpenTelemetry trace ID
- `span.id`: OpenTelemetry span ID
- `parent.id`: Parent span ID (if applicable)
- `instrumentation.name`: "agentops"
- `instrumentation.version`: AgentOps library version
- `instrumentation.library.name`: "openai_agents"
- `instrumentation.library.version`: Library version
### Workflow and Trace Attributes
- `workflow.name`: Name of the workflow or trace
- `workflow.step_type`: "trace" for trace spans
- `workflow.input`: Input to the workflow
- `workflow.final_output`: Final output from the workflow
### Agent Attributes
- `agent.name`: The name of the agent
- `agent.tools`: Comma-separated list of available tools
- `agent.handoffs`: Comma-separated list of handoff targets
- `agent.from`: Source agent in handoffs (used in HandoffSpanData)
- `agent.to`: Destination agent in handoffs (used in HandoffSpanData)
### LLM Attributes
- `gen_ai.system`: "openai" for all OpenAI spans
- `gen_ai.request.model`: Model used for generation
- `gen_ai.response.model`: Model that provided the response
- `gen_ai.prompt`: Input prompt or message
- `gen_ai.completion.0.role`: Role of the completion message (usually "assistant")
- `gen_ai.completion.0.content`: Content of the completion message
- `gen_ai.tool_call.0.0.name`: Name of the tool called (if applicable)
- `gen_ai.tool_call.0.0.arguments`: Arguments for the tool call (if applicable)
### Token Usage Attributes
- `gen_ai.usage.prompt_tokens`: Number of input tokens
- `gen_ai.usage.completion_tokens`: Number of output tokens
- `gen_ai.usage.total_tokens`: Total number of tokens
- `gen_ai.usage.reasoning_tokens`: Tokens used for reasoning (Response API)
- `gen_ai.usage.cache_read.input_tokens`: Cached input tokens (Response API)
## Span Lifecycle Management
The exporter handles span lifecycle with these stages:
1. **Start Events**:
- Create spans with `start_span()` (not using context manager)
- Store span references in tracking dictionaries
- Leave status as UNSET to indicate in-progress
2. **End Events**:
- Look up existing span by ID
- Update with final attributes
- Set appropriate status and end the span manually
3. **Error Handling**:
- Set status to ERROR for spans with errors
- Add error type and message as attributes
- Record exceptions with `record_exception()`
## OpenTelemetry Span Kinds
Span kinds map to OpenTelemetry concepts:
- `AgentSpanData` → `SpanKind.CONSUMER`
- `FunctionSpanData` → `SpanKind.CLIENT`
- `GenerationSpanData` → `SpanKind.CLIENT`
- `ResponseSpanData` → `SpanKind.CLIENT`
- `HandoffSpanData` → `SpanKind.INTERNAL`
@@ -0,0 +1,144 @@
# OpenAI Agents Tracing API Integration
This document provides an overview of how AgentOps integrates with the OpenAI Agents SDK tracing system.
## OpenAI Agents Tracing API Overview
The OpenAI Agents SDK provides a comprehensive tracing system that allows you to monitor and instrument agent activities. AgentOps integrates with this system to capture and forward trace data to its backend.
## Core Integration Methods
### 1. `add_trace_processor(processor)`
The main integration point that allows external systems like AgentOps to receive trace events:
```python
from agents import add_trace_processor
from agentops.instrumentation.openai_agents.processor import OpenAIAgentsProcessor
processor = OpenAIAgentsProcessor()
add_trace_processor(processor)
```
### 2. `set_trace_processors(processors)`
Replaces all current processors with a new list:
```python
from agents import set_trace_processors
set_trace_processors([my_processor1, my_processor2])
```
### 3. `set_tracing_disabled(disabled)`
Globally enables/disables tracing:
```python
from agents import set_tracing_disabled
set_tracing_disabled(True) # Disable tracing
```
### 4. `set_tracing_export_api_key(api_key)`
Sets the API key for the backend exporter:
```python
from agents import set_tracing_export_api_key
set_tracing_export_api_key("your-api-key")
```
## Span Creation Methods
The SDK provides specialized methods for creating different types of spans:
1. **`agent_span(name, handoffs, tools, output_type, ...)`**
- Creates spans for agent operations
- Tracks agent name, available tools, potential handoffs
2. **`function_span(name, input, output, ...)`**
- Creates spans for function/tool calls
- Records function name, input arguments, and results
3. **`generation_span(input, output, model, model_config, usage, ...)`**
- Creates spans for LLM generations
- Records prompts, completions, model details, and token usage
4. **`response_span(response, ...)`**
- Lightweight span for capturing OpenAI API response metadata
5. **`handoff_span(from_agent, to_agent, ...)`**
- Tracks agent-to-agent handoffs
6. **`guardrail_span(name, triggered, ...)`**
- Records guardrail evaluations
7. **`custom_span(name, data, ...)`**
- Creates user-defined spans with arbitrary data
## Trace and Context Management
1. **`trace(workflow_name, trace_id, group_id, metadata, ...)`**
- Creates and manages a trace context
- Groups related spans into a logical trace/session
2. **`get_current_span()`**
- Returns the current active span
3. **`get_current_trace()`**
- Returns the current active trace
## How AgentOps Implements Integration
AgentOps integrates with this API through:
1. The `OpenAIAgentsProcessor` class that implements the `TracingProcessor` interface
2. The `create_span` context manager that ensures proper parent-child relationships between spans
3. The `AgentsInstrumentor` which registers the processor and adds additional instrumentation
This integration allows AgentOps to capture detailed information about agent execution, including:
- Agent operations and tool usage
- LLM requests and responses
- Token usage metrics
- Error information
- Agent-to-agent handoffs
### Trace Context Propagation
Our implementation ensures proper parent-child relationships between spans through:
1. **Context Manager Pattern**: Using `start_as_current_span()` to maintain the OpenTelemetry span context
2. **Parent Reference Tracking**: Storing parent span relationships and using them to create proper span hierarchies
3. **Trace Correlation Attributes**: Adding consistent attributes to help with querying:
- `agentops.original_trace_id`: Original trace ID from the Agents SDK
- `agentops.original_span_id`: Original span ID from the Agents SDK
- `agentops.parent_span_id`: Parent span ID for child spans
- `agentops.trace_hash`: Consistent hash based on the original trace ID
- `agentops.is_root_span`: "true" for spans without a parent
When querying spans for analysis:
1. Group spans by `agentops.original_trace_id` to find all spans in the same trace
2. Use `agentops.parent_span_id` to reconstruct the parent-child hierarchy
## Span Data Types
Several specialized span data types exist in the OpenAI Agents SDK to capture different operations:
- **AgentSpanData**: Captures agent execution data
- **FunctionSpanData**: Records tool/function calls
- **GenerationSpanData**: Records LLM generation details
- **ResponseSpanData**: Captures model response information
- **HandoffSpanData**: Tracks agent-to-agent handoffs
- **GuardrailSpanData**: Records guardrail evaluations
- **CustomSpanData**: For user-defined spans
## Processor Interface
The `TracingProcessor` interface defines methods processors must implement:
- `on_trace_start`: Called when a trace begins
- `on_trace_end`: Called when a trace ends
- `on_span_start`: Called when a span begins
- `on_span_end`: Called when a span completes
- `shutdown`: Called during application shutdown
- `force_flush`: Forces immediate processing of pending spans
The processor receives events from OpenAI Agents SDK's tracing system through these callback methods, translates them to OpenTelemetry spans, and sends them to the AgentOps backend for analysis and visualization.
@@ -0,0 +1,30 @@
"""
AgentOps Instrumentor for OpenAI Agents SDK
This module provides automatic instrumentation for the OpenAI Agents SDK when AgentOps is imported.
It implements a clean, maintainable implementation that follows semantic conventions.
IMPORTANT DISTINCTION BETWEEN OPENAI API FORMATS:
1. OpenAI Completions API - The traditional API format using prompt_tokens/completion_tokens
2. OpenAI Response API - The newer format used by the Agents SDK using input_tokens/output_tokens
3. Agents SDK - The framework that uses Response API format
The Agents SDK uses the Response API format, which we handle using shared utilities from
agentops.instrumentation.openai.
"""
from agentops.instrumentation.common import LibraryInfo
# Library information
_library_info = LibraryInfo(name="openai-agents")
LIBRARY_NAME = _library_info.name
LIBRARY_VERSION = _library_info.version
# Import after defining constants to avoid circular imports
from agentops.instrumentation.agentic.openai_agents.instrumentor import OpenAIAgentsInstrumentor # noqa: E402
__all__ = [
"LIBRARY_NAME",
"LIBRARY_VERSION",
"OpenAIAgentsInstrumentor",
]
@@ -0,0 +1,549 @@
"""Common utilities and constants for attribute processing.
This module contains shared constants, attribute mappings, and utility functions for processing
trace and span attributes in OpenAI Agents instrumentation. It provides the core functionality
for extracting and formatting attributes according to OpenTelemetry semantic conventions.
"""
from typing import Any, List, Dict, Optional
from agentops.logging import logger
from agentops.semconv import (
AgentAttributes,
WorkflowAttributes,
SpanAttributes,
InstrumentationAttributes,
ToolAttributes,
AgentOpsSpanKindValues,
ToolStatus,
)
from agentops.helpers import safe_serialize # Import safe_serialize
from agentops.instrumentation.common import AttributeMap, _extract_attributes_from_mapping
from agentops.instrumentation.common.attributes import get_common_attributes
from agentops.instrumentation.common.objects import get_uploaded_object_attributes
from agentops.instrumentation.providers.openai.attributes.response import get_response_response_attributes
from agentops.instrumentation.agentic.openai_agents import LIBRARY_NAME, LIBRARY_VERSION
from agentops.instrumentation.agentic.openai_agents.attributes.model import (
get_model_attributes,
get_model_config_attributes,
)
from agentops.instrumentation.agentic.openai_agents.attributes.completion import get_generation_output_attributes
# Attribute mapping for AgentSpanData
AGENT_SPAN_ATTRIBUTES: AttributeMap = {
AgentAttributes.AGENT_NAME: "name",
AgentAttributes.AGENT_TOOLS: "tools",
AgentAttributes.HANDOFFS: "handoffs",
WorkflowAttributes.WORKFLOW_INPUT: "input",
WorkflowAttributes.WORKFLOW_OUTPUT: "output",
}
# Attribute mapping for FunctionSpanData
FUNCTION_SPAN_ATTRIBUTES: AttributeMap = {
ToolAttributes.TOOL_NAME: "name",
ToolAttributes.TOOL_PARAMETERS: "input",
ToolAttributes.TOOL_RESULT: "output",
# AgentAttributes.AGENT_NAME: "name",
AgentAttributes.FROM_AGENT: "from_agent",
}
# Attribute mapping for HandoffSpanData
HANDOFF_SPAN_ATTRIBUTES: AttributeMap = {
AgentAttributes.FROM_AGENT: "from_agent",
AgentAttributes.TO_AGENT: "to_agent",
}
# Attribute mapping for GenerationSpanData
GENERATION_SPAN_ATTRIBUTES: AttributeMap = {
SpanAttributes.LLM_PROMPTS: "input",
}
# Attribute mapping for ResponseSpanData
RESPONSE_SPAN_ATTRIBUTES: AttributeMap = {
# Don't map input here as it causes double serialization
# We handle prompts manually in get_response_span_attributes
SpanAttributes.LLM_RESPONSE_MODEL: "model",
}
# Attribute mapping for TranscriptionSpanData
TRANSCRIPTION_SPAN_ATTRIBUTES: AttributeMap = {
# `input` and `input_format` are handled below
WorkflowAttributes.WORKFLOW_OUTPUT: "output",
}
# Attribute mapping for SpeechSpanData
SPEECH_SPAN_ATTRIBUTES: AttributeMap = {
WorkflowAttributes.WORKFLOW_INPUT: "input",
# `output` and `output_format` are handled below
# TODO `first_content_at` is not converted
}
# Attribute mapping for SpeechGroupSpanData
SPEECH_GROUP_SPAN_ATTRIBUTES: AttributeMap = {
WorkflowAttributes.WORKFLOW_INPUT: "input",
}
def _get_llm_messages_attributes(messages: Optional[List[Dict]], attribute_base: str) -> AttributeMap:
"""
Extracts attributes from a list of message dictionaries (e.g., prompts or completions).
Uses the attribute_base to format the specific attribute keys.
"""
attributes: AttributeMap = {}
if not messages:
return attributes
if not isinstance(messages, list):
logger.warning(
f"[_get_llm_messages_attributes] Expected a list of messages for base '{attribute_base}', got {type(messages)}. Value: {safe_serialize(messages)}. Returning empty."
)
return attributes
for i, msg_dict in enumerate(messages):
if isinstance(msg_dict, dict):
role = msg_dict.get("role")
content = msg_dict.get("content")
name = msg_dict.get("name")
tool_calls = msg_dict.get("tool_calls")
tool_call_id = msg_dict.get("tool_call_id")
# Common role and content
if role:
attributes[f"{attribute_base}.{i}.role"] = str(role)
if content is not None:
attributes[f"{attribute_base}.{i}.content"] = safe_serialize(content)
# Optional name for some roles
if name:
attributes[f"{attribute_base}.{i}.name"] = str(name)
# Tool calls (specific to assistant messages)
if tool_calls and isinstance(tool_calls, list):
for tc_idx, tc_dict in enumerate(tool_calls):
if isinstance(tc_dict, dict):
tc_id = tc_dict.get("id")
tc_type = tc_dict.get("type")
tc_function_data = tc_dict.get("function")
if tc_function_data and isinstance(tc_function_data, dict):
tc_func_name = tc_function_data.get("name")
tc_func_args = tc_function_data.get("arguments")
base_tool_call_key_formatted = f"{attribute_base}.{i}.tool_calls.{tc_idx}"
if tc_id:
attributes[f"{base_tool_call_key_formatted}.id"] = str(tc_id)
if tc_type:
attributes[f"{base_tool_call_key_formatted}.type"] = str(tc_type)
if tc_func_name:
attributes[f"{base_tool_call_key_formatted}.function.name"] = str(tc_func_name)
if tc_func_args is not None:
attributes[f"{base_tool_call_key_formatted}.function.arguments"] = safe_serialize(
tc_func_args
)
# Tool call ID (specific to tool_call_output messages)
if tool_call_id:
attributes[f"{attribute_base}.{i}.tool_call_id"] = str(tool_call_id)
else:
# If a message is not a dict, serialize its representation
attributes[f"{attribute_base}.{i}.content"] = safe_serialize(msg_dict)
return attributes
def get_common_instrumentation_attributes() -> AttributeMap:
"""Get common instrumentation attributes for the OpenAI Agents instrumentation.
This combines the generic AgentOps attributes with OpenAI Agents specific library attributes.
Returns:
Dictionary of common instrumentation attributes
"""
attributes = get_common_attributes()
attributes.update(
{
InstrumentationAttributes.LIBRARY_NAME: LIBRARY_NAME,
InstrumentationAttributes.LIBRARY_VERSION: LIBRARY_VERSION,
}
)
return attributes
def get_agent_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from an AgentSpanData object.
Agents are requests made to the `openai.agents` endpoint.
Args:
span_data: The AgentSpanData object
Returns:
Dictionary of attributes for agent span
"""
attributes = {}
attributes.update(get_common_attributes())
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.AGENT.value
# Get agent name directly from span_data
if hasattr(span_data, "name") and span_data.name:
attributes[AgentAttributes.AGENT_NAME] = str(span_data.name)
# Get handoffs directly from span_data
if hasattr(span_data, "handoffs") and span_data.handoffs:
attributes[AgentAttributes.HANDOFFS] = safe_serialize(span_data.handoffs)
if hasattr(span_data, "tools") and span_data.tools:
attributes[AgentAttributes.AGENT_TOOLS] = safe_serialize([str(getattr(t, "name", t)) for t in span_data.tools])
return attributes
def get_function_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from a FunctionSpanData object.
Functions are requests made to the `openai.functions` endpoint.
Args:
span_data: The FunctionSpanData object
Returns:
Dictionary of attributes for function span
"""
attributes = _extract_attributes_from_mapping(span_data, FUNCTION_SPAN_ATTRIBUTES)
attributes.update(get_common_attributes())
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.TOOL.value
# Determine tool status based on presence of error
if hasattr(span_data, "error") and span_data.error:
attributes[ToolAttributes.TOOL_STATUS] = ToolStatus.FAILED.value
else:
if hasattr(span_data, "output") and span_data.output is not None:
attributes[ToolAttributes.TOOL_STATUS] = ToolStatus.SUCCEEDED.value
else:
# Status will be set by exporter based on span lifecycle
pass
if hasattr(span_data, "from_agent") and span_data.from_agent:
attributes["agent.calling_tool.name"] = str(span_data.from_agent)
return attributes
def get_handoff_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from a HandoffSpanData object.
Handoffs are requests made to the `openai.handoffs` endpoint.
Args:
span_data: The HandoffSpanData object
Returns:
Dictionary of attributes for handoff span
"""
attributes = _extract_attributes_from_mapping(span_data, HANDOFF_SPAN_ATTRIBUTES)
attributes.update(get_common_attributes())
return attributes
def _extract_text_from_content(content: Any) -> Optional[str]:
"""Extract text from various content formats used in the Responses API.
Args:
content: Content in various formats (str, dict, list)
Returns:
Extracted text or None if no text found
"""
if isinstance(content, str):
return content
if isinstance(content, dict):
# Direct text field
if "text" in content:
return content["text"]
# Output text type
if content.get("type") == "output_text":
return content.get("text", "")
if isinstance(content, list):
text_parts = []
for item in content:
extracted = _extract_text_from_content(item)
if extracted:
text_parts.append(extracted)
return " ".join(text_parts) if text_parts else None
return None
def _build_prompt_messages_from_input(input_data: Any) -> List[Dict[str, Any]]:
"""Build prompt messages from various input formats.
Args:
input_data: Input data from span_data.input
Returns:
List of message dictionaries with role and content
"""
messages = []
if isinstance(input_data, str):
# Single string input - assume it's a user message
messages.append({"role": "user", "content": input_data})
elif isinstance(input_data, list):
for msg in input_data:
if isinstance(msg, dict):
role = msg.get("role")
content = msg.get("content")
if role and content is not None:
extracted_text = _extract_text_from_content(content)
if extracted_text:
messages.append({"role": role, "content": extracted_text})
return messages
def get_response_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from a ResponseSpanData object with full LLM response processing.
Responses are requests made to the `openai.responses` endpoint.
This function extracts not just the basic input/response mapping but also processes
the rich response object to extract LLM-specific attributes like token usage,
model information, content, etc.
TODO tool calls arrive from this span type; need to figure out why that is.
Args:
span_data: The ResponseSpanData object
Returns:
Dictionary of attributes for response span
"""
# Get basic attributes from mapping
attributes = _extract_attributes_from_mapping(span_data, RESPONSE_SPAN_ATTRIBUTES)
attributes.update(get_common_attributes())
# Process response attributes first to get all response data including instructions
if span_data.response:
response_attrs = get_response_response_attributes(span_data.response)
# Extract system prompt if present
system_prompt = response_attrs.get(SpanAttributes.LLM_OPENAI_RESPONSE_INSTRUCTIONS)
prompt_messages = []
# Add system prompt as first message if available
if system_prompt:
prompt_messages.append({"role": "system", "content": system_prompt})
# Remove from response attrs to avoid duplication
response_attrs.pop(SpanAttributes.LLM_OPENAI_RESPONSE_INSTRUCTIONS, None)
# Add conversation history from input
if hasattr(span_data, "input") and span_data.input:
prompt_messages.extend(_build_prompt_messages_from_input(span_data.input))
# Format prompts using existing function
if prompt_messages:
attributes.update(_get_llm_messages_attributes(prompt_messages, "gen_ai.prompt"))
# Remove any prompt-related attributes that might have been set by response processing
response_attrs = {
k: v for k, v in response_attrs.items() if not k.startswith("gen_ai.prompt") and k != "gen_ai.request.tools"
}
# Add remaining response attributes
attributes.update(response_attrs)
else:
# No response object, just process input as prompts
if hasattr(span_data, "input") and span_data.input:
prompt_messages = _build_prompt_messages_from_input(span_data.input)
if prompt_messages:
attributes.update(_get_llm_messages_attributes(prompt_messages, "gen_ai.prompt"))
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.LLM.value
return attributes
def get_generation_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from a GenerationSpanData object.
Generations are requests made to the `openai.completions` endpoint.
Args:
span_data: The GenerationSpanData object
Returns:
Dictionary of attributes for generation span
"""
attributes = _extract_attributes_from_mapping(span_data, GENERATION_SPAN_ATTRIBUTES)
attributes.update(get_common_attributes())
if SpanAttributes.LLM_PROMPTS in attributes:
raw_prompt_input = attributes.pop(SpanAttributes.LLM_PROMPTS)
formatted_prompt_for_llm = []
if isinstance(raw_prompt_input, str):
formatted_prompt_for_llm.append({"role": "user", "content": raw_prompt_input})
elif isinstance(raw_prompt_input, list):
temp_formatted_list = []
all_strings_or_dicts = True
for item in raw_prompt_input:
if isinstance(item, str):
temp_formatted_list.append({"role": "user", "content": item})
elif isinstance(item, dict):
temp_formatted_list.append(item)
else:
all_strings_or_dicts = False
break
if all_strings_or_dicts:
formatted_prompt_for_llm = temp_formatted_list
else:
logger.warning(
f"[get_generation_span_attributes] span_data.input was a list with mixed/unexpected content: {safe_serialize(raw_prompt_input)}"
)
if formatted_prompt_for_llm:
attributes.update(_get_llm_messages_attributes(formatted_prompt_for_llm, "gen_ai.prompt"))
if span_data.model:
attributes.update(get_model_attributes(span_data.model))
if span_data.output:
attributes.update(get_generation_output_attributes(span_data.output))
if span_data.model_config:
attributes.update(get_model_config_attributes(span_data.model_config))
attributes[SpanAttributes.AGENTOPS_SPAN_KIND] = AgentOpsSpanKindValues.LLM.value
return attributes
def get_transcription_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from a TranscriptionSpanData object.
This represents a conversion from audio to text.
Args:
span_data: The TranscriptionSpanData object
Returns:
Dictionary of attributes for transcription span
"""
from agentops import get_client
from agentops.client.api.types import UploadedObjectResponse
client = get_client()
attributes = _extract_attributes_from_mapping(span_data, TRANSCRIPTION_SPAN_ATTRIBUTES)
attributes.update(get_common_attributes())
if span_data.input:
prefix = WorkflowAttributes.WORKFLOW_INPUT
uploaded_object: UploadedObjectResponse = client.api.v4.upload_object(span_data.input)
attributes.update(get_uploaded_object_attributes(uploaded_object, prefix))
if span_data.model:
attributes.update(get_model_attributes(span_data.model))
if span_data.model_config:
attributes.update(get_model_config_attributes(span_data.model_config))
return attributes
def get_speech_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from a SpeechSpanData object.
This represents a conversion from audio to text.
Args:
span_data: The SpeechSpanData object
Returns:
Dictionary of attributes for speech span
"""
from agentops import get_client
from agentops.client.api.types import UploadedObjectResponse
client = get_client()
attributes = _extract_attributes_from_mapping(span_data, SPEECH_SPAN_ATTRIBUTES)
attributes.update(get_common_attributes())
if span_data.output:
prefix = WorkflowAttributes.WORKFLOW_OUTPUT
uploaded_object: UploadedObjectResponse = client.api.v4.upload_object(span_data.output)
attributes.update(get_uploaded_object_attributes(uploaded_object, prefix))
if span_data.model:
attributes.update(get_model_attributes(span_data.model))
if span_data.model_config:
attributes.update(get_model_config_attributes(span_data.model_config))
return attributes
def get_speech_group_span_attributes(span_data: Any) -> AttributeMap:
"""Extract attributes from a SpeechGroupSpanData object.
This represents a conversion from audio to text.
Args:
span_data: The SpeechGroupSpanData object
Returns:
Dictionary of attributes for speech group span
"""
attributes = _extract_attributes_from_mapping(span_data, SPEECH_GROUP_SPAN_ATTRIBUTES)
attributes.update(get_common_attributes())
return attributes
def get_span_attributes(span_data: Any) -> AttributeMap:
"""Get attributes for a span based on its type.
This function centralizes attribute extraction by delegating to type-specific
getter functions.
Args:
span_data: The span data object
Returns:
Dictionary of attributes for the span
"""
span_type = span_data.__class__.__name__
if span_type == "AgentSpanData":
attributes = get_agent_span_attributes(span_data)
elif span_type == "FunctionSpanData":
attributes = get_function_span_attributes(span_data)
elif span_type == "GenerationSpanData":
attributes = get_generation_span_attributes(span_data)
elif span_type == "HandoffSpanData":
attributes = get_handoff_span_attributes(span_data)
elif span_type == "ResponseSpanData":
attributes = get_response_span_attributes(span_data)
elif span_type == "TranscriptionSpanData":
attributes = get_transcription_span_attributes(span_data)
elif span_type == "SpeechSpanData":
attributes = get_speech_span_attributes(span_data)
elif span_type == "SpeechGroupSpanData":
attributes = get_speech_group_span_attributes(span_data)
else:
logger.debug(f"[agentops.instrumentation.openai_agents.attributes] Unknown span type: {span_type}")
attributes = {}
return attributes
@@ -0,0 +1,172 @@
"""Completion processing utilities for OpenAI Agents instrumentation.
This module handles completion content processing from both the Chat Completions API
and the OpenAI Response API formats, extracting messages, tool calls, function calls, etc.
"""
from typing import Any, Dict
from agentops.instrumentation.common.attributes import AttributeMap
from agentops.logging import logger
from agentops.helpers.serialization import model_to_dict
from agentops.semconv import (
SpanAttributes,
MessageAttributes,
)
from agentops.instrumentation.agentic.openai_agents.attributes.tokens import process_token_usage
def get_generation_output_attributes(output: Any) -> Dict[str, Any]:
"""Extract LLM response attributes from an `openai/completions` object.
Args:
output: The response object (can be dict, Response object, or other format)
Returns:
Dictionary of attributes extracted from the response in a consistent format
"""
# Convert model to dictionary for easier processing
response_dict = model_to_dict(output)
result: AttributeMap = {}
if not response_dict:
# Handle output as string if it's not a dict
if isinstance(output, str):
# For string output, just return the minimal set of attributes
return {}
return result
# Check for OpenAI Agents SDK response format (has raw_responses array)
if "raw_responses" in response_dict and isinstance(response_dict["raw_responses"], list):
result.update(get_raw_response_attributes(response_dict))
else:
# TODO base attributes for completion type
# Get completions or response API output attributes first
if "choices" in response_dict:
result.update(get_chat_completions_attributes(response_dict))
# Extract token usage from dictionary for standard formats
usage_attributes: AttributeMap = {}
if "usage" in response_dict:
process_token_usage(response_dict["usage"], usage_attributes)
result.update(usage_attributes)
# Extract token usage from Response object directly if dict conversion didn't work
if hasattr(output, "usage") and output.usage:
direct_usage_attributes: AttributeMap = {}
process_token_usage(output.usage, direct_usage_attributes)
result.update(direct_usage_attributes)
return result
def get_raw_response_attributes(response: Dict[str, Any]) -> Dict[str, Any]:
"""Extract attributes from OpenAI Agents SDK response format (with raw_responses).
This function handles the specific structure of OpenAI Agents SDK responses,
which include a raw_responses array containing the actual API responses.
This is the format used specifically by the Agents SDK, not the standard OpenAI API.
Args:
response: The OpenAI Agents SDK response dictionary (containing raw_responses array)
Returns:
Dictionary of attributes extracted from the Agents SDK response
"""
result: AttributeMap = {}
# Set the LLM system to OpenAI
result[SpanAttributes.LLM_SYSTEM] = "openai"
# Process raw responses
if "raw_responses" in response and isinstance(response["raw_responses"], list):
for i, raw_response in enumerate(response["raw_responses"]):
# Extract token usage from the first raw response
if "usage" in raw_response and isinstance(raw_response["usage"], dict):
usage_attrs: AttributeMap = {}
process_token_usage(raw_response["usage"], usage_attrs)
result.update(usage_attrs)
logger.debug(f"Extracted token usage from raw_responses[{i}]: {usage_attrs}")
# Extract output content
if "output" in raw_response and isinstance(raw_response["output"], list):
for j, output_item in enumerate(raw_response["output"]):
# Process message content
if "content" in output_item and isinstance(output_item["content"], list):
for content_item in output_item["content"]:
if content_item.get("type") == "output_text" and "text" in content_item:
# Set message content attribute using the standard convention
result[MessageAttributes.COMPLETION_CONTENT.format(i=j)] = content_item["text"]
# Process role
if "role" in output_item:
result[MessageAttributes.COMPLETION_ROLE.format(i=j)] = output_item["role"]
# Process tool calls
if "tool_calls" in output_item and isinstance(output_item["tool_calls"], list):
for k, tool_call in enumerate(output_item["tool_calls"]):
tool_id = tool_call.get("id", "")
# Handle function format
if "function" in tool_call and isinstance(tool_call["function"], dict):
function = tool_call["function"]
result[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=j, j=k)] = tool_id
result[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=j, j=k)] = function.get(
"name", ""
)
result[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=j, j=k)] = (
function.get("arguments", "")
)
return result
def get_chat_completions_attributes(response: Dict[str, Any]) -> Dict[str, Any]:
"""Get attributes from OpenAI Chat Completions API format (with choices array).
This function specifically handles the original Chat Completions API format
that uses a 'choices' array with 'message' objects, as opposed to the newer
Response API format that uses an 'output' array.
Args:
response: The response dictionary containing chat completions (with choices array)
Returns:
Dictionary of chat completion attributes
"""
result: AttributeMap = {}
if "choices" not in response:
return result
for i, choice in enumerate(response["choices"]):
if "finish_reason" in choice:
result[MessageAttributes.COMPLETION_FINISH_REASON.format(i=i)] = choice["finish_reason"]
message = choice.get("message", {})
if "role" in message:
result[MessageAttributes.COMPLETION_ROLE.format(i=i)] = message["role"]
if "content" in message:
content = message["content"] if message["content"] is not None else ""
result[MessageAttributes.COMPLETION_CONTENT.format(i=i)] = content
if "tool_calls" in message and message["tool_calls"] is not None:
tool_calls = message["tool_calls"]
for j, tool_call in enumerate(tool_calls):
if "function" in tool_call:
function = tool_call["function"]
result[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=i, j=j)] = tool_call.get("id")
result[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=i, j=j)] = function.get("name")
result[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=i, j=j)] = function.get(
"arguments"
)
if "function_call" in message and message["function_call"] is not None:
function_call = message["function_call"]
result[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=i)] = function_call.get("name")
result[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=i)] = function_call.get("arguments")
return result
@@ -0,0 +1,58 @@
"""Model information extraction for OpenAI Agents instrumentation.
This module provides utilities for extracting model information and parameters
from various object types, centralizing model attribute handling logic.
"""
from typing import Any, Dict
from agentops.semconv import SpanAttributes
from agentops.instrumentation.common.attributes import AttributeMap, _extract_attributes_from_mapping
# Parameter mapping dictionary for model parameters
MODEL_CONFIG_ATTRIBUTES: AttributeMap = {
SpanAttributes.LLM_REQUEST_TEMPERATURE: "temperature",
SpanAttributes.LLM_REQUEST_TOP_P: "top_p",
SpanAttributes.LLM_REQUEST_FREQUENCY_PENALTY: "frequency_penalty",
SpanAttributes.LLM_REQUEST_PRESENCE_PENALTY: "presence_penalty",
SpanAttributes.LLM_REQUEST_MAX_TOKENS: "max_tokens",
# TODO we need to establish semantic conventions for the following:
# tool_choice
# parallel_tool_calls
# truncation
# store
# language
# prompt
# turn_detection
SpanAttributes.LLM_REQUEST_INSTRUCTIONS: "instructions",
SpanAttributes.LLM_REQUEST_VOICE: "voice",
SpanAttributes.LLM_REQUEST_SPEED: "speed",
}
def get_model_attributes(model_name: str) -> Dict[str, Any]:
"""Get model name attributes for both request and response for consistency.
Args:
model_name: The model name to set
Returns:
Dictionary of model name attributes
"""
return {
SpanAttributes.LLM_REQUEST_MODEL: model_name,
SpanAttributes.LLM_RESPONSE_MODEL: model_name,
SpanAttributes.LLM_SYSTEM: "openai",
}
def get_model_config_attributes(model_config: Any) -> Dict[str, Any]:
"""Extract model configuration attributes using the model parameter mapping.
Args:
model_config: The model configuration object
Returns:
Dictionary of extracted model configuration attributes
"""
return _extract_attributes_from_mapping(model_config, MODEL_CONFIG_ATTRIBUTES)
@@ -0,0 +1,275 @@
"""Token processing and metrics for the OpenAI Agents instrumentation.
This module contains functions for processing token usage data from OpenAI responses,
including standardized handling of different API formats (Chat Completions API vs Response API)
and recording token usage metrics.
"""
import json
from typing import Any, Dict, Optional
from agentops.semconv import SpanAttributes
from agentops.logging import logger
def safe_parse(content: str) -> Optional[Dict[str, Any]]:
"""Safely parse JSON content from a string.
Args:
content: String content that might contain JSON
Returns:
Parsed dictionary if content is valid JSON, None otherwise
"""
if not isinstance(content, str):
return None
try:
# Try to parse the string as JSON
return json.loads(content)
except (json.JSONDecodeError, TypeError, ValueError):
# If parsing fails, log a debug message and return None
logger.debug(f"Failed to parse JSON content: {content[:100]}...")
return None
def extract_nested_usage(content: Any) -> Optional[Dict[str, Any]]:
"""Recursively extract usage data from potentially nested response structures.
Handles multiple nesting patterns:
1. Direct usage field at the top level
2. Usage nested in completion content JSON string
3. Usage nested in response.output[].content[].text
Args:
content: Any content object that might contain usage data
Returns:
Extracted usage dictionary or None if not found
"""
# Case: direct dictionary with usage field
if isinstance(content, dict) and "usage" in content:
return content["usage"]
# Case: JSON string that might contain usage
if isinstance(content, str):
parsed_data = safe_parse(content)
if parsed_data:
# Direct usage field in parsed JSON
if "usage" in parsed_data and isinstance(parsed_data["usage"], dict):
return parsed_data["usage"]
# Response API format with nested output structure
if "output" in parsed_data and isinstance(parsed_data["output"], list):
# Usage at top level in Response format
if "usage" in parsed_data:
return parsed_data["usage"]
# Case: complex nested structure with output array
# This handles the Response API format where usage is at the top level
if isinstance(content, dict):
if "output" in content and isinstance(content["output"], list):
if "usage" in content:
return content["usage"]
return None
def process_token_usage(
usage: Dict[str, Any], attributes: Dict[str, Any], completion_content: Optional[str] = None
) -> Dict[str, Any]:
"""Process token usage data from OpenAI responses using standardized attribute naming.
Args:
usage: Dictionary containing token usage data
attributes: Dictionary where attributes will be set
completion_content: Optional JSON string that may contain token usage info
Returns:
Dictionary mapping token types to counts for metrics
"""
# Result dictionary for metric recording
result = {}
# If usage is empty or None, use completion_content to find usage data
if not usage or (isinstance(usage, dict) and len(usage) == 0):
if completion_content:
logger.debug("TOKENS: Usage is empty, trying to extract from completion content")
extracted_usage = extract_nested_usage(completion_content)
if extracted_usage:
usage = extracted_usage
# Always set token usage attributes directly on the span to ensure they're captured
# For both Chat Completions API and Response API formats
# Helper to get an attribute from either a dict or an object
def get_value(obj, key):
if isinstance(obj, dict) and key in obj:
return obj[key]
elif hasattr(obj, key):
return getattr(obj, key)
return None
# Helper to check if an object has an attribute
def has_key(obj, key):
if isinstance(obj, dict):
return key in obj
return hasattr(obj, key)
# Process prompt/input tokens
if has_key(usage, "prompt_tokens"):
prompt_tokens = get_value(usage, "prompt_tokens")
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = prompt_tokens
result["prompt_tokens"] = prompt_tokens
elif has_key(usage, "input_tokens"):
input_tokens = get_value(usage, "input_tokens")
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = input_tokens
result["prompt_tokens"] = input_tokens
# Process completion/output tokens
if has_key(usage, "completion_tokens"):
completion_tokens = get_value(usage, "completion_tokens")
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = completion_tokens
result["completion_tokens"] = completion_tokens
elif has_key(usage, "output_tokens"):
output_tokens = get_value(usage, "output_tokens")
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = output_tokens
result["completion_tokens"] = output_tokens
# Process total tokens
if has_key(usage, "total_tokens"):
total_tokens = get_value(usage, "total_tokens")
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = total_tokens
result["total_tokens"] = total_tokens
# Process Response API specific token details using defined semantic conventions
# Process reasoning tokens (from Response API output_tokens_details)
output_tokens_details = None
if has_key(usage, "output_tokens_details"):
output_tokens_details = get_value(usage, "output_tokens_details")
if output_tokens_details:
# Handle both dict and object types
if isinstance(output_tokens_details, dict):
details = output_tokens_details
if "reasoning_tokens" in details:
attributes[SpanAttributes.LLM_USAGE_REASONING_TOKENS] = details["reasoning_tokens"]
result["reasoning_tokens"] = details["reasoning_tokens"]
elif hasattr(output_tokens_details, "reasoning_tokens"):
reasoning_tokens = output_tokens_details.reasoning_tokens
attributes[SpanAttributes.LLM_USAGE_REASONING_TOKENS] = reasoning_tokens
result["reasoning_tokens"] = reasoning_tokens
# Process cached tokens (from Response API input_tokens_details)
input_tokens_details = None
if has_key(usage, "input_tokens_details"):
input_tokens_details = get_value(usage, "input_tokens_details")
if input_tokens_details:
# Handle both dict and object types
if isinstance(input_tokens_details, dict):
details = input_tokens_details
if "cached_tokens" in details:
attributes[SpanAttributes.LLM_USAGE_CACHE_READ_INPUT_TOKENS] = details["cached_tokens"]
result["cached_input_tokens"] = details["cached_tokens"]
# Handle object with cached_tokens attribute
elif hasattr(input_tokens_details, "cached_tokens"):
cached_tokens = input_tokens_details.cached_tokens
attributes[SpanAttributes.LLM_USAGE_CACHE_READ_INPUT_TOKENS] = cached_tokens
result["cached_input_tokens"] = cached_tokens
# Log all token-related attributes that were set
token_attrs = {k: v for k, v in attributes.items() if k.startswith("gen_ai.usage")}
# If we still have no token attributes, try one more approach - look for nested output structure
if not token_attrs and completion_content:
try:
# Parse the completion content to see if we can find more deeply nested usage data
parsed_content = safe_parse(completion_content)
if parsed_content and isinstance(parsed_content, dict):
# If this is a Response API format, check for nested output structure
if "output" in parsed_content and isinstance(parsed_content["output"], list):
for output_item in parsed_content["output"]:
# Check if this has nested content with usage
if "content" in output_item and isinstance(output_item["content"], list):
for content_item in output_item["content"]:
if "text" in content_item:
# Try to parse this text for usage data
parsed_text = safe_parse(content_item["text"])
if parsed_text and "usage" in parsed_text:
logger.debug(f"Found deeply nested usage data: {parsed_text['usage']}")
# Process this usage data recursively
return process_token_usage(parsed_text["usage"], attributes)
except Exception as e:
logger.debug(f"Error during deep token extraction: {e}")
return result
def map_token_type_to_metric_name(token_type: str) -> str:
"""Maps token type names from SpanAttributes to simplified metric names.
Args:
token_type: Token type name, could be a full semantic convention or a simple name
Returns:
Simplified token type name for metrics
"""
# If token_type is a semantic convention (contains a dot), extract the last part
if isinstance(token_type, str) and "." in token_type:
parts = token_type.split(".")
token_type = parts[-1]
# Map to simplified metric names
if token_type == "prompt_tokens":
return "input"
elif token_type == "completion_tokens":
return "output"
elif token_type == "reasoning_tokens":
return "reasoning"
# Return as-is if no mapping needed
return token_type
def get_token_metric_attributes(usage: Dict[str, Any], model_name: str) -> Dict[str, Dict[str, Any]]:
"""Get token usage metric attributes from usage data.
Args:
usage: Dictionary containing token usage data
model_name: Name of the model used
Returns:
Dictionary mapping token types to metric data including value and attributes
"""
# Process all token types using our standardized processor
token_counts = process_token_usage(usage, {})
# Common attributes for all metrics
common_attributes = {
"model": model_name,
SpanAttributes.LLM_REQUEST_MODEL: model_name,
SpanAttributes.LLM_SYSTEM: "openai",
}
# Prepare metrics data for each token type
metrics_data = {}
for token_type, count in token_counts.items():
# Skip if no count
if not count:
continue
# Map token type to simplified metric name
metric_token_type = map_token_type_to_metric_name(token_type)
# Prepare the metric data
metrics_data[token_type] = {
"value": count,
"attributes": {
"token_type": metric_token_type,
**common_attributes,
},
}
return metrics_data
@@ -0,0 +1,469 @@
"""OpenAI Agents SDK Instrumentation Exporter for AgentOps
This module handles the conversion of Agents SDK spans to OpenTelemetry spans.
It manages the complete span lifecycle, attribute application, and proper span hierarchy.
See the README.md in this directory for complete documentation on:
- Span lifecycle management approach
- Serialization rules for attributes
- Structured attribute handling
- Semantic conventions usage
IMPORTANT FOR TESTING:
- Tests should verify attribute existence using MessageAttributes constants
- Do not check for the presence of SpanAttributes.LLM_COMPLETIONS
- Verify individual content/tool attributes instead of root attributes
"""
import json
from typing import Any, Dict, Optional
from opentelemetry import trace, context as context_api
from opentelemetry.trace import get_tracer, SpanKind, Status, StatusCode, NonRecordingSpan
from opentelemetry import trace as trace_api
from opentelemetry.sdk.trace import Span
from agentops.logging import logger
from agentops.semconv import (
CoreAttributes,
)
from agentops.instrumentation.common.attributes import (
get_base_trace_attributes,
get_base_span_attributes,
)
from agentops.instrumentation.agentic.openai_agents import LIBRARY_NAME, LIBRARY_VERSION
from agentops.instrumentation.agentic.openai_agents.attributes.common import (
get_span_attributes,
)
def log_otel_trace_id(span_type):
"""Log the OpenTelemetry trace ID for debugging and correlation purposes.
The hexadecimal OTel trace ID is essential for querying the backend database
and correlating local debugging logs with server-side trace data. This ID
is different from the Agents SDK trace_id and is the primary key used in
observability systems and the AgentOps dashboard.
This function retrieves the current OpenTelemetry trace ID directly from the
active span context and formats it as a 32-character hex string.
Args:
span_type: The type of span being exported for logging context
Returns:
str or None: The OpenTelemetry trace ID as a hex string, or None if unavailable
"""
current_span = trace.get_current_span()
if hasattr(current_span, "get_span_context"):
ctx = current_span.get_span_context()
if hasattr(ctx, "trace_id") and ctx.trace_id:
# Convert trace_id to 32-character hex string as shown in the API
otel_trace_id = f"{ctx.trace_id:032x}" if isinstance(ctx.trace_id, int) else str(ctx.trace_id)
logger.debug(f"[SPAN] Export | Type: {span_type} | TRACE ID: {otel_trace_id}")
return otel_trace_id
logger.debug(f"[SPAN] Export | Type: {span_type} | NO TRACE ID AVAILABLE")
return None
def get_span_kind(span: Any) -> SpanKind:
"""Determine the appropriate span kind based on span type."""
span_data = span.span_data
span_type = span_data.__class__.__name__
if span_type == "AgentSpanData":
return SpanKind.CONSUMER
elif span_type in ["FunctionSpanData", "GenerationSpanData", "ResponseSpanData"]:
return SpanKind.CLIENT
else:
return SpanKind.INTERNAL
def get_span_name(span: Any) -> str:
"""Get the name of the span based on its type and attributes."""
span_data = span.span_data
span_type = span_data.__class__.__name__
if hasattr(span_data, "name") and span_data.name:
return span_data.name
else:
return span_type.replace("SpanData", "").lower() # fallback
def _get_span_lookup_key(trace_id: str, span_id: str) -> str:
"""Generate a unique lookup key for spans based on trace and span IDs.
This key is used to track spans in the exporter and allows for efficient
lookups and management of spans during their lifecycle.
Args:
trace_id: The trace ID for the current span
span_id: The span ID for the current span
Returns:
str: A unique lookup key for the span
"""
return f"span:{trace_id}:{span_id}"
class OpenAIAgentsExporter:
"""Exporter for Agents SDK traces and spans that forwards them to OpenTelemetry.
This exporter is responsible for:
1. Creating and configuring spans
2. Setting span attributes based on data from the processor
3. Managing the span lifecycle
4. Using semantic conventions for attribute naming
5. Interacting with the OpenTelemetry API
6. Tracking spans to allow updating them when tasks complete
"""
def __init__(self, tracer_provider=None):
self.tracer_provider = tracer_provider
# Dictionary to track active spans by their SDK span ID
# Allows us to reference spans later during task completion
self._active_spans = {}
# Dictionary to track spans by trace/span ID for faster lookups
self._span_map = {}
def export_trace(self, trace: Any) -> None:
"""
Handle exporting the trace.
"""
tracer = get_tracer(LIBRARY_NAME, LIBRARY_VERSION, self.tracer_provider)
trace_id = getattr(trace, "trace_id", "unknown")
if not hasattr(trace, "trace_id"):
logger.debug("Cannot export trace: missing trace_id")
return
# Determine if this is a trace end event using status field
# We use the status field to determine if this is an end event
is_end_event = hasattr(trace, "status") and trace.status == StatusCode.OK.name
trace_lookup_key = _get_span_lookup_key(trace_id, trace_id)
attributes = get_base_trace_attributes(trace)
# For end events, check if we already have the span
if is_end_event and trace_lookup_key in self._span_map:
existing_span = self._span_map[trace_lookup_key]
span_is_ended = False
if isinstance(existing_span, Span) and hasattr(existing_span, "_end_time"):
span_is_ended = existing_span._end_time is not None
if not span_is_ended:
# Update with core attributes
for key, value in attributes.items():
existing_span.set_attribute(key, value)
# Handle error if present
if hasattr(trace, "error") and trace.error:
self._handle_span_error(trace, existing_span)
# Set status to OK if no error
else:
existing_span.set_status(Status(StatusCode.OK))
existing_span.end()
# Clean up our tracking resources
self._active_spans.pop(trace_id, None)
self._span_map.pop(trace_lookup_key, None)
return
# Create span directly instead of using context manager
span = tracer.start_span(name=trace.name, kind=SpanKind.INTERNAL, attributes=attributes)
# Add any additional trace attributes
if hasattr(trace, "group_id") and trace.group_id:
span.set_attribute(CoreAttributes.GROUP_ID, trace.group_id)
if hasattr(trace, "metadata") and trace.metadata:
for key, value in trace.metadata.items():
if isinstance(value, (str, int, float, bool)):
span.set_attribute(f"trace.metadata.{key}", value)
# Record error if present
if hasattr(trace, "error") and trace.error:
self._handle_span_error(trace, span)
# For start events, store the span for later reference
if not is_end_event:
self._span_map[trace_lookup_key] = span
self._active_spans[trace_id] = {
"span": span,
"span_type": "TraceSpan",
"trace_id": trace_id,
"parent_id": None,
}
else:
span.end()
def _get_parent_context(self, trace_id: str, span_id: str, parent_id: Optional[str] = None) -> Any:
"""Find the parent span context for proper span nesting.
This method checks:
1. First for an explicit parent ID in our span tracking dictionary
2. Then checks if the trace span is the parent
3. Falls back to the current active span context if no parent is found
Args:
trace_id: The trace ID for the current span
span_id: The span ID for the current span
parent_id: Optional parent span ID to look up
Returns:
The OpenTelemetry span context to use as parent
"""
parent_span_ctx = None
if parent_id:
# Try to find the parent span in our tracking dictionary
parent_lookup_key = f"span:{trace_id}:{parent_id}"
if parent_lookup_key in self._span_map:
parent_span = self._span_map[parent_lookup_key]
# Get the context from the parent span if it exists
if hasattr(parent_span, "get_span_context"):
parent_span_ctx = parent_span.get_span_context()
# If parent not found by span ID, check if trace span should be the parent
if not parent_span_ctx and parent_id is None:
# Try using the trace span as parent
trace_lookup_key = _get_span_lookup_key(trace_id, trace_id)
if trace_lookup_key in self._span_map:
trace_span = self._span_map[trace_lookup_key]
if hasattr(trace_span, "get_span_context"):
parent_span_ctx = trace_span.get_span_context()
# If we couldn't find the parent by ID, use the current span context as parent
if not parent_span_ctx:
# Get the current span context from the context API
ctx = context_api.get_current()
parent_span_ctx = trace_api.get_current_span(ctx).get_span_context()
return parent_span_ctx
def _create_span_with_parent(
self, name: str, kind: SpanKind, attributes: Dict[str, Any], parent_ctx: Any, end_immediately: bool = False
) -> Any:
"""Create a span with the specified parent context.
This centralizes span creation with proper parent nesting.
Args:
name: The name for the new span
kind: The span kind (CLIENT, SERVER, etc.)
attributes: The attributes to set on the span
parent_ctx: The parent context to use for nesting
end_immediately: Whether to end the span immediately
Returns:
The newly created span
"""
# Get tracer from provider
tracer = get_tracer(LIBRARY_NAME, LIBRARY_VERSION, self.tracer_provider)
# Create span with context so we get proper nesting
with trace_api.use_span(NonRecordingSpan(parent_ctx), end_on_exit=False):
span = tracer.start_span(name=name, kind=kind, attributes=attributes)
# Optionally end the span immediately
if end_immediately:
span.end()
return span
def export_span(self, span: Any) -> None:
"""Export a span to OpenTelemetry, creating or updating as needed.
This method decides whether to create a new span or update an existing one
based on whether this is a start or end event for a given span ID.
For start events:
- Create a new span and store it for later updates
- Leave status as UNSET (in progress)
- Do not end the span
- Properly set parent span reference for nesting
For end events:
- Look for an existing span to update
- If found and not ended, update with final data and end it
- If not found or already ended, create a new complete span with all data
- End the span with proper status
"""
if not hasattr(span, "span_data"):
return
span_data = span.span_data
span_type = span_data.__class__.__name__
span_id = getattr(span, "span_id", "unknown")
trace_id = getattr(span, "trace_id", "unknown")
parent_id = getattr(span, "parent_id", None)
# Check if this is a span end event
is_end_event = hasattr(span, "status") and span.status == StatusCode.OK.name
# Unique lookup key for this span
span_lookup_key = _get_span_lookup_key(trace_id, span_id)
attributes = get_base_span_attributes(span)
span_attributes = get_span_attributes(span_data)
attributes.update(span_attributes)
if is_end_event:
# Update all attributes for end events
attributes.update(span_attributes)
# Log the trace ID for debugging and correlation with AgentOps API
log_otel_trace_id(span_type)
# For start events, create a new span and store it (don't end it)
if not is_end_event:
# Process the span based on its type
# TODO span_name should come from the attributes module
span_name = get_span_name(span)
span_kind = get_span_kind(span)
# Get parent context for proper nesting
parent_span_ctx = self._get_parent_context(trace_id, span_id, parent_id)
# Create the span with proper parent context
otel_span = self._create_span_with_parent(
name=span_name, kind=span_kind, attributes=attributes, parent_ctx=parent_span_ctx
)
# Store the span for later reference
if not isinstance(otel_span, NonRecordingSpan):
self._span_map[span_lookup_key] = otel_span
self._active_spans[span_id] = {
"span": otel_span,
"span_type": span_type,
"trace_id": trace_id,
"parent_id": parent_id,
}
# Handle any error information
self._handle_span_error(span, otel_span)
# DO NOT end the span for start events - we want to keep it open for updates
return
# For end events, check if we already have the span
if span_lookup_key in self._span_map:
existing_span = self._span_map[span_lookup_key]
span_is_ended = False
if isinstance(existing_span, Span) and hasattr(existing_span, "_end_time"):
span_is_ended = existing_span._end_time is not None
if not span_is_ended:
# Update with final attributes
for key, value in attributes.items():
existing_span.set_attribute(key, value)
existing_span.set_status(
Status(StatusCode.OK if getattr(span, "status", "OK") == "OK" else StatusCode.ERROR)
)
self._handle_span_error(span, existing_span)
existing_span.end()
# Span already ended, create a new one (should be rare if logic is correct)
else:
logger.warning(
f"[Exporter] SDK span_id: {span_id} (END event) - Attempting to end an ALREADY ENDED span: {span_lookup_key}. Creating a new one instead."
)
self.create_span(span, span_type, attributes, is_already_ended=True)
# No existing span found for end event, create a new one
else:
logger.warning(
f"[Exporter] SDK span_id: {span_id} (END event) - No active span found for end event: {span_lookup_key}. Creating a new one."
)
self.create_span(span, span_type, attributes, is_already_ended=True)
self._active_spans.pop(span_id, None)
self._span_map.pop(span_lookup_key, None)
def create_span(
self, span: Any, span_type: str, attributes: Dict[str, Any], is_already_ended: bool = False
) -> None:
"""Create a new span with the provided data and end it immediately.
This method creates a span using the appropriate parent context, applies
all attributes, and ends it immediately since it's for spans that are
already in an ended state.
Args:
span: The span data from the Agents SDK
span_type: The type of span being created
attributes: The attributes to set on the span
"""
parent_ctx = None
if hasattr(span, "parent_id") and span.parent_id:
parent_ctx = self._get_parent_context(
getattr(span, "trace_id", "unknown"), getattr(span, "id", "unknown"), span.parent_id
)
name = get_span_name(span)
kind = get_span_kind(span)
self._create_span_with_parent(
name=name, kind=kind, attributes=attributes, parent_ctx=parent_ctx, end_immediately=True
)
def _handle_span_error(self, span: Any, otel_span: Any) -> None:
"""Handle error information from spans."""
if hasattr(span, "error") and span.error:
# Set status to error
status = Status(StatusCode.ERROR)
otel_span.set_status(status)
# Determine error message - handle various error formats
error_message = "Unknown error"
error_data = {}
error_type = "AgentError"
# Handle different error formats
if isinstance(span.error, dict):
error_message = span.error.get("message", span.error.get("error", "Unknown error"))
error_data = span.error.get("data", {})
# Extract error type if available
if "type" in span.error:
error_type = span.error["type"]
elif "code" in span.error:
error_type = span.error["code"]
elif isinstance(span.error, str):
error_message = span.error
elif hasattr(span.error, "message"):
error_message = span.error.message
# Use type() for more reliable class name access
error_type = type(span.error).__name__
elif hasattr(span.error, "__str__"):
# Fallback to string representation
error_message = str(span.error)
# Record the exception with proper error data
try:
exception = Exception(error_message)
error_data_json = json.dumps(error_data) if error_data else "{}"
otel_span.record_exception(
exception=exception,
attributes={"error.data": error_data_json},
)
except Exception as e:
# If JSON serialization fails, use simpler approach
logger.warning(f"Error serializing error data: {e}")
otel_span.record_exception(Exception(error_message))
# Set error attributes
otel_span.set_attribute(CoreAttributes.ERROR_TYPE, error_type)
otel_span.set_attribute(CoreAttributes.ERROR_MESSAGE, error_message)
def cleanup(self):
"""Clean up any outstanding spans during shutdown.
This ensures we don't leak span resources when the exporter is shutdown.
"""
# Clear all tracking dictionaries
self._active_spans.clear()
self._span_map.clear()
@@ -0,0 +1,107 @@
"""OpenAI Agents SDK Instrumentation for AgentOps
This module provides instrumentation for the OpenAI Agents SDK, leveraging its built-in
tracing API for observability. It captures detailed information about agent execution,
tool usage, LLM requests, and token metrics.
The implementation uses a clean separation between exporters and processors. The exporter
translates Agent spans into OpenTelemetry spans with appropriate semantic conventions.
The processor implements the tracing interface, collects metrics, and manages timing data.
We use the built-in add_trace_processor hook for all functionality. Streaming support
would require monkey-patching the run method of `Runner`, but doesn't really get us
more data than we already have, since the `Response` object is always passed to us
from the `agents.tracing` module.
TODO Calls to the OpenAI API are not available in this tracing context, so we may
need to monkey-patch the `openai` from here to get that data. While we do have
separate instrumentation for the OpenAI API, in order to get it to nest with the
spans we create here, it's probably easier (or even required) that we incorporate
that here as well.
"""
from typing import Collection
from opentelemetry import trace
from opentelemetry.instrumentation.instrumentor import BaseInstrumentor # type: ignore
from agentops.instrumentation.agentic.openai_agents import LIBRARY_VERSION
from agentops.logging import logger
from agentops.instrumentation.agentic.openai_agents.processor import OpenAIAgentsProcessor
from agentops.instrumentation.agentic.openai_agents.exporter import OpenAIAgentsExporter
class OpenAIAgentsInstrumentor(BaseInstrumentor):
"""An instrumentor for OpenAI Agents SDK that uses the built-in tracing API."""
_processor = None
_exporter = None
_default_processor = None
def __init__(self):
super().__init__()
self._tracer = None
self._is_instrumented_instance_flag = False
def instrumentation_dependencies(self) -> Collection[str]:
"""Return packages required for instrumentation."""
return ["openai-agents >= 0.0.1"]
def _instrument(self, **kwargs):
"""Instrument the OpenAI Agents SDK."""
if self._is_instrumented_instance_flag:
logger.debug("OpenAI Agents SDK already instrumented. Skipping.")
return
tracer_provider = kwargs.get("tracer_provider")
if self._tracer is None:
logger.debug("OpenAI Agents SDK tracer is None, creating new tracer.")
self._tracer = trace.get_tracer("agentops.instrumentation.openai_agents", LIBRARY_VERSION)
try:
self._exporter = OpenAIAgentsExporter(tracer_provider=tracer_provider)
self._processor = OpenAIAgentsProcessor(
exporter=self._exporter,
)
# Replace the default processor with our processor
from agents import set_trace_processors
from agents.tracing.processors import default_processor
if getattr(self, "_default_processor", None) is None:
self._default_processor = default_processor()
# Store reference to default processor for later restoration
set_trace_processors([self._processor])
self._is_instrumented_instance_flag = True
except Exception as e:
logger.warning(f"Failed to instrument OpenAI Agents SDK: {e}", exc_info=True)
def _uninstrument(self, **kwargs):
"""Remove instrumentation from OpenAI Agents SDK."""
if not self._is_instrumented_instance_flag:
logger.debug("OpenAI Agents SDK not currently instrumented. Skipping uninstrument.")
return
try:
# Clean up any active spans in the exporter
if hasattr(self, "_exporter") and self._exporter:
if hasattr(self._exporter, "cleanup"):
self._exporter.cleanup()
# Put back the default processor
from agents import set_trace_processors
if hasattr(self, "_default_processor") and self._default_processor:
set_trace_processors([self._default_processor])
self._default_processor = None
else:
logger.warning("OpenAI Agents SDK has no default processor to restore.")
self._processor = None
self._exporter = None
self._is_instrumented_instance_flag = False
logger.info("Successfully removed OpenAI Agents SDK instrumentation")
except Exception as e:
logger.warning(f"Failed to uninstrument OpenAI Agents SDK: {e}", exc_info=True)
@@ -0,0 +1,58 @@
from typing import Any
from opentelemetry.trace import StatusCode
from agentops.logging import logger
class OpenAIAgentsProcessor:
"""Processor for OpenAI Agents SDK traces.
This processor implements the TracingProcessor interface from the Agents SDK
and converts trace events to OpenTelemetry spans and metrics.
The processor does NOT directly create OpenTelemetry spans.
It delegates span creation to the OpenAIAgentsExporter.
"""
def __init__(self, exporter=None):
self.exporter = exporter
def on_trace_start(self, sdk_trace: Any) -> None:
"""Called when a trace starts in the Agents SDK."""
logger.debug(f"[agentops.instrumentation.openai_agents] Trace started: {sdk_trace}")
self.exporter.export_trace(sdk_trace)
def on_trace_end(self, sdk_trace: Any) -> None:
"""Called when a trace ends in the Agents SDK."""
# Mark this as an end event
# This is used by the exporter to determine whether to create or update a trace
sdk_trace.status = StatusCode.OK.name
logger.debug(f"[agentops.instrumentation.openai_agents] Trace ended: {sdk_trace}")
self.exporter.export_trace(sdk_trace)
def on_span_start(self, span: Any) -> None:
"""Called when a span starts in the Agents SDK."""
logger.debug(f"[agentops.instrumentation.openai_agents] Span started: {span}")
self.exporter.export_span(span)
def on_span_end(self, span: Any) -> None:
"""Called when a span ends in the Agents SDK."""
# Mark this as an end event
# This is used by the exporter to determine whether to create or update a span
span.status = StatusCode.OK.name
logger.debug(f"[agentops.instrumentation.openai_agents] Span ended: {span}")
self.exporter.export_span(span)
def shutdown(self) -> None:
"""Called when the application stops."""
pass
def force_flush(self) -> None:
"""Forces an immediate flush of all queued spans/traces."""
# We don't queue spans so this is a no-op
pass
@@ -0,0 +1,88 @@
# SmoLAgents Instrumentation
This module provides OpenTelemetry instrumentation for the SmoLAgents framework. It captures telemetry data from model operations, agent executions, and tool usage.
## Features
- Model operation tracking
- Text generation
- Token usage
- Streaming responses
- Latency metrics
- Agent execution monitoring
- Step-by-step execution
- Planning phases
- Tool usage
- Execution time
- Tool usage analytics
- Tool call patterns
- Success/failure rates
- Execution time
- Error tracking
## Usage
```python
from agentops import init
from agentops.instrumentation.smolagents import SmolagentsInstrumentor
# Initialize AgentOps with your API key
init(api_key="your-api-key")
# The instrumentation will be automatically activated
# All SmoLAgents operations will now be tracked
```
## Metrics Collected
1. Token Usage
- Input tokens
- Output tokens
- Total tokens per operation
2. Timing Metrics
- Operation duration
- Time to first token (streaming)
- Tool execution time
- Planning phase duration
3. Agent Metrics
- Step counts
- Planning steps
- Tools used
- Success/failure rates
4. Error Tracking
- Generation errors
- Tool execution errors
- Parsing errors
## Architecture
The instrumentation is built on OpenTelemetry and follows the same pattern as other AgentOps instrumentors:
1. Attribute Extractors
- Model attributes
- Agent attributes
- Tool call attributes
2. Wrappers
- Method wrappers for sync operations
- Stream wrappers for async operations
- Context propagation handling
3. Metrics
- Histograms for distributions
- Counters for events
- Custom attributes for filtering
## Contributing
When adding new features or modifying existing ones:
1. Follow the established pattern for attribute extraction
2. Maintain context propagation
3. Add appropriate error handling
4. Update tests and documentation
@@ -0,0 +1,12 @@
"""SmoLAgents instrumentation for AgentOps."""
from agentops.instrumentation.common import LibraryInfo
# Library information
_library_info = LibraryInfo(name="smolagents", default_version="1.16.0")
LIBRARY_NAME = _library_info.name
LIBRARY_VERSION = _library_info.version
from agentops.instrumentation.agentic.smolagents.instrumentor import SmolagentsInstrumentor # noqa: E402
__all__ = ["SmolagentsInstrumentor"]
@@ -0,0 +1,354 @@
"""Attribute extractors for SmoLAgents agent operations."""
from typing import Any, Dict, Optional, Tuple
import uuid
import json
from agentops.instrumentation.common.attributes import get_common_attributes
from agentops.semconv.agent import AgentAttributes
from agentops.semconv.tool import ToolAttributes
def get_agent_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from an agent execution call.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Extract agent instance information
agent_instance = None
if args and len(args) > 0:
agent_instance = args[0]
elif kwargs and "self" in kwargs:
agent_instance = kwargs["self"]
if agent_instance:
# Extract agent name
agent_name = getattr(agent_instance, "name", agent_instance.__class__.__name__)
attributes[AgentAttributes.AGENT_NAME] = agent_name
# Generate agent ID if not present
agent_id = getattr(agent_instance, "id", str(uuid.uuid4()))
attributes[AgentAttributes.AGENT_ID] = agent_id
# Extract agent role/type
attributes[AgentAttributes.AGENT_ROLE] = "executor"
# Extract tools information
tools = getattr(agent_instance, "tools", [])
if tools:
tool_names = []
for tool in tools:
tool_name = getattr(tool, "name", str(tool))
tool_names.append(tool_name)
attributes[AgentAttributes.AGENT_TOOLS] = json.dumps(tool_names)
else:
attributes[AgentAttributes.AGENT_TOOLS] = "[]"
# Extract managed agents information
managed_agents = getattr(agent_instance, "managed_agents", [])
if managed_agents:
managed_agent_names = []
for managed_agent in managed_agents:
agent_name = getattr(managed_agent, "name", managed_agent.__class__.__name__)
managed_agent_names.append(agent_name)
attributes[AgentAttributes.AGENT_MANAGED_AGENTS] = json.dumps(managed_agent_names)
else:
attributes[AgentAttributes.AGENT_MANAGED_AGENTS] = "[]"
# Extract input/task from args or kwargs
task_input = None
if args and len(args) > 1:
task_input = args[1]
elif kwargs and "task" in kwargs:
task_input = kwargs["task"]
elif kwargs and "prompt" in kwargs:
task_input = kwargs["prompt"]
if task_input:
attributes["agent.task"] = str(task_input)
# Extract return value/output
if return_value is not None:
attributes["agentops.entity.output"] = str(return_value)
except Exception:
# If extraction fails, continue with basic attributes
pass
return attributes
def get_agent_stream_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from an agent streaming call.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Extract reasoning/task information
if kwargs:
if "max_steps" in kwargs:
attributes["agent.max_steps"] = str(kwargs["max_steps"])
# Extract task/reasoning from various parameter names
task_info = None
for param_name in ["task", "prompt", "reasoning", "query"]:
if param_name in kwargs:
task_info = kwargs[param_name]
break
if task_info:
attributes["agent.reasoning"] = str(task_info)
# Extract from args
if args and len(args) > 1:
task_info = args[1]
attributes["agent.reasoning"] = str(task_info)
except Exception:
# If extraction fails, continue with basic attributes
pass
return attributes
def get_agent_step_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from an agent step execution.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Try to extract step information
step_number = getattr(args[0] if args else None, "step_count", None)
if step_number is not None:
attributes["agent.step_number"] = str(step_number)
# Extract step name/type
step_name = "ActionStep" # Default for smolagents
attributes["agent.name"] = step_name
# Extract return value
if return_value is not None:
attributes["agentops.entity.output"] = str(return_value)
except Exception:
# If extraction fails, continue with basic attributes
pass
return attributes
def get_tool_call_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from a tool call execution.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Generate tool execution ID
tool_id = str(uuid.uuid4())
attributes[ToolAttributes.TOOL_ID] = tool_id
# Extract tool information from various sources
tool_name = "unknown"
tool_description = "unknown"
tool_parameters = "{}"
# Try to extract from instance (first arg)
if args and len(args) > 0:
instance = args[0]
if hasattr(instance, "name"):
tool_name = instance.name
if hasattr(instance, "description"):
tool_description = instance.description
# Try to extract from kwargs
if kwargs:
if "tool_call" in kwargs:
tool_call = kwargs["tool_call"]
if hasattr(tool_call, "function"):
tool_name = tool_call.function.name
if hasattr(tool_call.function, "arguments"):
tool_parameters = tool_call.function.arguments
elif "name" in kwargs:
tool_name = kwargs["name"]
elif "function_name" in kwargs:
tool_name = kwargs["function_name"]
# Extract parameters
if "parameters" in kwargs:
tool_parameters = json.dumps(kwargs["parameters"])
elif "arguments" in kwargs:
tool_parameters = json.dumps(kwargs["arguments"])
elif "args" in kwargs:
tool_parameters = json.dumps(kwargs["args"])
# Set tool attributes
attributes[ToolAttributes.TOOL_NAME] = tool_name
attributes[ToolAttributes.TOOL_DESCRIPTION] = tool_description
attributes[ToolAttributes.TOOL_PARAMETERS] = tool_parameters
attributes[ToolAttributes.TOOL_STATUS] = "pending"
attributes[ToolAttributes.TOOL_OUTPUT_TYPE] = "unknown"
attributes[ToolAttributes.TOOL_INPUTS] = "{}"
# Extract return value
if return_value is not None:
attributes["tool.result"] = str(return_value)
attributes[ToolAttributes.TOOL_STATUS] = "success"
except Exception:
# If extraction fails, set basic attributes
attributes[ToolAttributes.TOOL_NAME] = "unknown"
attributes[ToolAttributes.TOOL_DESCRIPTION] = "unknown"
attributes[ToolAttributes.TOOL_ID] = str(uuid.uuid4())
attributes[ToolAttributes.TOOL_PARAMETERS] = "{}"
attributes[ToolAttributes.TOOL_STATUS] = "pending"
return attributes
def get_planning_step_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from a planning step execution.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Extract planning information
if kwargs:
if "planning_step" in kwargs:
step = kwargs["planning_step"]
attributes["agent.planning.step"] = str(step)
if "reasoning" in kwargs:
attributes["agent.planning.reasoning"] = str(kwargs["reasoning"])
# Extract return value
if return_value is not None:
attributes["agentops.entity.output"] = str(return_value)
except Exception:
# If extraction fails, continue with basic attributes
pass
return attributes
def get_managed_agent_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from a managed agent call.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Extract managed agent information
agent_instance = None
if args and len(args) > 0:
agent_instance = args[0]
elif kwargs and "agent" in kwargs:
agent_instance = kwargs["agent"]
if agent_instance:
# Extract agent details
agent_name = getattr(agent_instance, "name", agent_instance.__class__.__name__)
agent_id = getattr(agent_instance, "id", str(uuid.uuid4()))
agent_description = getattr(agent_instance, "description", "")
attributes[AgentAttributes.AGENT_NAME] = agent_name
attributes[AgentAttributes.AGENT_ID] = agent_id
attributes[AgentAttributes.AGENT_ROLE] = "managed"
attributes[AgentAttributes.AGENT_TYPE] = agent_instance.__class__.__name__
if agent_description:
attributes[AgentAttributes.AGENT_DESCRIPTION] = agent_description
# Check if this agent provides run summaries
attributes["agent.provide_run_summary"] = "false" # Default for smolagents
# Extract task information
task = None
if args and len(args) > 1:
task = args[1]
elif kwargs and "task" in kwargs:
task = kwargs["task"]
if task:
attributes["agent.task"] = str(task)
# Extract return value
if return_value is not None:
attributes["agentops.entity.output"] = str(return_value)
except Exception:
# If extraction fails, continue with basic attributes
pass
return attributes
@@ -0,0 +1,205 @@
"""Attribute extractors for SmoLAgents model operations."""
from typing import Any, Dict, Optional, Tuple
import json
from agentops.instrumentation.common.attributes import (
get_common_attributes,
)
from agentops.semconv.message import MessageAttributes
from agentops.semconv.span_attributes import SpanAttributes
def get_model_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from a model generation call.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Extract model name from various sources
model_name = "unknown"
# Try to get from kwargs
if kwargs:
if "model" in kwargs:
model_name = kwargs["model"]
elif kwargs.get("self") and hasattr(kwargs["self"], "model_id"):
model_name = kwargs["self"].model_id
# Try to get from args (instance is usually first arg in methods)
if model_name == "unknown" and args and len(args) > 0:
instance = args[0]
if hasattr(instance, "model_id"):
model_name = instance.model_id
# Set model attributes
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_name
# Extract messages from kwargs
if kwargs and "messages" in kwargs:
messages = kwargs["messages"]
if isinstance(messages, list):
for i, message in enumerate(messages):
message_dict = message
if hasattr(message, "to_dict"):
message_dict = message.to_dict()
elif hasattr(message, "__dict__"):
message_dict = message.__dict__
if isinstance(message_dict, dict):
# Set role
role = message_dict.get("role", "user")
attributes[MessageAttributes.PROMPT_ROLE.format(i=i)] = role
# Set content
content = message_dict.get("content", "")
if content:
attributes[MessageAttributes.PROMPT_CONTENT.format(i=i)] = str(content)
# Extract tools from kwargs
if kwargs and "tools_to_call_from" in kwargs:
tools = kwargs["tools_to_call_from"]
if tools and isinstance(tools, list):
for i, tool in enumerate(tools):
tool_name = getattr(tool, "name", "unknown")
tool_description = getattr(tool, "description", "")
attributes[MessageAttributes.TOOL_CALL_NAME.format(i=i)] = tool_name
if tool_description:
attributes[MessageAttributes.TOOL_CALL_DESCRIPTION.format(i=i)] = tool_description
# Extract additional parameters
if kwargs:
if "temperature" in kwargs:
attributes[SpanAttributes.LLM_REQUEST_TEMPERATURE] = kwargs["temperature"]
if "max_tokens" in kwargs:
attributes[SpanAttributes.LLM_REQUEST_MAX_TOKENS] = kwargs["max_tokens"]
if "stop_sequences" in kwargs:
attributes[SpanAttributes.LLM_REQUEST_STOP_SEQUENCES] = json.dumps(kwargs["stop_sequences"])
# Extract response attributes
if return_value:
try:
# Handle ChatMessage response
if hasattr(return_value, "content"):
attributes[MessageAttributes.COMPLETION_CONTENT.format(i=0)] = str(return_value.content)
if hasattr(return_value, "role"):
attributes[MessageAttributes.COMPLETION_ROLE.format(i=0)] = return_value.role
# Handle tool calls in response
if hasattr(return_value, "tool_calls") and return_value.tool_calls:
for j, tool_call in enumerate(return_value.tool_calls):
if hasattr(tool_call, "function"):
attributes[MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=j)] = (
tool_call.function.name
)
if hasattr(tool_call.function, "arguments"):
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=j)] = (
tool_call.function.arguments
)
if hasattr(tool_call, "id"):
attributes[MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=j)] = tool_call.id
# Extract token usage
if hasattr(return_value, "token_usage") and return_value.token_usage:
token_usage = return_value.token_usage
if hasattr(token_usage, "input_tokens"):
attributes[SpanAttributes.LLM_USAGE_PROMPT_TOKENS] = token_usage.input_tokens
if hasattr(token_usage, "output_tokens"):
attributes[SpanAttributes.LLM_USAGE_COMPLETION_TOKENS] = token_usage.output_tokens
if hasattr(token_usage, "total_tokens"):
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = token_usage.total_tokens
# Extract response ID
if hasattr(return_value, "raw") and return_value.raw:
raw_response = return_value.raw
if hasattr(raw_response, "id"):
attributes[SpanAttributes.LLM_RESPONSE_ID] = raw_response.id
except Exception:
# If we can't extract response attributes, continue with what we have
pass
except Exception:
# If extraction fails, return basic attributes
pass
return attributes
def get_stream_attributes(
args: Optional[Tuple] = None,
kwargs: Optional[Dict] = None,
return_value: Optional[Any] = None,
) -> Dict[str, Any]:
"""Extract attributes from a streaming model generation call.
Args:
args: Optional tuple of positional arguments
kwargs: Optional dict of keyword arguments
return_value: Optional return value from the function
Returns:
Dictionary of extracted attributes
"""
attributes = get_common_attributes()
try:
# Extract model name
model_name = "unknown"
if kwargs and kwargs.get("self") and hasattr(kwargs["self"], "model_id"):
model_name = kwargs["self"].model_id
elif args and len(args) > 0 and hasattr(args[0], "model_id"):
model_name = args[0].model_id
attributes[SpanAttributes.LLM_REQUEST_MODEL] = model_name
attributes["gen_ai.request.streaming"] = True
# Extract messages for streaming
if kwargs and "messages" in kwargs:
messages = kwargs["messages"]
if isinstance(messages, list):
for i, message in enumerate(messages):
message_dict = message
if hasattr(message, "to_dict"):
message_dict = message.to_dict()
elif hasattr(message, "__dict__"):
message_dict = message.__dict__
if isinstance(message_dict, dict):
role = message_dict.get("role", "user")
attributes[MessageAttributes.PROMPT_ROLE.format(i=i)] = role
content = message_dict.get("content", "")
if content:
attributes[MessageAttributes.PROMPT_CONTENT.format(i=i)] = str(content)
# Extract tools for streaming
if kwargs and "tools_to_call_from" in kwargs:
tools = kwargs["tools_to_call_from"]
if tools and isinstance(tools, list):
for i, tool in enumerate(tools):
tool_name = getattr(tool, "name", "unknown")
tool_description = getattr(tool, "description", "")
attributes[MessageAttributes.TOOL_CALL_NAME.format(i=i)] = tool_name
if tool_description:
attributes[MessageAttributes.TOOL_CALL_DESCRIPTION.format(i=i)] = tool_description
except Exception:
# If extraction fails, return basic attributes
pass
return attributes
@@ -0,0 +1,286 @@
"""SmoLAgents instrumentation for AgentOps."""
from typing import Dict, Any
from opentelemetry.trace import SpanKind
from opentelemetry.metrics import Meter
from wrapt import wrap_function_wrapper
from agentops.instrumentation.common import CommonInstrumentor, StandardMetrics, InstrumentorConfig
from agentops.logging import logger
# Library info for tracer/meter
LIBRARY_NAME = "agentops.instrumentation.smolagents"
LIBRARY_VERSION = "0.1.0"
# Import attribute handlers
try:
from agentops.instrumentation.agentic.smolagents.attributes.agent import (
get_agent_attributes,
get_tool_call_attributes,
get_planning_step_attributes,
get_agent_step_attributes,
get_agent_stream_attributes,
get_managed_agent_attributes,
)
from agentops.instrumentation.agentic.smolagents.attributes.model import (
get_model_attributes,
get_stream_attributes,
)
except ImportError:
# Fallback functions if imports fail
def get_agent_attributes(*args, **kwargs):
return {}
def get_tool_call_attributes(*args, **kwargs):
return {}
def get_planning_step_attributes(*args, **kwargs):
return {}
def get_agent_step_attributes(*args, **kwargs):
return {}
def get_agent_stream_attributes(*args, **kwargs):
return {}
def get_managed_agent_attributes(*args, **kwargs):
return {}
def get_model_attributes(*args, **kwargs):
return {}
def get_stream_attributes(*args, **kwargs):
return {}
class SmolagentsInstrumentor(CommonInstrumentor):
"""Instrumentor for SmoLAgents library."""
def __init__(self):
"""Initialize the SmoLAgents instrumentor."""
# Create instrumentor config
config = InstrumentorConfig(
library_name=LIBRARY_NAME,
library_version=LIBRARY_VERSION,
wrapped_methods=[], # We use custom wrapping
metrics_enabled=True,
dependencies=["smolagents >= 1.0.0", "litellm"],
)
super().__init__(config)
def _create_metrics(self, meter: Meter) -> Dict[str, Any]:
"""Create metrics for the instrumentor.
Returns a dictionary of metric name to metric instance.
"""
# Create standard metrics for LLM operations
return StandardMetrics.create_standard_metrics(meter)
def _custom_wrap(self, **kwargs):
"""Apply custom wrapping for SmoLAgents.
This is called after normal wrapping, but we use it for all wrapping
since we don't have normal wrapped methods.
"""
# Core agent operations
wrap_function_wrapper("smolagents.agents", "CodeAgent.run", self._agent_run_wrapper(self._tracer))
wrap_function_wrapper("smolagents.agents", "ToolCallingAgent.run", self._agent_run_wrapper(self._tracer))
# Tool calling operations
wrap_function_wrapper(
"smolagents.agents", "ToolCallingAgent.execute_tool_call", self._tool_execution_wrapper(self._tracer)
)
# Model operations with proper model name extraction
wrap_function_wrapper("smolagents.models", "LiteLLMModel.generate", self._llm_wrapper(self._tracer))
wrap_function_wrapper("smolagents.models", "LiteLLMModel.generate_stream", self._llm_wrapper(self._tracer))
logger.info("SmoLAgents instrumentation enabled")
def _agent_run_wrapper(self, tracer):
"""Wrapper for agent run methods."""
def wrapper(wrapped, instance, args, kwargs):
# Get proper agent name - handle None case
agent_name = getattr(instance, "name", None)
if not agent_name: # Handle None, empty string, or missing attribute
agent_name = instance.__class__.__name__
span_name = f"{agent_name}.run"
with tracer.start_as_current_span(
span_name,
kind=SpanKind.CLIENT,
) as span:
# Extract attributes
attributes = get_agent_attributes(args=(instance,) + args, kwargs=kwargs)
# Fix managed agents attribute
if hasattr(instance, "managed_agents") and instance.managed_agents:
managed_agent_names = []
for agent in instance.managed_agents:
name = getattr(agent, "name", None)
if not name: # Handle None case for managed agents too
name = agent.__class__.__name__
managed_agent_names.append(name)
attributes["agent.managed_agents"] = str(managed_agent_names)
else:
attributes["agent.managed_agents"] = "[]"
for key, value in attributes.items():
if value is not None:
span.set_attribute(key, value)
try:
result = wrapped(*args, **kwargs)
# Set output attribute
if result is not None:
span.set_attribute("agentops.entity.output", str(result))
return result
except Exception as e:
span.record_exception(e)
raise
return wrapper
def _tool_execution_wrapper(self, tracer):
"""Wrapper for tool execution methods."""
def wrapper(wrapped, instance, args, kwargs):
# Extract tool name for better span naming
tool_name = "unknown"
if args and len(args) > 0:
tool_call = args[0]
if hasattr(tool_call, "function"):
tool_name = tool_call.function.name
span_name = f"tool.{tool_name}" if tool_name != "unknown" else "tool.execute"
with tracer.start_as_current_span(
span_name,
kind=SpanKind.CLIENT,
) as span:
# Extract tool information from kwargs or args
tool_params = "{}"
# Try to extract tool call information
if args and len(args) > 0:
tool_call = args[0]
if hasattr(tool_call, "function"):
if hasattr(tool_call.function, "arguments"):
tool_params = str(tool_call.function.arguments)
# Extract attributes
attributes = get_tool_call_attributes(args=(instance,) + args, kwargs=kwargs)
# Override with better tool information if available
if tool_name != "unknown":
attributes["tool.name"] = tool_name
attributes["tool.parameters"] = tool_params
for key, value in attributes.items():
if value is not None:
span.set_attribute(key, value)
try:
result = wrapped(*args, **kwargs)
# Set success status and result
span.set_attribute("tool.status", "success")
if result is not None:
span.set_attribute("tool.result", str(result))
return result
except Exception as e:
span.set_attribute("tool.status", "error")
span.record_exception(e)
raise
return wrapper
def _llm_wrapper(self, tracer):
"""Wrapper for LLM generation methods with proper model name extraction."""
def wrapper(wrapped, instance, args, kwargs):
# Extract model name from instance
model_name = getattr(instance, "model_id", "unknown")
# Determine if this is streaming
is_streaming = "generate_stream" in wrapped.__name__
operation = "generate_stream" if is_streaming else "generate"
span_name = f"litellm.{operation} ({model_name})" if model_name != "unknown" else f"litellm.{operation}"
with tracer.start_as_current_span(
span_name,
kind=SpanKind.CLIENT,
) as span:
# Extract attributes
if is_streaming:
attributes = get_stream_attributes(args=(instance,) + args, kwargs=kwargs)
else:
attributes = get_model_attributes(args=(instance,) + args, kwargs=kwargs)
# Ensure model name is properly set
attributes["gen_ai.request.model"] = model_name
for key, value in attributes.items():
if value is not None:
span.set_attribute(key, value)
try:
result = wrapped(*args, **kwargs)
# Extract response attributes if available
if result and hasattr(result, "content"):
span.set_attribute("gen_ai.completion.0.content", str(result.content))
if result and hasattr(result, "token_usage"):
token_usage = result.token_usage
if hasattr(token_usage, "input_tokens"):
span.set_attribute("gen_ai.usage.prompt_tokens", token_usage.input_tokens)
if hasattr(token_usage, "output_tokens"):
span.set_attribute("gen_ai.usage.completion_tokens", token_usage.output_tokens)
return result
except Exception as e:
span.record_exception(e)
raise
return wrapper
def _custom_unwrap(self, **kwargs):
"""Remove custom wrapping from SmoLAgents.
This method removes all custom wrapping we applied.
"""
# Unwrap all instrumented methods
from opentelemetry.instrumentation.utils import unwrap
try:
unwrap("smolagents.agents", "CodeAgent.run")
except Exception as e:
logger.debug(f"Failed to unwrap CodeAgent.run: {e}")
try:
unwrap("smolagents.agents", "ToolCallingAgent.run")
except Exception as e:
logger.debug(f"Failed to unwrap ToolCallingAgent.run: {e}")
try:
unwrap("smolagents.agents", "ToolCallingAgent.execute_tool_call")
except Exception as e:
logger.debug(f"Failed to unwrap ToolCallingAgent.execute_tool_call: {e}")
try:
unwrap("smolagents.models", "LiteLLMModel.generate")
except Exception as e:
logger.debug(f"Failed to unwrap LiteLLMModel.generate: {e}")
try:
unwrap("smolagents.models", "LiteLLMModel.generate_stream")
except Exception as e:
logger.debug(f"Failed to unwrap LiteLLMModel.generate_stream: {e}")
logger.info("SmoLAgents instrumentation disabled")
@@ -0,0 +1,258 @@
"""Stream wrapper for SmoLAgents model streaming responses."""
import time
import uuid
from typing import Any, Generator, Optional
from opentelemetry.trace import Status, StatusCode, Span
from agentops.semconv.message import MessageAttributes
from agentops.semconv.agent import AgentAttributes
from agentops.semconv.tool import ToolAttributes
from .attributes.model import get_stream_attributes
from agentops.semconv.span_attributes import SpanAttributes
def model_stream_wrapper(tracer):
"""Wrapper for model streaming methods.
Args:
tracer: OpenTelemetry tracer
Returns:
Wrapped function
"""
def wrapper(wrapped, instance, args, kwargs):
messages = kwargs.get("messages", [])
model_id = instance.model_id if hasattr(instance, "model_id") else "unknown"
with tracer.start_as_current_span(
name=f"{model_id}.generate_stream", attributes=get_stream_attributes(model_id=model_id, messages=messages)
) as span:
try:
# Start streaming
stream = wrapped(*args, **kwargs)
first_token_received = False
start_time = time.time()
accumulated_text = ""
# Process stream
for chunk in stream:
if not first_token_received:
first_token_received = True
span.set_attribute("gen_ai.time_to_first_token", time.time() - start_time)
# Accumulate text and update attributes
if hasattr(chunk, "content") and chunk.content:
accumulated_text += chunk.content
span.set_attribute(MessageAttributes.COMPLETION_CONTENT.format(i=0), accumulated_text)
span.set_attribute(MessageAttributes.COMPLETION_TYPE.format(i=0), "text")
yield chunk
# Set final attributes
span.set_attribute("gen_ai.streaming_duration", time.time() - start_time)
span.set_status(Status(StatusCode.OK))
except Exception as e:
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
return wrapper
def agent_stream_wrapper(tracer):
"""Wrapper for agent streaming methods.
Args:
tracer: OpenTelemetry tracer
Returns:
Wrapped function
"""
def wrapper(wrapped, instance, args, kwargs):
task = kwargs.get("task", args[0] if args else "unknown")
agent_type = instance.__class__.__name__
agent_id = str(uuid.uuid4())
with tracer.start_as_current_span(
name=f"{agent_type}.run_stream",
attributes={
AgentAttributes.AGENT_ID: agent_id,
AgentAttributes.AGENT_NAME: agent_type,
AgentAttributes.AGENT_ROLE: "executor",
AgentAttributes.AGENT_REASONING: task,
},
) as span:
try:
# Initialize counters
step_count = 0
planning_steps = 0
tools_used = set()
start_time = time.time()
# Process stream
stream = wrapped(*args, **kwargs)
for step in stream:
step_count += 1
# Track step types
if hasattr(step, "type"):
if step.type == "planning":
planning_steps += 1
elif step.type == "tool_call":
tools_used.add(step.tool_name)
# Add tool-specific attributes
span.set_attribute(ToolAttributes.TOOL_NAME, step.tool_name)
if hasattr(step, "arguments"):
span.set_attribute(ToolAttributes.TOOL_PARAMETERS, step.arguments)
# Update span attributes
span.set_attribute("agent.step_count", step_count)
span.set_attribute("agent.planning_steps", planning_steps)
span.set_attribute(AgentAttributes.AGENT_TOOLS, list(tools_used))
yield step
# Set final attributes
span.set_attribute("agent.execution_time", time.time() - start_time)
span.set_status(Status(StatusCode.OK))
except Exception as e:
span.set_status(Status(StatusCode.ERROR, str(e)))
raise
return wrapper
class SmoLAgentsStreamWrapper:
"""Wrapper for streaming responses from SmoLAgents models."""
def __init__(
self,
stream: Generator,
span: Span,
model_id: Optional[str] = None,
):
"""Initialize the stream wrapper.
Args:
stream: The original generator from the model
span: The OpenTelemetry span to track the stream
model_id: Optional model identifier
"""
self._stream = stream
self._span = span
self._model_id = model_id
self._chunks_received = 0
self._full_content = []
self._tool_calls = []
self._current_tool_call = None
self._token_count = 0
def __iter__(self):
"""Iterate over the stream."""
return self
def __next__(self):
"""Get the next chunk from the stream."""
try:
chunk = next(self._stream)
self._process_chunk(chunk)
return chunk
except StopIteration:
self._finalize_stream()
raise
def _process_chunk(self, chunk: Any) -> None:
"""Process a chunk from the stream.
Args:
chunk: The chunk to process
"""
self._chunks_received += 1
# Handle ChatMessageStreamDelta objects
if hasattr(chunk, "content") and chunk.content:
self._full_content.append(chunk.content)
# Handle tool calls in chunks
if hasattr(chunk, "tool_calls") and chunk.tool_calls:
for tool_call in chunk.tool_calls:
if tool_call.id not in [tc["id"] for tc in self._tool_calls]:
self._tool_calls.append(
{
"id": tool_call.id,
"type": tool_call.type,
"name": tool_call.function.name,
"arguments": tool_call.function.arguments,
}
)
# Track token usage if available
if hasattr(chunk, "token_usage") and chunk.token_usage:
if hasattr(chunk.token_usage, "output_tokens"):
self._token_count += chunk.token_usage.output_tokens
# Update span with chunk information
self._span.add_event(
"stream_chunk_received",
{
"chunk_number": self._chunks_received,
"chunk_content_length": len(chunk.content) if hasattr(chunk, "content") and chunk.content else 0,
},
)
def _finalize_stream(self) -> None:
"""Finalize the stream and update span attributes."""
# Combine all content chunks
full_content = "".join(self._full_content)
# Set final attributes on the span
attributes = {
MessageAttributes.COMPLETION_CONTENT.format(i=0): full_content,
"stream.chunks_received": self._chunks_received,
"stream.total_content_length": len(full_content),
}
# Add tool calls if any
if self._tool_calls:
for j, tool_call in enumerate(self._tool_calls):
attributes.update(
{
MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=0, j=j): tool_call["id"],
MessageAttributes.COMPLETION_TOOL_CALL_TYPE.format(i=0, j=j): tool_call["type"],
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=0, j=j): tool_call["name"],
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=0, j=j): str(tool_call["arguments"]),
}
)
# Add token usage if tracked
if self._token_count > 0:
attributes[SpanAttributes.LLM_USAGE_STREAMING_TOKENS] = self._token_count
self._span.set_attributes(attributes)
def close(self) -> None:
"""Close the stream wrapper."""
if hasattr(self._stream, "close"):
self._stream.close()
def wrap_stream(
stream: Generator,
span: Span,
model_id: Optional[str] = None,
) -> SmoLAgentsStreamWrapper:
"""Wrap a streaming response from a SmoLAgents model.
Args:
stream: The original generator from the model
span: The OpenTelemetry span to track the stream
model_id: Optional model identifier
Returns:
SmoLAgentsStreamWrapper: The wrapped stream
"""
return SmoLAgentsStreamWrapper(stream, span, model_id)
@@ -0,0 +1,15 @@
"""Xpander SDK instrumentation for AgentOps."""
from agentops.instrumentation.agentic.xpander.instrumentor import XpanderInstrumentor
from agentops.instrumentation.agentic.xpander.trace_probe import (
wrap_openai_call_for_xpander,
is_xpander_session_active,
get_active_xpander_session,
)
__all__ = [
"XpanderInstrumentor",
"wrap_openai_call_for_xpander",
"is_xpander_session_active",
"get_active_xpander_session",
]
@@ -0,0 +1,112 @@
"""Xpander context management for session tracking."""
import time
import threading
from typing import Any, Dict, Optional
class XpanderContext:
"""Context manager for Xpander sessions with nested conversation spans."""
def __init__(self):
self._sessions = {} # session_id -> session_data
self._workflow_spans = {} # session_id -> active workflow span
self._agent_spans = {} # session_id -> active agent span
self._conversation_spans = {} # session_id -> active conversation span
self._conversation_counters = {} # session_id -> conversation counter
self._lock = threading.Lock()
def start_session(self, session_id: str, agent_info: Dict[str, Any], workflow_span=None, agent_span=None) -> None:
"""Start a new session with agent info."""
with self._lock:
self._sessions[session_id] = {
"agent_name": agent_info.get("agent_name", "unknown"),
"agent_id": agent_info.get("agent_id", "unknown"),
"task_input": agent_info.get("task_input"),
"phase": "planning",
"step_count": 0,
"total_tokens": 0,
"tools_executed": [],
"start_time": time.time(),
}
if workflow_span:
self._workflow_spans[session_id] = workflow_span
if agent_span:
self._agent_spans[session_id] = agent_span
# Initialize conversation counter
self._conversation_counters[session_id] = 0
def start_conversation(self, session_id: str, conversation_span) -> None:
"""Start a new conversation within the session."""
with self._lock:
self._conversation_spans[session_id] = conversation_span
self._conversation_counters[session_id] = self._conversation_counters.get(session_id, 0) + 1
def end_conversation(self, session_id: str) -> None:
"""End the current conversation."""
with self._lock:
if session_id in self._conversation_spans:
del self._conversation_spans[session_id]
def has_active_conversation(self, session_id: str) -> bool:
"""Check if there's an active conversation for this session."""
with self._lock:
return session_id in self._conversation_spans
def get_conversation_counter(self, session_id: str) -> int:
"""Get the current conversation counter."""
with self._lock:
return self._conversation_counters.get(session_id, 0)
def get_session(self, session_id: str) -> Optional[Dict[str, Any]]:
"""Get session data."""
with self._lock:
return self._sessions.get(session_id)
def update_session(self, session_id: str, updates: Dict[str, Any]) -> None:
"""Update session data."""
with self._lock:
if session_id in self._sessions:
self._sessions[session_id].update(updates)
def end_session(self, session_id: str) -> None:
"""End a session."""
with self._lock:
if session_id in self._sessions:
del self._sessions[session_id]
if session_id in self._workflow_spans:
del self._workflow_spans[session_id]
if session_id in self._agent_spans:
del self._agent_spans[session_id]
if session_id in self._conversation_spans:
del self._conversation_spans[session_id]
if session_id in self._conversation_counters:
del self._conversation_counters[session_id]
def get_workflow_phase(self, session_id: str) -> str:
"""Detect current workflow phase based on state."""
with self._lock:
session = self._sessions.get(session_id, {})
if session.get("tools_executed", []):
return "executing"
elif session.get("step_count", 0) > 0:
return "executing"
else:
return "planning"
def get_workflow_span(self, session_id: str):
"""Get the active workflow span for a session."""
with self._lock:
return self._workflow_spans.get(session_id)
def get_agent_span(self, session_id: str):
"""Get the active agent span for a session."""
with self._lock:
return self._agent_spans.get(session_id)
def get_conversation_span(self, session_id: str):
"""Get the active conversation span for a session."""
with self._lock:
return self._conversation_spans.get(session_id)
@@ -0,0 +1,877 @@
"""Xpander SDK instrumentation for AgentOps.
This module provides instrumentation for the Xpander SDK, which uses JSII to convert
TypeScript code to Python at runtime. The instrumentation tracks agent sessions,
tool executions, and LLM interactions.
MODIFIED VERSION: Using existing AgentOps utilities where possible while keeping
runtime-specific instrumentation logic that cannot be replaced.
REPLACEMENTS MADE:
✅ Span creation: Using tracer.make_span() instead of manual span creation
✅ Error handling: Using _finish_span_success/_finish_span_error utilities
✅ Attribute management: Using existing SpanAttributeManager
✅ Serialization: Using safe_serialize and model_to_dict utilities
✅ Attribute setting: Using _update_span utility
RUNTIME-SPECIFIC LOGIC KEPT (Cannot be replaced):
❌ Method wrapping: Runtime method creation requires custom hooks
❌ Context persistence: XpanderContext must handle runtime object lifecycle
❌ Agent detection: Custom logic for dynamically created agents
"""
import logging
import time
import json
from typing import Any, Optional
from opentelemetry.metrics import Meter
from opentelemetry.trace import SpanKind as OTelSpanKind
from opentelemetry import trace
# Use existing AgentOps utilities
from agentops.instrumentation.common import (
CommonInstrumentor,
InstrumentorConfig,
StandardMetrics,
)
from agentops.instrumentation.common.span_management import SpanAttributeManager
from agentops.instrumentation.common.wrappers import _finish_span_success, _finish_span_error, _update_span
from agentops.helpers.serialization import safe_serialize, model_to_dict
from agentops.sdk.core import tracer
from agentops.instrumentation.agentic.xpander.context import XpanderContext
from agentops.semconv import SpanAttributes, SpanKind, ToolAttributes
from agentops.semconv.message import MessageAttributes
# Use existing OpenAI attribute extraction patterns (lazy import to avoid circular imports)
# from agentops.instrumentation.providers.openai.attributes.common import (
# get_response_attributes,
# )
logger = logging.getLogger(__name__)
_instruments = ("xpander-sdk >= 1.0.0",)
# Use existing AgentOps utility instead of custom implementation
def safe_set_attribute(span, key: str, value: Any) -> None:
"""Set attribute on span using existing AgentOps utility."""
try:
_update_span(span, {key: value})
except Exception as e:
logger.warning(f"Failed to set attribute {key}: {e}")
class XpanderInstrumentor(CommonInstrumentor):
"""Instrumentor for Xpander SDK interactions."""
def __init__(self, config: Optional[InstrumentorConfig] = None):
if config is None:
config = InstrumentorConfig(
library_name="xpander-sdk", library_version="1.0.0", dependencies=_instruments, metrics_enabled=True
)
super().__init__(config)
self._context = XpanderContext()
self._tracer = None
# Use existing AgentOps attribute manager
self._attribute_manager = SpanAttributeManager("xpander-service", "production")
def _get_session_id_from_agent(self, agent) -> str:
"""Generate consistent session ID from agent."""
# First try to get memory_thread_id from agent context if available
if hasattr(agent, "memory_thread_id"):
return f"session_{agent.memory_thread_id}"
# Check for execution context
if hasattr(agent, "execution") and hasattr(agent.execution, "memory_thread_id"):
return f"session_{agent.execution.memory_thread_id}"
# Fallback to agent-based ID
agent_name = getattr(agent, "name", "unknown")
agent_id = getattr(agent, "id", "unknown")
return f"agent_{agent_name}_{agent_id}"
def _extract_session_id(self, execution, agent=None) -> str:
"""Extract session ID from execution data."""
if isinstance(execution, dict):
if "memory_thread_id" in execution:
return f"session_{execution['memory_thread_id']}"
elif "thread_id" in execution:
return f"session_{execution['thread_id']}"
elif "session_id" in execution:
return f"session_{execution['session_id']}"
# Fallback to agent-based ID if available
if agent:
return self._get_session_id_from_agent(agent)
# Last resort fallback
return f"session_{int(time.time())}"
def _extract_tool_name(self, tool_call) -> str:
"""Extract tool name from tool call."""
# Handle different tool call formats
if hasattr(tool_call, "function_name"):
return tool_call.function_name
elif hasattr(tool_call, "function") and hasattr(tool_call.function, "name"):
return tool_call.function.name
elif hasattr(tool_call, "name"):
return tool_call.name
elif isinstance(tool_call, dict):
if "function" in tool_call:
return tool_call["function"].get("name", "unknown")
elif "function_name" in tool_call:
return tool_call["function_name"]
elif "name" in tool_call:
return tool_call["name"]
# Try to extract from string representation
import re
patterns = [
r'function[\'"]\s*:\s*[\'"]([^\'"]+)[\'"]',
r'name[\'"]\s*:\s*[\'"]([^\'"]+)[\'"]',
r"([a-zA-Z_][a-zA-Z0-9_]*)\.tool",
r'function_name[\'"]\s*:\s*[\'"]([^\'"]+)[\'"]',
]
tool_str = str(tool_call)
for pattern in patterns:
match = re.search(pattern, tool_str, re.IGNORECASE)
if match:
return match.group(1)
return "unknown"
def _extract_tool_params(self, tool_call) -> dict:
"""Extract tool parameters from tool call."""
# Handle different parameter formats
if hasattr(tool_call, "function") and hasattr(tool_call.function, "arguments"):
try:
args = tool_call.function.arguments
if isinstance(args, str):
return json.loads(args)
elif isinstance(args, dict):
return args
except (json.JSONDecodeError, AttributeError):
pass
elif hasattr(tool_call, "arguments"):
try:
args = tool_call.arguments
if isinstance(args, str):
return json.loads(args)
elif isinstance(args, dict):
return args
except (json.JSONDecodeError, AttributeError):
pass
elif isinstance(tool_call, dict):
if "function" in tool_call:
args = tool_call["function"].get("arguments", "{}")
try:
return json.loads(args) if isinstance(args, str) else args
except json.JSONDecodeError:
pass
elif "arguments" in tool_call:
args = tool_call["arguments"]
try:
return json.loads(args) if isinstance(args, str) else args
except json.JSONDecodeError:
pass
return {}
def _extract_llm_data_from_messages(self, messages) -> dict:
"""Extract LLM metadata from messages."""
data = {}
if isinstance(messages, dict):
# Direct model and usage fields
if "model" in messages:
data["model"] = messages["model"]
if "usage" in messages:
data["usage"] = messages["usage"]
# Check in choices array (OpenAI format)
if "choices" in messages and messages["choices"]:
choice = messages["choices"][0]
if "message" in choice:
message = choice["message"]
if "model" in message:
data["model"] = message["model"]
elif isinstance(messages, list):
# Look for assistant messages with metadata
for msg in messages:
if isinstance(msg, dict) and msg.get("role") == "assistant":
if "model" in msg:
data["model"] = msg["model"]
if "usage" in msg:
data["usage"] = msg["usage"]
break
# Try to extract from any nested structures
if not data and hasattr(messages, "__dict__"):
msg_dict = messages.__dict__
if "model" in msg_dict:
data["model"] = msg_dict["model"]
if "usage" in msg_dict:
data["usage"] = msg_dict["usage"]
return data
def _extract_and_set_openai_message_attributes(self, span, messages, result, agent=None):
"""Extract and set OpenAI message attributes from messages and response."""
try:
# Manual extraction since we don't need the OpenAI module for this
# Try to get the agent's current message history for prompts
agent_messages = []
if agent and hasattr(agent, "messages"):
agent_messages = getattr(agent, "messages", [])
elif agent and hasattr(agent, "conversation_history"):
agent_messages = getattr(agent, "conversation_history", [])
elif agent and hasattr(agent, "history"):
agent_messages = getattr(agent, "history", [])
# Also try to extract messages from the messages parameter itself
if isinstance(messages, list):
# If messages is a list of messages, use it directly
agent_messages.extend(messages)
elif isinstance(messages, dict) and "messages" in messages:
# If messages contains a messages key
agent_messages.extend(messages.get("messages", []))
# Set prompt messages (input to LLM)
prompt_index = 0
for msg in agent_messages[-10:]: # Get last 10 messages to avoid huge context
if isinstance(msg, dict):
role = msg.get("role", "user")
content = msg.get("content", "")
# Handle different content formats
if content and isinstance(content, str) and content.strip():
safe_set_attribute(span, MessageAttributes.PROMPT_ROLE.format(i=prompt_index), role)
safe_set_attribute(
span, MessageAttributes.PROMPT_CONTENT.format(i=prompt_index), content[:2000]
)
prompt_index += 1
elif content and isinstance(content, list):
# Handle multi-modal content
content_str = str(content)[:2000]
safe_set_attribute(span, MessageAttributes.PROMPT_ROLE.format(i=prompt_index), role)
safe_set_attribute(span, MessageAttributes.PROMPT_CONTENT.format(i=prompt_index), content_str)
prompt_index += 1
elif hasattr(msg, "content"):
# Handle object with content attribute
content = getattr(msg, "content", "")
role = getattr(msg, "role", "user")
if content and isinstance(content, str) and content.strip():
safe_set_attribute(span, MessageAttributes.PROMPT_ROLE.format(i=prompt_index), role)
safe_set_attribute(
span, MessageAttributes.PROMPT_CONTENT.format(i=prompt_index), str(content)[:2000]
)
prompt_index += 1
# Set completion messages (response from LLM)
completion_index = 0
response_data = result if result else messages
# Handle different response formats
if isinstance(response_data, dict):
choices = response_data.get("choices", [])
for choice in choices:
message = choice.get("message", {})
role = message.get("role", "assistant")
content = message.get("content", "")
if content:
safe_set_attribute(span, MessageAttributes.COMPLETION_ROLE.format(i=completion_index), role)
safe_set_attribute(
span, MessageAttributes.COMPLETION_CONTENT.format(i=completion_index), content[:2000]
)
# Handle tool calls in the response
tool_calls = message.get("tool_calls", [])
for j, tool_call in enumerate(tool_calls):
tool_id = tool_call.get("id", "")
tool_name = tool_call.get("function", {}).get("name", "")
tool_args = tool_call.get("function", {}).get("arguments", "")
if tool_id:
safe_set_attribute(
span, MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=completion_index, j=j), tool_id
)
if tool_name:
safe_set_attribute(
span,
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=completion_index, j=j),
tool_name,
)
if tool_args:
safe_set_attribute(
span,
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(i=completion_index, j=j),
tool_args[:500],
)
safe_set_attribute(
span,
MessageAttributes.COMPLETION_TOOL_CALL_TYPE.format(i=completion_index, j=j),
"function",
)
completion_index += 1
elif hasattr(response_data, "choices"):
# Handle response object with choices attribute
choices = getattr(response_data, "choices", [])
for choice in choices:
message = getattr(choice, "message", None)
if message:
role = getattr(message, "role", "assistant")
content = getattr(message, "content", "")
if content:
safe_set_attribute(span, MessageAttributes.COMPLETION_ROLE.format(i=completion_index), role)
safe_set_attribute(
span,
MessageAttributes.COMPLETION_CONTENT.format(i=completion_index),
str(content)[:2000],
)
# Handle tool calls
tool_calls = getattr(message, "tool_calls", [])
for j, tool_call in enumerate(tool_calls):
tool_id = getattr(tool_call, "id", "")
function = getattr(tool_call, "function", None)
if function:
tool_name = getattr(function, "name", "")
tool_args = getattr(function, "arguments", "")
if tool_id:
safe_set_attribute(
span,
MessageAttributes.COMPLETION_TOOL_CALL_ID.format(i=completion_index, j=j),
tool_id,
)
if tool_name:
safe_set_attribute(
span,
MessageAttributes.COMPLETION_TOOL_CALL_NAME.format(i=completion_index, j=j),
tool_name,
)
if tool_args:
safe_set_attribute(
span,
MessageAttributes.COMPLETION_TOOL_CALL_ARGUMENTS.format(
i=completion_index, j=j
),
str(tool_args)[:500],
)
safe_set_attribute(
span,
MessageAttributes.COMPLETION_TOOL_CALL_TYPE.format(i=completion_index, j=j),
"function",
)
completion_index += 1
except Exception as e:
logger.error(f"Error extracting OpenAI message attributes: {e}")
def _wrap_init_task(self, original_method):
"""Wrap init_task and add_task to create agent span hierarchy."""
instrumentor = self
def wrapper(self, execution=None, input=None, **kwargs):
# Normalize parameters - handle both add_task(input=...) and init_task(execution=...)
if execution is None and input is not None:
# add_task call with input parameter - normalize to execution format
if isinstance(input, str):
execution = {"input": {"text": input}}
else:
execution = {"input": input}
elif execution is None:
# Neither execution nor input provided - create empty execution
execution = {}
# Extract session ID and agent info
session_id = instrumentor._extract_session_id(execution)
agent_name = getattr(self, "name", "unknown")
agent_id = getattr(self, "id", "unknown")
# Check if session already exists
existing_session = instrumentor._context.get_session(session_id)
if existing_session:
# Session already exists, just continue
# Call with original parameters
if input is not None:
result = original_method(self, input=input, **kwargs)
else:
result = original_method(self, execution)
return result
# Extract task input
task_input = None
if isinstance(execution, dict):
if "input" in execution:
input_data = execution["input"]
if isinstance(input_data, dict) and "text" in input_data:
task_input = input_data["text"]
elif isinstance(input_data, str):
task_input = input_data
# Create top-level conversation/session span - this is the ROOT span
conversation_span_attributes = {
SpanAttributes.AGENTOPS_ENTITY_NAME: f"Session - {agent_name}",
"xpander.span.type": "session",
"xpander.session.name": f"Session - {agent_name}",
"xpander.agent.name": agent_name,
"xpander.agent.id": agent_id,
"xpander.session.id": session_id,
}
session_span, session_ctx, session_token = tracer.make_span(
operation_name=f"session.{agent_name}",
span_kind=SpanKind.AGENT, # Use AGENT kind for the root session span
attributes=conversation_span_attributes,
)
# Set task input on session span
if task_input:
safe_set_attribute(session_span, SpanAttributes.AGENTOPS_ENTITY_INPUT, task_input[:1000])
safe_set_attribute(session_span, "xpander.session.initial_input", task_input[:500])
# Create workflow span as child of session span (this will be the main execution span)
trace.set_span_in_context(session_span)
workflow_span_attributes = {
"xpander.span.type": "workflow",
"xpander.workflow.phase": "planning",
"xpander.agent.name": agent_name,
"xpander.agent.id": agent_id,
"xpander.session.id": session_id,
"agent.name": agent_name,
"agent.id": agent_id,
}
workflow_span, workflow_ctx, workflow_token = tracer.make_span(
operation_name=f"workflow.{agent_name}",
span_kind=SpanKind.WORKFLOW,
attributes=workflow_span_attributes,
)
# No separate agent span - workflow span contains all agent info
# Initialize workflow state with persistent spans
agent_info = {
"agent_name": agent_name,
"agent_id": agent_id,
"task_input": task_input,
"thread_id": execution.get("memory_thread_id") if isinstance(execution, dict) else None,
}
instrumentor._context.start_session(session_id, agent_info, workflow_span, None) # No agent span
# Store the session span as well
instrumentor._context.start_conversation(session_id, session_span)
try:
# Execute original method - don't end agent span here, it will be ended in retrieve_execution_result
# Call with original parameters
if input is not None:
result = original_method(self, input=input, **kwargs)
else:
result = original_method(self, execution)
return result
except Exception as e:
# Use existing AgentOps error handling utilities
_finish_span_error(workflow_span, e)
raise
return wrapper
def _wrap_run_tools(self, original_method):
"""Wrap run_tools to create execution phase tool spans."""
instrumentor = self
def wrapper(self, tool_calls, payload_extension=None):
session_id = instrumentor._get_session_id_from_agent(self)
current_session = instrumentor._context.get_session(session_id)
# Update workflow state
step_num = (current_session.get("step_count", 0) + 1) if current_session else 1
instrumentor._context.update_session(
session_id,
{
"step_count": step_num,
"phase": "executing",
"tools_executed": (current_session.get("tools_executed", []) if current_session else [])
+ [instrumentor._extract_tool_name(tc) for tc in tool_calls],
},
)
# Get current span context (should be the LLM span)
current_span = trace.get_current_span()
# Create execution phase span as child of current LLM span
execution_span_context = trace.set_span_in_context(current_span) if current_span else None
with instrumentor._tracer.start_as_current_span(
"xpander.execution",
kind=OTelSpanKind.INTERNAL,
context=execution_span_context,
attributes={
SpanAttributes.AGENTOPS_SPAN_KIND: SpanKind.TASK,
"xpander.span.type": "execution",
"xpander.workflow.phase": "executing",
"xpander.step.number": step_num,
"xpander.step.tool_count": len(tool_calls),
"xpander.session.id": session_id,
},
) as execution_span:
# Execute tools and create individual tool spans
results = []
conversation_finished = False
for i, tool_call in enumerate(tool_calls):
tool_name = instrumentor._extract_tool_name(tool_call)
tool_params = instrumentor._extract_tool_params(tool_call)
# Check if this is the conversation finish tool
if tool_name == "xpfinish-agent-execution-finished":
conversation_finished = True
start_time = time.time()
# Create tool span as child of execution span
tool_span_context = trace.set_span_in_context(execution_span)
with instrumentor._tracer.start_as_current_span(
f"tool.{tool_name}",
kind=OTelSpanKind.CLIENT,
context=tool_span_context,
attributes={
SpanAttributes.AGENTOPS_SPAN_KIND: SpanKind.TOOL,
ToolAttributes.TOOL_NAME: tool_name,
ToolAttributes.TOOL_PARAMETERS: str(tool_params)[:500],
"xpander.span.type": "tool",
"xpander.workflow.phase": "executing",
"xpander.tool.step": step_num,
"xpander.tool.index": i,
},
) as tool_span:
# Execute single tool
single_result = original_method(self, [tool_call], payload_extension)
results.extend(single_result)
# Record tool execution details
execution_time = time.time() - start_time
safe_set_attribute(tool_span, "xpander.tool.execution_time", execution_time)
# Add tool result if available
if single_result:
result_summary = f"Executed successfully with {len(single_result)} results"
safe_set_attribute(tool_span, "xpander.tool.result_summary", result_summary)
# Store actual result data using existing AgentOps utilities
try:
result_content = ""
for i, result_item in enumerate(single_result):
# Handle xpander_sdk.ToolCallResult objects specifically
if hasattr(result_item, "__class__") and "ToolCallResult" in str(type(result_item)):
# Extract the actual result content from ToolCallResult
try:
if hasattr(result_item, "result") and result_item.result is not None:
actual_result = result_item.result
if isinstance(actual_result, str):
result_content += actual_result[:1000] + "\n"
else:
result_content += safe_serialize(actual_result)[:1000] + "\n"
elif hasattr(result_item, "data") and result_item.data is not None:
result_content += safe_serialize(result_item.data)[:1000] + "\n"
else:
# Fallback: try to find any content attribute
for attr_name in ["content", "output", "value", "response"]:
if hasattr(result_item, attr_name):
attr_value = getattr(result_item, attr_name)
if attr_value is not None:
result_content += safe_serialize(attr_value)[:1000] + "\n"
break
else:
# If no content attributes found, indicate this
result_content += "ToolCallResult object (no extractable content)\n"
except Exception as attr_e:
logger.debug(f"Error extracting from ToolCallResult: {attr_e}")
result_content += "ToolCallResult object (extraction failed)\n"
# Handle regular objects and primitives
elif isinstance(result_item, (str, int, float, bool)):
result_content += str(result_item)[:1000] + "\n"
elif hasattr(result_item, "__dict__"):
# Convert objects to dict using existing utility
result_dict = model_to_dict(result_item)
result_content += safe_serialize(result_dict)[:1000] + "\n"
else:
# Use safe_serialize for consistent conversion
result_content += safe_serialize(result_item)[:1000] + "\n"
if result_content.strip():
final_content = result_content.strip()[:2000]
safe_set_attribute(tool_span, ToolAttributes.TOOL_RESULT, final_content)
else:
safe_set_attribute(
tool_span, ToolAttributes.TOOL_RESULT, "No extractable content found"
)
except Exception as e:
logger.error(f"Error setting tool result: {e}")
safe_set_attribute(
tool_span, ToolAttributes.TOOL_RESULT, f"Error capturing result: {e}"
)
else:
safe_set_attribute(tool_span, "xpander.tool.result_summary", "No results returned")
# If conversation is finished, mark for session closure
if conversation_finished:
# Since session span is now the conversation span, we need to close all spans
# when the conversation finishes
pass # Session closure will be handled in retrieve_execution_result
return results
return wrapper
def _wrap_add_messages(self, original_method):
"""Wrap add_messages to create LLM spans with proper parent-child relationship."""
instrumentor = self
def wrapper(self, messages):
session_id = instrumentor._get_session_id_from_agent(self)
current_session = instrumentor._context.get_session(session_id)
current_phase = instrumentor._context.get_workflow_phase(session_id)
workflow_span = instrumentor._context.get_workflow_span(session_id)
# Create LLM span as child of workflow span (not conversation span)
# The hierarchy should be: session -> agent/workflow -> LLM -> execution -> tools
llm_span_context = trace.set_span_in_context(workflow_span) if workflow_span else None
# Call original method first to get the actual OpenAI response
result = original_method(self, messages)
# Now create a span that captures the LLM interaction with the actual response data
with instrumentor._tracer.start_as_current_span(
f"llm.{current_phase}",
kind=OTelSpanKind.CLIENT,
context=llm_span_context,
attributes={
SpanAttributes.AGENTOPS_SPAN_KIND: SpanKind.LLM,
"xpander.span.type": "llm",
"xpander.workflow.phase": current_phase,
"xpander.session.id": session_id,
},
) as llm_span:
# Extract and set OpenAI message data from the messages and response
instrumentor._extract_and_set_openai_message_attributes(llm_span, messages, result, self)
# Extract and set LLM metadata from the result if possible
llm_data = instrumentor._extract_llm_data_from_messages(result if result else messages)
if llm_data:
if "model" in llm_data:
safe_set_attribute(llm_span, SpanAttributes.LLM_REQUEST_MODEL, llm_data["model"])
safe_set_attribute(llm_span, SpanAttributes.LLM_RESPONSE_MODEL, llm_data["model"])
if "usage" in llm_data:
usage = llm_data["usage"]
if "prompt_tokens" in usage:
safe_set_attribute(llm_span, SpanAttributes.LLM_USAGE_PROMPT_TOKENS, usage["prompt_tokens"])
if "completion_tokens" in usage:
safe_set_attribute(
llm_span, SpanAttributes.LLM_USAGE_COMPLETION_TOKENS, usage["completion_tokens"]
)
if "total_tokens" in usage:
safe_set_attribute(llm_span, SpanAttributes.LLM_USAGE_TOTAL_TOKENS, usage["total_tokens"])
# Update workflow state
instrumentor._context.update_session(
session_id,
{
"total_tokens": (current_session.get("total_tokens", 0) if current_session else 0)
+ usage["total_tokens"]
},
)
return result
return wrapper
def _wrap_is_finished(self, original_method):
"""Wrap is_finished to track workflow completion."""
instrumentor = self
def wrapper(self):
result = original_method(self)
if result:
session_id = instrumentor._get_session_id_from_agent(self)
# Update session to finished state
instrumentor._context.update_session(session_id, {"phase": "finished", "end_time": time.time()})
return result
return wrapper
def _wrap_extract_tool_calls(self, original_method):
"""Wrap extract_tool_calls to track tool planning."""
def wrapper(self, messages):
result = original_method(self, messages)
return result
return wrapper
def _wrap_report_execution_metrics(self, original_method):
"""Wrap report_execution_metrics to track metrics."""
def wrapper(self, llm_tokens=None, ai_model=None):
result = original_method(self, llm_tokens, ai_model)
return result
return wrapper
def _wrap_retrieve_execution_result(self, original_method):
"""Wrap retrieve_execution_result to finalize agent and workflow spans."""
instrumentor = self
def wrapper(self):
session_id = instrumentor._get_session_id_from_agent(self)
current_session = instrumentor._context.get_session(session_id)
workflow_span = instrumentor._context.get_workflow_span(session_id)
session_span = instrumentor._context.get_conversation_span(session_id) # This is now the root session span
try:
# Execute and capture result
result = original_method(self)
# Add workflow summary to the persistent workflow span
if workflow_span and current_session:
safe_set_attribute(
workflow_span, "xpander.workflow.total_steps", current_session.get("step_count", 0)
)
safe_set_attribute(
workflow_span, "xpander.workflow.total_tokens", current_session.get("total_tokens", 0)
)
safe_set_attribute(
workflow_span, "xpander.workflow.tools_used", len(current_session.get("tools_executed", []))
)
# Calculate total execution time
start_time = current_session.get("start_time", time.time())
execution_time = time.time() - start_time
safe_set_attribute(workflow_span, "xpander.workflow.execution_time", execution_time)
safe_set_attribute(workflow_span, "xpander.workflow.phase", "completed")
# Set result details on session and workflow spans
if result:
result_content = ""
if hasattr(result, "result"):
result_content = str(result.result)[:1000]
# Set on session span (root span)
if session_span and result_content:
safe_set_attribute(session_span, SpanAttributes.AGENTOPS_ENTITY_OUTPUT, result_content)
safe_set_attribute(session_span, "xpander.session.final_result", result_content)
if hasattr(result, "memory_thread_id"):
safe_set_attribute(session_span, "xpander.session.thread_id", result.memory_thread_id)
if workflow_span:
if result_content:
safe_set_attribute(workflow_span, "xpander.result.content", result_content)
if hasattr(result, "memory_thread_id"):
safe_set_attribute(workflow_span, "xpander.result.thread_id", result.memory_thread_id)
# Add session summary to session span
if session_span and current_session:
safe_set_attribute(
session_span, "xpander.session.total_steps", current_session.get("step_count", 0)
)
safe_set_attribute(
session_span, "xpander.session.total_tokens", current_session.get("total_tokens", 0)
)
safe_set_attribute(
session_span, "xpander.session.tools_used", len(current_session.get("tools_executed", []))
)
start_time = current_session.get("start_time", time.time())
execution_time = time.time() - start_time
safe_set_attribute(session_span, "xpander.session.execution_time", execution_time)
# Close all spans - session span should be closed last
if workflow_span:
_finish_span_success(workflow_span)
workflow_span.end()
if session_span:
_finish_span_success(session_span)
session_span.end()
return result
except Exception as e:
# Mark spans as failed and close them in proper order
if workflow_span:
_finish_span_error(workflow_span, e)
workflow_span.end()
if session_span:
_finish_span_error(session_span, e)
session_span.end()
raise
finally:
# Clean up session
instrumentor._context.end_session(session_id)
return wrapper
def _instrument(self, **kwargs):
"""Instrument the Xpander SDK."""
try:
# Import xpander modules
from xpander_sdk import Agent
# Set up tracing using existing AgentOps tracer
self._tracer = tracer.get_tracer()
# Attribute manager already initialized in __init__
# Wrap Agent methods
Agent.add_task = self._wrap_init_task(Agent.add_task)
Agent.init_task = self._wrap_init_task(Agent.init_task) # Also wrap init_task for completeness
Agent.run_tools = self._wrap_run_tools(Agent.run_tools)
Agent.add_messages = self._wrap_add_messages(Agent.add_messages)
Agent.is_finished = self._wrap_is_finished(Agent.is_finished)
Agent.extract_tool_calls = self._wrap_extract_tool_calls(Agent.extract_tool_calls)
Agent.report_execution_metrics = self._wrap_report_execution_metrics(Agent.report_execution_metrics)
Agent.retrieve_execution_result = self._wrap_retrieve_execution_result(Agent.retrieve_execution_result)
except ImportError:
logger.debug("Xpander SDK not available")
except Exception as e:
logger.error(f"Failed to instrument Xpander SDK: {e}")
def _uninstrument(self, **kwargs):
"""Uninstrument the Xpander SDK."""
pass
def _create_metrics(self, meter: Meter) -> StandardMetrics:
"""Create metrics for Xpander instrumentation."""
return StandardMetrics(
requests_active=meter.create_up_down_counter(
name="xpander_requests_active",
description="Number of active Xpander requests",
),
requests_duration=meter.create_histogram(
name="xpander_requests_duration",
description="Duration of Xpander requests",
unit="s",
),
requests_total=meter.create_counter(
name="xpander_requests_total",
description="Total number of Xpander requests",
),
requests_error=meter.create_counter(
name="xpander_requests_error",
description="Number of Xpander request errors",
),
)
@@ -0,0 +1,86 @@
"""Xpander trace probe for automatic instrumentation activation.
This module provides automatic instrumentation for Xpander SDK when imported.
It should be imported early in the application lifecycle to ensure all
Xpander interactions are captured.
"""
import logging
from agentops.instrumentation.agentic.xpander.instrumentor import XpanderInstrumentor
logger = logging.getLogger(__name__)
# Global instrumentor instance
_instrumentor = None
def activate_xpander_instrumentation():
"""Activate Xpander instrumentation."""
global _instrumentor
if _instrumentor is None:
try:
_instrumentor = XpanderInstrumentor()
_instrumentor.instrument()
logger.info("Xpander instrumentation activated successfully")
except Exception as e:
logger.error(f"Failed to activate Xpander instrumentation: {e}")
_instrumentor = None
return _instrumentor
def deactivate_xpander_instrumentation():
"""Deactivate Xpander instrumentation."""
global _instrumentor
if _instrumentor is not None:
try:
_instrumentor.uninstrument()
logger.info("Xpander instrumentation deactivated successfully")
except Exception as e:
logger.error(f"Failed to deactivate Xpander instrumentation: {e}")
finally:
_instrumentor = None
def get_instrumentor():
"""Get the active instrumentor instance."""
return _instrumentor
# Stub functions for backward compatibility
def wrap_openai_call_for_xpander(openai_call_func, purpose="general"):
"""Backward compatibility stub - functionality now handled by auto-instrumentation."""
logger.debug(f"wrap_openai_call_for_xpander called with purpose: {purpose}")
return openai_call_func
def is_xpander_session_active():
"""Check if xpander session is active."""
return _instrumentor is not None
def get_active_xpander_session():
"""Get active xpander session."""
return _instrumentor._context if _instrumentor else None
# Convenience functions for cleaner OpenAI integration
def wrap_openai_analysis(openai_call_func):
"""Wrap OpenAI calls for analysis/reasoning steps."""
return wrap_openai_call_for_xpander(openai_call_func, "analysis")
def wrap_openai_planning(openai_call_func):
"""Wrap OpenAI calls for planning steps."""
return wrap_openai_call_for_xpander(openai_call_func, "planning")
def wrap_openai_synthesis(openai_call_func):
"""Wrap OpenAI calls for synthesis/summary steps."""
return wrap_openai_call_for_xpander(openai_call_func, "synthesis")
# Note: Auto-activation is now handled by the main AgentOps instrumentation system
# activate_xpander_instrumentation()
@@ -0,0 +1,3 @@
"""Version information for xpander instrumentation."""
__version__ = "1.0.0"
+65
View File
@@ -0,0 +1,65 @@
# AgentOps Instrumentation Common Module
The `agentops.instrumentation.common` module provides shared utilities for OpenTelemetry instrumentation across different LLM service providers.
## Core Components
### Attribute Handler Example
Attribute handlers extract data from method inputs and outputs:
```python
from typing import Optional, Any, Tuple, Dict
from agentops.instrumentation.common.attributes import AttributeMap
from agentops.semconv import SpanAttributes
def my_attribute_handler(args: Optional[Tuple] = None, kwargs: Optional[Dict] = None, return_value: Optional[Any] = None) -> AttributeMap:
attributes = {}
# Extract attributes from kwargs (method inputs)
if kwargs:
if "model" in kwargs:
attributes[SpanAttributes.MODEL_NAME] = kwargs["model"]
# ...
# Extract attributes from return value (method outputs)
if return_value:
if hasattr(return_value, "model"):
attributes[SpanAttributes.LLM_RESPONSE_MODEL] = return_value.model
# ...
return attributes
```
### `WrapConfig` Class
Config object defining how a method should be wrapped:
```python
from agentops.instrumentation.common.wrappers import WrapConfig
from opentelemetry.trace import SpanKind
config = WrapConfig(
trace_name="llm.completion", # Name that will appear in trace spans
package="openai.resources", # Path to the module containing the class
class_name="Completions", # Name of the class containing the method
method_name="create", # Name of the method to wrap
handler=my_attribute_handler, # Function that extracts attributes
span_kind=SpanKind.CLIENT # Type of span to create
)
```
### Wrapping/Unwrapping Methods
```python
from opentelemetry.trace import get_tracer
from agentops.instrumentation.common.wrappers import wrap, unwrap
# Create a tracer and wrap a method
tracer = get_tracer("openai", "0.0.0")
wrap(config, tracer)
# Later, unwrap the method
unwrap(config)
```

Some files were not shown because too many files have changed in this diff Show More