141 lines
3.9 KiB
Plaintext
141 lines
3.9 KiB
Plaintext
---
|
|
title: LiteLLM
|
|
description: "Track and analyze your LiteLLM calls across multiple providers with AgentOps"
|
|
---
|
|
|
|
AgentOps provides seamless integration with [LiteLLM](https://github.com/BerriAI/litellm), allowing you to automatically track all your LLM API calls across different providers through a unified interface.
|
|
|
|
## Installation
|
|
|
|
<CodeGroup>
|
|
```bash pip
|
|
pip install agentops litellm
|
|
```
|
|
```bash poetry
|
|
poetry add agentops litellm
|
|
```
|
|
```bash uv
|
|
uv pip install agentops litellm
|
|
```
|
|
</CodeGroup>
|
|
|
|
## Setting Up API Keys
|
|
|
|
Before using LiteLLM with AgentOps, you need to set up your API keys. You can obtain:
|
|
- **Provider API Keys**: From your chosen LLM provider (OpenAI, Anthropic, Google, etc.)
|
|
- **AGENTOPS_API_KEY**: From your [AgentOps Dashboard](https://app.agentops.ai/)
|
|
|
|
Then to set them up, you can either export them as environment variables or set them in a `.env` file.
|
|
|
|
<CodeGroup>
|
|
```bash Export to CLI
|
|
export OPENAI_API_KEY="your_openai_api_key_here"
|
|
export ANTHROPIC_API_KEY="your_anthropic_api_key_here"
|
|
export AGENTOPS_API_KEY="your_agentops_api_key_here"
|
|
```
|
|
```txt Set in .env file
|
|
OPENAI_API_KEY="your_openai_api_key_here"
|
|
ANTHROPIC_API_KEY="your_anthropic_api_key_here"
|
|
AGENTOPS_API_KEY="your_agentops_api_key_here"
|
|
```
|
|
</CodeGroup>
|
|
|
|
Then load the environment variables in your Python code:
|
|
|
|
```python
|
|
from dotenv import load_dotenv
|
|
import os
|
|
|
|
# Load environment variables from .env file
|
|
load_dotenv()
|
|
|
|
# Set up environment variables with fallback values
|
|
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
|
|
os.environ["ANTHROPIC_API_KEY"] = os.getenv("ANTHROPIC_API_KEY")
|
|
os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY")
|
|
```
|
|
|
|
## Usage
|
|
|
|
The simplest way to integrate AgentOps with LiteLLM is to set up the success_callback.
|
|
|
|
```python
|
|
import litellm
|
|
from litellm import completion
|
|
|
|
# Configure LiteLLM to use AgentOps
|
|
litellm.success_callback = ["agentops"]
|
|
|
|
# Make completion requests with LiteLLM
|
|
response = completion(
|
|
model="gpt-3.5-turbo",
|
|
messages=[{"role": "user", "content": "Hello, how are you?"}]
|
|
)
|
|
|
|
print(response.choices[0].message.content)
|
|
```
|
|
|
|
## Examples
|
|
|
|
<CodeGroup>
|
|
```python Streaming
|
|
import litellm
|
|
from litellm import completion
|
|
|
|
# Configure LiteLLM to use AgentOps
|
|
litellm.success_callback = ["agentops"]
|
|
|
|
# Make a streaming completion request
|
|
response = completion(
|
|
model="gpt-4",
|
|
messages=[{"role": "user", "content": "Write a short poem about AI."}],
|
|
stream=True
|
|
)
|
|
|
|
# Process the streaming response
|
|
for chunk in response:
|
|
if chunk.choices[0].delta.content:
|
|
print(chunk.choices[0].delta.content, end="", flush=True)
|
|
print() # Add a newline at the end
|
|
```
|
|
|
|
```python Multi-Provider
|
|
import litellm
|
|
from litellm import completion
|
|
|
|
# Configure LiteLLM to use AgentOps
|
|
litellm.success_callback = ["agentops"]
|
|
|
|
# OpenAI request
|
|
openai_response = completion(
|
|
model="gpt-4",
|
|
messages=[{"role": "user", "content": "What are the advantages of GPT-4?"}]
|
|
)
|
|
|
|
print("OpenAI Response:", openai_response.choices[0].message.content)
|
|
|
|
# Anthropic request using the same interface
|
|
anthropic_response = completion(
|
|
model="anthropic/claude-3-opus-20240229",
|
|
messages=[{"role": "user", "content": "What are the advantages of Claude?"}]
|
|
)
|
|
|
|
print("Anthropic Response:", anthropic_response.choices[0].message.content)
|
|
|
|
# All requests across different providers are automatically tracked by AgentOps
|
|
```
|
|
</CodeGroup>
|
|
|
|
|
|
## More Examples
|
|
<CardGroup cols={2}>
|
|
<Card title="LiteLLM Quickstart Notebook" icon="notebook" href="/v2/examples/litellm" />
|
|
</CardGroup>
|
|
|
|
For more information on integrating AgentOps with LiteLLM, refer to the [LiteLLM documentation on AgentOps integration](https://docs.litellm.ai/docs/observability/agentops_integration).
|
|
|
|
<script type="module" src="/scripts/github_stars.js"></script>
|
|
<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>
|
|
<script type="module" src="/scripts/button_heartbeat_animation.js"></script>
|
|
<script type="css" src="/styles/styles.css"></script>
|