99 lines
3.4 KiB
Plaintext
99 lines
3.4 KiB
Plaintext
---
|
|
title: 'LiteLLM'
|
|
description: 'AgentOps for observing LiteLLM'
|
|
---
|
|
{/* SOURCE_FILE: examples/litellm/litellm_example.ipynb */}
|
|
|
|
_View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/examples/litellm/litellm_example.ipynb'} target={'_blank'}>Github</a>_
|
|
|
|
# AgentOps for observing LiteLLM
|
|
|
|
We can use AgentOps to observe LiteLLM, a lightweight library for working with large language models. This integration allows you to monitor and log the performance of your LiteLLM applications, providing insights into their behavior and efficiency.
|
|
LiteLLM integration extends observability to the different agent libraries which rely on LiteLLM and hence make it possible to observe the agents built using these libraries.
|
|
|
|
[See our LiteLLM docs](https://docs.agentops.ai/v1/integrations/litellm)
|
|
|
|
First let's install the required packages
|
|
|
|
|
|
|
|
## Installation
|
|
<CodeGroup>
|
|
```bash pip
|
|
pip install -U agentops litellm python-dotenv
|
|
```
|
|
```bash poetry
|
|
poetry add -U agentops litellm python-dotenv
|
|
```
|
|
```bash uv
|
|
uv pip install -U agentops litellm python-dotenv
|
|
```
|
|
</CodeGroup>
|
|
|
|
Then import them
|
|
|
|
|
|
```python
|
|
import litellm
|
|
import agentops
|
|
import os
|
|
from dotenv import load_dotenv
|
|
```
|
|
|
|
Next, we'll set our API keys. There are several ways to do this, the code below is just the most foolproof way for the purposes of this notebook. It accounts for both users who use environment variables and those who just want to set the API Key here in this notebook.
|
|
|
|
[Get an AgentOps API key](https://agentops.ai/settings/projects)
|
|
|
|
1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...
|
|
|
|
2. Replace `<your_agentops_key>` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!
|
|
|
|
LiteLLM allows you to use several models including from OpenAI, Llama, Mistral, Claude, Gemini, Gemma, Dall-E, Whisper, and more all using the OpenAI format. To use a different model all you need to change are the API KEY and model (litellm.completion(model="...")).
|
|
|
|
|
|
```python
|
|
load_dotenv()
|
|
os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")
|
|
os.environ["OPENAI_API_KEY"] = os.getenv(
|
|
"OPENAI_API_KEY", "your_openai_api_key_here"
|
|
) # or the provider of your choosing
|
|
```
|
|
|
|
|
|
```python
|
|
agentops.init(auto_start_session=False)
|
|
tracer = agentops.start_trace(trace_name="LiteLLM Example", tags=["litellm-example", "agentops-example"])
|
|
```
|
|
|
|
Note: AgentOps requires that you call LiteLLM completions differently than the LiteLLM's docs mention
|
|
Instead of doing this -
|
|
|
|
```python
|
|
from litellm import completion
|
|
completion()
|
|
```
|
|
|
|
You should do this -
|
|
|
|
```python
|
|
import litellm
|
|
litellm.completion()
|
|
```
|
|
|
|
|
|
```python
|
|
messages = [{"role": "user", "content": "Write a 12 word poem about secret agents."}]
|
|
response = litellm.completion(model="gpt-4", messages=messages) # or the model of your choosing
|
|
print(response.choices[0].message.content)
|
|
```
|
|
|
|
|
|
```python
|
|
agentops.end_trace(tracer, end_state="Success")
|
|
```
|
|
|
|
|
|
<script type="module" src="/scripts/github_stars.js"></script>
|
|
<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>
|
|
<script type="module" src="/scripts/button_heartbeat_animation.js"></script>
|
|
<script type="module" src="/scripts/adjust_api_dynamically.js"></script> |