154 lines
4.9 KiB
Plaintext
154 lines
4.9 KiB
Plaintext
---
|
|
title: 'Concurrent Traces Example'
|
|
description: 'Managing multiple concurrent traces and sessions'
|
|
mode: "wide"
|
|
---
|
|
|
|
_View Notebook on <a href={'https://github.com/AgentOps-AI/agentops/blob/main/examples/multi_session_llm.ipynb'} target={'_blank'}>Github</a>_
|
|
|
|
# Multiple Concurrent Traces
|
|
This example demonstrates how to run multiple traces (sessions) concurrently using both the modern trace-based API and the legacy session API for backwards compatibility.
|
|
|
|
First let's install the required packages:
|
|
|
|
```bash
|
|
pip install -U openai
|
|
pip install -U agentops
|
|
pip install -U python-dotenv
|
|
```
|
|
|
|
Then import them:
|
|
|
|
```python
|
|
from openai import OpenAI
|
|
import agentops
|
|
import os
|
|
from dotenv import load_dotenv
|
|
```
|
|
|
|
Next, we'll set our API keys. There are several ways to do this, the code below is just the most foolproof way for the purposes of this example. It accounts for both users who use environment variables and those who just want to set the API Key here.
|
|
|
|
[Get an AgentOps API key](https://agentops.ai/settings/projects)
|
|
|
|
1. Create an environment variable in a .env file or other method. By default, the AgentOps `init()` function will look for an environment variable named `AGENTOPS_API_KEY`. Or...
|
|
|
|
2. Replace `<your_agentops_key>` below and pass in the optional `api_key` parameter to the AgentOps `init(api_key=...)` function. Remember not to commit your API key to a public repo!
|
|
|
|
```python
|
|
load_dotenv()
|
|
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") or "<your_openai_key>"
|
|
AGENTOPS_API_KEY = os.getenv("AGENTOPS_API_KEY") or "<your_agentops_key>"
|
|
```
|
|
|
|
Initialize AgentOps. We'll disable auto-start to manually create our traces:
|
|
|
|
```python
|
|
agentops.init(AGENTOPS_API_KEY, auto_start_session=False)
|
|
client = OpenAI()
|
|
```
|
|
|
|
## Modern Trace-Based Approach
|
|
|
|
The recommended approach uses `start_trace()` and `end_trace()`:
|
|
|
|
```python
|
|
# Create multiple concurrent traces
|
|
trace_1 = agentops.start_trace("user_query_1", tags=["experiment_a"])
|
|
trace_2 = agentops.start_trace("user_query_2", tags=["experiment_b"])
|
|
|
|
print(f"Trace 1 ID: {trace_1.span.get_span_context().trace_id}")
|
|
print(f"Trace 2 ID: {trace_2.span.get_span_context().trace_id}")
|
|
```
|
|
|
|
## LLM Calls with Automatic Tracking
|
|
|
|
With the modern implementation, LLM calls are automatically tracked without needing special session assignment:
|
|
|
|
```python
|
|
# LLM calls are automatically tracked and associated with the current context
|
|
messages_1 = [{"role": "user", "content": "Hello from trace 1"}]
|
|
response_1 = client.chat.completions.create(
|
|
model="gpt-3.5-turbo",
|
|
messages=messages_1,
|
|
temperature=0.5,
|
|
)
|
|
|
|
messages_2 = [{"role": "user", "content": "Hello from trace 2"}]
|
|
response_2 = client.chat.completions.create(
|
|
model="gpt-3.5-turbo",
|
|
messages=messages_2,
|
|
temperature=0.5,
|
|
)
|
|
```
|
|
|
|
## Using Context Managers
|
|
|
|
You can also use traces as context managers for automatic cleanup:
|
|
|
|
```python
|
|
with agentops.start_trace("context_managed_trace") as trace:
|
|
response = client.chat.completions.create(
|
|
model="gpt-3.5-turbo",
|
|
messages=[{"role": "user", "content": "Hello from context manager"}],
|
|
temperature=0.5,
|
|
)
|
|
# Trace automatically ends when exiting the context
|
|
```
|
|
|
|
## Using Decorators
|
|
|
|
For even cleaner code, use decorators:
|
|
|
|
```python
|
|
@agentops.trace
|
|
def process_user_query(query: str):
|
|
response = client.chat.completions.create(
|
|
model="gpt-3.5-turbo",
|
|
messages=[{"role": "user", "content": query}],
|
|
temperature=0.5,
|
|
)
|
|
return response.choices[0].message.content
|
|
|
|
# Each function call creates its own trace
|
|
result_1 = process_user_query("What is the weather like?")
|
|
result_2 = process_user_query("Tell me a joke")
|
|
```
|
|
|
|
## Legacy Session API (Backwards Compatibility)
|
|
|
|
For backwards compatibility, the legacy session API is still available:
|
|
|
|
```python
|
|
# Legacy approach - still works but not recommended for new code
|
|
session_1 = agentops.start_session(tags=["legacy-session-1"])
|
|
session_2 = agentops.start_session(tags=["legacy-session-2"])
|
|
|
|
# Legacy sessions work the same way as before
|
|
session_1.end_session(end_state="Success")
|
|
session_2.end_session(end_state="Success")
|
|
```
|
|
|
|
## Ending Traces
|
|
|
|
End traces individually or all at once:
|
|
|
|
```python
|
|
# End specific traces
|
|
agentops.end_trace(trace_1, "Success")
|
|
agentops.end_trace(trace_2, "Success")
|
|
|
|
# Or end all active traces at once
|
|
# agentops.end_trace(end_state="Success")
|
|
```
|
|
|
|
## Key Differences from Legacy Multi-Session Mode
|
|
|
|
1. **No mode switching**: You can create multiple traces without entering a special "multi-session mode"
|
|
2. **Automatic LLM tracking**: LLM calls are automatically associated with the current execution context
|
|
3. **No exceptions**: No `MultiSessionException` or similar restrictions
|
|
4. **Cleaner API**: Use decorators and context managers for better code organization
|
|
5. **Backwards compatibility**: Legacy session functions still work for existing code
|
|
|
|
If you look in the AgentOps dashboard, you will see multiple unique traces, each with their respective LLM calls and events properly tracked.
|
|
|