155 lines
3.8 KiB
Plaintext
155 lines
3.8 KiB
Plaintext
---
|
|
title: Ollama
|
|
description: "AgentOps provides first class support for Ollama"
|
|
---
|
|
|
|
import CodeTooltip from '/snippets/add-code-tooltip.mdx'
|
|
import EnvTooltip from '/snippets/add-env-tooltip.mdx'
|
|
|
|
[Ollama](https://ollama.com) is a lightweight, open-source tool for running and managing LLM models. Track your Ollama model calls with AgentOps.
|
|
|
|
<Steps>
|
|
<Step title="Install the AgentOps SDK">
|
|
<CodeGroup>
|
|
```bash pip
|
|
pip install agentops ollama
|
|
```
|
|
```bash poetry
|
|
poetry add agentops ollama
|
|
```
|
|
</CodeGroup>
|
|
</Step>
|
|
<Step title="Install the Ollama SDK">
|
|
<CodeGroup>
|
|
```bash pip
|
|
pip install ollama
|
|
```
|
|
```bash poetry
|
|
poetry add ollama
|
|
```
|
|
</CodeGroup>
|
|
</Step>
|
|
<Step title="Add 3 lines of code">
|
|
<CodeTooltip/>
|
|
<CodeGroup>
|
|
```python python
|
|
import agentops
|
|
import ollama
|
|
|
|
agentops.init(<INSERT YOUR API KEY HERE>)
|
|
agentops.start_session()
|
|
|
|
ollama.pull("<MODEL NAME>")
|
|
|
|
response = ollama.chat(model='mistral',
|
|
messages=[{
|
|
'role': 'user',
|
|
'content': 'What are the benefits of using AgentOps for monitoring LLMs?',
|
|
}]
|
|
)
|
|
print(response['message']['content'])
|
|
...
|
|
# End of program (e.g. main.py)
|
|
agentops.end_session("Success")
|
|
```
|
|
</CodeGroup>
|
|
<EnvTooltip />
|
|
<CodeGroup>
|
|
```python .env
|
|
# Alternatively, you can set the API key as an environment variable
|
|
AGENTOPS_API_KEY=<YOUR API KEY>
|
|
```
|
|
</CodeGroup>
|
|
Read more about environment variables in [Advanced Configuration](/v1/usage/advanced-configuration)
|
|
</Step>
|
|
<Step title="Run your Agent">
|
|
Execute your program and visit [app.agentops.ai/drilldown](https://app.agentops.ai/drilldown) to observe your Agent! 🕵️
|
|
<Tip>
|
|
After your run, AgentOps prints a clickable url to console linking directly to your session in the Dashboard
|
|
</Tip>
|
|
<div/>
|
|
</Step>
|
|
</Steps>
|
|
|
|
## Full Examples
|
|
|
|
<CodeGroup>
|
|
```python basic completion
|
|
import ollama
|
|
import agentops
|
|
|
|
agentops.init(<INSERT YOUR API KEY HERE>)
|
|
|
|
ollama.pull("<MODEL NAME>")
|
|
response = ollama.chat(
|
|
model="<MODEL NAME>",
|
|
max_tokens=1024,
|
|
messages=[{
|
|
"role": "user",
|
|
"content": "Write a haiku about AI and humans working together"
|
|
}]
|
|
)
|
|
|
|
print(response['message']['content'])
|
|
agentops.end_session('Success')
|
|
```
|
|
|
|
```python streaming
|
|
import agentops
|
|
import ollama
|
|
|
|
async def main():
|
|
agentops.init(<INSERT YOUR API KEY HERE>)
|
|
ollama.pull("<MODEL NAME>")
|
|
|
|
stream = ollama.chat(
|
|
model="<MODEL NAME>",
|
|
messages=[{
|
|
'role': 'user',
|
|
'content': 'Write a haiku about monitoring AI agents',
|
|
}],
|
|
stream=True
|
|
)
|
|
|
|
for chunk in stream:
|
|
print(chunk['message']['content'], end='')
|
|
|
|
agentops.end_session('Success')
|
|
```
|
|
|
|
```python conversation
|
|
import ollama
|
|
import agentops
|
|
|
|
agentops.init(<INSERT YOUR API KEY HERE>)
|
|
ollama.pull("<MODEL NAME>")
|
|
|
|
messages = [
|
|
{
|
|
'role': 'user',
|
|
'content': 'What is AgentOps?'
|
|
},
|
|
{
|
|
'role': 'assistant',
|
|
'content': 'AgentOps is a monitoring and observability platform for LLM applications.'
|
|
},
|
|
{
|
|
'role': 'user',
|
|
'content': 'Can you give me 3 key features?'
|
|
}
|
|
]
|
|
|
|
response = ollama.chat(
|
|
model="<MODEL NAME>",
|
|
messages=messages
|
|
)
|
|
print(response['message']['content'])
|
|
agentops.end_session('Success')
|
|
```
|
|
</CodeGroup>
|
|
|
|
<script type="module" src="/scripts/github_stars.js"></script>
|
|
<script type="module" src="/scripts/scroll-img-fadein-animation.js"></script>
|
|
<script type="module" src="/scripts/button_heartbeat_animation.js"></script>
|
|
<script type="css" src="/styles/styles.css"></script>
|