Open-source observability for your LLM application
Get started »
OpenLLMetry is a set of extensions built on top of OpenTelemetry that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions – Datadog, Honeycomb, and others.
It’s built and maintained by Traceloop under the Apache 2.0 license.
The repo contains standard OpenTelemetry instrumentations for LLM providers and Vector DBs, as well as a Traceloop SDK that makes it easy to get started with OpenLLMetry, while still outputting standard OpenTelemetry data that can be connected to your observability stack.
If you already have OpenTelemetry instrumented, you can just add any of our instrumentations directly.
🚀 Getting Started
The easiest way to get started is to use our SDK.
For a complete guide, go to our docs.
Install the SDK:
pip install traceloop-sdk
Then, to start instrumenting your code, just add this line to your code:
from traceloop.sdk import Traceloop Traceloop.init(app_name="your_app_name")
That’s it. You’re now tracing your code with OpenLLMetry!
If you’re running this locally, you may want to disable batch sending, so you can see the traces immediately:
Traceloop.init(app_name="your_app_name", disable_batch=True)
Now, you need to decide where to export the traces to.
⏫ Supported destinations
- Traceloop
- Datadog
- New Relic
- Honeycomb
- SigNoz
See our docs for instructions on how to connect to each one.
🪗 What do we instrument?
OpenLLMetry can instrument everything that OpenTelemetry already instruments – so things like your DB, API calls, and more. On top of that, we built a set of custom extensions that instrument things like your calls to OpenAI or Anthropic, or your Vector DB like Pinecone, Chroma, or Weaviate.
LLM Providers
- OpenAI / Azure OpenAI
- Anthropic
- Cohere
- Replicate
- HuggingFace
- Vertex AI (GCP)
- Bedrock (AWS)
Vector DBs
- Pinecone
- Chroma
- Weaviate
- Milvus
Frameworks
- LangChain
- Haystack
- LlamaIndex
🌱 Contributing
Whether it’s big or small, we love contributions ❤️ Check out our guide to see how to get started.
Not sure where to get started? You can:
