CheckOps
OpenLLMetry
OpenLLMetry is an Apache-2.0 set of OpenTelemetry extensions from Traceloop that instrument LLM providers, vector databases and frameworks, sending traces to any OTel backend.
- Open source
- Free
- LLM & agent evaluation
- AI-enabled testing
- LLM Observability
- Tracing
- Python
- Production monitoring & observability
Latest public signals
Not refreshed yet. Refresh reads the public site and, when the entry has a GitHub repository, the latest release.
Introduction
OpenLLMetry is open-source observability for GenAI and LLM applications based on OpenTelemetry, maintained by Traceloop. It provides instrumentations for LLM providers, vector databases and frameworks, so LLM calls appear as standard spans with prompts, completions, token counts and latency.
Because it emits OpenTelemetry, traces can go to any compatible backend, such as Datadog, Honeycomb, Grafana Tempo, New Relic, SigNoz or Traceloop’s own platform. The Traceloop SDK initialises everything in one call, and decorators mark workflows and tasks.
In the quality loop it makes LLM behaviour visible in the observability stack teams already run.
The source code is at traceloop/openllmetry on GitHub under the Apache-2.0 licence, with about 7,500 stars; the most recent commit was on 5 November 2026.
Features
- OTel-native: Standard spans and attributes.
- Provider instrumentations: Many LLM APIs.
- Vector DB instrumentations: Popular vector stores.
- Framework support: LangChain, LlamaIndex and others.
- Any backend: Export to existing APM tools.
- Workflow decorators: Group spans logically.
Pros
- No lock-in: Uses open standards.
- Free to use: Open source under the Apache-2.0 licence, so the core tool costs nothing to use.
- Actively maintained: The repository had commits as recent as 5 November 2026.
- Community: About 7,500 GitHub stars.
Cons
- Backend needed: You still need somewhere to store traces.
- Prompt data in traces: Sensitive content needs controls.
- Evaluation is separate: Tracing only, unless using Traceloop’s platform.
Usage and Links (Example)
from traceloop.sdk import Traceloop
Traceloop.init(app_name="support-bot", api_endpoint="http://otel-collector:4318")
Resources:
Pricing
OpenLLMetry is free and open source under the Apache-2.0 licence.
Recommended for?
- Teams with existing OpenTelemetry pipelines.
Conclusion
OpenLLMetry is the standards-based way to trace LLM apps.
Configure content redaction before sending prompts to shared backends.
On CheckOps, OpenLLMetry is filed under the Production monitoring & observability stage. Related tools listed here include Langfuse, Laminar, Helicone, AgentOps, Arize Phoenix and Opik. Before standardising on one, compare recent maintenance activity, the pricing model as your usage grows, and the integrations you rely on today. Information on this page was verified against the official website, documentation and repository on 11 November 2026.
Alternatives
Other tools on CheckOps that cover similar ground. Open a page, or compare one with OpenLLMetry side by side.
LLangfuseLangfuse is an open-source platform for LLM traces, prompts, and evals. Self-hosting is free; Langfuse Cloud is paid after a hobby tier.Compare with OpenLLMetry
LLaminarLaminar is an Apache-2.0 open-source observability platform for AI agents with OpenTelemetry-native tracing, evaluations and natural-language signals that flag agent failures.Compare with OpenLLMetry
HHeliconeHelicone is an Apache-2.0 open-source AI gateway and LLM observability platform that logs requests, costs and latency with one line of code, plus prompt experiments and evaluations.Compare with OpenLLMetry
AAgentOpsAgentOps is an MIT Python SDK and app for monitoring AI agents, tracking LLM calls, costs, tool use and sessions, with replays for debugging agent runs.Compare with OpenLLMetry
AArize PhoenixArize Phoenix is an open-source (Elastic License 2.0) AI observability and evaluation tool for tracing, evaluating and improving LLM apps and agents, with a hosted Arize AX tier.Compare with OpenLLMetry
OOpikOpik by Comet is an Apache-2.0 open-source platform to trace, evaluate and monitor LLM apps, RAG systems and agents, with test suites, LLM-as-judge evals and a free cloud tier.Compare with OpenLLMetry