CheckOps home

Tools directory

T

CheckOps

TruLens

TruLens is an MIT-licensed, OpenTelemetry-native library for tracing and evaluating LLM apps and agents with explainable LLM judges, maintained by Snowflake.

Visit website

Latest public signals

Not refreshed yet. Refresh reads the public site and, when the entry has a GitHub repository, the latest release.

Visit Website

Introduction

TruLens is an open-source library for evaluating and tracing AI agents. The README says it “finds where your agent fails and where you can cut cost without losing quality”. It is MIT-licensed at truera/trulens and OpenTelemetry-native.

The site notes that since its acquisition, Snowflake actively maintains TruLens in open source. Version 2.12 added tracking and evaluation of complete multi-turn conversations.

Features

  1. Decorator instrumentation: Instrument any app with a decorator.
  2. LLM judges: Score each step with judges that explain themselves.
  3. RAG triad: Groundedness, context relevance and answer relevance.
  4. Tool and MCP evals: Tool calling quality and MCP span tracing.
  5. Version comparison: Compare scores, latency and cost per app version.
  6. OTel traces: Portable to any OTLP backend.
  7. Multi-turn: Evaluate whole conversations (v2.12).

Pros

  1. Free and MIT: No licence cost.
  2. Explainable judges: Judges give reasons, not just numbers.
  3. Cost awareness: Highlights cheaper versions that keep quality.
  4. OTel-native: Fits existing observability.
  5. Maintained: Backed by Snowflake.

Cons

  1. Judge cost: LLM judges cost tokens.
  2. Python only: Python library.
  3. Self-hosted UI: You run the dashboard.
  4. Judges can be wrong: Validate against human labels, as the project itself encourages.
  5. Ecosystem shift: Ownership changed through acquisition.

Install from PyPI (pip install trulens), wrap your app with the TruLens instrumentation decorator, define feedback functions such as groundedness, and run the dashboard to compare versions. See the getting-started guide on trulens.org.

Resources:

Pricing

TruLens is free and open source under MIT. Model calls for judges are billed by your provider.

  1. RAG and agent developers wanting explainable metrics.
  2. Teams on OpenTelemetry who want evals on the same traces.
  3. Snowflake users looking for supported open source.
  4. Not a fit for non-Python stacks.

Conclusion

TruLens is a mature open-source eval and tracing library with explainable judges and OTel-native traces.

Calibrate judges against human labels before trusting them as release gates.

Alternatives

Other tools on CheckOps that cover similar ground. Open a page, or compare one with TruLens side by side.