CheckOps home

Tools directory

L

CheckOps

LangWatch

LangWatch is an Apache-2.0 open-source platform for agent observability, evals and agent simulation testing, with prompt management, red-teaming and voice-agent testing.

Visit website

Latest public signals

Not refreshed yet. Refresh reads the public site and, when the entry has a GitHub repository, the latest release.

Visit Website

Introduction

LangWatch describes itself as open-source agent observability and evals for AI platform teams. The repository langwatch/langwatch is Apache-2.0 licensed. The platform covers tracing, evaluation, agent simulation testing, prompt management, an AI gateway and governance.

Its agentic AI testing feature runs realistic user scenarios against your agent to catch issues before production. The site also lists voice AI testing, LLM red-teaming, instant evals over production history, and “Langy”, described as an automated AI engineer that reads traces, writes tests and opens a pull request.

Features

  1. Agent simulations: Run realistic user scenarios against agents.
  2. LLM evaluation: Measure response quality and accuracy.
  3. Observability: Trace every agent step, cost and latency.
  4. Mockable tools: Tool calls, skills and MCP servers are traced and can be mocked or fixtured for deterministic runs.
  5. Voice and red teaming: Test voice agents and simulate attacks.
  6. Prompt management: Version and A/B test prompts with GitHub sync.
  7. Deployment: Cloud (EU, US, UK, APAC), self-hosted with Docker or Kubernetes, or hybrid.

Pros

  1. Open source: Apache-2.0, self-host in minutes per the site.
  2. Testing focus: Simulations go beyond passive tracing.
  3. Deterministic options: Fixtured tool calls help reproducible tests.
  4. Free tier: Developer plan at €0.
  5. Regional hosting: Multiple data regions.

Cons

  1. Broad product: Gateway and governance features add complexity.
  2. Event-based billing: Costs grow with events beyond allowances.
  3. Euro pricing: Convert for budgets.
  4. Simulations cost tokens: Each simulated conversation uses models.
  5. Small free limits: 3 scenarios, simulations and custom evals on Developer.

Start on the free cloud or self-host with Docker, install the SDK, send traces, then write a scenario that simulates a user conversation against your agent and add a judge to score it. Run scenarios in CI before release. See langwatch.ai docs.

Resources:

Pricing

From LangWatch’s pricing page:

  1. Developer: €0, free forever. 50k events a month, 14-day data access, 2 users, 3 scenarios, 3 simulations and 3 custom evals.
  2. Growth: €29 per core seat a month. 200k events included, then €5 per 100k, 30-day retention, unlimited lite users.
  3. Enterprise and self-managed options: see site.
  1. Teams testing conversational agents with simulated users.
  2. Organisations wanting open-source, self-hosted observability.
  3. Voice agent builders needing pre-launch tests.
  4. Not a fit if you want only a tiny eval library.

Conclusion

LangWatch pairs open-source observability with real agent testing through simulations and fixtured tools.

Watch event volumes and start with a few high-value scenarios.

Alternatives

Other tools on CheckOps that cover similar ground. Open a page, or compare one with LangWatch side by side.