Tools / Category

LLM & agent evaluation

Libraries and platforms for evaluating LLM applications and agents: metrics, datasets, traces and regression checks.

5 tools

D
DeepEval
DeepEval is an open-source LLM evaluation library with pytest-style asserts. Confident AI is the optional paid cloud from the same team.
- llm-agent-evaluation
- free
- open-source
- ai-testing
Add DeepEval to compare
L
Langfuse
Langfuse is an open-source platform for LLM traces, prompts, and evals. Self-hosting is free; Langfuse Cloud is paid after a hobby tier.
- llm-agent-evaluation
- free
- open-source
- saas
Add Langfuse to compare
L
LangSmith
LangSmith is LangChain's platform for observability and evaluation of LLM applications and agents, including offline and online evals.
- llm-agent-evaluation
- saas
- paid
- free-version
Add LangSmith to compare
P
Promptfoo
Promptfoo is an open-source CLI for LLM evals and red teaming. The community tool is free; enterprise features are quoted.
- llm-agent-evaluation
- free
- open-source
- ai-testing
Add Promptfoo to compare
R
Ragas
Ragas is an open-source library for systematically evaluating LLM applications, including RAG systems and agents.
- llm-agent-evaluation
- free
- open-source
Add Ragas to compare

View in the tools directory