Arize Phoenix

Visit Website

Introduction

Phoenix is Arize’s open-source AI observability and evaluation project. The site describes it as a way to trace, evaluate and improve AI agents, running locally or self-hosted. The repository Arize-ai/phoenix is licensed under the Elastic License 2.0 (ELv2), which is source-available rather than an OSI-approved licence.

Arize also sells Arize AX, a managed AI engineering platform with free, Pro and Enterprise tiers. Phoenix can be started locally with one command (uvx arize-phoenix serve).

Features

1. Tracing: Capture traces from LLM apps and agents, based on OpenTelemetry instrumentation.
2. Evaluations: Run LLM-as-judge and code evaluations on traces and datasets.
3. Datasets and experiments: Compare versions of prompts and apps.
4. Local or self-hosted: Run on a laptop or your own infrastructure.
5. Coding agent setup: The site shows adding tracing with a coding agent through an npx installer.
6. Managed AX: Hosted tiers for teams.

Pros

1. Free to self-host: No licence fee for Phoenix.
2. Quick start: One command to run locally.
3. Popular: Over 12,000 GitHub stars at review.
4. Path to managed: Move to Arize AX if needed.
5. OpenTelemetry-based: Portable instrumentation.

Cons

1. ELv2 licence: Restrictions apply, such as not offering it as a managed service; check with legal.
2. Self-hosting effort: You run storage and upgrades.
3. Judge costs: LLM evaluations cost tokens.
4. Two products: Phoenix and AX differ in features.
5. Observability-first: Testing workflows need setup.

Usage and Links (Example)

Start Phoenix locally, from the official site:

uvx arize-phoenix serve

Instrument your app with the Phoenix or OpenInference libraries, send traces, then create datasets and run evaluations in the UI or from Python.

Resources:

• Phoenix website
• GitHub repository
• Docs
• Arize pricing

Pricing

Phoenix is free to self-host under ELv2. From Arize’s pricing page for the managed AX product:

1. AX Free: Free. 25k trace spans a month, 1 GB ingestion, 15-day retention, unlimited users and evals.
2. AX Pro: $50 a month. 50k spans, 10 GB, 30-day retention.
3. AX Enterprise: Custom.

Recommended for?

1. Teams wanting self-hosted LLM tracing and evals at no licence cost.
2. Developers who want a local UI for debugging agents.
3. Organisations planning to grow into managed observability.
4. Not a fit if you require an OSI-approved open-source licence.

Conclusion

Phoenix is a popular, easy-to-start tool for tracing and evaluating LLM apps, with a managed path through Arize AX.

Read the ELv2 terms before embedding or hosting it for others.