Arize Phoenix
A self-hostable tool for tracing, evaluating and experimenting with LLM applications and agents.
These buttons open the developer's own site, repository or store listing in a new tab. wares.gg does not host downloads.
About Arize Phoenix
Phoenix collects traces from LLM applications so you can see each step a request takes, then run evaluations on the results, including built-in evals. It also covers prompt management, datasets and experiments, and feedback annotations on traces.
It can run from a terminal, in Docker or on Kubernetes, and there is also a hosted Phoenix Cloud. Arize sells a separate managed platform, Arize AX, for teams that want more. Integrations include coding agents and a TypeScript CLI.
Strengths
- Tracing, evals, prompts and experiments in one tool
- Runs locally, in Docker or on Kubernetes
- Hosted cloud option for teams that do not want to self-host
Limitations
- Source-available license rather than an OSI-approved one
- Some advanced features are in the separate paid Arize AX product
Details
- Pricing
- FreemiumPhoenix is free to self-host; Arize AX is a separate managed paid platform.
- License
- Proprietary (Elastic-2.0)
- Developer
- Arize AI
- Platforms
- Web, Self-hosted
- How it runs
- Self-hosted, Hosted service
- Best suited for
- Developers debugging and evaluating LLM apps on their own infrastructure
- Categories
- AI developer tools
- Last verified
- Added
- Sources
Alternatives to Arize Phoenix
Compare allSoftware that can replace Arize Phoenix for an important use case, and what changes if you switch.
Langfuse
An open-source platform for tracing, evaluating and managing prompts for LLM applications and AI agents.
Langfuse offers tracing, prompt management and evals under an open-source license, with a hosted or self-hosted option, but requires instrumenting your app with its SDK.
LangSmith
A hosted platform for tracing, monitoring and evaluating LLM applications and AI agents.
LangSmith is a proprietary hosted service with step-by-step agent tracing, online evals and PagerDuty alerts, and self-hosting it requires Kubernetes rather than a local Docker run.
Opik
An open-source platform from Comet for debugging, evaluating and monitoring LLM applications and agents.
Opik from Comet is open source and self-hostable with tracing, automated evals and dashboards, though self-hosting means running several services and contributors must sign a CLA.
MLflow
An open-source platform for tracing, evaluating and managing LLM applications, agents and machine learning models.
MLflow is Apache-2.0 licensed with OpenTelemetry tracing, 50+ built-in metrics and prompt optimization, and also covers classic model training, though its broad feature set takes time to learn.
Helicone
An open-source AI gateway and observability platform for routing, logging and debugging LLM requests.
Helicone works as an Apache-2.0 AI gateway that logs requests and tracks costs across providers, with less emphasis on experiments, and its roadmap is uncertain after joining Mintlify.
Weights & Biases
A hosted platform for tracking machine learning experiments, evaluating models and tracing LLM applications.
Weights & Biases is a proprietary hosted platform combining experiment tracking and LLM tracing, so data goes to its service instead of staying on your own infrastructure.
Arize Phoenix as an alternative
Listings that name Arize Phoenix as an alternative.
DeepEval
An open-source Python framework for unit-testing and evaluating the outputs of large language model applications.
Arize Phoenix combines evals with tracing and experiments in a self-hostable tool, but uses a source-available license rather than an OSI-approved one.
Evidently
An open-source framework for evaluating, testing and monitoring LLM applications, RAG systems and ML models.
Arize Phoenix provides tracing, evals and experiments for LLM apps with a hosted option, under a source-available license instead of Apache 2.0.
Promptfoo
A command-line tool for testing prompts, evaluating model output and red-teaming LLM applications and agents.
Arize Phoenix offers tracing, evals, prompts and experiments in one self-hostable tool, under a source-available license instead of an open-source one, with some features in the paid Arize AX product.
Similar software
Related functionality, not necessarily a direct replacement.
lm-evaluation-harness
EleutherAI's command-line framework for few-shot evaluation of language models on many benchmark tasks.
LiteLLM
A self-hosted AI gateway that calls over 100 LLM providers through one OpenAI-compatible API.
