Alternatives to Arize Phoenix
A self-hostable tool for tracing, evaluating and experimenting with LLM applications and agents. The listings below can replace it for an important use case. Each note says what changes if you switch.
The original
Arize Phoenix
A self-hostable tool for tracing, evaluating and experimenting with LLM applications and agents.
Replacements
Listings that take over the same core job as Arize Phoenix.
Langfuse
An open-source platform for tracing, evaluating and managing prompts for LLM applications and AI agents.
Langfuse offers tracing, prompt management and evals under an open-source license, with a hosted or self-hosted option, but requires instrumenting your app with its SDK.
LangSmith
A hosted platform for tracing, monitoring and evaluating LLM applications and AI agents.
LangSmith is a proprietary hosted service with step-by-step agent tracing, online evals and PagerDuty alerts, and self-hosting it requires Kubernetes rather than a local Docker run.
Opik
An open-source platform from Comet for debugging, evaluating and monitoring LLM applications and agents.
Opik from Comet is open source and self-hostable with tracing, automated evals and dashboards, though self-hosting means running several services and contributors must sign a CLA.
MLflow
An open-source platform for tracing, evaluating and managing LLM applications, agents and machine learning models.
MLflow is Apache-2.0 licensed with OpenTelemetry tracing, 50+ built-in metrics and prompt optimization, and also covers classic model training, though its broad feature set takes time to learn.
Helicone
An open-source AI gateway and observability platform for routing, logging and debugging LLM requests.
Helicone works as an Apache-2.0 AI gateway that logs requests and tracks costs across providers, with less emphasis on experiments, and its roadmap is uncertain after joining Mintlify.
Weights & Biases
A hosted platform for tracking machine learning experiments, evaluating models and tracing LLM applications.
Weights & Biases is a proprietary hosted platform combining experiment tracking and LLM tracing, so data goes to its service instead of staying on your own infrastructure.
Also worth comparing
These listings name Arize Phoenix as their own alternative, so the relationship runs both ways.
DeepEval
An open-source Python framework for unit-testing and evaluating the outputs of large language model applications.
Arize Phoenix combines evals with tracing and experiments in a self-hostable tool, but uses a source-available license rather than an OSI-approved one.
Evidently
An open-source framework for evaluating, testing and monitoring LLM applications, RAG systems and ML models.
Arize Phoenix provides tracing, evals and experiments for LLM apps with a hosted option, under a source-available license instead of Apache 2.0.
Promptfoo
A command-line tool for testing prompts, evaluating model output and red-teaming LLM applications and agents.
Arize Phoenix offers tracing, evals, prompts and experiments in one self-hostable tool, under a source-available license instead of an open-source one, with some features in the paid Arize AX product.
Similar software
Related functionality, not a direct replacement.
lm-evaluation-harness
EleutherAI's command-line framework for few-shot evaluation of language models on many benchmark tasks.
LiteLLM
A self-hosted AI gateway that calls over 100 LLM providers through one OpenAI-compatible API.
