Alternatives to Evidently
An open-source framework for evaluating, testing and monitoring LLM applications, RAG systems and ML models. The listings below can replace it for an important use case. Each note says what changes if you switch.
The original
Evidently
An open-source framework for evaluating, testing and monitoring LLM applications, RAG systems and ML models.
Replacements
Listings that take over the same core job as Evidently.
DeepEval
An open-source Python framework for unit-testing and evaluating the outputs of large language model applications.
DeepEval focuses on LLM outputs as unit tests in Python, without Evidently's coverage of classic ML models and drift monitoring.
MLflow
An open-source platform for tracing, evaluating and managing LLM applications, agents and machine learning models.
MLflow is also Apache-2.0 and covers both LLM apps and classic models, adding tracing, prompt versioning and experiment management, but a team setup needs a hosted server.
Opik
An open-source platform from Comet for debugging, evaluating and monitoring LLM applications and agents.
Opik offers LLM tracing, automated evaluations and dashboards in a self-hostable platform, but focuses on LLM apps and agents rather than classic ML models.
Promptfoo
A command-line tool for testing prompts, evaluating model output and red-teaming LLM applications and agents.
Promptfoo tests prompts and model output from the command line and adds red-teaming for prompt injection and PII leaks, but does not cover classic ML monitoring.
Arize Phoenix
A self-hostable tool for tracing, evaluating and experimenting with LLM applications and agents.
Arize Phoenix provides tracing, evals and experiments for LLM apps with a hosted option, under a source-available license instead of Apache 2.0.
Langfuse
An open-source platform for tracing, evaluating and managing prompts for LLM applications and AI agents.
Langfuse centres on tracing, prompt management and evals for LLM apps in a hosted or self-hosted platform, without classic ML drift checks.
Similar software
Related functionality, not a direct replacement.
Weights & Biases
A hosted platform for tracking machine learning experiments, evaluating models and tracing LLM applications.
TensorBoard
A visualization toolkit for inspecting machine learning training runs, metrics and model graphs.
LangSmith
A hosted platform for tracing, monitoring and evaluating LLM applications and AI agents.
lm-evaluation-harness
EleutherAI's command-line framework for few-shot evaluation of language models on many benchmark tasks.
