Alternatives to LangSmith
A hosted platform for tracing, monitoring and evaluating LLM applications and AI agents. The listings below can replace it for an important use case. Each note says what changes if you switch.
The original
LangSmith
A hosted platform for tracing, monitoring and evaluating LLM applications and AI agents.
Replacements
Listings that take over the same core job as LangSmith.
Langfuse
An open-source platform for tracing, evaluating and managing prompts for LLM applications and AI agents.
Langfuse is open source with tracing, prompt management and evals, and can be self-hosted without Kubernetes or used as a hosted service.
Arize Phoenix
A self-hostable tool for tracing, evaluating and experimenting with LLM applications and agents.
Arize Phoenix runs tracing, evals and experiments locally, in Docker or on Kubernetes, with a hosted option, under a source-available license.
Opik
An open-source platform from Comet for debugging, evaluating and monitoring LLM applications and agents.
Opik is an open-source, self-hostable platform from Comet with tracing, automated evaluations and dashboards for RAG and agent workflows.
MLflow
An open-source platform for tracing, evaluating and managing LLM applications, agents and machine learning models.
MLflow is Apache-2.0 with provider-independent OpenTelemetry tracing and 50+ evaluation metrics, but you host and maintain the server yourself.
Helicone
An open-source AI gateway and observability platform for routing, logging and debugging LLM requests.
Helicone is an Apache-2.0 gateway that logs LLM requests and tracks costs across providers, with a self-host option, but its roadmap is uncertain after joining Mintlify.
Weights & Biases
A hosted platform for tracking machine learning experiments, evaluating models and tracing LLM applications.
Weights & Biases is also a proprietary hosted platform, adding experiment tracking for classic ML training alongside LLM tracing.
Also worth comparing
These listings name LangSmith as their own alternative, so the relationship runs both ways.
Promptfoo
A command-line tool for testing prompts, evaluating model output and red-teaming LLM applications and agents.
LangSmith is a proprietary hosted service focused on production agent tracing with online evals and alerts, replacing Promptfoo's local command-line testing, and self-hosting it requires Kubernetes.
Similar software
Related functionality, not a direct replacement.
DeepEval
An open-source Python framework for unit-testing and evaluating the outputs of large language model applications.
Langflow
Visual builder for creating and deploying AI agent workflows.
Evidently
An open-source framework for evaluating, testing and monitoring LLM applications, RAG systems and ML models.
