Langfuse vs LangSmith vs Braintrust: How to Choose
In the Langfuse vs LangSmith decision, pick Langfuse if you want open source you can self-host for free, and LangSmith if your agents run on LangChain or LangGraph and you want a managed platform. Pick Braintrust if evaluation and experiments, not tracing, are the center of your workflow.
All three are good tools. The harder question is who will build and maintain the eval set they run. On IrisAgent's Support Model Arena, GPT-5.6 Luna answers 97.6% of answerable support tickets correctly, yet declines only 85% of the tickets it should refuse. A generic accuracy score hides that gap. Only an eval set built from your own traffic shows it.
A disclosure before we start: IrisAgent makes IrisEvals, an evaluation product that overlaps with part of what these tools do. We have kept the comparison to facts we could verify on each vendor's own docs and pricing pages, linked below, and we say plainly where each tool is the better choice.
Key Takeaways
Langfuse wins on openness. The core is MIT licensed and free to self-host, and tracing is built on OpenTelemetry.
LangSmith wins for LangChain and LangGraph teams. It comes from the LangChain team and offers cloud, hybrid, or self-hosted deployment on Enterprise.
Braintrust wins when evals come first. It pairs versioned datasets and online scoring with an MIT-licensed scorer library and an agent that helps build evals.
All three price differently. Langfuse meters units, LangSmith meters seats and traces, and Braintrust meters processed data and scores.
None of them owns your eval set for you. Budget for the people who define criteria, curate the set, and keep it current.
The Short Answer: Which Team Should Pick Which Tool
Start from your constraints, not the feature list. Most teams can rule out one option in five minutes.
Choose Langfuse if data residency or budget pushes you toward self-hosting, if you use several frameworks, or if you want to read and modify the code. The open-source edition includes tracing, evaluation, prompt management, and datasets.
Choose LangSmith if your agents are built on LangChain or LangGraph and your team would rather not run infrastructure. Tracing those frameworks is the shortest path in the market, and it also accepts OpenTelemetry traces from other stacks.
Choose Braintrust if your team thinks in experiments: compare prompt and model versions on datasets, then score production traces with the same scorers. It is the most evaluation-centric of the three.
Choose none of them yet if no one on your team will own the eval set. A tracing tool without maintained datasets and calibrated scorers becomes an expensive log viewer.
If your agents answer customers, you can also see what a finished eval looks like before you choose. Send 10 to 20 example conversations and get a sample eval report with graded results, explanations, and the same prompts replayed on leading models with price.
Langfuse vs LangSmith vs Braintrust at a Glance
Every cell below comes from the vendor's own documentation or pricing page as of November 18, 2026. Where we could not verify a detail, we left it out rather than guess.
Langfuse | LangSmith | Braintrust | |
|---|---|---|---|
Tracing | Built on OpenTelemetry. Native Python and JavaScript SDKs, plus 100+ integrations | SDK, framework integrations, or any standard OpenTelemetry client | SDKs for Python, TypeScript, Go, Java, Ruby, and C#. Accepts OpenTelemetry traces |
Datasets | Datasets for experiments. Run prompts and models against them in the UI | Datasets of examples with optional reference outputs and metadata | Versioned datasets of test cases for repeatable evaluations |
Scorers | LLM-as-a-judge, decision-model, and code evaluators (Python or TypeScript), plus annotation queues | Human, code, LLM-as-judge, and pairwise evaluators, plus annotation queues | Code and LLM-as-a-judge scorers, open-source autoevals library, and human review |
Online evaluation | Rules with filters and a sampling rate score new production observations. Batch runs cover past data | LLM-as-judge evaluators on production traces, with filters, sampling, and weekly spend limits | Scoring rules at span, trace, or session scope, run asynchronously with a sampling rate |
Self-hosting and license | MIT core (except "ee" folders). Free to self-host. Enterprise license adds audit logs, retention policies, and project-level RBAC | Commercial. Developer and Plus are cloud only. Hybrid or self-hosted on Enterprise | Commercial SaaS on all plans. Bring-your-own-cloud data plane on Enterprise. Autoevals is MIT licensed |
Framework fit | Framework-neutral. OpenAI SDK, LangChain, LlamaIndex, and more | Built by the LangChain team. Integrates LangChain, LangGraph, OpenAI Agents, Vercel AI SDK, CrewAI, PydanticAI, and more | Integrations include OpenAI, Anthropic, LangChain, and Vercel AI SDK |
Pricing | Hobby free (50k units, 2 users). Core $29/month, Pro $199/month, 100k units included, then $8 per 100k. Enterprise $2,499/month | Developer free (1 seat, 5k base traces). Plus $39 per seat per month with 10k base traces. Enterprise custom | Starter free (1 GB, 10k scores). Pro $249/month (5 GB, 50k scores). Enterprise custom |
Sources for the table, all official vendor pages:
Langfuse: pricing, self-hosted pricing and license, observability overview, online evaluation, GitHub license
LangSmith: pricing, evaluation concepts, online evaluations, OpenTelemetry tracing, integrations
Braintrust: pricing, deployment options, online scoring, datasets, SDKs, autoevals on GitHub
Two pricing notes. First, the meters differ, so compare on your own volume. A trace-heavy agent with few users favors seat pricing, while a large team with modest traffic favors usage pricing. Second, retention varies by plan: LangSmith keeps base traces 14 days and extended traces 180 days, Braintrust Starter keeps data 14 days and Pro 30 days, and Langfuse ranges from 30 days on Hobby to 3 years on Pro.
Where Each Tool Genuinely Wins
Langfuse: Openness and Control
Langfuse is the clearest choice when you need to own the deployment. You can self-host all core features for free with Docker Compose or Kubernetes, and its tracing follows OpenTelemetry, which lowers lock-in if you later switch backends. Online evaluation is complete: LLM-as-a-judge, decision-model, and code evaluators all run against production traffic with filters and sampling.
The tradeoff is operational. Self-hosting means your team runs the stack, upgrades it, and secures it. Features such as audit logs, data retention policies, and project-level RBAC need the Enterprise license, so regulated teams should price that in.
LangSmith: The LangChain-Native Path
If your agents run on LangChain or LangGraph, LangSmith removes the most friction. Its evaluation model is also well structured: offline evals on curated datasets with reference outputs, online evals on live traces without them, pairwise comparisons, and annotation queues for structured human review. Online evaluators can target the runs that matter most, such as conversations with negative user feedback, and each evaluator can carry a weekly spend limit.
It is not LangChain-only. LangSmith accepts traces from any OpenTelemetry client and lists integrations from the OpenAI Agents SDK to the Vercel AI SDK. The constraint is deployment: if you must self-host, that means the Enterprise plan.
Braintrust: Evaluation as the Core Workflow
Braintrust treats evaluation as the main event rather than an add-on to tracing. Datasets are versioned, experiments compare versions side by side, and the same scorers run online against production traces at span, trace, or session scope without adding latency. Its open-source autoevals library ships ready-made heuristic, LLM-as-a-judge, and RAG scorers you can use even outside Braintrust.
Braintrust also has the strongest answer to the eval-set question among the three. Its Loop agent can suggest and save scorers and generate datasets from log patterns, asking for approval before it changes saved objects. The tradeoffs are a higher entry price for the paid plan and no free self-hosted option.
The Question These Comparisons Skip: Who Builds and Maintains the Eval Set?
Every tool above gives you the machinery for evals. None of them knows what a correct answer looks like for your product. Someone on your team still has to decide that, and keep deciding it as the product changes.
The Arena numbers above show why it matters. A model that answers 97.6% of answerable tickets correctly but refuses only 85% of the tickets it should not answer will look strong on a set built only from answerable questions. You only see the weakness if someone deliberately puts "should decline" cases in the set. Our guide to building a golden dataset from production logs covers how to do that.
Before you sign with any vendor, put a name next to each of these jobs:
Define the criteria. Turn your quality bar into written pass and fail rules for each workflow.
Curate the set. Sample production traffic across every intent, not just the common path, and redact it.
Add the failures. Pull in thumbs-downs, escalations, and corrected answers every week.
Calibrate the judge. Check LLM-as-a-judge verdicts against expert reviews before you trust the scores.
Set sampling and budgets. Decide what share of production traffic to score online and what that costs.
Review drift. Compare eval scores against production signals so a stale set does not hide a regression.
Act on failures. Turn a failing score into a prompt, retrieval, or model fix, then rerun the set.
If every job has an owner, any of the three tools will serve you well. If the list has gaps, the gaps will cost more than the license. For the fundamentals behind these jobs, see our plain-English guide to AI evals.
Evaluation also depends on tracing. If you have not decided what a trace should capture for a customer-facing agent, start with what to monitor in production for support AI agents.
When IrisEvals Is the Better Fit, and When It Is Not
This section is about our own product, so read it with that in mind.
IrisEvals takes on the jobs in the list above rather than giving your engineers tools to do them. It drafts evaluation criteria from your prompts, policies, and real conversations, grades 100% of production LLM calls, keeps a golden eval set generated from live traffic and refreshed automatically, and turns failures into verified pull requests in your repo. It never sits on your request path, so evals add no latency.
IrisEvals is likely the better fit when:
Your AI answers customers, and one bad answer costs revenue, trust, or compliance.
You are already paying people to review AI conversations by hand.
No engineer has time to own criteria, datasets, and judge calibration.
You need to compare frontier, in-house, or fine-tuned models on your own traffic, with cost and latency.
Langfuse, LangSmith, or Braintrust is likely the better fit when:
Your engineers want code-first evals in CI and trace-level debugging that they own.
You need a general-purpose observability platform across many kinds of LLM apps.
You want to self-host open-source software (Langfuse) or stay inside the LangChain ecosystem (LangSmith).
Many teams run both: an observability platform for engineering, and an evaluation layer that owns answer quality. Compare model results on real support work in the Support Model Arena.
Next Steps
Pick the constraint that matters most (self-hosting, framework, or an evaluation-first workflow) and shortlist one tool. Then write down who owns each eval job before you sign. If that list has gaps, see how IrisEvals runs evals on your production traffic, or send us 10 to 20 conversations for a free sample eval report.
Frequently Asked Questions
What is the main difference between Langfuse and LangSmith?
Langfuse is open source under the MIT license (except its ee folders) and free to self-host, with tracing built on OpenTelemetry. LangSmith is a commercial platform from the LangChain team: Developer and Plus plans run in the cloud, and hybrid or self-hosted deployment needs Enterprise. Both cover tracing, datasets, offline and online evaluation, and annotation queues. Choose on deployment needs and framework fit first, because the core evaluation features are closer than most comparisons suggest.
Is LangSmith only for LangChain applications?
No. LangSmith accepts traces from any OpenTelemetry-compatible client, and its integrations page lists frameworks well beyond LangChain, including the OpenAI Agents SDK, Vercel AI SDK, CrewAI, and PydanticAI. That said, LangChain and LangGraph teams get the smoothest setup, since the same company builds both. If your stack mixes several frameworks and you want neutrality, Langfuse or Braintrust may feel more natural.
What are the best Langfuse alternatives?
The two most common Langfuse alternatives are LangSmith, for teams on LangChain or LangGraph who prefer a managed platform, and Braintrust, for teams that treat evaluation and experiments as the core workflow. If your main need is answer quality for a customer-facing agent rather than a general observability platform, an evaluation service such as IrisEvals, made by IrisAgent, is another option. Match the choice to who will own your eval set.
Is Braintrust better than LangSmith for evaluation?
It depends on your workflow. Braintrust is the more evaluation-centric product: versioned datasets, experiments, online scoring at span, trace, or session scope, an open-source scorer library, and an agent that helps build scorers and datasets. LangSmith offers a similarly complete evaluation model, including pairwise evaluators and annotation queues, with the advantage of native LangChain and LangGraph tracing. Run both on the same 50 production examples before deciding.
Can I self-host Langfuse, LangSmith, or Braintrust?
Langfuse is free to self-host, with Docker Compose for local use and Kubernetes for production, and an Enterprise license for features like audit logs. LangSmith offers hybrid and self-hosted deployment on its Enterprise plan only. Braintrust runs as SaaS on every plan, and its Enterprise plan adds a bring-your-own-cloud option, where the data plane runs in your cloud and Braintrust operates it.
How much do Langfuse, LangSmith, and Braintrust cost?
As of November 2026, Langfuse has a free Hobby plan, Core at $29 per month, Pro at $199 per month, and Enterprise at $2,499 per month, with usage above 100k units billed from $8 per 100k. LangSmith Plus is $39 per seat per month with 10k base traces, after a free one-seat Developer plan. Braintrust Pro is $249 per month with 5 GB of processed data and 50k scores, after a free Starter plan.
Loading the form. Prefer email? Reach us at info@irisagent.com or call +1 617 249 3312.