LangSmith
LangSmith is an agent evaluation and observability platform by LangChain that records traces, runs offline and online evaluations and compares application versions.
Written by
What LangSmith does
LangSmith is an evaluation and observability platform for language-model applications and agents. It records nested traces, prompts, tool calls, latency and feedback, then lets teams turn production failures into datasets for repeatable offline experiments.
LangSmith supports code evaluators, LLM judges, pairwise comparisons, summary metrics, human annotation and online scoring of production traces. Builders can compare experiments, establish regression thresholds and run evaluations through Python, TypeScript, pytest, Vitest or GitHub workflows without adopting the LangChain application framework.
Who LangSmith is for
- Teams that want traces, datasets and evaluations in one workflow
- Regression testing for agents with multiple tool calls or conversation turns
- Organizations already using LangChain or LangGraph but needing production evaluation
Who should pick something else
- Teams requiring the whole platform under an open source license, because only client libraries and related components are open
- Large collaborative teams staying on the Developer plan, because that plan includes one free seat
- Projects that cannot send traces to a hosted service unless they qualify for an enterprise self-hosted deployment
LangSmith: one strength, one limit
Strength. LangSmith's evaluation docs cover code evaluators, LLM judges, pairwise comparisons and online scoring of production traces, and say they work with any framework.
Limit. The Developer plan includes only one free seat and a limited monthly base-trace allowance; additional collaboration and usage require paid service.
What LangSmith costs
The Developer plan includes one seat and a trace allowance. Paid plans add collaboration and administration, with further charges based on traces, storage and selected compute services.
Current prices: www.langchain.com . Prices change often, so check the maker's own page before you buy.
LangSmith with AI coding agents
LangSmith's documentation publishes an llms.txt index, and LangSmith offers APIs plus evaluation runs through Python, TypeScript, pytest, Vitest or GitHub workflows.
Questions people ask
Does LangSmith require LangChain or LangGraph?
No. LangSmith's evaluation page says tracing and evaluation work with any framework, although LangChain and LangGraph have native integrations.
Alternatives to LangSmith
- Braintrust: Braintrust is an AI evaluation and observability platform by Braintrust that connects datasets, experiments, scorers, production traces and human review.
Compare LLM evals and tracing tools
All three have a free plan.
Langfuse uses the MIT license; Braintrust and LangSmith are proprietary.
More in LLM evals and tracing
Braintrust is an AI evaluation and observability platform by Braintrust that connects datasets, experiments, scorers, production traces and human review.
Langfuse is an open source LLM engineering platform by Langfuse that provides tracing, prompt management, datasets, experiments, and online and offline evaluation.