LangSmith

LangSmith is an agent evaluation and observability platform by LangChain that records traces, runs offline and online evaluations and compares application versions.

Written by

What LangSmith does

LangSmith is an evaluation and observability platform for language-model applications and agents. It records nested traces, prompts, tool calls, latency and feedback, then lets teams turn production failures into datasets for repeatable offline experiments.

LangSmith supports code evaluators, LLM judges, pairwise comparisons, summary metrics, human annotation and online scoring of production traces. Builders can compare experiments, establish regression thresholds and run evaluations through Python, TypeScript, pytest, Vitest or GitHub workflows without adopting the LangChain application framework.

Who LangSmith is for

  • Teams that want traces, datasets and evaluations in one workflow
  • Regression testing for agents with multiple tool calls or conversation turns
  • Organizations already using LangChain or LangGraph but needing production evaluation

Who should pick something else

  • Teams requiring the whole platform under an open source license, because only client libraries and related components are open
  • Large collaborative teams staying on the Developer plan, because that plan includes one free seat
  • Projects that cannot send traces to a hosted service unless they qualify for an enterprise self-hosted deployment

LangSmith: one strength, one limit

Strength. LangSmith's evaluation docs cover code evaluators, LLM judges, pairwise comparisons and online scoring of production traces, and say they work with any framework.

Limit. The Developer plan includes only one free seat and a limited monthly base-trace allowance; additional collaboration and usage require paid service.

What LangSmith costs

The Developer plan includes one seat and a trace allowance. Paid plans add collaboration and administration, with further charges based on traces, storage and selected compute services.

Current prices: www.langchain.com . Prices change often, so check the maker's own page before you buy.

LangSmith with AI coding agents

LangSmith's documentation publishes an llms.txt index, and LangSmith offers APIs plus evaluation runs through Python, TypeScript, pytest, Vitest or GitHub workflows.

Questions people ask

Does LangSmith require LangChain or LangGraph?

No. LangSmith's evaluation page says tracing and evaluation work with any framework, although LangChain and LangGraph have native integrations.

Alternatives to LangSmith

Compare LLM evals and tracing tools

All three have a free plan.

Langfuse uses the MIT license; Braintrust and LangSmith are proprietary.

Compare all 3

More in LLM evals and tracing