You've built an agent. It works when you test it. Then it ships, and a week later someone says "it gave a weird answer yesterday" - and you have no idea which of the six LLM calls, two tool calls, and one retriever lookup inside that request actually went wrong, because all you have is a final output and a shrug. That gap - between "it worked in my terminal" and "I can see exactly what happened in production" - is what LangSmith exists to close.
The short version
LangSmith is an observability and evaluation platform for LLM applications and agents, built by the LangChain team. It's framework-agnostic - it works whether you're using LangChain, LangGraph, or nothing but raw API calls - and it does three things well: it traces every step of a request so you can see what actually happened, it lets you evaluate outputs against real test cases instead of eyeballing them, and it gives you a Prompt Hub for versioning and iterating on prompts without redeploying code.
Same request, two different levels of visibility: a black box versus a step-by-step record of everything that happened.
How it relates to LangChain and LangGraph
These three tools solve different problems and are commonly used together, but none of them require the others. LangChain gives you standardized building blocks - model calls, tool wrappers, vector store integrations. LangGraph gives you control flow - the ability to loop, branch, and pause an agent's execution as a graph instead of a straight-line script. LangSmith gives you visibility into what either of those actually did once it's running, plus a way to test whether it's doing it well.
If you've read the LangGraph 101 or LangChain 101 guides on this site, think of it this way: those two are about building the thing. LangSmith is about knowing whether the thing you built is actually working.
Why "it worked when I tested it" isn't good enough
LLM apps fail differently from normal software. A traditional bug is usually reproducible - same input, same broken output, every time. An LLM-based agent can work perfectly on the ten examples you tried by hand and then quietly misbehave on the eleventh, because the failure mode isn't a crash, it's a subtly wrong tool call, a hallucinated fact, or a retrieval step that pulled the wrong document. Nothing throws an exception. The request just... returns something wrong, and looks fine at a glance.
Print statements and manual spot-checks don't scale past a demo. Once real traffic is hitting your app, you need a record of every step of every request - what went in, what came out, how long it took, and what it cost - so that when something looks off, you can actually go look instead of guessing.
What you get, concretely
In practice, LangSmith's dashboard organizes everything into projects (so dev, staging, and production traffic never get mixed together), and inside each project you see a running feed of traces - one per request, each expandable into every nested LLM call, tool call, and retriever step that made it up, with latency, token counts, and errors attached to each one. From there, you can pull interesting or broken examples into a dataset, and run evaluations against that dataset automatically whenever you change a prompt or a model - covered in detail later in this series.
Who this is for
This series assumes you're already building something with an LLM - a chatbot, an agent, a RAG pipeline - in Python or JavaScript, and that you have (or are about to create) a free LangSmith account at smith.langchain.com. You don't need to already be using LangChain or LangGraph; LangSmith's tracing SDK works standalone.