Streaming Output and Tool Calls

An agent that's calling three tools and reasoning between each one can easily take ten or fifteen seconds to finish. Call invoke() and wait for the whole thing, and to a user that looks indistinguishable from a hung request. Streaming is how you fix that, not by making the agent faster, but by showing progress as it happens instead of making people stare at a spinner.

Three things you could stream, and three stream modes

LangGraph's .stream() method (and its async twin, .astream()) takes a stream_mode argument that controls what you get back on each yield:

• "values" - the full graph state after every node finishes. Simple to reason about, but you get the whole state object each time, which can be more than you need.
• "updates" - just the diff: which node ran and what it changed. Good for building a live "here's what the agent is doing" log without re-processing the full state each step.
• "messages" - individual LLM tokens as they're generated, the same token-by-token feel as a ChatGPT-style interface. This is what you want for a chat UI.

for chunk in app.stream(
    {"messages": [("user", "What's the weather in Austin?")]},
    config,
    stream_mode="updates",
):
    print(chunk)

You can also pass a list of modes, like stream_mode=["updates", "messages"], and get both interleaved, which is exactly what most real chat UIs do: token-by-token text for the response, plus discrete "calling get_weather..." events for tool calls.

Showing tool calls specifically

With stream_mode="updates", a tool call shows up as a node update the moment that node starts, before its result comes back. That's the hook for showing something like "Searching the web..." or "Checking your calendar..." in a UI: you don't need to wait for the tool to finish, you just react to the update event that says the tool node is now running.

Streaming and persistence work together, not against each other

Streaming doesn't change anything about checkpointing from the memory and persistence guide. Every node's output still gets saved to the checkpointer as it completes, streaming is just a second thing that happens at the same time, a way of observing the run as it goes rather than only after it's done. If the connection drops mid-stream, the checkpoint is still there and you can resume the run exactly like any other interrupted graph.

A note on async

If you're serving multiple users at once, use astream() over an async web framework rather than the sync stream(). The API is identical, just with async for instead of for, but it means one slow agent run doesn't block every other request your server is handling.