What an agent trace should hold
One user request becomes one trace. Inside it, a root span for the agent, and under it a child span for each step: the model call that chose a tool, the tool call with its arguments and result, a retrieval, a handoff to another agent, the next model call. Each LLM span carries the model, the provider, input and output tokens, cached tokens, latency and any error. A multi-turn conversation ties several traces together as a session or thread.
With that tree you can open a failed run and see the step where it went wrong: a tool that returned nothing, a loop of the same call, a model that ignored the tool's answer. Without it you only see a slow or expensive request.
The OpenTelemetry names for agent steps
OpenTelemetry's GenAI semantic conventions, now kept in their own repository and still marked Development, name the spans most tools use:
invoke_agent {gen_ai.agent.name}andcreate_agentfor the agent itself;execute_tool {gen_ai.tool.name}for each tool call;- a span per model call, with
gen_ai.operation.name,gen_ai.provider.nameandgen_ai.request.model; gen_ai.usage.input_tokens,gen_ai.usage.output_tokensandgen_ai.usage.cache_read.input_tokensfor tokens;gen_ai.conversation.idto join the turns of one conversation, and MCP spans named after the MCP method.
There is no cost attribute. OpenTelemetry records tokens, not dollars, so every tool prices tokens with its own table, and two tools can show different costs for the same run.
AI agent observability tools that draw the run
How each tool shows an agent, from its own docs on November 19, 2026. All but Griffin take OTLP traces, so the OTel names above arrive intact.
| Tool | Agent view | MCP | Cost | OTLP |
|---|---|---|---|---|
| Langfuse | Agent graphs drawn from nested observations; sessions | MCP tracing | Per generation | Yes |
| LangSmith | Runs for every step; threads link multi-turn sessions | Not stated | Per call | Yes |
| Arize | Agent Trajectory graph (AX); sessions (Phoenix) | Not stated | Rolled up to trace and project | Yes |
| Datadog | Agent decisions and handoffs | MCP clients auto-instrumented | Per LLM request | Yes |
| Braintrust | Nested llm, tool and task spans | MCP server tracing | Child to parent | Yes |
| W&B Weave | Multi-turn timeline with nested tool calls; threads | MCP client and server | Per call | Yes |
| Opik | Agent graphs (LangGraph, Google ADK); threads | Not stated | Per span | Yes |
| Laminar | LLM calls, tool use and control flow; sessions | Not stated | Every LLM span | Yes |
| Traceloop | @agent and @tool decorators (OpenLLMetry) | MCP instrumentation | Costs by property | Yes |
| Griffin | None: one record per call, grouped by use case | No | Per call, by use case | No |
Prices for all of them, per 100,000 spans and for a sample month, are on the pricing data page; the ten main tools are compared card by card on the home page.
LLM tracing vs agent observability
LLM tracing grew out of logging single model calls: a prompt, a response, tokens and latency. That is enough for a chatbot that calls one model once. An agent calls models in a loop and acts between calls, so the useful record is the loop. The tools above all started from tracing and added agent views; a proxy that only sees model calls (Helicone, Portkey) shows the calls but not the tool steps between them unless you group them into sessions yourself.
What to look for
- A tree, not a list. Nested spans for tool calls under the model call that chose them.
- Sessions or threads for multi-turn agents.
- Cost that rolls up from each LLM span to the run (Arize and Braintrust say so in their docs).
- OTLP in, so you can switch tools without re-instrumenting.
- Evaluations on whole runs, not only single answers: Traceloop, for one, ships evaluators for tool selection and agent goals.
- Retention long enough to compare this week's runs with last month's.
Where cost per use case fits
Traces tell you why a run failed. A different question is what each feature of your product spends on AI each day. Griffin answers that one: it records every call's model, tokens, cost and latency, grouped by use case, without spans, and replays real calls on cheaper models with a judge (replay grading) before routing a use case that passed. It doesn't draw agent runs, so teams that need the tree use one of the tools above beside it. Griffin is made by Winding AI, Inc., which publishes this site.
Questions
What is the difference between LLM tracing and agent observability?
LLM tracing records model calls. Agent observability records the whole run around them: which tools the agent picked, what each returned, where it handed off, and how many turns it took, so you can see why a run went wrong, not only which call was slow.
Which AI agent observability tools are open source?
Langfuse (MIT core), Opik, Laminar and OpenLLMetry, Traceloop's SDK (all Apache-2.0) are open source. Arize Phoenix is free to self-host under the Elastic License 2.0.
Does OpenTelemetry record what an agent run costs?
No. The GenAI conventions record input, output and cached tokens per call, but there is no cost attribute, so each tool multiplies tokens by its own price list.