14 AI Agent Observability Tools Compared (2026 Pricing)

AI agent observability tools record what an agent did on every run: each model call, tool call and retrieval step, with its inputs, outputs, latency, token cost and a quality score. That lets you find the bad runs and see why they went wrong. This guide compares 14 of them on the things that decide most purchases: the free tier and starting price, the unit you are billed on, whether you can self-host, and whether they accept OpenTelemetry data. Every price was read on the vendor's own pricing page on 10 November 2026. After the table come a one-line verdict per tool, what a setup must capture, and how to choose by team situation. Gigabit is not a partner of any of these vendors and earns nothing from your choice.

The 14 tools at a glance

Prices are US dollars per month, read on each vendor's pricing page on 10 November 2026. "Free" means a hosted free tier unless the row says the software itself is free to run. Vendors change plans often, so confirm the figure on the linked page before you budget.

Three changes since our last check matter more than any price. Galileo's pricing page now redirects to Splunk, after Cisco completed its purchase of Galileo in May 2026. Traceloop's site says it is joining ServiceNow, with OpenLLMetry staying open source. Helicone was bought by Mintlify in March 2026 and put into maintenance mode; its site now says it is no longer part of Mintlify, with details to follow. If you choose Helicone, Galileo or Traceloop today, ask the vendor in writing what the roadmap and support terms are.

Why agents need more than ordinary monitoring

Uptime and error-rate dashboards stay green while an agent gets worse, because a wrong answer returns HTTP 200 like a right one. Quality drifts when a vendor changes a model, a policy document goes stale or traffic shifts, and nobody notices for weeks; why AI systems break after launch covers the five ways it happens. Agents add a second problem: a run is a chain of steps, and small per-step error rates multiply. When a multi-step AI agent fails in production does the arithmetic. Observability exists so you can see inside the chain.

What agent observability must capture

Use this as a requirements list when you evaluate any tool, including one you build yourself.

A quick test for any setup: pick one run from yesterday and answer these five questions in under five minutes. What did the user ask? What did the agent retrieve, and which tools did it call, in what order, with what arguments? What did each step return? What did the run cost and how long did it take? Which model and prompt version produced it? If any answer needs a developer and a log search, the setup is not finished. Our AI operations runbook explains how to write the trace-reading section around that test.

On scores, two posts go deeper than this one: what to measure in an AI agent eval suite for which metrics should block a deploy, and calibrating LLM-as-judge for automated scoring of live runs you can trust.

The open standard: OpenTelemetry GenAI conventions

OpenTelemetry is the vendor-neutral standard most infrastructure monitoring already uses. Its GenAI semantic conventions define spans, metrics and events for generative AI: model call spans, token usage metrics and agent spans, with operation names such as invoke_agent and execute_tool. Their status, checked on 10 November 2026, is still "Development", so attribute names can still change. Some content fields are opt-in; system instructions, for example, are not recorded unless you choose to. The conventions are a format, not a product: you still need a backend to store, search and score the traces. Instrumenting against them is the cheapest insurance against being locked into one vendor.

Which tool is best for what

All 14 record token counts and cost per model call, so cost tracking alone will not separate them. The verdicts below are about fit: who each tool was built for, and where it is weak.

• Langfuse. The default pick if you want open source with a real cloud option: tracing, prompt management, evals and datasets in one tool. The unit counts every observation, so busy agents use units fast.
• LangSmith. The natural fit if you build on LangChain or LangGraph, which come from the same company. Per-seat pricing gets expensive as the team grows, and self-hosting needs an Enterprise contract.
• Arize Phoenix and AX. Phoenix runs on a laptop or in your cluster, and AX is the managed upgrade. Note that Phoenix is source-available under the Elastic License, not an OSI open-source licence.
• W&B Weave. Worth it if your team already lives in Weights & Biases for training. Data-based pricing at $0.10 per MB over the allowance makes large traces costly.
• Braintrust. Built around experiments and scores, with production logs next to them. The best choice when an eval suite gates every release; the paid tier starts high at $249.
• Helicone. The fastest start: point your model calls at its gateway and you see cost and latency per request. Its ownership and roadmap are unsettled, so self-host it or keep an exit plan.
• Datadog LLM Observability. Agent traces next to your APM and infrastructure data. Because only LLM spans are billed, an agent's tool and retrieval steps add nothing to the bill.
• Galileo (Splunk). Strong on evaluation metrics such as hallucination checks, and now part of Splunk Observability Cloud. A fit for Splunk shops; small teams will find it heavy.
• Opik. The cheapest paid cloud plan here at $19 for 100k spans, and the whole platform is Apache 2.0 if you self-host. Comet's eval tooling comes with it.
• Laminar. OTel-native and aimed at agents that browse or act, with one-line set-up for Browser Use, Stagehand and the Vercel AI SDK. The free tier keeps only 7 days.
• AgentOps. Session replay for agent frameworks, and Pro includes unlimited log retention. The free tier's 5,000 events suits testing rather than production.
• Traceloop and OpenLLMetry. OpenLLMetry is the useful part for most teams: free OTel instrumentation that can send to Traceloop or many other backends. The hosted free tier keeps data for only 24 hours.
• HoneyHive. Enterprise focus, with a hybrid option where HoneyHive runs the control plane and your data stays in your cloud. No self-serve paid tier is published.
• Pydantic Logfire. General OpenTelemetry observability that also handles agents, with 10 million free records a month. Best for Python teams, especially those using Pydantic AI.

Retention differs as much as price. On the free tiers, it ranges from 24 hours (Traceloop) and 7 days (Helicone, Laminar) to 15 days (Datadog, Arize AX), 30 days (Langfuse, Logfire, HoneyHive) and 60 days (Opik). LangSmith keeps base traces 14 days and extended traces 180 days; Langfuse Pro gives 3 years of data access; AgentOps Pro lists unlimited log retention; Opik sells retention up to 400 days; Datadog sells 30, 60 and 90-day retention as add-ons. If you need a year of traces for audits, price that in before anything else.

Price on your own volume

Because the units differ, count your own. Take a typical run and list its steps. An agent that makes three model calls, two tool calls and one retrieval produces one trace and six spans per run. At 20,000 runs a month, that is 140,000 units on Langfuse, which counts the trace and every observation, and about 120,000 spans on span-billed tools such as Opik, Arize AX or Galileo. On Datadog it is 60,000 LLM spans, because only the model calls are billed. Request-based and data-based pricing need a different count again: model requests per run for Helicone, gigabytes of trace data for Braintrust, Weave and Laminar, and records for Logfire. Per-seat pricing (LangSmith) adds a second meter: multiply by everyone who needs to read traces, not only the engineers. Run that arithmetic for your top two candidates before you compare headline prices.

How to choose

Pick by situation, not by feature list. The products above overlap heavily on core tracing, and the differences that decide most choices are hosting, the pricing unit and what each was built around.

• One agent, small team, want traces this week. Opik Pro Cloud ($19), Langfuse Core ($29) and Laminar Starter ($30) are the cheapest paid plans; Logfire's free tier gives one user 10 million records. Helicone's gateway is a base-URL change, but read its status note first.
• Data must stay on your own infrastructure (health, finance, strict EU customers). Self-host Langfuse (MIT), Opik or Laminar (Apache 2.0) at no licence cost, or Phoenix under the Elastic License. For a vendor-supported private deployment, LangSmith, Braintrust, Arize AX, HoneyHive, Logfire and Weave offer one on enterprise terms. Self-hosting means you run the database and its upgrades. For agents handling PHI, read HIPAA-compliant AI agents first.
• Built on LangChain or LangGraph. LangSmith fits most naturally, since one company makes both.
• Quality is the main worry and evals gate every release. Braintrust is built around experiments and scores; Langfuse, Opik and Phoenix also run evals and cost less to start. Build the test set first: how to build an eval set from support tickets.
• Already paying for Datadog. Its LLM Observability keeps agent traces beside the rest of your monitoring, and tool and retrieval steps are not billed. If you want a Datadog alternative for agents only, Langfuse, Opik or Phoenix cost less at low volume; Logfire covers general OTel monitoring too.
• Already on Weights & Biases or Splunk. Weave and Galileo keep agent traces next to the tools you already pay for.
• Want to keep options open. Instrument against the OpenTelemetry GenAI conventions, or with OpenLLMetry, and check in each vendor's own docs that it accepts them before you commit.
• Nobody on the team has time to read traces. A tool records problems; someone still has to look at them every week. If you would rather hand that off, Managed AI Operations runs the monitoring, eval regressions and model updates for you.

One honest note: a single low-volume agent can start with structured logs in your own database, keyed by a trace ID, and a weekly review of a sample. Buy a tool when you find yourself reading traces every week and the logs slow you down.

Questions to ask any vendor

Copy these into your evaluation sheet and get the answers in writing:

1. What exactly is a billable unit, and how many will one of our runs produce?
2. Where is our data stored, and can we choose the region?
3. Can we redact or mask PII before data leaves our servers?
4. What is the retention on the plan we would buy, and what does a longer one cost?
5. Can we export raw traces through an API or in OpenTelemetry format?
6. Can we attach eval scores and user feedback to a trace after the run?
7. Does a multi-step run show tool calls and sub-agents nested in order?
8. Who at the vendor can see our traces, and is that logged?

What Gigabit sets up in production agents

Observability is one of six layers in every agent we ship, alongside a model tier, orchestration, retrieval, evals and guardrails, as our production AI agent stack describes. In that stack every action is logged with full context, cost and latency are tracked per run, and drift is surfaced on dashboards. Permissions are bounded per step, low-confidence states escalate to a person, and rollback paths are rehearsed. Before an agent ships we build a scored test set from real cases, including the awkward ones, and every later change to prompt, model or retrieval runs against it. No green run, no deploy.

A Gigabit Agents build is a flat fee from $8,000 per agent, and that fee includes evals and observability rather than treating them as change orders. It covers testing against real cases, guardrails and escalation paths, deployment and 30 days of tuning. After that, Managed AI Operations from $3,000 a month handles monitoring, eval regressions, model updates and new intents.