The terms each tool uses
The tools name the same things differently, so here is each term once.
- Span: one unit of work, such as a model call, a tool call or a retrieval step. Spans are the building blocks of traces. Langfuse calls a span an observation, and LangSmith calls it a run.
- LLM span: in Datadog's billing, a single call to a model provider such as OpenAI or Anthropic.
- Trace: all the spans of one request, joined by one trace ID. For an agent, LangSmith records each turn as its own trace: the user's message and everything the agent does in response.
- Session: the traces of one user interaction, such as a chat thread. Langfuse calls it a session, and LangSmith calls it a thread.
- Score: the result of evaluating a model's output, written by an evaluator or a person.
The difference between a session and a trace matters when you decide what to keep. A one-turn session is one trace. A support chat with ten turns is ten traces in one session. A sampler that keeps a share of traces decides from the trace ID, so it can keep turn three of a chat and drop turn four. The conversation you need to read then has gaps.
What each tool traces
A tool sees the model request alone, the agent's steps, or the agent's steps inside the whole request. That decides what you can debug, because a tool cannot help with a step it never received. Some tools log the model request and little else. Most record the agent's steps: each model call, tool call and retrieval step as spans in one trace. A few keep that trace inside the rest of the request, down to the database query a tool call made, when your services pass trace context along.
The defaults matter here. Langfuse accepts OpenTelemetry traces, but by default its SDK exports only the LLM-relevant spans, and to send everything you set a permissive filter. Datadog records the whole agent workflow and bills only the LLM spans in it. To see the service request around the agent, Datadog customers add its APM.
To check a tool, send it one real agent session. Then look for the tool calls, the retrieval and the database query under them in a single trace.
How the billing unit decides whether you sample
A per-trace bill grows with traffic. A per-unit or per-gigabyte bill also grows when the agent takes more steps or sends longer prompts. When the bill grows faster than the value of old traces, teams turn on sampling.
Every hosted tool bills on a unit, and each one counts something different. So the same agent session costs a different number of units in each tool.
Take one session of one turn: four model calls, eight tool calls and one evaluation score. Langfuse counts every trace, observation and score as a unit, so the session is about 14 units if each call is one observation. LangSmith counts the session as one trace, however many steps it has. Datadog counts four LLM spans and does not bill the tool calls. Helicone counts requests, so it sees the four model requests.
Tools that bill by the gigabyte, such as Braintrust and Oodle, count the bytes of every span. Agent spans carry prompts and responses, so they are large, and a per-gigabyte bill grows with prompt size.
What keeping every session costs on each tool
For 1 million sessions a month, each the one-turn session above, keeping every session costs $9,610 or more on Arize AX, the most of the tools priced here. Each figure is from the tool's own pricing page, as of October 2026, on the plan a team at this volume would start with. Oodle bills by the gigabyte ingested, starting at $1 per GB, so its bill depends on the size of your sessions. Measure one of your own sessions to work it out.
| Tool and plan | One session counts as | A month of sessions | Cost a month | Retention included |
|---|---|---|---|---|
| Oodle Agent Observability Pro | Its size in GB | Your sessions' total size | Ingest-based, from $1 per GB | 30 days |
| Helicone Pro | 4 requests | 4 million requests | $294.50, plus storage | 1 month |
| Langfuse Core | 14 units | 14 million units | $991 | 90 days to view |
| Datadog Agent Observability Pro, annual | 4 LLM spans | 4 million LLM spans | $1,525 | 15 days |
| LangSmith Plus, one seat | 1 trace | 1 million traces | $4,989 | 14 days |
| Arize AX Pro | 12 spans or more | 12 million spans | From $9,610, plus $3 a GB past 10 GB | 30 days |
The working, tool by tool:
- Helicone: past the first 10,000, requests cost from $0.0007 down to $0.00002 each. Its tiers price 4 million requests at $215.50, plus $79 for Pro, which makes $294.50. Storage starts at $3.25 a GB, so 50 GB stored would add $137.50, for $432 in all.
- Langfuse: 14 million units on Core's graduated rates is $29 for the plan, $72 for the units up to 1 million, $630 for 1 to 10 million and $260 for 10 to 14 million.
- Datadog: 4 million LLM spans is 3.9 million above the 100,000 included, or 390 blocks of 10,000. At $3.50 a block on an annual commitment that is $1,365, plus $160 for the plan. Month to month it is $1,838, and on demand $2,190.
- LangSmith: 990,000 traces above the 10,000 included, at 0.005 LSU each, where 1 LSU is $1, is $4,950, plus $39 for the seat. If an online evaluator writes the score, the trace upgrades to extended retention, which adds $2,500 at 0.0025 LSU a trace.
- Arize AX: AX Pro is $50 a month and includes 50,000 spans and 10 GB. As of November 2026, each span past that costs $0.0008 and each GB $3. At 12 million spans, 11,950,000 are past the 50,000, which is $9,560, plus $50 for Pro. More spans a session, or more than 10 GB a month, adds to that. Self-hosted Phoenix has no license fee, and you pay for the servers.
A bill by the gigabyte grows when sessions are large. Take a tool-heavy agent whose eight tool calls each return a 200 KB document. Its session is about 2 MB, so 1 million sessions is 2,000 GB, and a per-gigabyte bill grows with every one of them.
Datadog's bill stays at $1,525, because it bills the four model calls and not the tool spans. Langfuse's stays at $991, because it counts steps, not bytes. A long conversation resent with each of the four model calls is billed by the byte each time. Datadog says you "Only pay based on the volume of LLM spans, even as context grows, reasoning gets heavier, and agents use more tools."
Why teams sample
Every tool here can sample, and teams turn it on to cut cost. LangSmith's sampling docs say sampling helps "balance observability needs with cost considerations." Langfuse's default rate is 1, so it collects every trace until you lower it. Oodle's pricing is set so that sampling is optional.
Most built-in samplers decide when the trace starts. That is called head sampling. In OpenTelemetry's words, head sampling makes the decision "as early as possible", from the trace ID and a percentage. Langfuse's SDK, for example, samples "on the trace level" in your code. So head sampling cannot make sure the failing sessions are kept. At a 10% rate, the session a customer complains about is gone nine times out of ten.
Tail sampling decides after the trace is complete, so it can keep every trace with an error and a share of the rest. The OpenTelemetry Collector does this with its tail sampling processor, which "keeps spans in memory while it waits to make a sampling decision". It also needs every span of a trace to reach the same Collector instance. Any tool here that accepts OpenTelemetry can receive its spans from such a Collector. Inside LangSmith, an automation rule can upgrade the traces it matches to extended retention, which keeps the failures longer at a higher trace price.
A session where the agent gave a wrong answer while every call succeeded has no error for a rule to catch. No sampler can promise to keep the session that failed, so the number to check for each tool is what keeping every session costs at your volume.
How far back each tool lets you debug an incident
Default retention runs from seven days to forever (the table below has each one), and what differs more is what changes a trace's retention and what happens when it runs out. Retention decides how old a trace can be when you open it to explain an incident, and whether the tool meets a retention rule you must follow.
- LangSmith keeps base traces for 14 days and extended traces for 180 days, the SaaS maximum since September 14, 2026. An online evaluator or an automation rule can upgrade a trace to extended, which raises its cost. Matching a single run upgrades the whole trace.
- Langfuse Cloud limits how far back you can view by plan. That window does not delete data, and older traces can be opened again after an upgrade to a longer window. Deletion needs a retention policy, set per project with a minimum of 3 days (on the Cloud Pro and Enterprise plans, and an Enterprise feature when self-hosted). Self-hosted Langfuse keeps everything by default.
- Phoenix, self-hosted, keeps everything until you set
PHOENIX_DEFAULT_RETENTION_POLICY_DAYS. - Datadog keeps traces for 15 days and sells 30, 60 and 90 days as paid add-ons, priced per 10,000 LLM spans a month.
- Oodle Agent Observability Pro keeps traces 30 days, and keeping them longer is an added fee, up to multi-year retention.
Check which events upgrade a trace's retention, and what the tool does with a trace when its time is up.
Running a tool yourself
If hosted units or retention cost too much, the other route is to run the tool yourself. The licenses run from MIT and the Apache License to the source-available Elastic License, and what you operate runs from one service on SQLite to a set of databases or your own Kubernetes cluster.
Tools sold as open source on this list use three kinds of license. Langfuse's core is MIT, Helicone and Opik use the Apache License (Helicone, Opik), and Phoenix uses the Elastic License 2.0. The Elastic License 2.0 is source-available: Phoenix is free to self-host, but the license bars offering it to others as a hosted service.
The services differ more than the licenses.
- Langfuse runs two application containers with PostgreSQL, ClickHouse, Redis or Valkey, and S3 or other blob storage. Its Docker Compose setup is a "Single VM without high availability, scaling, or backups", so production means Kubernetes or a cloud template.
- Phoenix is a web UI, a trace collector and a SQL database. SQLite suits local and single-user use, and PostgreSQL suits production.
- Helicone self-hosts with PostgreSQL, ClickHouse and MinIO, and its Docker guide packs them into one all-in-one image. To proxy requests, you run its AI Gateway beside it.
- LangSmith self-hosting is an add-on to the Enterprise plan and runs in your own Kubernetes cluster.
For a small team where nobody wants to own observability infrastructure, the count of services is the real cost. A hosted plan, checked against your unit count, is often the lower-cost route.
How the LLM observability tools compare
The tools differ most in the unit they bill on, what keeping every session costs on that unit, and how long they keep traces by default. The figures below are as of October 2026, from each vendor's own pages, for the 1 million sessions priced above.
| Tool | What it traces by default | Billed on | Cost to keep 100% of 1 million sessions a month | Can it keep sessions based on how they ended? | Default retention | Free tier | Run it yourself |
|---|---|---|---|---|---|---|---|
| Oodle | Agent session, and the request around it when your services pass trace context and send their traces to Oodle | GB ingested | Ingest-based, from $1 per GB | Keeps every session, and sampling is optional | 30 days on Pro | None for agent observability | Hosted by Oodle. Bring your own cloud (BYOC) and bring your own bucket (BYOB) are Enterprise options |
| Langfuse | Agent session (LLM-relevant spans from its SDK) | Units: traces, observations and scores | $991 on Core | Not with its SDK sampler, which decides per trace | Cloud: 30 days to view (Hobby), 90 (Core), 3 years (Pro). Self-hosted: forever | 50,000 units a month | Yes, MIT core, free without usage limits |
| LangSmith | Agent session | Seats plus traces | $4,989 on Plus | Its sampler is a percentage, and an automation rule can keep matching traces longer | 14 days base, 180 days extended | One seat, 5,000 base traces a month | Enterprise add-on only |
| Arize Phoenix and Arize AX | Agent session over OpenTelemetry | Phoenix: free. AX: spans and GB per plan | Phoenix: your servers. AX: $9,610 or more on Pro, plus $3 a GB past 10 GB | Head sampling is the OpenTelemetry SDK default | Phoenix: forever. AX: 15 or 30 days | AX: 25,000 spans and 1 GB a month | Phoenix, under the Elastic License 2.0 |
| Helicone | Model requests | Requests and GB stored | $294.50 on Pro, plus storage | Not stated on its pages | 7 days to forever, by plan | 10,000 requests and 1 GB | Yes, Apache License |
| Datadog Agent Observability | Agent session, and the request around it with Datadog APM | LLM spans only | $1,525 on Pro, annual | Not stated on its pages | 15 days | 40,000 LLM spans a month | No |
The tools, one by one
Oodle
Oodle is agent observability for teams running agents at production volume. Its cost comes from how it is built. Traces go straight to object storage in Oodle's own columnar format, and queries run on serverless compute that spins up per query and shuts down after. So every session is kept in full, for your retention period (30 days on Pro, longer for an added fee).
Agent traces are kept on one platform with logs, metrics and application traces. When your services pass trace context, one trace runs from the API request through the agent and its tools to the database. Every session is analyzed at scale using statistics, ML classifiers and a small model for sentiment. Oodle's agent then checks each issue it finds against the evidence and removes the noise.
Signals tag each session as it arrives, and insights run once a day, as a short list each morning of what to fix. Fello, a real estate technology company, sends 6 million agent traces a day to Oodle with none sampled, at about a third of the cost of Langfuse.
Oodle has simple, ingest-based pricing, starting at $1 per GB, and is cost efficient. There are no seat or query fees. Agent Observability Pro includes 30 days of retention, keeping data longer is an added fee, and there is no fee to bring it back. There is no free tier for agent observability. Because Oodle bills by the gigabyte, a session with large tool results costs more on Oodle than on a tool that bills per model call.
Oodle runs as a hosted service. Bring your own cloud (BYOC) and bring your own bucket (BYOB) are Enterprise options, for agent observability as well as logs, metrics and traces. It is not open source. Setup is OpenTelemetry pointed at Oodle, and teams on the Langfuse SDK change only the endpoint. You can try Oodle on the playground without signing up.
Langfuse
Langfuse is for teams that want open source, self-hosting or a framework-neutral tool. Its core is MIT, with
enterprise features under a separate license in its ee directories. The copyright is held by
ClickHouse, Inc., since Langfuse is now part of
ClickHouse. Self-hosting is free, and Langfuse says you can
"Self-host all core Langfuse features for free without any limitations."
On Langfuse Cloud, the Core plan is $29 a month with 100,000 units, then $8 per
100,000 units, falling with volume (as of October 2026).
The limit is the unit. A session with many steps uses many units, and every score an evaluator writes counts too. Before you choose Cloud, count the units in one of your real sessions.
LangSmith
LangSmith is for teams building on LangChain or LangGraph, though it does not require them: it accepts OpenTelemetry from any application. Because it bills per trace, one execution of the agent costs the same however many steps it takes. As of October 2026, the Developer plan is free for one seat and 5,000 base traces a month. Plus is $39 a seat a month for 10,000 base traces, then pay as you go.
The limits are seats, which count every invited user, and retention. Base traces last 14 days, and features that upgrade a trace to 180 days add to its cost. Self-hosting is for Enterprise customers only.
Arize Phoenix and Arize AX
Phoenix is for teams that want to start locally and trace over OpenTelemetry. Arize calls it "our open-source, local-first platform", and it is the lightest tool here to run: one service on SQLite to start. Its license is the Elastic License 2.0, as Arize's docs state, with self-hosting "free and fully permitted".
The limit is scale on your own database. For high-volume ingestion, Arize's architecture page points to Arize AX, its managed product on its own database. As of October 2026, AX Free includes 25,000 spans and 1 GB a month with 15 days of retention. AX Pro is $50 a month and raises those to 50,000 spans, 10 GB and 30 days. Past those, each span costs $0.0008 and each GB $3, as of November 2026.
Helicone
Helicone is for teams that want every model request, its latency and its cost logged with almost no code. You can change the base URL so requests go through Helicone's proxy, or log asynchronously so Helicone is not on the request's critical path. It is under the Apache License.
As of October 2026, the Hobby plan is free with 10,000 requests and 1 GB of storage, and Pro is $79 a month plus usage. The same page carries a banner: "Helicone is no longer part of Mintlify. Details to come soon." Its view starts from the model request, so check whether you also need the tool calls and the work around them in the same trace.
Datadog Agent Observability
Datadog Agent Observability, formerly LLM Observability, is for teams already on Datadog. It is a standalone product, and teams that also run Datadog APM can follow an agent span into the service trace on the same trace ID. Its docs also describe Insights, which looks for recurring cost and reliability problems in the traces it receives.
Agent Observability bills per LLM span, so it does not bill tool, retrieval or workflow spans. As of October 2026, Pro starts at $160 a month on an annual commitment for 100,000 LLM spans. Each extra 10,000 LLM spans costs $3.50 on an annual commitment, $4.20 month to month and $5 on demand. It is held back by the 15-day default retention and by the need for APM, a separate product, to see the request around the agent.
Other AI observability tools on the lists
Braintrust is often named by teams that build around evaluation. It bills by the gigabyte of data processed and by the score, with 1 GB and 10,000 scores free, then $4 a GB and $2.50 per 1,000 scores (as of October 2026). Opik, from Comet, is under the Apache License. LiteLLM and Portkey are gateways: they route your requests to the model APIs and log each request and its cost. Weights & Biases Weave, Grafana Cloud, Honeycomb and New Relic suit teams already on those platforms.
How each tool handles LLM cost tracking
You rarely need to build cost tracking yourself. LangSmith, Langfuse and Helicone price model calls for you from token counts and a price table, and Datadog's dashboards show cost next to latency and usage.
- LangSmith records token use and cost for major providers automatically, and takes custom costs for tool calls and retrieval.
- Langfuse tracks the cost of every LLM call. It uses the cost you send, or works it out from the model and the token counts.
- Helicone estimates cost from the model in the response and a price table, with no instrumentation beyond the base URL.
The OpenTelemetry standard leaves the price to the tool. Its GenAI semantic conventions, at "Status: Development", record token counts with no cost attribute.
A gateway such as LiteLLM or Helicone's proxy gives you accounting for every request without touching each call site, at the cost of one more service in the request path. For the coding agents your engineers use, Claude Code and Codex, Oodle's AI cost management tracks spend by user, model, team, repository and session. Whichever you choose, check a week of tracked cost against your model provider's invoice before you trust it.
Which tool fits which team
| Your situation | Start with |
|---|---|
| You want every agent session kept at production volume, billed by the gigabyte on object storage, next to your logs, metrics and traces | Oodle |
| You build on LangChain or LangGraph and want a hosted tool | LangSmith |
| You want open source, and can run PostgreSQL, ClickHouse, Redis and object storage | Langfuse, self-hosted |
| You want a hosted tool, open source underneath, at moderate volume | Langfuse Cloud |
| You want to start on a laptop with one service | Arize Phoenix |
| You want model requests and their cost logged by changing a base URL | Helicone |
| You already run Datadog APM | Datadog Agent Observability |
How to test two tools before you commit
Pick two tools from the table and run them side by side for a week of real traffic. Instrument with OpenTelemetry, and use the Collector to send the same spans to both.
- Count what one of your sessions becomes on each bill: traces, units, spans, requests or gigabytes.
- Work out whether either tool would push you to sample at your volume, and which sessions you would lose.
- Open a session from the start of the week, and find out when it would expire.
- Check each tool's tracked cost against your model provider's invoice for the same week.




