FIND US ONProduct HuntAgent ObservabilitySee how we help you debug AI agents, and share your feedback.

LLM cost monitoring for agents: catch a runaway session before it runs up the bill

Check a per-session budget in your own code before every model and tool call. Spend alerts, provider limits and traces report LLM cost after it is spent.

Cover with the words "Stop the call before it is sent". A session budget meter crossing its max_usd limit. Call 9 was sent. Call 10 is estimated, checked before sending and not sent, raising BudgetExceeded.
On this page
  1. Why AI agent cost varies each time you run the same task
  2. What spend alerts, provider limits and LLM cost monitoring can stop
  3. AI agent cost control: a budget on each session, checked before every call
  4. Use the step and retry limits your framework already has
  5. Catch a loop before the step limit does
  6. Find the expensive sessions that stayed under the cap
  7. How to tell the budget works
  8. Finding why a session looped, with Oodle
  9. Common questions
  10. Sources

TL;DR

  • Check a per-session budget in your own code before every model and tool call, because alerts, provider limits and traces report cost afterward.
  • Loops and retries make sessions run away, and each extra step costs more because every model call resends the history so far.
  • A provider's monthly hard limit stops every session in the project at once, and only after the month's money is gone.
  • A spend alert sends a notification while API traffic continues, so it stops no session at all.
  • Reserve each call's estimated cost before it is sent and settle it after, so parallel calls in one session cannot spend past the budget together.
  • Stop a session that repeats a tool call with the same result, and keep the step cap as a backstop above your longest valid sessions.

When the same agent task costs a few cents on most sessions and many dollars on a few, the expensive sessions usually ran many more steps than the rest, through a loop or a stack of retries. Start by finding the code that sends the agent's model and tool requests. It is the one place a limit can act before the next call goes out. LLM cost monitoring on your traces then finds the costly ones that stayed under the limit, and those tell you where to set it.

Why AI agent cost varies each time you run the same task

The same task costs different amounts because the number of steps varies, and each later step costs more than the one before it. An agent loop calls the model, and if the model asks for tools, the loop runs them, appends the results and calls the model again. Each of those calls sends the conversation so far. Anthropic's docs describe the input on each turn as one that "Contains all previous conversation history plus the current user message", and tool results count toward it. So the tenth call in a session pays again for everything the first nine added.

Assume step 1 sends about 2,000 input tokens and each step adds about 2,000 more of tool output and model reply to the history, and leave out the system prompt. Step 10 then sends about 20,000 input tokens, and the first ten steps send about 110,000 in total. A session that loops to 30 steps sends about 930,000. Three times the steps sends more than eight times the input tokens. Those numbers are arithmetic, not a measurement, but the shape holds for any chat API that is sent the whole history on each call.

Prompt caching lowers the price of that history without changing the shape. On Anthropic's API, cache read tokens are billed at 0.1 times the base input price on most models, and less on some newer ones (as of October 2026). A cached history costs a tenth or less as much on each call, and it is still sent and billed on every call.

Loops are how a session ends up with many more steps than it needs. The agent calls the same tool with the same arguments, gets the same error or the same empty result, and tries again. One write-up on agent failure modes describes an agent in a retry loop that "keeps issuing the same tool call because it has no mechanism to recognize that the call already failed".

One team on the OpenAI Agents SDK wrote in its issue tracker that "an agent loop triggered thousands of API calls unexpectedly". A CrewAI feature request describes an agent that keeps "calling the exact same tool", such as search_web("rust").

Retries add attempts below the agent, where you may not see them. The OpenAI Python SDK retries certain errors "2 times by default", including rate limits, server errors and timeouts (as of October 2026).

A framework can also retry the step. CrewAI's max_retry_limit, the number of retries when an error occurs, defaults to 2. If CrewAI retries a step whose model call already used the SDK's two retries, one step can make up to nine attempts: three from the SDK for each of the framework's three tries. Not every failed attempt is billed. But a step retried after its model calls succeeded, because a tool or a parser failed next, pays for those calls again, with the longer history.

What spend alerts, provider limits and LLM cost monitoring can stop

A provider's spend controls work on the month and the account, so they cannot pick out one session. OpenAI's spend alert sends a notification while API traffic continues. Its hard limit is a "monthly organization or project cap". When spend reaches it, affected API requests "return a 429 error" until someone raises the limit or the next monthly cycle begins.

OpenAI also warns that hard limits "can interrupt production traffic" and that "Enforcement is not instantaneous". Anthropic's spend limits are also monthly, set for the organization or for each workspace. Both are as of October 2026.

So a hard limit stops every session in the project at once, after the month's money is gone, and an alert stops nothing. Keep both as a backstop for the account. Set the hard limit above what all your sessions should spend in a month, so that it catches a bug rather than a busy week.

A tracing tool sees a call's cost only after the call is made. It works out that cost from the tokens the call used and a model price table. Its alerts send a notification, such as a Slack message or a webhook call, after the spans arrive. A developer on OpenAI's forum wrote: "logging isn’t enforcement", and the trace "shows you what happened after the fact".

AI agent cost control: a budget on each session, checked before every call

The budget check has to run inside the session, before the next call is made. A session budget is a limit on what one agent session may spend, kept by your own code. It counts dollars or tokens, and also model calls, tool calls and elapsed time, because a loop of small calls can run for a long time before its cost stands out.

Before each model call, estimate the most the call can cost. That is the input tokens you are about to send at the input price, plus the call's maximum output tokens at the output price. If the spend so far plus that estimate is over the budget, do not make the call. After the call, replace the estimate with the cost from the usage the provider returned.

Your framework may already keep the count. The OpenAI Agents SDK tracks requests and tokens for every run, and its docs say you can use that count to "monitor costs, enforce limits, or record analytics".

Parallel calls need one more step. When sub-agents or parallel tool calls share a session, each can see the same remaining budget and go ahead, so together they spend past it. Reserve each call's estimate before it is sent and settle it after, so a second call sees the first one's share as spent.

The guard below keeps one budget per session and makes that reservation. The prices are your own per-token rates, and send is whatever makes the model request and returns its token usage.

python
import threading


class BudgetExceeded(Exception):
    pass


class SessionBudget:
    """One per agent session, shared by every model call in it."""

    def __init__(self, max_usd, max_model_calls):
        self.max_usd = max_usd
        self.max_model_calls = max_model_calls
        self.spent_usd = 0.0
        self.reserved_usd = 0.0
        self.model_calls = 0
        self.lock = threading.Lock()  # parallel calls share one budget

    def reserve(self, estimate_usd):
        with self.lock:
            if self.model_calls >= self.max_model_calls:
                raise BudgetExceeded("model call limit")
            if self.spent_usd + self.reserved_usd + estimate_usd > self.max_usd:
                raise BudgetExceeded("dollar limit")
            self.model_calls += 1
            self.reserved_usd += estimate_usd

    def settle(self, estimate_usd, actual_usd):
        with self.lock:
            self.reserved_usd -= estimate_usd
            self.spent_usd += actual_usd


def call_model(budget, send, input_tokens, max_output_tokens, in_price, out_price):
    # Worst case for this call: everything you send, plus the most it may write.
    estimate = input_tokens * in_price + max_output_tokens * out_price
    budget.reserve(estimate)
    actual = estimate  # a call that raised may still be billed: count its worst case
    try:
        response = send(max_output_tokens)
        actual = (response.input_tokens * in_price
                  + response.output_tokens * out_price)
        return response
    finally:
        budget.settle(estimate, actual)

The lock works within one process. A session spread across workers needs the same reserve-and-settle on one shared store, such as an atomic counter in your database or cache.

Tool calls get the same treatment: a counter, and a price where the tool itself is paid.

Put the check in the code that makes the call, such as a wrapper around your model client or your tool runner, and not in the prompt. A model that is told it has a budget can still ignore it, and a check in your code cannot be skipped. A reply on the same forum thread makes that case for a gate "outside the agent loop entirely (non-bypassable)".

Decide what happens at the limit before a session reaches it. A soft limit below the hard one can log a warning, switch the rest of the session to a smaller model, or ask a person to approve more. At the hard limit, end the session with a status your code and your dashboards both understand, such as budget_exceeded. Return what the agent has so far, and keep the trace, because that session is the one you will want to read.

Use the step and retry limits your framework already has

Most frameworks already cap steps and retries, so check that each cap is on and set where you want it. The defaults below are as of October 2026.

Setting Where Default What happens at the limit
max_turns OpenAI Agents SDK 10 (version 0.24.0) Raises MaxTurnsExceeded. max_turns=None turns the limit off
recursion_limit LangGraph, in the config you invoke the graph with Not given on the error's docs page Stops the graph with GRAPH_RECURSION_LIMIT
max_iter CrewAI agent 20 The agent "must provide its best answer"
max_retry_limit CrewAI agent 2 No more retries for that error
max_execution_time, max_rpm CrewAI agent None, so off Time and request-rate limits, only when you set them
max_retries OpenAI Python SDK 2 The error reaches your code

In your own code, a cap set to None, or set very high to get one long session through, is no cap. Because max_iter ends a loop with a best answer instead of an error, the session can look like a normal one on a dashboard.

A step cap is a blunt tool, though. It counts steps, not cost, so it cannot tell a long valid session from a loop. The same CrewAI feature request says a blind max_iter counter "kills valid 15-step workflows at step 11". Keep the step cap as a backstop, set above your longest valid sessions, and let the budget and a loop check do the main work.

Catch a loop before the step limit does

A loop shows in the calls well before it shows in the bill, so check for it on each tool call. Keep a count, per session, of each tool name with its arguments. When the same call comes back with the same result a set number of times (three is a reasonable start), stop the session or tell the model that the call already failed and what to try instead. For a tool that writes, pays or notifies, check the repeated name and arguments before running the call again, not after its result.

Arguments that drift need a second check. An agent can search for "notebook pc", then "portable computer", then "laptop", and learn nothing new each time. So also count calls since the session last made progress, such as a new result, a changed plan or a step finished, and stop when that count passes your limit.

Bound retries in your own code the same way. Retry only errors that can clear, such as timeouts, rate limits and server errors, with backoff and a small limit. Never retry a quota or billing error. And where your framework already retries the step, turn the SDK's own retries down, so the two layers do not multiply.

Find the expensive sessions that stayed under the cap

To find the expensive sessions that stayed under the cap, add up cost per session from your traces. A cap stops the worst session, but it says nothing about the sessions just under it, which can still cost many times a normal one. Tracing tools already price each call from its token usage and a price table, so the session total is a sum, as long as every call carries the session id. The price table makes the total an estimate, which can differ a little from the provider's bill.

Then read the distribution of session cost for each agent, not the average. The loops and retry storms are at the top of the distribution. Sort by cost, open the most expensive sessions, and read what the agent did. Tool-call counts point the same way. OpenTelemetry defines a metric that counts tool calls per agent invocation, gen_ai.invoke_agent.tool_calls, and an invocation with many times its usual count is worth opening before you price it.

That distribution is also how you set the budget. Put the hard limit above the cost of the most expensive sessions that finished correctly, and below the sessions that looped. Set it again when the agent changes, because a new tool or a longer prompt moves every session's cost. All of this needs every session. A costly session you sampled out to keep the tracing bill down cannot be opened later.

How to tell the budget works

Test the budget before you trust it, the way you would test any other limit in production code.

Before release

  • Force a loop in staging. Give the agent a tool that always returns the same error. The session should stop at the loop check or the budget, with a status such as loop_detected or budget_exceeded, and the trace should show which check fired.
  • Run parallel calls in one session. Send several at once against a small budget, and check that the settled spend never passes it.

In production

  • Count the sessions that hit each limit, per agent. A jump after a release is a regression. Sessions that hit the cap and were doing valid work mean the cap is too low.
  • Read a few capped sessions each week. Each one is either a loop you can fix at its cause or a sign the limit is in the wrong place.

Finding why a session looped, with Oodle

Oodle shows why an agent looped. A budget ends a runaway session, but it does not say why.

Traces go straight to object storage in Oodle's own columnar format, and queries run on serverless compute that spins up per query and shuts down after. Every agent trace is kept for your retention period (30 days on Pro, longer for an added fee).

Oodle identifies loops, tool failures, outages and stalled conversations out of the box, as each session arrives. Every session is analyzed at scale using statistics, ML classifiers and a small model for sentiment. Oodle's agent then checks each issue it finds against the evidence and removes the noise. An Oodle loop finding shows the drifting kind: one tool called six times in one turn, with no new information between the calls.

Oodle is observability built for AI agents. Once a day, Oodle's insights give a short list each morning of what went wrong, how often and why, with the traces behind each finding attached. A finding opens Claude Code or Cursor with the failing traces in the prompt, so the fix starts from the session that looped. The stop itself stays in your agent's own budget. Cost insights are in Oodle's AI Usage Overview dashboards.

See your own telemetry in Oodle

Start debugging in minutes.

Start freeBook a demo

Common questions

How do I catch a runaway AI agent run before its cost blows up?

Put a budget on each session in your agent's own code, and check it before every model and tool call. A call that would pass the budget is then never made. Add a loop check that stops a session repeating the same tool call, and keep your provider's spend limit as a backstop for the account. A spend alert and a trace both report the cost after it is spent.

Why does the same agent task cost a different amount each time?

The number of steps varies, and each step costs more than the last, because every model call resends the session's history. If each step adds about the same amount to the history, a session that loops to three times the steps sends more than eight times the input tokens. Retries in the SDK and the framework add more attempts.

Will my provider's spend limit stop a runaway agent session?

Not on its own. As of October 2026, OpenAI's hard limit is a "monthly organization or project cap", and Anthropic's limits are monthly, for the organization or a workspace. A hard limit stops every session once the month's money is spent, and an alert sends a notification while API traffic continues.

Will a hard cap stop long sessions that are valid?

It can, if it is set too low. Set it above the cost of each agent's most expensive sessions that finished correctly, put a soft limit below it, and count the sessions that hit it, so you see when it starts catching valid work.

Does my tracing tool stop an expensive session?

It prices each call after the call is made and can send an alert, so it reports spend that has already happened. Use it to find the expensive sessions and to set your budget, and keep the stop in your agent's code.

Sources

Share