Why do token costs grow so much faster for agents than for chatbots?

A chatbot sends one user message and receives one reply. An agent running a multi-step task sends the growing conversation history plus tool outputs on every intermediate call, so context length compounds across steps. A ten-step workflow can consume an order of magnitude more tokens than ten independent chat turns.

What is prompt caching and when does it actually help?

Prompt caching lets the model provider reuse the computed state of a static prefix, so you pay a reduced rate for those tokens on repeat calls. It works best when your system prompt or a large document stays constant across many requests. ProjectDiscovery cut LLM costs by 59% within weeks by applying caching to their security testing agent's repeating context.

How should I set a hard cost limit without breaking legitimate long-running tasks?

Set limits at the task level, not the session level. Estimate the maximum token budget for each task type during load testing, add a safety margin of around 20%, and fail the task with a structured error when the budget is exceeded. This lets short tasks complete cheaply while blocking only the runaway ones.

Which model should handle which parts of an agentic workflow?

Route planning, tool selection, and result synthesis to a large frontier model. Route classification, filtering, intent detection, and repeated sub-tasks to a smaller, cheaper model. The decision boundary is whether the step requires open-ended reasoning or whether a capable smaller model can produce a reliable answer within a well-defined scope.

Agent Token Economics: Engineering Cost Efficiency Before Your First Production Deployment

What this article gives you

By the time you finish reading, you will know how to measure what your agent actually spends per task, why that number grows faster than you expect at scale, and which three techniques cut the largest share of waste before you deploy. You will also have a pattern for setting hard cost limits that stop runaway agents without killing legitimate work.

Why agent token economics are not chatbot token economics

A chatbot sends a message and receives a reply. An agent runs a loop: it reasons, calls a tool, reads the result, updates its plan, and repeats. Every iteration carries the full conversation history forward. By step five of a ten-step task, the model is reading everything that came before it on every single call.

This is the quadratic problem. Context length grows with each step, and cost scales with context length. According to research from Antino and Zylos, LLM API calls account for 70 to 85 percent of total agentic AI cost, and agents make three to ten times more LLM calls than simple chatbots. Unoptimized production agents regularly reach $10 to $100 per session in API spend.

That ceiling matters because 88 percent of organisations are experimenting with agents, but only 23 percent are actively scaling, and Gartner projects 40 percent of agentic AI projects will be cancelled by 2027. Uncontrolled token spend is one of the clearest routes to that outcome.

Measuring what you actually spend

You cannot optimise what you have not measured. Instrument every LLM call before you write a single optimisation. Record input tokens, output tokens, model name, task type, and step number. Aggregate by task run, not by request, so you can see the full cost of one unit of work.

The step number field is particularly useful. If your cost-per-task spikes after step three, that tells you the context is accumulating faster than the task is progressing, which points directly at context compaction as the right fix.

flowchart TD
    A[Task starts] --> B[Record: tokens in, tokens out, step number]
    B --> C{Step limit reached?}
    C -- No --> D[Agent calls tool]
    D --> E[Append result to context]
    E --> B
    C -- Yes --> F[Emit cost summary]
    F --> G{Within budget?}
    G -- Yes --> H[Task complete]
    G -- No --> I[Fail with structured error]

Store these records somewhere you can query them by task type. After one week of load testing, you will have a cost distribution per task type that you can use to set per-task budgets and to forecast monthly spend at your target volume.

Three techniques that move the number

Prompt caching

If your system prompt is long, your agent pays full price for it on every call. Prompt caching lets the model provider store the computed representation of a static prefix and charge a fraction of the normal input rate for subsequent calls that reuse it.

ProjectDiscovery applied this to their security testing agent Neo in April 2026. The result was a 59 percent reduction in LLM costs within the first few days, reaching 70 percent by day ten. One developer account dropped from $720 per month to $72. The technique requires that the cacheable content sits at the start of the prompt and stays stable across calls, so structure your prompts with the static system instructions first and the variable task context after.

Model routing

Not every step in an agentic workflow needs a frontier model. Planning and synthesis benefit from the best reasoning available. Classification, intent detection, filtering, and repeated sub-tasks often produce equally reliable results from a model that costs a tenth as much.

Build a routing layer that maps task types to model tiers. The decision criterion is whether the step requires open-ended reasoning or whether it fits inside a well-defined scope where a smaller model can be reliable. Test each route with representative inputs before you trust it in production.

flowchart TD
    A[Incoming step] --> B{Step type?}
    B -- Planning / synthesis --> C[Large frontier model]
    B -- Classification / filtering --> D[Small fast model]
    B -- Tool call parsing --> D
    C --> E[Emit result + token count]
    D --> E
    E --> F[Aggregate to task budget]

Context compaction

The most direct attack on the quadratic growth problem is to stop the context from growing unchecked. At each step, summarise completed sub-tasks into a compact record and drop the raw turn history. The agent retains what it needs to continue without carrying every token from the beginning.

This requires a small summarisation call, which adds a modest cost at each checkpoint, but that cost is fixed per step rather than proportional to total history length. For tasks beyond five or six steps, compaction almost always pays for itself. Agentic RAG patterns use a version of this: rather than stuffing full documents into context, they retrieve only the fragment relevant to the current step.

Setting hard cost limits

Optimisation reduces average spend. Limits bound the worst case. Both are necessary.

Set a token budget at the task level during load testing. Take the 95th-percentile cost for each task type, add 20 percent, and enforce that as a hard ceiling. When an agent exceeds its budget mid-task, emit a structured error with the task ID, the step number, and the token counts. Do not silently truncate context; that produces unpredictable behaviour that is harder to debug than a clean failure.

Platform-level tooling, for example Prefactor, can enforce these limits outside the agent code so they cannot be bypassed by a runaway loop. Whether you use dedicated tooling or implement the check in your orchestration layer, the limit needs to be enforced somewhere the agent cannot override it.

Wire the budget check into your agent observability pipeline so that every breach appears in your alerting. A pattern of budget breaches on one task type is a signal to revisit the workflow design, not just the budget figure.

What production deployments show

The numbers from teams that have gone through this process are consistent. Klarna’s customer service agent resolved 2.3 million conversations in its first month and reached $60 million in annual cost savings, with response times falling from eleven minutes to under two minutes. That scale is only viable if the per-conversation token cost is predictable and bounded. Morgan Stanley’s DevGen.AI agent processed nine million lines of legacy code and saved 280,000 developer hours in five months, handling the kind of long-context work where unoptimised token spend would have made the economics unworkable.

These are not outliers. They are what happens when multi-agent systems are designed with cost instrumentation from the start rather than retrofitted after the first billing surprise.

The engineering work is not complicated. Instrument first, then cache, route, and compact. Set limits before you deploy. The agentic AI architecture decisions that affect cost are mostly made at design time, which is exactly why you want this framework in place before production, not after.

Where to start

Run your candidate workflow in a staging environment with full token logging for at least 500 task runs across your expected task-type mix. That data will tell you where the cost is concentrated and which of the three techniques applies first. Take the agent readiness assessment to identify the gaps in your deployment plan before token spend becomes a fire drill.