Long-context AI agents need strict token budgets

A long-context AI agent should start with a $100 monthly token cap and a per-job ceiling: published list prices put 1,000 repeated 100,000-token requests between $11 and $220 before tools, storage, or engineering costs.

FuturPulse analysis · 19 October 2026

How much should an AI agent cost?

For a new production agent, budget for a successful completed job, not for a single chat message. A sensible starting control is a low double-digit dollar monthly pilot budget per active user, paired with a hard spend cap per job. Raise that ceiling only after the agent proves it completes work that is worth more than the extra spend.

The answer changes with the task. A support-routing agent can be cheap because it needs little history. A coding, research, legal-review, or document-analysis agent may repeatedly carry a large file, tool results, and prior messages. Tokens are the chunks of text a model reads and bills by. Long context means putting far more of those chunks into each request.

The important unit is therefore cost per completed outcome. A cheap-looking call becomes expensive if the agent loops, rereads its history, or fails after several tool attempts. The SWE-Effi research project found “expensive failures” in coding agents, including one setup where an unresolved attempt used more than four times the resources of a successful one. The study’s analysis is a useful warning against judging an agent by its answer rate alone.

Set two limits before launch:

• Job limit: the maximum tokens, tool calls, and elapsed time for one task.
• Monthly limit: the amount the product can spend before it slows, routes work to a cheaper model, or asks for approval.

Those guardrails make a bill predictable. They also force a business decision: is a $2 agent run acceptable for a task that saves a worker 10 minutes, or must it stay below 20 cents There is no universal right answer, but there is a wrong one: allowing a long-running agent to spend without a measured stopping rule.

What does a long-context request actually cost?

The comparison below answers a narrower, useful question: what do published token rates imply when an agent sends 100,000 input tokens and returns 2,000 output tokens That resembles a single full-document turn, not an entire autonomous job. It deliberately excludes tool charges, storage, networking, retries, taxes, and platform margins.

OpenAI lists separate Standard short-context and long-context rates for GPT-6 Luna. This calculation uses its listed long-context Standard prices: $0.20 per million input tokens and $0.75 per million output tokens. OpenAI’s pricing table does not state the token boundary between those columns on that page, so treat the result as the published long-context scenario rather than an assumed threshold.

Anthropic’s cache documentation lists Claude Haiku 5.5 at $0.10 per million input tokens for prompts up to 100,000 tokens and $0.50 per million output tokens. It lists Claude Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens. Google lists Gemini 3.8 Flash at $0.75 input and $3.75 output per million tokens through 31 December 2026. Google’s price page says those rates double from 1 January 2027.

Our calculation assumes every request carries the same 100,000-token context and produces 2,000 tokens. It multiplies published per-million input and output rates by those token counts. It is not a performance benchmark or a claim that the models complete equivalent work.

The table gives a procurement range, not a model ranking. It shows why “long context” is too broad for a budget. A $0.011 question and a $0.22 question can both include 100,000 tokens, yet their list-price cost differs by 20 times. Measure task quality before selecting the cheaper row.

Why do agent costs rise faster than expected?

An agent is a loop. It reads state, chooses a tool, sees a result, and repeats. If every new model call includes the old conversation, prior tool output is billed again. That makes total input grow much faster than a simple count of steps suggests.

Augment Code explains this as quadratic growth in a naïve loop: accumulating history is sent repeatedly, rather than once. Its guide says a 20-step run can consume more than 10 times the tokens implied by a basic per-step estimate. That is a design claim, not a universal tariff, but the billing mechanism is clear.

Suppose an agent begins with a 30,000-token repository brief. If it adds test logs, command output, and reasoning after every turn, the next call may contain both the brief and the growing trail. A model provider charges the input again unless cache pricing applies. Tool output can therefore matter as much as the user’s original prompt.

That is why a monthly budget should have a second line item for failures. A run that edits the wrong files, retries a broken command, or searches repeatedly may create no value while consuming its highest token total. The SWE-Effi researchers argue that time and tokens must sit beside task accuracy when judging agent systems. Their resource-constrained evaluation frames that trade-off directly.

When does caching make long context affordable?

Caching is most useful when many questions reuse the same large prefix, such as a policy library, codebase snapshot, contract set, or durable system instructions. It does not make new text free. It reduces the rate for a previously processed matching prefix during the cache’s allowed lifetime.

Anthropic says its default prompt-cache lifetime is five minutes, refreshed when reused. Claude Sonnet 5.5 charges $2.50 per million tokens to write a five-minute cache and $0.10 per million tokens for cache hits or refreshes. Anthropic’s pricing table also lists a $10-per-million output rate, which remains payable when the agent generates a response.

By our calculation, a 100,000-token Sonnet 5.5 prefix with a 2,000-token answer costs $0.27 when it is first written to the five-minute cache. A later matching request costs $0.03: one cent for the cached input and two cents for output. That is a ninefold reduction for the repeated request, but only if the reusable prefix matches and the follow-up arrives before expiry.

Google also publishes context-cache pricing. Gemini 3.8 Flash lists $0.075 per million cached tokens through 31 December 2026, plus $0.50 per million tokens per hour for cache storage during that period. Google’s page distinguishes cache use and storage, so teams should include both in workloads that keep large context resident.

Cache first when the material is stable and the questions are many. Do not cache indiscriminately. Frequently changing state reduces reuse, while a cache that includes sensitive or customer-specific material needs the same access controls as the underlying source.

Should an agent use full context or retrieval?

Use full context when the agent genuinely needs relationships across a bounded body of material. Use retrieval when most questions need only a small subset, the public record is much larger than a context window, or repeated full-context calls would break the job budget.

Retrieval-augmented generation, usually shortened to RAG, searches a knowledge collection and inserts selected passages into the prompt. Its appeal is simple: fewer input tokens usually mean a lower model bill. But retrieval can miss a passage or lose connections between far-apart sections. A full-context approach removes that retrieval step, but bills every included token.

MindStudio’s long-context explainer makes the useful distinction: flat per-token pricing makes the cost predictable as prompts deepen, not automatically cheap. Its older model prices should not be used for a current budget. The current vendor price cards above are the appropriate basis for procurement.

The decision should be tested with representative work. Give both designs the same questions, document set, model, output cap, and success measure. Then compare completed-task quality, token use, response time, and human correction. A lower token bill is not a saving if it causes a worker to redo the task.

How should teams set long-context agent limits?

Start with a small workload and record every billable component. Model tokens are usually the largest visible line, but tools can dominate an agent that browses, executes code, or retrieves files. OpenAI, for example, lists web search at $10 per 1,000 calls plus model-rated search-content tokens, while a 1 GB hosted container is $0.03 per 20-minute session. OpenAI’s tool pricing makes clear why a token-only estimate is incomplete.

Use a routing policy rather than one model for every turn:

• Use a low-cost model for classification, extraction, and choosing a tool.
• Reserve a more expensive model for synthesis, difficult planning, or final review.
• Cap output length, because output rates can exceed input rates.
• Summarize or discard stale tool logs before the next model call.
• Stop after a fixed number of failed tool actions and hand off to a person.

Also test whether each standing instruction earns its place. A study released in revised form on 29 September 2026 found that repository-level AGENTS.md context files did not generally improve coding-agent success rates in its evaluation, while increasing inference cost by more than 20% on average. The AGENTS.md study does not prove all context files are wasteful, but it supports measuring their effect rather than treating longer prompts as automatically better.

A practical monthly model is simple: expected jobs × median cost per completed job, then add a failure allowance and tool allowance. Use a percentile cap as well. The median hides rare runaway sessions, which are often where a small pilot loses control of its budget.

What changed in long-context agent pricing?

Long-context costs are changing through both published price schedules and implementation choices. The timeline separates what providers have already documented from dates when a published future rate is due to replace a promotional one.

• 12 February 2026
  The first version of the AGENTS.md research paper was submitted. Its later revisions tested whether repository context files improved coding-agent work, giving teams a reason to audit fixed prompt material rather than assume it is useful. The arXiv record shows the submission history.
• 17 March 2026
  A paper on trajectory reduction was created for publication, addressing ways to reduce the work an agent carries through a software task. The paper concerns agent behaviour, not vendor API pricing. The research paper describes trajectory reduction as its subject.
• 29 September 2026
  The third version of the AGENTS.md paper was posted. It reported more than 20% average inference-cost growth from the context files it evaluated, without general task-success improvement. The revised abstract states both findings.
• 30 November 2026
  Google’s current Gemini 4 Argon promotional paid price is scheduled to end. Google lists $2 input and $10 output per million tokens through that date, then $4 and $20 from 1 December 2026. Google’s published schedule is a future price change, not a price already in force on 19 October 2026.
• 31 December 2026
  Google’s published Gemini 3.8 Flash promotional rates are scheduled to end. The page lists $0.75 input and $3.75 output per million tokens through that date, with higher rates from 1 January 2027. The Gemini pricing page sets out that planned change.

What we could not verify?

No public price card can say what a particular agent should cost for every business. The missing facts are the agent’s real token distribution, completion rate, retry rate, tool-call count, cache-hit rate, support burden, and the financial value of a correct result. Those require production traces or a controlled pilot.

Public model prices also do not establish that similarly named models deliver equal quality on the same long-context task. The model vendors, the team operating the agent, and independent evaluators could settle that with reproducible task sets, full token logs, tool charges, and human review results.

Sources

• OpenAI API pricing
• Anthropic prompt caching documentation
• Google Gemini Developer API pricing
• SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints
• Augment Code guide to AI agent loop token costs
• MindStudio guide to flat-rate long-context pricing
• Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents