The Token Subsidy Is Ending, and Agents Are Token Multiplication Machines

2026-07-18 // Theory

Unit prices for intelligence are collapsing while enterprise AI bills explode. Both are true because agents changed the metric from cost-per-prompt to cost-per-task — and the mathematics of stateless LLMs, context bloat, and GPU memory eviction mean the tokens per task are growing faster than the price per token is falling.

Part 3 of the Innovation Plateau series.

For two years, AI felt cheap because someone else was paying. Flat-rate pricing hid the true cost of compute — the Uber playbook. Just as VC money once bought you a $10 ride across any city, flat-rate AI subscriptions ($20/month consumer, $200/month pro) were priced for light chatbot use and then colonized by agentic pipelines running 30–50 API calls per task. SemiAnalysis found single $200/month plans burning up to $14,000 of API-equivalent compute per month — a 70x gap, quietly funded by everyone who just wanted help writing emails.

That subsidy is now being unwound. Through late 2025 and 2026, Anthropic, OpenAI, and GitHub all restructured toward metered, usage-based billing; some Copilot users watched monthly bills jump from €67 to €966.

The Paradox

Here’s the paradox: unit prices are collapsing — equivalent-capability inference is getting ~10x cheaper per year, a 1,000x drop since 2021 — and yet enterprise AI bills keep exploding, with Fortune 500 firms heading past $30M/year. Both are true because the metric that matters changed from cost per prompt to cost per completed task, and agents multiplied the tokens per task faster than the price per token fell.

Fig. 01 // The paradox — unit prices fall, task costs rise (indexed, 2023 = 100)

Era	Example Models	Input / 1M tokens	Output / 1M tokens
Late 2022	GPT-3 class	~$20.00	—
2026 frontier	Claude Opus 4.6, GPT-5.5	~$5.00	~$25–30
2026 mid-tier	Claude Sonnet 4.6, GPT-4o	~$2.50–3.00	~$10–15
2026 economy	Llama 3.2 3B, Gemini Flash-Lite	~$0.06–0.08	~$0.30–0.90

Agents Are Token Multiplication Machines

Why do agents cost so much? Because LLMs are stateless. Every API call re-reads the entire context from scratch — the system prompt, the instructions, and the accumulated history of everything the agent has done so far. Agents run in loops (plan, act, observe, retry), and every retry appends the failed attempt to the context. Industry traces put agentic workloads at 5–30x the tokens of a chatbot query, stretching to 1,000x in complex pipelines; a single coding agent can average 85,000 tokens of context per turn.

Fig. 02 // Context bloat — the same history is re-billed on every call

Musical Chairs with GPU Memory

The industry’s answer is KV caching: store the computed representations of a prompt so re-sending it costs ~90% less. On paper, agents re-use 95–99% of their context. In production, physical hit rates are far worse — because caches live in GPU DRAM, DRAM is capped at 1–2 TB per node, and it’s shared across thousands of concurrent users. Let your agent idle for five minutes while a parallel subtask finishes, and the provider evicts your cache to seat someone else. When you resume: cache miss, full re-prefill, full price. It’s musical chairs with GPU memory, and the constant eviction-and-recompute cycle destroys the unit economics of agents at scale.

The Bottom Line

The subsidy that made AI feel cheap is being withdrawn at exactly the moment agents are multiplying consumption. Cheaper tokens don’t help when the token count per task grows faster than the price falls — and the hardware realities of GPU memory mean the caching tricks that should fix this break down under real agentic workloads. This is the economic engine of the adoption plateau; the reason coding is the exception that proves the rule is part 2.