DeepSeek's recent decision to drastically cut pricing on its V4-Pro model by 75% should have been unequivocally good news for enterprise AI vendors and developers. Instead, many are discovering that cheaper models don’t automatically translate into healthier margins.
The reason is simple: While inference costs plummet, agent systems are voraciously consuming tokens faster than prices are declining. For the last 2 decades, software economics was dictated by the same rule. Infra became cheaper every year whereas applications became more capable. AI was initially hypothesized to follow the same pattern. As frontier models improved and token prices dropped, many assumed inference would become a negligible operating expense.That assumption has begun crumbling exponentially.
A chatbot usually turns one user question into one model call. An agent turns it into a chain of planning, retrieval, tool use, verification, summarization, and follow-up decisions. The user sees one answer. The vendor pays for the loop. That is the 100x problem: The same user-visible request can cost a lot more to serve as an agentic workflow than as a chatbot or retrieval-augmented generation (RAG) response. In longer-running workflows, the multiplier is higher. Falling model prices help, but they do not fix a product architecture that turns one prompt into dozens of billable operations.
The scale of what is now at stake is clear in how model providers themselves are pricing developer relationships. OpenAI's proposed program to give every Y Combinator startup $2 million in API credits — a number that would have funded an entire seed round in any prior tech cycle, and when the same cohort got by on a few thousand dollars of AWS credits — is less a recruiting perk than an admission of what it now costs to run an AI-native company through its first year of product. For established enterprises retrofitting agents into existing product lines, the absolute numbers are larger still.
What token amplification is
In a single-turn chatbot, one user message produces roughly one model call. Input-to-billed ratio is about 1:5.
In a multi-step agent rolled out across customer support, sales operations, finance, legal review, and engineering, that ratio routinely lands at 1:700 or higher. Every loop iteration carries forward the cumulative conversation, tool outputs, and reasoning traces. Each step appends; nothing is dropped.
A "simple" agent query like “What did our top customer ask about last week?” typically touches seven priced operations before returning an answer:
-
User prompt (~50 tokens)
-
System prompt and tool definitions (~3,000 tokens, repeated on every call)
-
Retrieval (~5,000 tokens of context)
-
Model call #1 — tool selection (8,000 in / 200 out)
-
Tool execution (~4,000 tokens returned)
-
Model call #2 — summarization (12,000 in / 400 out)
-
Model call #3 — follow-up decision (12,400 in / 100 out)
One sentence in, roughly 35,000 input tokens billed. Somewhere between $0.10 and $0.40 per query on a frontier model. Multiply that by a million queries a month — the table-stakes volume for any enterprise B2B feature — and the line item is six figures.
Why this breaks the existing AI business model
The dominant pricing story for enterprise AI has been seat-based SaaS: Pay per-user per-month, deliver agent capability, capture margin. That model assumes a reasonably bounded cost-per-user.
Token amplification breaks the assumption. A power user running 50 agent invocations a day on a $40/seat plan can cost more in inference than the plan charges. Token amplification shatters the traditional SaaS pricing model. When a power user’s daily agent activity costs more in inference than their monthly subscription fee, vendor gross margins turn negative, a paradox that compounds as customers deepen their agent adoption, the very usage curve vendors are selling to their boards. Several vendors are now privately reporting negative gross margins on heavy users, mirroring recent cloud expenditure reports from the Bessemer 'Supernova' cohort, where the correlation between AI-agent adoption and gross margin contraction has moved from a theoretical risk to a primary P&L headwind.
The visible symptoms have started leaking into public coverage. Bloomberg this week documented a widening gap between Salesforce's Agentforce marketing demos and the capabilities actually shipping to customers. This is the kind of gap that opens predictably when promised functionality is technically possible but uneconomical to serve at the price the seat plan implies. Salesforce is the most-watched case, not a unique one.
"For my team, the cost of compute is far beyond the costs of the employees." — Bryan Catanzaro, VP of Applied Deep Learning, Nvidia
The strategic implication is not "AI is expensive." It is that the dominant business model assumed by most AI-native company plans does not survive contact with agentic workloads.
A simple example
Consider an enterprise software vendor charging $40 per-user per-month for an AI-enabled support assistant. A traditional chatbot might cost only a few cents per user per day in inference, leaving healthy gross margins.
Now replace that chatbot with a fully agentic workflow capable of investigating tickets, querying internal systems, drafting responses, validating outputs, and escalating exceptions. If a heavy user executes 50 to 100 agent requests per day, inference consumption can increase by an order of magnitude. What was once a negligible infrastructure cost becomes a material operating expense.
This creates an unusual dynamic: The customers receiving the most value from the product are often the customers generating the highest inference costs. In extreme cases, vendors can find themselves with their most engaged users contributing the least profit. The result is a growing realization across enterprise software that agent adoption and margin expansion are no longer automatically aligned.
Agent orchestration is the new moat
The technical responses are known and converging. They are not novel, but they are critical for survival
-
Cost-aware routing: This technique involves a small classifier model that decides which tier (Haiku, Sonnet, Opus equivalents) handles each query. Well-tuned routers cut inference bills by around 60% without any degradation in quality
-
Prompt caching: Anthropic, OpenAI, and Google now offer 75 to 90% discounts on cached prefixes.
-
Context discipline: You can truncate tool outputs, prune reasoning traces, and cap tool depth to prevent your agent from going down a rabbit hole
-
Speculative decoding: for self-hosted deployments, this technique guarantees 2 to 3X effective throughput on the same GPUs.
"Organizations using orchestration-led governance report stronger productivity gains — a holistic orchestration layer is associated with six times greater productivity impact than compliance‑only approaches" — IBM
The companies building this layer well are starting to look less like microservice operators and more like financial trading systems: Every routing decision priced, every path with its own P&L, every tenant on a metered budget.
What enterprise leaders should actually do
Four moves separate the companies that will still have margin in 24 months from the ones that won't:
-
Make inference cost a first-class metric. Track it per-feature, per-tenant, per-query class the same way cloud cost was tracked starting in the mid-2010s.
-
Budget like a media buyer. Set cost-per-thousand-queries ceilings per feature. Cap them. Alert on overruns. Engineering will not enforce this on its own.
-
Treat the router as core infrastructure, not an optimization. It is the new load balancer.
-
Audit prompts quarterly. A 4,000-token system prompt that grew organically over six months is a six-figure bill in slow motion. Most teams have never read their own production prompts end to end.
-
Negotiate volume commits early. Frontier-model vendors now offer reserved-instance-style prepaid commits at substantial discounts. List price is the worst price any enterprise will ever pay.
The next 24 months
The structural shift underneath agentic AI is not that it is expensive. As DeepSeek's price cut today underscores, frontier inference unit costs are dropping roughly 3X per year, and the curve is not slowing.
The shift is that amplification is outrunning the price cuts. Cutting per-token costs 75% does not help a company whose agents are doing 700X more tokens per user query than its pricing model assumed. For the first time since the cloud era began, architecture decisions are again financial decisions in real time. A prompt redesign is a margin event. A poorly bound agent loop is an outage with a credit card attached.
The companies that survive the next 24 months of AI infrastructure pricing will not be the ones running the cheapest model. They will be the ones whose agents are smart and know what they cost to think.
That is the 100X problem. And it is arriving faster than the price cuts can hide it.
Maitreyi Chatterjee is a senior software engineer at a big tech company.
Devansh Agarwal works as an ML engineer at a leading tech company.
