Per-token prices have collapsed since 2022, yet enterprise AI bills keep climbing. And it's the gap between the two where the next phase of the AI trade could be won or lost.
Anyone still telling their board that artificial intelligence is getting cheaper has half the story, because at present, the price of a single token - the unit of text a model reads or writes - has been in freefall.
McKinsey's Andreessen Horowitz reckons the cost of running a model of equivalent capability drops roughly tenfold every year.
Epoch AI, which tracks this more granularly, puts the decline anywhere between ninefold and 900-fold annually depending on the task, with a median around 50 times a year.
And yet the bills landing on chief financial officers' desks are getting bigger, not smaller?
That's because, in reality, cheaper tokens have not made companies spend less, but instead made them ask the tech to do vastly more, and the cost of finishing a job has climbed even as the cost of each token has fallen.
Microsoft chief exec Satya Nadella named this early last year, dusting off a 160-year-old piece of economics - the Jevons paradox, where efficiency gains drive consumption up rather than down.
The signal is not the token price but what a completed task costs, and that number has been climbing.
This is the argument at the centre of a July conversation published by McKinsey Quarterly with Seattle software firm Pay-i.
A necessary disclosure: Pay-i sells the exact measurement software the interview argues every enterprise now needs, so the thesis doubles as a sales pitch.
That does not make it wrong, but it means the numbers deserve their own scrutiny - and most of them survive it.
"Tokens are not value. Tokens are the bill," Pay-i chief executive and co-founder David Tepper said.
"The bill tells you what you spent. It does not tell you whether you should have spent it."
That gap between the invoice and the outcome is what has changed the conversation, along with who is starting it.
"Eighteen months ago, the call came from an engineering leader who wanted to instrument cost," he said.
“Today, it is from a CFO or a CIO, and they are not asking how to reduce AI spend. They are asking how to justify it.”
The question is no longer whether AI pays.
"Which of my 47 agents are worth it, and how would I know?" he said.
Two academic papers, neither of them Pay-i's, put numbers behind the claim.
A Stanford Digital Economy Lab and Microsoft Research paper published on arXiv in April.
Stanford argues that fully agentic coding tasks consume around 1,000 times more tokens than a comparable chat or reasoning session - and that the cost is driven by what the model reads, not what it writes.
A separate January study quantified where those tokens go, finding that the iterative checking and repairing stage - not the first draft - accounts for close to 60% of consumption.
Tepper reduces the deployment decision to a single inequality.
"An agent adds value when the probability of success is greater than the time it takes a human to verify the work divided by the time it takes a human to do the work themselves," Tepper said.
Run that on a task that takes two hours to complete but six minutes to check, and the agent needs to succeed just five times in 100 to come out ahead.
"For a large class of enterprise work, the inequality is far more forgiving than people assume," he said.
The catch is that the clean version only works when failure costs nothing.
A document-processing agent that gets it wrong produces a bad document you throw away.
"A customer service agent that tells a customer they can refund a nonrefundable trip has changed the environment," he said.
Recovery costs now stack on top of the original spend - the agency tax, verification plus rework, and the sum most firms have never done, use case by use case.
"The deeper implication is that the cost of a gen AI use case is no longer a number," he said.
"It is a distribution with likelihoods, expected values, and percentiles."
The real argument
Some of Pay-i's headline figures rest on its own telemetry and cannot be checked independently - the claim that spending crosses a capacity cliff around US$3 million a year with a single provider, or that agents string together three-and-a-half different models per task.
Treat those as one firm's read, not settled fact.
The bigger dispute is whether any of this adds up to a bubble.
"Beyond all of that, the AI bubble narrative will quietly die," he said.
“There is no bubble bursting here.”
Sequoia's David Cahn has pressed the question of the annual revenue needed to justify the hardware, JPMorgan's Michael Cembalest has tracked how narrowly returns are concentrated in a handful of AI-linked stocks, and MIT researchers found 95% of enterprise AI pilots producing no measurable profit impact.
The bulls have the cash flows.
Hyperscaler capital spending is guided at roughly $725 billion for 2026, Nvidia's data-centre revenue is running north of $190 billion a year, and enterprise generative AI spend leapt from $11.5 billion to $37 billion in twelve months.
Both camps are working from real data. The duller reading is that the returns are there but invisible, because nobody built the gauges to read them.
"The enterprises that mistake 'We cannot see the ROI' for 'There is no ROI' will look back on this period the way the laggards looked back on cloud in 2014," he said.
It is, conveniently, the position that sells the most software - yet it's also the one the evidence supports.
