by Matty Smith
Those who think sending in AI agents should make things cheaper and more efficient may be in for some sticker shock. A new paper looks at token consumption patterns in agentic tasks, seeing agents more than willing to splurge.
“How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks”—whose authors include Lab Director Erik Brynjolfsson, Faculty Lead Sandy Pentland, and Postdoctoral Fellow Jiaxin Pei—finds agentic tasks to be “uniquely expensive, consuming 1000x more tokens than code reasoning and code chat.”
The high cost is in input tokens rather than output. In code reasoning, the user gives a task, the model reasons, and outputs an answer. But an agent reads the task, gets a response, then has to re-read everything (original prompt and response) before the next action, then re-reads all of that plus the new response before the next action… building one big pricey context snowball.
Making matters more difficult, the paper finds token usage to be extremely challenging to predict.
Agents are not capable of predicting their own token costs. This is the fundamental bottleneck for result-based pricing for agents. You can’t really price the agent well unless you can figure out the cost, but now you only see the token costs after everything is done.
The paper points to several reasons why agents can’t predict their own spending, including that the agent can’t know ahead of time how much context it will accumulate and that agent trajectories are “inherently stochastic.” Even when running the same agent on the same task, costs varied by up to 30x.
The paper also found that models consistently underestimate token spend. And humans aren’t much help, as task difficulty doesn’t always align.
“It’s like the classic Moravec’s Paradox,” Jiaxin said, shortly before we googled Moravec’s Paradox. “AI agents and human coders work in very different ways, so we end up seeing that some tasks simple for humans might be very hard for agents.”
Jiaxin feels the cost issues need to be solved at both model level and agent harness level, and luckily there are projects in the works at the Lab—which a lot of firms should find useful, as we imagine “kid Skee-ballin’ for the slap bracelet” isn’t a sound business strategy.
This feature originally appeared in our email newsletter, the DigDig.
Sign up to receive updates on everything going on at the Lab!
