Artificial Analysis co-founders Micah Hill-Smith and George Cameron use their Intelligence Index and new AA Briefcase agentic benchmark to unpack a paradox: token prices for a fixed level of intelligence keep falling 5-10x per year, yet single agentic knowledge-work tasks now cost $20 or more. The driver is that agents run hundreds of tool calls over messy, realistic environments, and the vast majority of what you pay for is input tokens.
Micah opened by skipping the usual justification for why intelligence-and-cost trade-offs matter — "I would be shocked if I needed to convince anyone in this room why the cost of intelligence is an important topic" in mid-2026. Artificial Analysis is an AI benchmarking company that tests everything in the stack developers and companies care about: chips, cloud infrastructure, models, and agents, measuring how smart they are, how fast, and how much they cost. They publish that data publicly and work directly with labs including OpenAI, Google, and Nvidia.
The core framing is the intelligence frontier — what today's smartest models can reliably do. Most tasks we foreseeably want AI to do are still beyond it, which is why so much attention chases the latest frontier models. But the set of tasks inside the frontier grows every month, and for those tasks the intelligence-cost trade-off is decisive: by not using the smartest model for everything, "you can spend 10, 100, a thousand times less to get the same work done by the AI."
Artificial Analysis publishes the Intelligence Index, a synthesis across nine different evals — now at version 4.1 — mixing agentic tasks and hard reasoning Q&A. They pitch it as the best one number for the AI race, while noting that if one number were enough they wouldn't publish the rest of the website. At the time of the talk, Claude Fable 5 sat on top, with a "not currently available" tag Micah joked they could now remove.
Plotting the smartest model per lab over the last few years shows OpenAI and Anthropic trading blows, with release dots clustering tighter on the right as the pace accelerates. Isolating the best overall model versus the best open-weights model reveals a consistent 3-to-9-month gap that has held for three years — neither the "open weights have caught up" nor the "further than ever" headlines are true. The implication: within nine months of Mythos being announced, someone is predicted to give away an open model just as smart.
Token prices "have continued to fall by 5 to 10x every year for each fixed level of intelligence" — plotted as bands of 10 intelligence-index points on a log y-axis, each line drops fast. Micah stressed that a 10-point gap is decisive: it is incredibly hard to find any task where the dumber model outperforms. Yet even as per-token cost at the frontier stays consistent, cost per task is climbing.
Breaking it out: GPQA Diamond, a reasoning eval where models don't act as agents, is largely solved and costs from fractions of a cent up to ~50 cents per answer. But the coding agent index and the new AA Briefcase eval push beyond $20 for a single task, with the most expensive Briefcase task several times that. Notably, Claude Sonnet 5 burns an enormous number of tokens, making it nearly as expensive as Claude Fable 5 on Briefcase. The frontier keeps moving, so we can ask harder things — and spend enormously more per task — even as cost per token keeps falling.
George took over to explain why AI feels more expensive than ever. AA Briefcase is their new agentic knowledge-work benchmark: four private scenarios, each representing weeks of human-equivalent professional work, graded on rubric correctness, analytical quality, and presentation. To stay realistic, models aren't handed clean inputs — they must dig through thousands of files, messy Excel, unstructured and structured documents, hundred-page reports, emails, and Slack messages. On a commercial due-diligence example, GPT-4o produced a basic slide, o3 a few bullet points, while Opus 4.8 and Fable 5 went far deeper in rigor and presentation.
The four cost drivers are token price, number of turns in the agent trajectory, token usage/efficiency, and prompt caching — the last named potentially most important. Token prices span two orders of magnitude between frontier models like Claude Fable 5 and usable workhorses like DeepSeek V4 Flash and GPT-OSS-120B. Agents now make hundreds of tool calls, and Claude Sonnet 5 (released the day before) used over 200,000 output tokens per task. But the total-token breakdown is almost entirely input tokens, so the real lever is the input-token cache-hit price: cache discounts run ~90% typically, from ~80% to 99% across models and providers, and that swing changes total task cost by multiples. "Let's start with the cache-hit price when thinking about the cost of an agentic task."
George closed with what he called the most important chart for understanding the AI landscape in 2026: intelligence versus cost per task. In 2025 the defining chart was a simple Intelligence Index bar chart; now the field is wrestling with trade-offs. He offered two archetypes. The first is a task with no ceiling on intelligence — more intelligent means better output — which he argued covers most professional knowledge work (strategy analysis, cost-saving analysis, even writing a job description). For these, you follow the Pareto line and decide how much you'll pay for extra intelligence.
The second archetype has a ceiling — a factual question like "how much did I spend on Stripe fees last month," where a smarter model gives no better answer. There you find the minimum level of intelligence that completes the task and choose the cheapest model that clears it. He wrapped up with a recruiting pitch: "We're Artificial Analysis. We're hiring."
The vast majority of the things that we foreseeably want AI to do, the models are still far too dumb to do.
By choosing to not use the smartest model for every single thing, you can spend 10, 100, a thousand times less to get the same work done by the AI.
Token prices have continued to fall by 5 to 10x every year for each fixed level of intelligence.
The vast majority of tokens to complete longrunning agentic tasks are input tokens.
| Time | Topic |
|---|---|
| 00:00 | Intro: Artificial Analysis and the cost of intelligence |
| 03:09 | Intelligence Index v4.1; Claude Fable 5 on top |
| 04:12 | Open-weights vs frontier: a consistent 3-9 month gap |
| 05:45 | Token prices fall 5-10x/year at fixed intelligence |
| 07:18 | Cost per task rising; GPQA Diamond to $20+ Briefcase tasks |
| 08:51 | George: the AA Briefcase agentic knowledge-work benchmark |
| 12:36 | Four cost drivers; input tokens dominate; prompt caching |
| 17:52 | The 2026 chart: intelligence vs cost and two task archetypes |