Do frontier labs have a sustainable business selling frontier inference?
Sam Altman doesn’t seem worried.
On the other hand, public market investors are worried that frontier labs will turn out to be a zero and succumb to demand for low-margin open-source models from China.
These concerns are in-line with recent observations from the market.
- Frontier intelligence demand is a function of Leading The Frontier, but labs are allocating less compute than ever on generating it through inaccessibly-expensive training runs.
- AI spend is falling despite rising token usage.
- @Silicon_Data: “The model layer seems to be going through a period of commodification as non-frontier models saturate everyday usage.”
- @coinbase “fine tune[d] an open weight model (Qwen3.5-9B) … and it out performed frontier models, at much lower cost and speed, for a specialized task.”
- @ibab at River AI: “Again and again we’ve seen that custom models designed for your own internal workflows outperform top frontier models on performance, at a fraction of the cost (1 percent or less)”
This blog has been suspicious of frontier lab economics for a while now: Frontier labs
- require the world to be meta-surprising to sustain frontier pricing
- either have high margins or don’t participate in the surplus they create
- are actually in the business of selling durable goods, and their best customers today are also their worst (i.e., least defensible) tomorrow
Given this recent market data, I wanted to try to formalize the intuition that labs selling tokens are actually in the business of selling durable goods. I used this claim to apply the Coase Conjecture to explain why open source may be temporarily helping labs retain pricing power and margins, to the confusion of Anthropic investors (can someone please help @coatuemgmt?).
Specifically, and contrary to Dario, the claim is that actually users of Anthropic models are in the business of “industrial-scale distillation” of Claude, but for the production of zero- or trivially-low marginal cost software or models. It’s true ‘distillation’ is usually used in the context of producing lower-cost open models, but from an economic standpoint whether frontier intelligence is used to produce software or models is irrelevant: We only care about costs and service level.
You could say businesses are solving a constrained optimization problem: minimize the total cost of building, validating, maintaining, and operating an intelligent system, subject to an acceptable service level.
Thinking through the scaled applications of intelligence today:
- AI customer service (Sierra, Decagon)
- transactional law (Harvey, Crosby)
- AI interviewers (Listen Labs)
- classification (Jev, Decision API, River, Coinbase)
it sure seems like a large class of these applications can be studied as stochastic optimal control problems: A business
- observes its situation
- chooses an action, and
- receives uncertain outcomes.
While inputs to the control problem may change––different customer requests, machine issues, or demand fluctuations––these surprises may not require any newly invented decision procedure. [This is why I posed that labs really need the world to have problems that have surprises about surprises, so that fixed control patterns break and as such would require continual frontier model usage.]
So, how much frontier intelligence that a firm buys today must also be bought tomorrow?
Indeed, in the course of operating firms accumulate experience and develop procedures that make production cheaper. AI enhances this effect, and potentially makes its own usage less necessary over time.
To formalize this, it helps to consider the economic history of this idea; namely that Firms Learn by Doing.
Firms learn by doing Link to heading
In 1962 Nobel-prize winning economist Kenneth Arrow challenged how economists account for economic progress. Prev growth models supposed improvements in productive knowledge were exogenous: technology and productive capacity improved, but models did not explain where that improvement came from. In his Economic Implications of Learning by Doing he proposed that economic activity itself generates experience, which improves productive knowledge and can be embodied in new capital goods. In short, Doing Things helps you Do Things Better, so technical progress becomes an outcome of production itself.
This is the lightcone techno-capital antecedant to Satya’s
This [learning] loop becomes the new IP of the firm. I think of it as a hill climbing machine. And unlike most assets, it compounds. Every improved workflow generates better training signal, which accelerates the accumulation of tacit knowledge unique to the firm
My estimation is that the core hill climbing exercise of the firm is not necessarily (or even primarily for most firms!) in the development of distinctive IP that would be competitive with other methods–-most firms do not observe distinct economic phenomena strictly necessary for distinct IP–but rather in the minimization of COGS, which very likely requires training signal of the firm.
To see whether firms could build low-cost controller capital, I ran a series of simulations.
Frontier models build their software replacements Link to heading
The entirety of the simulation results are presented in my explorations (written entirely by GPT-6 Astra) and summarized with the help of GPT-6 Astra here. This is the only section with entirely AI-written prose.
My initial simulations demonstrated that while a task can be difficult for a model to solve repeatedly in prose, it can be straightforward for a program that the same model writes. In small inventory problems, even inexpensive models built reusable programs that matched an exact optimizer on the tested cases. This demonstrated inexpensive control, but didn’t resolve the more interesting question around cases where frontier intelligence is initially necessary to solve problems.
I then moved to a harder setting: running a business with 6 production units over 60 days. Each day, the controller allocates effort between production and maintenance. Production completes more work but wears down equipment. Maintenance consumes resources and its benefits can arrive after a delay. Units that share resources can affect one another. Finally, demand fluctuates, and the abstract controller––either AI or software––receives imperfect measurements of equipment condition. By construction, doing the most productive thing today can make tomorrow’s production worse.
The business experiments used GPT-6 Astra as the frontier model and GPT-6 Luna as the inexpensive comparator, both with medium reasoning through OpenRouter. The main study covered twelve business environments, with two independent learning attempts per business, and compared five learning approaches. It involved more than 8,000 operating simulations, each designed to cover sixty days of production and maintenance decisions. These supplied learning experience, tested proposed procedures, and evaluated completed controllers; unsuccessful attempts were retained. The separate final comparison tested four frontier-developed programs across two businesses in 192 fresh operating simulations—11,520 simulated business-days. The experiments therefore involved substantial repeated learning and testing, although the limited variety of businesses—particularly the two in the final comparison—restricts how broadly we can generalize.
The controller knows its objective and available actions, but it is not given the underlying mechanism: how effort translates into output, how wear accumulates, or how maintenance changes future productivity. It must work from observations and experience, which it can preserve in code, memory, and procedures. Crucially, while the business faces new circumstances every day, its experience can reveal lasting relationships that may be used to operate the business in the future.
In these businesses, frontier models were better at developing operating procedures than inexpensive models in the observed sample. Giving the inexpensive models more experience helped, and frontier-generated lessons sometimes helped further, but did not establish reliable catch-up. The more interesting finding was what the frontier models produced: controllers that could allocate production and maintenance effort without consulting a model every day.
To examine that possibility directly, I began a separate experiment with frontier models making the daily decisions, then allowed them to develop reusable procedures from operating experience. All four resulting systems eventually ran entirely as software. Demand still fluctuated, equipment still wore down, and maintenance still required judgment about future consequences. What disappeared was the need to purchase model inference to respond to those circumstances. The programs could vary their actions as conditions changed without rewriting their underlying decision procedure.
On fresh operating scenarios, every program achieved higher average performance than its corresponding frontier model making daily decisions. The frontier operator cost roughly $5 in model charges per simulated sixty-day business run; the software required only ordinary computation. The software performed better in our trials, but our small sample and conservative statistical test could not rule out that it performs meaningfully worse on average across further runs of the same businesses. Additionally, the experiment did not establish that equally useful software could not have been built before accumulating experience. But it demonstrated the proposed mechanism: intelligence used to develop a business’s operating procedures can become capital that continues working after the model stops being called.
So who owns the controller? Link to heading
Frontier models help businesses turn experience into inexpensive execution capital. Today’s even outcomes-pricing app-layer stands in the way of this compression. As firms move operations to AI, they effectively operationalize Kenneth Arrow’s 1962 insight: operating provides value directly––through the provision of service to customers––and through the accumulation of experience that makes operating tomorrow cheaper.
Of course, this exercise operates entirely in the domain of well-defined control problems. There are certainly domains where the principal exercise is not in optimizing well-formed control but rather discovering the paradigm or structure of the problem itself––@sebkrier’s recent treatment of this is relevant. Even if there were a healthy expanding universe of new tasks, such existence doesn’t tell us the rate at which these new tasks succumb to simpler cost-efficient operation. This is likely why Anthropic is entering drug discovery: where it can capture returns to frontier intelligence itself without concern for distillation. Vertical integration is a strong strategy of the application layer as well: if customers can use your output to produce a cheaper controller, stop selling AI output and begin selling commerce unlocked, e.g., as posed by @PossibilityResult at Variant.
Sadly, the simulations studied here do not show how broadly this dynamic extends. Berkeley’s Ben Recht––with his focus on a surprising effectiveness of simple controllers––may offer a path here. Nevertheless, what this simulation work does show is that a business can use frontier intelligence to distill a capability without purchasing frontier intelligence each time it exercises that capability.
The frontier may well continue to find valuable work––even as its customers become better at making each particular job its last.