The Most Expensive AI Model Is the One You Didn’t Need
There is a growing conversation about the cost of artificial intelligence. Tokens cost money. GPUs cost money. AI subscriptions cost money. Enterprise licensing costs money. As organizations put AI into more business processes, those costs scale surprisingly quickly.
All of that is true, and I think most of the conversation is happening at the wrong layer. Before asking what an AI model costs, we should ask two simpler questions: Did this task need an AI model at all, and if it did, did it need that model?
That distinction matters more as AI moves out of experimentation and into enterprise architecture.
Start With Whether the Answer Is Already Knowable
An employee asks an AI assistant, “How much PTO do I have left?”
One way to answer it is to send the question, the employee’s profile, the relevant HR policies, and assorted context to a frontier reasoning model and ask it to work out the answer. Another is to recognize the intent, check that the employee is authorized to see the value, retrieve the balance from the system of record, and return it.
The first approach feels more like AI. The second is the better architecture, and cost is the least interesting reason why.
The second approach doesn’t have to infer the answer. A PTO balance is not a judgment call. It is a value that already exists in a system designed to be authoritative about it. That source data can still be wrong, stale, or poorly governed, but routing it through a language model doesn’t make it more authoritative. It introduces another opportunity for a confident, plausible, incorrect answer to a question that has a verifiable answer.
In an HR context, that isn’t just a budget problem. It can become a compliance problem. And in adoption terms, it may be worse than either. Employees extend trust to a new system slowly and withdraw it all at once. One wrong PTO balance can cost more than a year of frontier tokens.
Now consider a different question: “Based on our purchasing history, current inventory, outstanding orders, and supplier performance, where are we most exposed to supply-chain risk next quarter?”
Nothing in any system of record holds that answer. We are asking something to reason across sources, identify relationships, weigh uncertainty, and explain its conclusions. That is exactly where a capable reasoning model can earn its cost several times over.
The architectural mistake is treating both questions as the same kind of problem because the user happened to type both of them into the same box.
This is one of the principles shaping how we’re designing Ori, the intelligence layer we’re building for the Origo platform. I don’t want Ori’s architecture organized around a particular model. I want it organized around the problem being solved.
A business request enters Ori, where a lightweight routing layer determines intent and evaluates what it knows about the request. From there, policy and confidence determine whether the request should follow a deterministic path, use a smaller model, or escalate to a more capable reasoning model.
Sometimes the right choice is a frontier model. Sometimes it’s a small, inexpensive one. Sometimes it’s an open-source model running on infrastructure we control. And sometimes it’s no additional language model at all.
The Router Is the Hard Part
What that diagram hides is the hardest part of this architecture: the router is itself a classification problem.
Understanding intent well enough to route a request usually costs a model call of its own. It can be a small, fast, inexpensive model, but any honest accounting needs to put that cost above the fan-out rather than pretending the decision about which model to use happens for free.
More importantly, the router’s two failure modes aren’t symmetric.
Sending an easy question to an expensive reasoning model wastes money. Sending a difficult question to a model that isn’t capable of handling it can produce something much worse: an answer that is wrong in a way nothing downstream necessarily detects.
One failure shows up on an invoice. The other can show up in a business decision someone already made.
That changes how I think the router should be designed. A router optimized primarily for cost is backwards. The safer posture is to bias toward escalation and take the savings only where the system has sufficient confidence. If a request maps to a known intent, the necessary information comes from an authoritative source, the permission model is unambiguous, and the system knows how to complete the transaction reliably, use the cheaper path. When those conditions aren’t met, escalate.
That approach produces less impressive savings in a spreadsheet. It’s also the version I’d be willing to put in front of an actual business process.
This reframes what the router is for. Its primary job isn’t to save money. Its job is to determine how a question should be answered and to recognize when it isn’t sufficiently confident in that decision. The savings become a byproduct of getting that judgment right rather than the objective itself.
From the user’s perspective, none of this should be visible. They asked Ori a question and received an answer. Behind that interaction, however, the system may have made several decisions about data, permissions, tools, models, compute, and confidence before producing it.
And that creates a metric I think will matter much more than raw token consumption.
I think of it as the reasoning rate: the percentage of requests that genuinely require generative reasoning rather than deterministic retrieval, business logic, or a known workflow.
I could put an estimate here for what percentage of Ori’s future requests I expect to be deterministic lookups rather than reasoning problems. It would also be made up. We haven’t collected the production telemetry yet. What matters architecturally is that we collect it, because that ratio will determine far more about inference economics than the price printed on any model provider’s website.
If 1,000 requests enter the system and 600 can be resolved reliably through authoritative data and deterministic logic, sending all 1,000 through frontier reasoning means 600 expensive inference calls didn’t need to happen. If only 100 can be resolved that way, the economics look very different.
Either way, now we have something worth measuring.
Alongside reasoning rate, I also want to understand the escalation rate: how often the system begins on a less expensive path and determines that it isn’t sufficiently confident to continue there. Over time, those measurements tell us much more than how many tokens we bought. They tell us whether the architecture is actually making good decisions about where intelligence belongs.
Model-Agnostic by Design, and What That Costs
AI models are improving quickly. The best model for a workload today probably won’t be the best one six months from now. Pricing changes, context windows grow, open-source models improve, and specialized models appear.
An application tightly coupled to one provider turns those changes into architectural problems. Put the model behind an orchestration boundary and it becomes a replaceable component instead. But abstraction isn’t free. Orchestration costs engineering time. Every model you support needs to be evaluated. Different providers implement tool calling, structured output, context management, caching, and other capabilities differently.
The counterargument to the title of this article is a good one: the most expensive architecture is the abstraction layer you didn’t need yet. Genuine model-agnosticism can easily become a lowest-common-denominator architecture that avoids some of the best capabilities of the models you’re paying to use. The answer isn’t to abstract everything simply for the sake of abstraction.
The trade is worth making at the boundary, not everywhere.
Ori shouldn’t care whether intelligence ultimately comes from a commercial API, a hosted open-source model, or something that hasn’t been released yet. Its business logic should care about what the user is trying to accomplish, what data is required, what the user is authorized to see, how much reasoning the problem needs, and what levels of accuracy and latency are acceptable.
Inside a particular implementation path, using a provider’s best capabilities is perfectly reasonable. That’s an implementation detail we can afford to rewrite.
What we can’t afford is an application whose business logic assumes a particular vendor’s pricing page.
Once that separation exists, the economic question becomes much easier to ask: What is the least expensive reliable way to accomplish this particular task?
That’s cost management as an architectural discipline rather than a procurement exercise.
A Free Model Is Not Free Inference
Open source introduces its own misconception. If the model is free, the AI must be nearly free.
It isn’t.
Someone still has to run it. GPUs cost money whether they’re serving requests or sitting idle, and that idle cost is the part many comparisons conveniently omit. Infrastructure has to be provisioned, monitored, patched, secured, and maintained. Models have to be deployed and updated. Capacity has to account for peaks rather than just averages.
That means the honest comparison isn’t simply one per-token price against another. It’s the cost of usage-based inference compared with the fully loaded cost of infrastructure capable of handling your peak demand while still being paid for during your quietest hours.
For sporadic usage, paying an API provider for actual consumption can be dramatically cheaper than owning inference capacity. At sufficient scale and predictable utilization, that equation can reverse.
The crossover point is a real number. It is specific to an organization’s usage curve, model requirements, latency expectations, infrastructure costs, and operational overhead. It should be calculated rather than assumed.
That’s another reason I don’t think the infrastructure decision should be permanent. An organization might begin with usage-based inference because the math says so. As workloads become predictable, routine tasks can move to smaller and less expensive models. Eventually, sufficient utilization may justify hosting certain open-source models internally while continuing to pay for frontier models where their capabilities provide enough business value to warrant the expense.
The important thing isn’t picking one of those strategies today. It’s building an architecture that allows you to pick a different one tomorrow.
The Objective Isn’t the Smartest Answer
If every business interaction becomes a frontier-model call, cheaper tokens only postpone the bill. If every employee needs another AI license for every application, the problem moves into licensing. If every workload lands on dedicated GPUs regardless of utilization, it moves into infrastructure.
The better opportunity is to build systems that know when intelligence is required, how much of it the problem deserves, and when they aren’t sure. That means separating the application from the model, treating models as computational resources rather than the center of the architecture, and combining traditional software engineering, governed enterprise data, APIs, automation, retrieval, and AI instead of assuming AI should replace all of them.
There will be business problems where spending more on intelligence is unquestionably the correct decision. If a reasoning model can identify a multimillion-dollar project risk, uncover a purchasing pattern that humans missed, or help an organization make a materially better decision, arguing about a few cents or dollars of inference is missing the point.
There will also be thousands of questions where the answer already exists in a database.
Good architecture should know the difference.
Sometimes the smartest use of AI is knowing when not to use AI.
The future of enterprise AI probably won’t belong to whoever uses the biggest model the most. It will belong to the organizations that learn to spend expensive intelligence only where the problem actually deserves it.