Once model spend can fund a meaningful share of an engineer’s salary, I treat it as a product and hiring decision rather than a cloud-cost problem. The choice depends on which requests require the better model, what weaker output costs the team, and whether an engineer can remove enough paid inference to justify the hire.
Consider Kojo, an invented composite of founders I have worked alongside. At 10:40 p.m. in Accra, he was holding a cold cup of coffee and comparing two spreadsheets on his laptop. One showed the latest AI invoice. The other showed how many months remained before his company would need either new revenue or new funding.
His team had planned to hire an engineer to own integrations and reliability. That month’s model usage had risen high enough to consume most of the money reserved for the role.
Kojo had three possible calls. Move every request to a cheaper model and accept weaker answers. Keep the better model and delay the hire. Or hire now, hoping the engineer could redesign the product before the runway narrowed further.
The wrong choice could leave him with an unreliable product or no one to repair it.
Separate valuable output from expensive habit
The first mistake would have been comparing models by price per token alone.
Kojo’s product drafted procurement recommendations from messy supplier documents. A cheaper model could handle extraction, classification and routine formatting. It struggled when two suppliers used different units, when a contract contradicted an invoice, or when the source material left a key field unclear.
Those failures rarely appeared in a polished demo. They appeared when a customer relied on the answer.
That distinction changed the spreadsheet. Kojo stopped asking, “Which model should run the product?” He asked, “Which decisions deserve the expensive model?”
He reviewed a week of requests and divided them by consequence. Low-risk transformations went to the cheaper model. Ambiguous comparisons, recommendations and exceptions stayed with the stronger one. A small set required human approval, much like the decision in why Kwame shipped an AI agent with human approval.
This gave him a routing policy tied to the work. Price still mattered, but only after he knew what a weak answer could cost.
The 2026 AI Engineering Survey’s emphasis on cost-sensitive, multi-model usage fits what founders are already discovering in practice: one model rarely deserves every request. The useful unit of comparison is the cost of completing a task to the required standard.
Price the correction loop
A cheap answer becomes expensive when a person must inspect it, rewrite it and explain it to a customer.
Kojo sampled failed outputs from both models. For each one, he recorded what happened next. Did the system retry? Did a team member review it? Did the customer reopen the task? Did the failure create support work?
The cheaper model won on routine requests. On ambiguous supplier comparisons, its apparent saving disappeared after retries and manual correction.
There was another trap. The strongest model sometimes produced confident language when the evidence was incomplete. Better output did not remove the need for product controls. Kojo still needed citations, uncertainty handling and a clear approval step where the consequence justified one.
This is where model evaluation becomes product work. A founder needs a small test set drawn from real failure modes, then a score that reflects the business consequence. “The answer sounded good” cannot carry a hiring decision.
I would track four numbers before changing the team plan: successful task cost, retry rate, minutes of human correction and the proportion of requests that genuinely need the stronger model. Those numbers reveal whether model spend reflects customer value or a product architecture that sends every task through the most expensive path.
Decide what the engineer would change
Hiring becomes rational when the engineer has a defined mechanism for reducing cost or increasing reliable output.
“Improve the AI stack” is too vague. Kojo wrote down the actual work: build routing, cache repeated context, reduce oversized prompts, create evaluations from production failures, and add deterministic code around calculations the model should never improvise.
Then he estimated the effect conservatively. If the hire only monitored invoices and swapped model names, the role would not pay for itself. If the engineer could reduce repeat calls, keep cheap tasks on cheap models and prevent costly customer-facing errors, the calculation changed.
The timing still mattered. Hiring introduces salary, onboarding and management load before the improvements arrive. When runway excludes a role the product appears to need, the roadmap has to absorb that truth. I explored that tension in what happens when nine months of runway excludes a critical hire.
With less than a week before he needed to decide, Kojo paused the full-time offer. He contracted a senior engineer for a tightly scoped routing and evaluation sprint, kept the stronger model on high-consequence requests, and moved routine transformations to the cheaper one.
That bought evidence, not comfort.
Make the next invoice explain itself
The following month, Kojo could point to each class of model spend and say why it existed. Routine work had a cost ceiling. Ambiguous work had a quality threshold. High-consequence recommendations had an approval step. The proposed engineering role had a list of changes it would own.
He still had an unresolved decision. If usage continued growing, a full-time hire could become necessary. But the next choice would rest on observed task economics instead of one frightening invoice.
Before approving your next model bill, take the ten most expensive request types and assign each one a consequence, a minimum acceptable output and a fallback. Then write the engineer’s first three changes beside them.
If those changes cannot be named, the invoice has exposed a product decision that the job description has not yet earned.
Comments
No comments yet.