When should you use a small language model instead of a frontier model in production?
Default to a frontier model while you're figuring out what 'good' looks like — its broad capability lets you prototype fast without fighting the model. Move a task to a smaller model once it's well-scoped and high-volume, because that's where cost and latency dominate and a small model tuned to one job can match frontier quality at a fraction of the price. The frontier stays the right call for open-ended reasoning, low-volume work, and requirements that are still moving. The decision isn't 'which model is smartest' — it's which model is the cheapest, fastest way to clear the quality bar your specific task actually needs.
Model ArchitectureEnterprise AI
Prototype on the frontier
A frontier model — the largest, most capable general models — is the right default when you don’t yet know what your task needs. Its headroom means you can get to a working prototype without fighting the model’s limits, and it tells you what the ceiling on quality looks like before you optimize for anything else. At low volume, the per-call price is noise, so there’s no reason to trade capability for savings you won’t notice.
When a small model wins
The economics flip once a task is well-scoped and runs at high volume. A smaller model — often open-weight and fine-tuned to one job — can hit the same quality bar as a frontier model on that narrow task while costing far less per call and answering faster. The clearest version of this on the show: Intercom cut roughly $250K a month by moving a high-volume support workload off a frontier model onto a smaller open model. When a single task dominates your bill and its “right answer” is stable enough to tune against, that’s the signal to shrink.
When the frontier still wins
Reach back for the frontier when the work is open-ended reasoning, genuinely varied, or still changing shape. A small model tuned to yesterday’s task is brittle when requirements move, and it can’t improvise the way a large general model can. Low-volume, high-stakes decisions also favor the frontier, because the cost argument for shrinking never kicks in and the capability margin is cheap insurance. This is the two-extremes reality of the current stack: broad foundations at one end, specialized efficiency at the other, and most production systems end up using both.
The migration path
The practical pattern is prototype-then-shrink: build on a frontier model, use it to define what “good enough” means with real evaluations, and only then distill or fine-tune down to a smaller model once the task is stable and the volume justifies the engineering. Do it in the wrong order — committing to a small model before you know the target — and you optimize for a bar you haven’t set yet. The frontier is how you discover the requirement; a small model is how you serve it cheaply once you have.
From the conversation
This explainer is drawn from these episodes — each carries its full transcript.
