Optimization has an ordering, and getting it wrong is expensive in both directions. Reach for heavy machinery too early and you spend weeks tuning a model for a task a clearer prompt would have fixed; refuse to climb and you pay frontier rates forever for work a smaller model could carry. The discipline is a ladder where each rung is earned by the one below it.
Rung 1 — find the slice
Before anything is optimized, know what the work is. Capture every call, group calls into the conversations they belong to, and cluster conversations into named task families. Spend concentrates: on our own development estate the top three patterns account for 46% of spend — a dev-set number, ours, and not a market study, but the concentration itself is the point. You cannot optimize a workload you cannot name.
Rung 2 — prove it holds
For the fat-head slice, learn the skill from its own traces: the tools it takes, the judgment it needs, the format its output must satisfy. Then replay held-out work through the skill-and-smaller-model pair and score it against the frontier baseline. Cheap interventions come first here too — a tighter instruction or an output contract often moves the number before any route change does. But whatever the intervention, it earns its place against the same held-out eval, or it does not ship.
Rung 3 — own the weights
The top rung is training: distilling a proven slice into weights you serve yourself. We have not built it, and we say so plainly — today it is a roadmap entry, not a product. What rungs 1 and 2 hand it is exactly what training needs and nothing else produces: the corpus, grouped and cost-attributed; the slice worth the money; and an acceptance criterion the eval has already exercised. Training before those exist is how ML projects burn quarters.
Why the ordering is the product
Every rung shares one gate: a held-out eval and a human ratify. That is what makes the ladder safe to climb and safe to descend — demote is ungated, and the frontier model is always the failover. The ladder is not a metaphor for ambition. It is a rule about evidence: no rung is climbed until the one below it has produced the proof the next one runs on.