The model is no longer the product decision — the system around it is. In the last 48 hours, GitHub Copilot Auto has reportedly added the ability for developers to bias routing toward cost, quality, or a middle ground, and OpenAI’s builder-facing updates keep pointing toward infrastructure you call directly rather than model demos you admire. Put together, these are not feature announcements. They’re a signal that platform providers are handing cost, latency, and reliability tradeoffs to the people actually shipping product — and most teams aren’t set up to make those calls well yet.
Why it matters
For most of 2023-2025, the default builder move was: pick the best model you can afford, wrap a prompt around it, ship. That era is ending. With routing controls now sitting at the developer layer and token prices still swinging wildly across providers, the choice of which model runs on which request is becoming a live, per-request decision — not a one-time architecture pick. That raises the ceiling on what a good team can do, and it raises the floor for what a sloppy team gets exposed for: silent quality regressions, margin leakage, or both.
What actually changed
- Routing is now a dial, not a default. Copilot Auto reportedly lets teams choose whether requests optimize for cost, quality, or a balance — a control previously left to the model provider or hidden behind a fixed model tier.
- OpenAI is shipping infrastructure, not just intelligence. Recent builder-facing API updates emphasize calling production infra directly, suggesting more of the ChatGPT-style stack (not just the model) is becoming available to third-party products.
- Price dispersion is still extreme. One pricing tracker puts frontier LLM token costs at an index of 16 as of September 18, 2026 — down 84% from the March 2023 baseline of 100. But a separate 2026 pricing roundup finds LLM API costs spanning more than 600x across major models, from $0.05 to $30 per million input tokens. The cheapest blended option tracked comes in around $0.010 per 1M tokens; other major APIs sit far above that.
- Evals remain the bottleneck, not the model. Builder commentary keeps flagging the same gap: teams that under-invest in evaluation can swap in a cheaper or faster model and have no way of knowing reliability quietly dropped until users complain.
What this means if you ship product this quarter
If your roadmap touches an LLM feature between now and end of quarter, three decisions just moved from “nice to have” to “do this before launch.”
- Treat routing as a product spec, not an ops config. If your platform (Copilot-style tools, OpenAI’s API, or your own gateway) lets you bias toward cost or quality, that bias should be tied to a business decision — a free-tier user gets the cheap path, a paying enterprise customer gets the quality path — not left as a default someone set once and forgot.
- Build evals before you build routing. You cannot safely route between cheap and expensive models, or swap providers to chase the 600x price spread, without a baseline that tells you when quality drops. At minimum: a small labeled test set per use case, run automatically on every model or prompt change, with a clear pass/fail threshold before it ships.
- Model tiering and caching are now first-order architecture, not cleanup work. With per-token costs ranging from roughly $0.05 to $30 per million input tokens depending on the model, the difference between a naive single-model integration and a tiered system (cheap model first, escalate on uncertainty, cache repeated queries) can be the difference between a feature that scales and one that quietly burns margin.
- Don’t assume falling average prices mean your bill is falling. The frontier pricing index dropping 84% since 2023 is a real trend, but it describes the frontier, not your specific stack. If you’re on a premium model for correctness-critical tasks, that trendline may not apply to you at all — check your own usage data, not the index.
The honest tradeoff
More control is also more surface area for mistakes. Teams that lack eval discipline will now have more knobs to misconfigure — a cost-biased routing setting can silently degrade output quality in ways that don’t show up until a customer complaint or a support ticket spike. The counter-argument to “more control is good” is simple: control without measurement is just a new way to break things faster. If you adopt routing controls this quarter, adopt evals in the same sprint, not the next one.
Bottom line
The API layer is maturing faster than most teams’ internal practices. Model choice used to be the hard part; now it’s system design — routing, caching, tiering, and evals — that separates products that scale profitably from ones that quietly erode trust or margin. If you ship AI features this quarter, budget time for evals and routing logic with the same seriousness you’d give the model integration itself.