If you’re shipping an LLM-powered product this quarter, the headline isn’t GPT-6.1 Sol — it’s that your eval pipeline is about to break and your cost math is probably wrong. OpenAI’s DevDay 2026 announcements, a quiet rate-limit change from Atlassian, and a hard deadline on OpenAI’s Evals platform all point to the same shift: the industry is moving from demo-grade AI features to production-grade AI systems, and the tooling gap between the two is where most teams will get stuck in the next 90 days.
Why it matters
OpenAI used DevDay 2026 to announce GPT-6.1 Sol, computer-use tooling for the Agents API, cloud-based Codex environments, and a new Decisions API. Read together, these are less about model intelligence and more about reducing the glue code teams write to turn a model into a working agent. That’s genuinely useful — but it also means more of your product’s logic now lives inside a vendor’s orchestration layer, which raises the stakes on evaluation, cost tracking, and failure handling. At the same time, OpenAI is shutting down its own Evals platform (read-only October 31, 2026; fully gone November 30, 2026), which tells you even OpenAI doesn’t think a vendor-hosted eval tool is where this should live long-term.
What actually changed
- Agent tooling got more native. Computer-use support in the Agents API and a dedicated Decisions API mean fewer custom integrations for multi-step agent workflows — but also less visibility into what’s happening between steps.
- Atlassian 10x’d its Forge LLM throughput, raising rate limits from 50,000 TPM to 500,000 TPM. That’s a concrete signal that platform vendors are now building for production-volume AI traffic, not pilot-scale usage.
- OpenAI’s Evals platform is being sunset, with a one-month read-only window before full shutdown. Teams using it for regression testing need a migration plan now, not in October.
- Pricing keeps dropping, but not evenly. A cited snapshot for one new model tier shows $2 per million input tokens, $0.10 per million cached input tokens, and $10 per million output tokens — a 20x spread between cached and output pricing that changes how you should architect prompts.
| Signal | Number | What it means for shipping |
|---|---|---|
| Atlassian Forge rate limit | 50,000 → 500,000 TPM | Production-volume agent apps are now viable on Forge without hitting throttling |
| OpenAI Evals shutdown | Read-only Oct 31, 2026 / off Nov 30, 2026 | Move regression evals into your own CI now |
| Sample model pricing | $2/M input, $0.10/M cached, $10/M output | Cache aggressively; output tokens are the real cost driver |
What this means if you ship product this quarter
Treat evals as infrastructure, not a side project. With OpenAI’s Evals platform shutting down by late November, any team relying on it for regression testing needs a replacement in their own CI pipeline before the read-only window hits. Waiting until October 31 to start is too late — exporting test sets and rebuilding the scoring logic takes longer than people expect.
Stop pricing your feature by token cost alone. The $2 / $0.10 / $10 spread in the cited pricing snapshot shows cached input tokens cost 20x less than output tokens. That means your real lever isn’t picking a cheaper model — it’s redesigning prompts to maximize cache hits and minimize verbose output. A feature that looks affordable at the token-price level can still blow your budget if it generates long completions or retries on failure.
Budget for retries and tool-call failures, not just API calls. As agent workflows get more native (computer use, Decisions API), end-to-end task cost — including retries, failed tool calls, and fallback paths — becomes the number that actually matters, not the headline per-token price. Instrument this before launch, not after a surprise bill.
Check your rate limits against real usage, not pilot usage. Atlassian’s jump from 50,000 to 500,000 TPM on Forge is a reminder that platform-level throughput limits can quietly cap your product’s growth. If you’re building on any vendor platform, confirm your production-volume ceiling now, before a launch spike finds it for you.
The honest counter-argument
None of this is confirmed as universally true across every vendor or workload — rate limits, pricing tiers, and eval migration paths vary by platform and contract, and some teams will find their current setup is already fine. The risk of over-reacting is real: rebuilding eval infrastructure or re-architecting prompts for cache efficiency takes engineering time that might be better spent elsewhere if your current volume is low. The practical move is to audit, not panic — check your actual usage against the new limits and deadlines above, and act only where the numbers say you need to.