Can your company move a production AI workflow to a different model in one week? If yes, you own your AI stack. If no, a vendor owns part of it. That matters because the model is the part of the stack that loses value fastest. The cost of a fixed level of AI capability falls about 13x per year. The parts that keep their value are specific to your company: identity, permissions, data access, audit, evals, and workflows. Own those. Rent the models.
The One-Week Model Swap Test
Ask your team to switch one production workflow to a different model, then answer seven questions:
- Can we change the model in configuration, without code changes in the application?
- Do we have an eval set of at least 100 real cases for this workflow, with pass or fail rules?
- Can we run the new model on that set and compare quality, latency, and cost in one day?
- Do user permissions and data rules stay the same after the switch?
- Do audit logs keep the same format, so compliance reports still work?
- Can we send 5% of live traffic to the new model and roll back in minutes?
- Do we know the cost per task, not only the cost per token, for both models?
Seven yes answers mean you own your control layer. Each no shows where a vendor owns part of it. Most companies fail questions 2 and 7 first: they have no eval set from their own work, and no cost per task.
Why the Test Matters: The Price of Capability Falls Fast
Epoch AI measured the cheapest cost to reach a fixed score on five benchmarks from 2023 to 2026. The cost fell about 47% per quarter on average. Compounded, that is about 13x per year.
| Benchmark | Average cost decline per quarter |
|---|---|
| Chess puzzles | 43.0% |
| Mystery game | 44.0% |
| GPQA Diamond (graduate science questions) | 47.0% |
| AIME (math competition) | 47.1% |
| FrontierMath, tiers 1-3 | 53.1% |
| Average | 47.0% |
No other technology Epoch compared has fallen this fast.
One example shows the scale. In January 2025, OpenAI's o3 was the first model in Epoch's data to score 75% on GPQA Diamond, at about $0.30 per question. By July 2026, GPT-5.6 Luna matched that score for about $0.0004 per question. Epoch calls it a 725-fold drop in under 18 months.
Two details matter for buyers. First, this is the cost to reach a score, not the price per token. A model can cost more per token and still be cheaper per correct answer. Second, the fall is fastest right after a level first becomes possible. Epoch measured about 66% per quarter in the first quarter after a new state-of-the-art level, slowing to about 32% per quarter two years later.
The Cheap Tier Is Good Enough for Most Tasks
The newest capability still sells at a premium, and for some tasks it is worth it. The premium is short-lived, though.
Epoch's capability index shows that open-weight models trailed the best closed models by about 4 months from January to May 2026. Under a stricter test, the gap was about 6 months. Epoch says the true gap is probably a bit larger, because closed labs hold back their best models and open models score lower on private benchmarks.
So the practical rule is simple. Use the cheapest model that passes your evals for each task. Pay for the frontier model only where your own tests show it changes the result. Re-run the test every quarter, because the answer changes.
Own, Rent, Keep Replaceable
Split the stack by how fast each part loses value.
| Layer | What it does | Decision |
|---|---|---|
| Identity and permissions | SSO, roles, and attribute rules per user and per agent | Own |
| Data access and tools | Connectors and tools scoped to each user's rights | Own |
| Audit and traces | Every prompt, tool call, write, and approval, kept per your retention rules | Own |
| Evals | Test cases from your own work, run on every model change | Own |
| Workflows, prompts, skills | How your company does the work | Own |
| Model gateway and routing | One API in front of all models, with budgets and fallbacks | Own the configuration |
| Inference | Running the model | Rent, from more than one provider |
| Models | Frontier and open-weight | Keep replaceable |
Everything marked "own" is specific to your company. A new model does not make it worse, and a vendor cannot give it to your competitor. Everything marked "rent" or "replaceable" gets cheaper each quarter. You want to capture that fall, not be locked out of it.
This is the buyer's control point, not a product category. Vendors will add governance features to their own platforms, and many are useful. The test is whether your controls still work the day you change vendors. Our post on open AI strategy covers routing, traces, and memory in more depth, and how to pick an agent platform without lock-in covers vendor selection.
The essay The Commanding Heights by Will Manidis and Joe Roberts makes the investor's version of this argument: the model commoditizes, and value goes to whoever controls the channel the work passes through. Inside a company, that channel is the path every AI request takes. It should belong to the company.
A Reference Path for Every AI Request
Every AI request in the company follows one path:
- The user or agent signs in through your identity provider. The request carries the user's identity, not a shared API key.
- A policy check runs. It decides which data, tools, and actions this user may reach. See permission-aware AI agents.
- Tools run with the user's rights. Connectors see only what the user may see.
- The gateway picks the model. Routing uses cost, capability, data rules, and budgets. See LLM routing in production.
- Every step is logged. Prompts, tool calls, outputs, approvals, and cost go to traces you keep. See AI agent audit trails.
- Evals score the output on a sample of live traffic, and on the full test set before any model change.
The same path works for ChatGPT Enterprise, Copilot, Claude, Gemini, or an open-weight model on your own hardware. The model sits at step 4 only. When a vendor retires a model, the change stays at step 4 too. Our model deprecation runbook covers that case.
Pass the Swap Test With OpenNash
OpenNash Work is the governed AI layer for your company. It works with the AI you already license, with any model provider, and with open-weight models on your own cloud. You own the code, the logs, and the configuration.
Our forward-deployed engineers do the work with your team:
- Connect identity. Every AI request carries the user's identity and rights.
- Connect your systems with read and write rights per user, and nothing more.
- Turn on audit. Every prompt, tool call, and write is logged and kept on your terms.
- Write evals. We build test sets from your own work, so each model change is measured.
- Route by cost. Each task goes to the cheapest model that passes its evals.
Book a call with an OpenNash forward-deployed engineer or email us at hello@opennash.com. In the first session, we run the swap test with you and review your top AI use cases and data. Then we show you which ones deliver the highest business value and the fastest ROI.