An Operating Model for Enterprise AI: From One Workflow to a Repeatable System

A new Director of AI needs an operating system that can keep shipping useful automation after today's winning model changes.

Grok, Muse, GLM, MiniMax, Kimi, Qwen, gpt-oss, and the major closed model families are improving on different release calendars. In bounded workflows such as classification, extraction, routing, and structured drafting, several open-weight models now warrant evaluation alongside closed models. Inference providers make that comparison possible without requiring an internal GPU platform. A company can test with a frontier closed model, validate alternatives against the same eval set, and move repeatable production volume to the lowest-cost approved route that clears the bar.

The operating model has five connected parts: an inference portfolio, embedded subject-matter expertise, a continuous improvement loop, an approved coding-agent layer, and ROI governance. Each part should make the next production workflow easier to ship than the first.

1. Build an Inference Portfolio Before You Build Agents

Start with the company's existing cloud boundary. If the business is already concentrated in AWS, Azure, or Google Cloud, begin with Amazon Bedrock, Microsoft Foundry, or Vertex AI. That path usually reduces the number of new identity, networking, billing, and security decisions required for the first governed workload.

It does not remove the need for diligence. Amazon Bedrock documents explicit data-retention modes, including a zero-retention mode that blocks models requiring retention. Microsoft says Foundry direct models are stateless and do not use prompts or completions to train base models, but some features store application state and abuse-monitoring terms vary. Google documents training restrictions and the configuration details required for zero data retention on Vertex AI.

The procurement rule is simple: "not used for training" is not the same as "not retained." Verify the exact endpoint, feature, region, monitoring mode, contract, and logging path. For healthcare or financial data, retention is only one row in the control matrix. Identity, network isolation, encryption, residency, auditability, subcontractors, and a BAA where required also belong there.

Then add at least one independent inference partner. Fireworks, Together AI, and Baseten give the team faster access to open models, dedicated deployments, and different price-performance options. Keep one independent route so a single hyperscaler never becomes the company's only place to test or serve intelligence.

Your internal model catalog should record:

• Provider, model version, license, and deployment type.
• Approved data classes, regions, and retention terms.
• Context limits, tool support, latency, and rate limits.
• Input, cached-input, output, and serving cost.
• Workflow eval score and cost per accepted task.
• Fallback route, owner, and rollback procedure.

What the Price Spread Looks Like at Enterprise Volume

Use a common workload to make the procurement decision concrete. The table below assumes 1 billion input tokens and 100 million output tokens per month. Prices were checked on July 17, 2026. The calculation uses public standard API rates and excludes cache discounts, batch discounts, tools, retries, infrastructure, and human review.

OpenAI publishes the GPT-5.6 rates on its API pricing page, xAI lists Grok 4.5 at $2 input and $6 output per million tokens, and the Together AI pricing catalog lists the open-model routes. Fireworks also publishes GLM-5.2 at $1.40 input and $4.40 output.

The listed price spread from GPT-5.6 Sol to gpt-oss-120B is about 38x on this token shape. Capability, latency, retries, serving configuration, and review burden can change the production result. If a lower-cost model passes one high-volume workflow, avoided inference spend may offset part of the evaluation and routing investment. Keep the frontier route wherever the cheaper option fails the workflow's acceptance tests.

This is why our enterprise model-routing strategy measures cost per accepted task instead of treating token price as the answer.

2. Embed Builders With Subject-Matter Experts

Agent construction is easier than it was two years ago. The basic loop is familiar: give a model instructions and tools, let it act, inspect the result, and repeat until the task finishes or a stop condition fires.

The hard part is teaching the system how the company actually works.

A finance agent needs to know why one invoice exception is harmless and another requires the controller. A healthcare agent needs the real escalation path, not a generic summary of policy. A support agent needs to know which source wins when the help center, CRM note, and current incident banner disagree.

Do not ask subject-matter experts to "provide requirements" and return three weeks later with a demo. Attach a builder to the operating team. Watch the best operator handle real cases. Capture the decision points, source systems, exceptions, approvals, and evidence required for acceptance.

An invoice exception workflow shows what this produces. The operator explains which mismatches can be cleared, which require the controller, and what evidence must appear in the case record. The builder captures those decisions in a version-controlled SKILL.md, adds worked examples from normal and adversarial cases, and defines tool permissions and idempotency rules. The finance team then owns the acceptance criteria and eval cases with the AI team. The resulting skill, tools, tests, and operating history remain useful when the underlying model changes.

This is also the forward-deployed engineering model OpenNash uses. Our engineers work beside operators long enough to understand the job, then encode that knowledge into software the client owns.

3. Make Production Traces Feed a Continuous Improvement Loop

No agent is finished at launch. Inputs change, policies change, upstream systems change, and employees find uses the original team did not anticipate.

The minimum production loop is:

1. Trace every model call, retrieval, tool invocation, approval, retry, latency event, and final outcome.
2. Run deterministic checks on all traffic and model-based or human evaluation on a risk-weighted sample.
3. Route low-confidence, high-value, or policy-sensitive cases to a reviewer.
4. Convert failures and surprising successes into labeled offline eval cases.
5. Test prompt, tool, model, and skill changes against the growing regression set.
6. Release through a canary and watch quality, cost, latency, escalation, and business metrics.

This pattern is supported by several mature toolchains. LangSmith describes the exact online-to-offline loop: production traces surface failures, failures become dataset examples, and offline experiments validate the fix. Langfuse connects tracing, datasets, experiments, evaluation, and human feedback, while its CLI exposes traces, prompts, datasets, scores, and other API resources to coding agents. Arize Phoenix provides tracing, evaluations, prompt iteration, datasets, and experiments on OpenTelemetry and OpenInference.

Choose one based on deployment, retention, integration, and operator needs. Traces provide observability; their value compounds when production evidence becomes regression tests and tested releases.

Our guide to production evals for agentic systems goes deeper on the evaluation layer.

4. Give the Team an Approved Coding Agent

Coding agents compress the distance between an operational finding and a tested change. A team can inspect traces, query an evaluation platform through its CLI, add a regression case, patch a prompt or tool, run the suite, and prepare a reviewable change in one working session.

Codex, Claude Code, or a comparable coding agent should be governed as part of the AI platform. OpenAI's Codex use cases include understanding large codebases, adding evals, running migrations, reviewing changes, and saving repeated workflows as skills. Those are the mechanics the transformation team needs.

Procure the tool through an approved business or enterprise surface. OpenAI states that business data is not used to train its models by default and that qualifying API organizations can request zero data retention. Those are separate controls, and the exact Codex surface still needs security review. Apply the same test to every coding-agent vendor: training policy, retention, repository access, shell permissions, network access, secrets, audit logs, identity, and administrator controls.

Then give the agent a harness: repository instructions, scoped credentials, tests, linters, evaluation commands, sandbox rules, and human review before production. Our coding harness guide explains why this structure matters more than raw autonomy.

5. Run AI as an ROI Portfolio

Leadership funds programs that produce measurable operating results. Give the steering group a clear reason to keep funding this one.

Before building a workflow, write down the baseline:

For each workflow, report four numbers together: accepted tasks, business value created, full operating cost, and incident or escalation rate. Full cost includes model calls, inference infrastructure, tools, review labor, retries, monitoring, and support.

These measures expose technically impressive agents that never change the operation. They also discourage promises about headcount reduction before anyone has measured the work. Better early targets are capacity returned, cycle time reduced, backlog absorbed, errors prevented, and hiring pressure avoided. Finance can verify those results, and operators can see how they were produced.

The Director of AI should review the portfolio monthly with finance and business-unit owners. Double down on workflows that create accepted value. Repair those with strong demand but weak reliability. Stop funding work that cannot name an owner, baseline, and acceptance rule.

The First Deployment: Three Decision Gates

Do not launch five disconnected pilots. Build one thin shared platform around one valuable workflow, then reuse it.

Discovery gate: map cloud and data constraints, approve candidate inference routes, and interview the leaders and operators closest to high-friction work. Discovery is complete when one workflow has a business owner, a measured baseline, an acceptance rule, and a documented data boundary.

Shadow gate: embed a builder with the operators, write the skill and tool contracts, assemble the first eval set, instrument traces, and run on real cases without taking production action. Move forward when the workflow clears its quality, security, escalation, and cost thresholds.

Production gate: launch with approval controls, run a weekly edge-case review, compare at least one open and one closed model, and publish the first ROI report. Expand only after the business owner accepts the results and the team can name which platform components will transfer to workflow two.

At day 90, the deliverable is not a demo. It is one operating workflow plus an inference, knowledge, evaluation, delivery, and governance system that makes the second workflow cheaper and faster to ship.

How OpenNash Can Help

OpenNash is a U.S.-based forward-deployed engineering team. This operating model reflects our work with enterprise teams on inference strategy, sensitive-data controls, business-unit discovery, agent delivery, evals, observability, and operating handoff.

We embed with the people doing the work, prove one workflow, and leave the client with the code, skill files, eval datasets, traces, runbooks, and ownership model required to keep improving it.

If you are a Director or VP of AI building this operating system, book a 30-minute working session or email [email protected]. Bring one workflow, its monthly volume, the current approval constraints, and the business definition of accepted work. We will map the production boundary, acceptance metric, and largest delivery risk with you.