Under $50 a Day: Cost Engineering for Production AI Workflows

Your AI agent demo was impressive. It researched prospects, drafted emails, and even scheduled follow-ups. The CEO loved it. Engineering high-fived. Then someone ran the numbers on what it would cost to handle 500 customers a day instead of 5, and the project quietly moved to the "revisit later" column.

This is the most common way AI agents die. Not from technical failure - from cost surprise. The POC ran on the best model with verbose prompts and no budget controls, and nobody did the multiplication until it was too late.

Running a production agent for $50 a day or less is entirely achievable. It requires the same discipline you'd apply to any other infrastructure cost.

What Agents Cost

Before you can optimize, you need to understand where the money goes. Every agent interaction has a cost anatomy:

Input tokens (your prompts, system instructions, context) + Output tokens (the model's response) = per-call cost. Multiply by the number of calls per task, tasks per day, and days per month.

Here's what common agent patterns cost at list API pricing (February 2026; Claude Sonnet 4.5 at $3/$15 and Claude Haiku 4.5 at $1/$5 per million input/output tokens, GPT-4o at $2.50/$10):

Notice the more-than-10x cost difference per task between data extraction on Haiku and research on GPT-4o. That gap is your optimization surface.

The numbers above assume everything works on the first try. In production, it won't. Failed tool calls, malformed outputs that need re-parsing, and agents that loop because they misunderstood the task - these all multiply your costs. A practical analysis from OpenAI's agent guide emphasizes that cost and latency tradeoffs should be designed in from the start, not bolted on later.

The Four Levers of Agent Cost Engineering

Lever 1: Model Routing (The Biggest Win)

Not every step in your agent's workflow needs the smartest model. A customer support agent might handle these steps:

1. Classify the ticket (billing? technical? cancellation?)
2. Retrieve relevant docs (search knowledge base)
3. Draft a response (write the reply)
4. Check for policy compliance (verify the reply is safe)

Steps 1, 2, and 4 are classification and retrieval tasks. They don't need Claude Sonnet or GPT-4o. A much cheaper model usually handles them fine.

A practical routing setup:

Tier 1 (cheap/fast): Classification, extraction, simple Q&A
  → Claude Haiku 4.5 ($1/$5 per MTok) or GPT-4o mini ($0.15/$0.60 per MTok)

Tier 2 (mid-range): Drafting, summarization, moderate reasoning
  → Claude Sonnet ($3/$15 per MTok) or GPT-4o ($2.50/$10 per MTok)

Tier 3 (expensive): Complex reasoning, creative writing, multi-step planning
  → Claude Opus 4.5 ($5/$25 per MTok) or o1 ($15/$60 per MTok) - use sparingly

The logic is simple: a cheap model making a routing decision adds pennies but saves dollars.

In practice, you implement this with a simple classifier at the start of your pipeline. The classifier itself runs on your cheapest model. If it's wrong 5% of the time, that's still cheaper than running every request through your most expensive model.

Lever 2: Prompt Caching (90% Input Savings)

Your system prompt is often one of your biggest token expenses, and you pay for it on every single API call.

A typical agent system prompt runs 2,000-4,000 tokens. If your agent makes 5 calls per task and handles 200 tasks a day, that's 1,000 API calls - and you're sending the same system prompt in every one. That's 2-4 million tokens per day just in repeated instructions.

Both Anthropic and OpenAI now offer prompt caching that stores your system prompt server-side and charges a fraction of the normal input rate for cache hits. Anthropic's implementation charges 90% less for cached tokens after the first request. OpenAI's approach is similar.

The implementation is straightforward:

# Instead of sending 3,000 tokens of system prompt every call,
# mark it as cacheable. First call pays full price,
# subsequent calls pay ~10% for the cached portion.

response = client.messages.create(
    model="claude-sonnet-4-5-20250929",
    system=[{
        "type": "text",
        "text": your_long_system_prompt,
        "cache_control": {"type": "ephemeral"}
    }],
    messages=messages
)

For a support agent making 1,000 calls/day with a 3,000-token system prompt, caching cuts the system prompt cost from roughly $9/day to $0.90/day at Sonnet rates. That's real money over a month.

Lever 3: Token Budgets and Circuit Breakers

The scariest cost scenario isn't a busy day - it's a stuck agent. When an agent enters a retry loop (tool returns an error, agent tries again with full context, fails again, tries again), costs compound fast. Each retry includes the entire conversation history, which grows with each attempt.

One attempt plus three retries doesn't cost 4x the first attempt. Because each retry sends all previous attempts as context, it costs much more:

That's a nearly 10x cost increase over the first attempt, versus 4x if each retry started fresh. Google's Site Reliability Engineering handbook describes this as cascading failure - the same pattern that takes down distributed systems applies to agent cost overruns.

The fix is straightforward:

1. Set a max token budget per agent run (e.g., 50,000 tokens total). When you hit it, stop and return a partial result.
2. Limit retry attempts (3 max, with exponential backoff).
3. Implement circuit breakers: if an agent fails N times in a row, stop trying and alert a human.

class AgentBudget:
    def __init__(self, max_tokens=50_000, max_retries=3):
        self.max_tokens = max_tokens
        self.max_retries = max_retries
        self.tokens_used = 0
        self.retries = 0

    def can_continue(self):
        return (self.tokens_used < self.max_tokens
                and self.retries < self.max_retries)

    def record_call(self, input_tokens, output_tokens):
        self.tokens_used += input_tokens + output_tokens

This isn't sophisticated engineering. It's the same pattern you'd use for any API with metered billing. But most agent frameworks don't include it by default, and most tutorials skip it entirely.

Lever 4: Batching and Async Processing

Not every agent task needs a real-time response. If you're processing customer feedback, enriching CRM records, or generating reports, you can batch requests and take advantage of lower pricing tiers.

OpenAI's Batch API offers 50% cost reduction for requests that can tolerate a 24-hour turnaround. Anthropic offers similar batch pricing through their Message Batches API. For workflows where latency doesn't matter - nightly data enrichment, weekly report generation, bulk email drafting - this is free money.

Even without formal batch APIs, you can reduce costs by:

• Combining similar requests: Instead of 10 separate "classify this ticket" calls, send one call with 10 tickets and ask the model to classify all of them. Most models handle this well up to about 20 items per batch.
• Pre-filtering: Don't send everything to the LLM. Use simple rules (regex, keyword matching, heuristics) to handle the easy 40% before the agent sees anything.
• Response length limits: Always set max_tokens. An agent drafting a support reply doesn't need 4,000 tokens of output. Cap it at 500 and you cut output costs by 80%.

A Worked Daily Budget

Here is a worked budget for a customer support agent handling 300 tickets per day. It assumes about half of each call's input is a cached system prompt billed at a 90% discount, and that failed calls cost about 1.5x a normal call to retry:

That's $4.64 per day, well under $50. You could run about ten agents like this within a $50 daily budget.

The research agent is more expensive. Let's budget one that handles 50 deep research tasks per day:

Combined: $8.49 per day for both agents. A $50/day budget leaves room for roughly 5x growth before you need to worry.

Three Costs Teams Overlook

API token costs are the obvious line item. But three other costs quietly eat your budget:

Tool call overhead. Every time your agent calls a tool (search API, database query, calculator), it generates a round-trip: the model outputs a tool call, you execute it, and you send the result back. That result goes into the context window for the next call. In tool-heavy agents, tool result context can take up a large share of the token budget rather than actual reasoning.

The fix: summarize tool results before injecting them. If your search API returns 5,000 tokens of results, condense to 500 before sending to the agent. You can use a cheap model for this summarization step.

Observability costs. You need logging and monitoring to debug production agents. Services like Langfuse (open source) or Braintrust let you track every call, but storing and querying traces has its own cost. Budget $20-50/month for observability tooling, or self-host Langfuse on your existing infrastructure.

The "just in case" model. Teams often keep a powerful model as a fallback "just in case" the cheaper one fails. This is fine as an architecture pattern, but watch the fallback rate. If 30% of requests escalate to your expensive model, your routing isn't working - it's just adding an extra cheap call before every expensive one. Track your routing hit rate weekly. Aim for 80%+ of requests resolved by the cheap tier.

What to Do Monday Morning

If you're running agents in production (or heading there), here's the priority order:

Week 1: Measure. You can't optimize what you don't track. Add token counting to every API call. Log input tokens, output tokens, model used, and whether the call succeeded. A spreadsheet works fine at this stage.

Week 2: Cache. Enable prompt caching on your highest-volume endpoints. This is typically a one-line change and delivers the best ROI of any optimization.

Week 3: Route. Identify which agent steps need your expensive model. Move classification, extraction, and simple validation to your cheapest model. Test quality to make sure it holds.

Week 4: Budget. Implement per-run token budgets and retry limits. Set up alerts for when individual runs exceed 2x the expected cost. This protects you from the runaway agent scenario that turns a $5 day into a $500 day.

The teams that get agent economics right treat LLM calls like database queries - something you monitor, optimize, and budget for. The teams that don't treat them like magic boxes and act surprised when the invoice arrives.

Your agent doesn't need to be cheap. It needs to be predictable. The $50/day budget isn't a constraint - it's a design requirement that forces you to build something that scales.