Last Tuesday, a client asked me to build an "AI agent" to route their support tickets. After digging into their actual requirements, we shipped a 47-line Python script with three if-statements. It runs in 200ms, costs nothing, and handles 94% of their volume perfectly.

The other 6%? That's where things get interesting.

This is the conversation I have weekly in 2026: someone wants an agent when they need a workflow, or they're wrestling with brittle automation when an agent would solve the problem in an afternoon. The line between these approaches isn't about technology - it's about understanding what kind of problem you actually have.

The Three Patterns You're Actually Choosing Between

Forget the marketing terms. In production, you're choosing between three distinct patterns:

Pattern 1: Linear Automation Input → Step A → Step B → Step C → Output

Every path is predetermined. If you can draw it as a flowchart with finite branches, this is what you have. Examples: data pipelines, scheduled reports, form submissions.

Pattern 2: Conditional Routing (Workflows) Input → Classifier → [Route A | Route B | Route C] → Output

An LLM or rule engine decides which path to take, but each path is still deterministic. The model makes one decision, then code handles the rest. Examples: support ticket triage, document classification, intent detection.

Pattern 3: Agentic Loops Input → [LLM decides action → Execute → Observe → Repeat until done] → Output

The model controls the entire execution flow, choosing tools and determining when the task is complete. This is Simon Willison's definition: "An LLM agent runs tools in a loop to achieve a goal."

Here's the decision matrix I use:

Question If Yes → If No →
Can I enumerate all possible paths? Linear/Routing Agent
Does path selection require language understanding? Routing/Agent Linear
Do later steps depend on earlier results in unpredictable ways? Agent Linear/Routing
Is the input format highly variable? Routing/Agent Linear
Do I need the system to recover from partial failures? Agent Linear/Routing

Most systems should be Pattern 1 or 2. Pattern 3 is the exception, not the default.

The Agent Tax: What Autonomy Actually Costs

Every time I see "just add an agent" as a solution, I think about the numbers from our last production deployment:

Deterministic workflow (ticket routing):

  • Latency: 180ms average
  • Cost: $0.002 per execution
  • Failure rate: 0.3%
  • Debugging time: 5 minutes (read the logs)

Agentic version (complex support resolution):

  • Latency: 4.2 seconds average
  • Cost: $0.08 per execution
  • Failure rate: 7%
  • Debugging time: 45 minutes (trace the reasoning chain)

That's 23x latency, 40x cost, and 23x failure rate. For the right problem, it's absolutely worth it. For the wrong problem, you've built an expensive, slow, unreliable system when a bash script would have worked.

Chip Huyen puts it well: "The journey from 0 to 60 is easy, whereas progressing from 60 to 100 becomes exceedingly challenging." The first demo is always impressive. The production system that handles edge cases without hallucinating is a different beast entirely.

The agent tax compounds in three ways:

1. Latency compounds with loop iterations Each tool call is a round trip. An agent that needs 5 tool calls to complete a task is inherently slower than a workflow that makes those same 5 calls in parallel with predetermined orchestration.

2. Cost compounds with reasoning tokens Claude and GPT-4 charge for thinking. An agent reasoning through "should I search the web or check the database first?" costs money that a hardcoded routing decision doesn't.

3. Errors compound through the loop A 95% accurate model making 5 sequential decisions has an overall accuracy of 77% (0.95^5). This is why Anthropic's agent guide emphasizes starting with the simplest solution.

When Agents Actually Make Sense

I'm not anti-agent. I'm anti-wrong-tool-for-the-job. Here are the scenarios where agentic patterns earn their cost:

The decision space is too large to enumerate

If your routing logic would require 500 if-statements to handle all the cases, you don't have a routing problem - you have a reasoning problem. An agent that understands intent and selects from a tool library is more maintainable than brittle conditional logic.

Example: A research assistant that can search the web, query internal databases, read documents, and synthesize findings. You can't predefine which tools to use in which order because it depends on what the user asks and what each intermediate result reveals.

Intermediate results change the plan

When the output of step 2 determines whether you need step 3a, 3b, or to go back and redo step 1, you need dynamic planning.

Example: Code debugging. The agent reads an error, hypothesizes a cause, checks the relevant file, finds it's not the issue, forms a new hypothesis, and continues until resolved. The path can't be predetermined because it depends on what the code actually contains.

Recovery requires judgment

When failures happen and the system needs to decide whether to retry, try an alternative, or escalate - and that decision requires understanding context - you need an agent.

Example: Customer service that can attempt self-resolution through multiple channels (check order status, process refund, schedule callback) and gracefully escalate when it's clear the issue needs human judgment.

The "How to Know" Test

Hamel Husain's evaluation framework offers a useful heuristic: if you can clearly define success criteria for individual steps but not for the overall task, an agent might be appropriate. If you can define overall success clearly, a workflow with checkpoints is probably better.

The Progression Path: Start Simple, Add Autonomy

Here's the pattern I recommend to every client:

Step 1: Build it as a deterministic workflow first

Even if you think you need an agent, start with hardcoded logic. This forces you to understand your problem deeply. You'll discover which parts are actually predictable (most of them) and which truly need dynamic reasoning.

Step 2: Identify the specific failure modes

Run your deterministic workflow against real data. Where does it fail? Is it:

  • Classification failures? The LLM misunderstands the input
  • Path coverage gaps? Legitimate cases your routing doesn't handle
  • Context requirements? Decisions that need information from earlier steps

Each failure mode has a different solution. Don't reach for agents until you know exactly what problem you're solving.

Step 3: Add autonomy only where needed

Often, you'll find that adding a single LLM decision point to your workflow - what LangChain calls the "Routing" pattern - solves 90% of your failures without the complexity of full agents.

The progression typically looks like:

Hardcoded → Rule-based routing → LLM classification → 
Conditional workflows → Orchestrator-workers → Full agents

Stop at the simplest level that meets your requirements. Most production systems should stop at "conditional workflows" - an LLM makes one routing decision, then deterministic code handles execution.

Step 4: Add guardrails before adding autonomy

The "lethal trifecta" for agent security is: access to private data + exposure to untrusted content + ability to exfiltrate. Before giving your agent more tools, ask whether you've created a system that an attacker can manipulate.

Practical guardrails:

  • Loop limits (no more than N iterations)
  • Cost caps (abort if cost exceeds $X)
  • Tool whitelists (agent can only use approved tools)
  • Output validation (check results before acting on them)
  • Human checkpoints (require approval for high-stakes actions)

The 2026 Reality Check

We're past the "pilot project" phase. Organizations shipping agentic systems in 2026 have learned some hard lessons:

Lesson 1: Observability is non-negotiable

You cannot debug an agent by reading code. You need traces of every decision, tool call, and intermediate result. If you can't reconstruct why your agent did what it did, you can't fix it when it breaks.

Lesson 2: Evaluation is harder than building

From Hamel's evals FAQ: "Generic metrics (BERTScore, ROUGE, cosine similarity) are NOT useful for most AI applications." You need task-specific evaluations that match your actual success criteria. This is where most agent projects fail - not in building, but in knowing whether they work.

Lesson 3: The "just add an agent" reflex is expensive

Every week I see projects that started as agent-first and had to be rebuilt as deterministic workflows when costs exploded or reliability tanked. The reverse - starting simple and adding complexity - is almost always cheaper.

Lesson 4: Production agents are mostly workflow with islands of agency

The most successful agentic systems I've seen aren't pure agents. They're deterministic orchestration with specific agentic components where dynamic reasoning is genuinely needed. Think of it as 80% workflow, 20% agent - not the other way around.

Anthropic's cookbook demonstrates this pattern: use prompt chaining and parallelization (deterministic) for the predictable parts, and reserve agentic loops for the genuinely uncertain decisions.

Making the Call

Here's my actual decision process when a client asks about automation:

  1. Can you write down the rules? If yes, start with deterministic automation.

  2. Do you need language understanding for routing? If yes, add an LLM classifier at the entry point. Still a workflow, just with smarter routing.

  3. Do decisions depend on intermediate results you can't predict? If yes, consider agentic patterns for that specific component.

  4. Is the cost/latency/reliability tradeoff acceptable? Run the numbers before committing.

  5. Do you have the observability and evaluation infrastructure? If no, build that first.

The answer is almost never "build an agent." It's usually "add intelligence at the specific point where hardcoded logic breaks down."

That support ticket system I mentioned at the start? The 94% that the if-statements handle stays deterministic. The 6% that needs judgment routes to a small agentic component with three tools: search knowledge base, check order status, and escalate to human. Tight scope, clear guardrails, specific purpose.

That's the pattern that works in 2026. Not agents everywhere. Not automation everywhere. The right tool for each part of the problem.