The Bitter Lesson for AI Agents: What to Build Into a Harness and What to Plan to Delete

Rich Sutton's essay "The Bitter Lesson" says that general methods which scale with computation beat methods built on human knowledge of a domain. For AI agents, this means that harness parts which patch a weakness of today's model will expire, often within one model generation. Parts that become more useful as models improve, such as evals, tool design, and context engineering, should get most of your engineering time.

What the Bitter Lesson Says

Sutton published The Bitter Lesson on March 13, 2019. It looks back at 70 years of AI research and finds the same pattern many times:

1. Researchers build their knowledge of the domain into their systems.
2. In Sutton's words, "this always helps in the short term, and is personally satisfying to the researcher."
3. Then "in the long run it plateaus and even inhibits further progress."
4. Finally, a different approach that scales computation through search and learning makes the breakthrough.

Sutton names the two methods that keep scaling. He writes: "The two methods that seem to scale arbitrarily in this way are search and learning". His advice: "we should build in only the meta-methods that can find and capture this arbitrary complexity". A meta-method is a general way to learn or search. It is not a rule about the domain.

Sutton does not say that human knowledge and general methods must conflict. He writes: "These two need not run counter to each other, but in practice they tend to". The rest of this article keeps that point in view.

The Cycle From Chess to Agent Harnesses

The table below puts Sutton's examples next to two recent examples from agent engineering. Each row shows the hand-built approach, the general method that replaced it, and the source.

The time scale is different. Go needed search-based success "a further 20 years" after chess. The agent examples changed within months. Calling the agent cases the same lesson is our interpretation. Sutton did not write about agents, and neither company makes this claim.

The 2012 ImageNet paper reports top-5 error of 18.9% on the 2010 competition test data. That result came from learned features, not hand-built ones. Sutton's point is not that the hand-built methods were bad. They were the best available until computation made the general method practical.

How the Cycle Shows Up in Agent Harnesses

A harness is the code around a model that runs the agent loop, provides tools, and manages context. Scaffolding is the part of the harness that makes up for a specific limit of the current model. For a step-by-step build, see our agent harness guide.

Anthropic: context resets came and went

Anthropic's harness design post describes a three-agent harness with a planner, a generator, and an evaluator. It defines three terms that matter here:

• Context anxiety: models "begin wrapping up work prematurely as they approach what they believe is their context limit."
• Compaction: earlier parts of the conversation "are summarized in place so the same agent can keep going on a shortened history."
• Context reset: "clearing the context window entirely and starting a fresh agent," with a structured handoff of state and next steps.

With Claude Sonnet 4.5, compaction alone was not enough, "so context resets became essential to the harness design." Resets had a cost. They add "orchestration complexity, token overhead, and latency to each harness run". When Opus 4.5 arrived, the author wrote: "I was able to drop context resets from this harness entirely". For more about the remaining technique, see our post on context compaction for agents.

The same post says that prompt engineering and harness design both raised performance well above baseline. Then it says, "but both eventually hit ceilings". That sentence is the Bitter Lesson at the scale of one project.

Cognition: two rules that changed

Cognition's 2025 post Don't Build Multi-Agents gives the first example. It says: "In 2024, many models were really bad at editing code". Teams used a separate "edit apply model" to apply changes. By 2025, "the edit decision-making and applying are more often done by a single model in one action."

The second example is the multi-agent rule itself. The 2025 post told builders to "share full agent traces, not just individual messages". It warned that "conflicting decisions carry bad results". About 10 months later, Cognition published Multi-Agents That Work. Cognition gives this reason: "models have become way more naturally 'agentic'". It adds that models "intuitively understand tool use, their own context limits, and how to distill their context for collaborators".

This was a narrowing, not a reversal. Cognition writes: "Our original observations still hold today for parallel-writer swarms". The new position covers patterns where "writes stay single-threaded". One piece of advice did flip. In 2026, Cognition says review works best "when the coding and review agents do not share any context beforehand". In 2025, it told builders to share full traces. Cognition reports that its review agent "catches an average of 2 bugs per PR, of which roughly 58% are severe." This is a company self-report.

The Lesson Is Not "Build No Harness"

A short version of the Bitter Lesson says "wait for the next model and write no scaffolding." Anthropic's own numbers do not support that version.

The full harness cost $200 ÷ $9 = 22.2 times more and took 360 ÷ 20 = 18 times longer. Anthropic describes it as "over 20x more expensive." The extra cost bought a much better result. The harness did not lose to the model. Only one part of it, the context reset, expired.

Anthropic's earlier guide, Building Effective Agents, gives the rule for adding parts: "you should consider adding complexity only when it demonstrably improves outcomes". It also gives an order of work. First, "start with simple prompts, optimize them with comprehensive evaluation". Then "add multi-step agentic systems only when simpler solutions fall short". This rule works in both directions. Add a part when an eval shows it helps. Remove it when an eval shows it no longer helps. Our post on harness A/B testing shows how to separate the effect of a scaffold change from the effect of a model change.

Build vs. Plan to Delete: Sorting Your Agent Stack

The table below is OpenNash's synthesis of the sources. No source gives this table as a list. The "permissions and data access" row is our editorial judgment, based on Anthropic's advice about guardrails and sandboxes.

Context engineering means choosing what information the model sees, and in what form, at each step. Cognition contrasts it with prompt engineering: "Context engineering is more durable." Our context engineering guide covers the methods.

One tool example shows why tool design lasts. Anthropic's SWE-bench agent made mistakes with relative file paths. The team required absolute paths in the tool. After that, "the model used this method flawlessly". The fix improved the interface. It did not depend on a model weakness. For a related problem, see why more MCP tools can make agents worse.

Workflows are not on the delete list. Building Effective Agents defines workflows as systems "orchestrated through predefined code paths". It says they give "predictability and consistency for well-defined tasks". A fixed path that exists because the business process is fixed is durable. A fixed path that exists only because the model could not plan is compensating.

How to Sort Your Own Stack

Use this procedure on an existing agent before the next model upgrade.

1. List every component in the harness: prompts, tools, agents, retries, filters, and context steps.
2. For each component, write down the model weakness it patches, if any.
3. Mark components with a named weakness as "compensating." Mark the rest as "durable."
4. For each compensating component, write one eval that measures the weakness directly.
5. When a new model ships, run that eval with the component removed.
6. If the eval passes without the component, delete it.
7. If the eval fails, keep the component and run the test again on the next model.
8. Spend new engineering time on durable components first.

A small registry in code keeps this list honest. The example below is illustrative. Replace the eval names with your own.

from dataclasses import dataclass

@dataclass
class HarnessPart:
    name: str
    kind: str              # "durable" or "compensating"
    compensates_for: str   # model weakness; empty if durable
    removal_eval: str      # eval that must pass without this part

PARTS = [
    HarnessPart("eval_suite", "durable", "", ""),
    HarnessPart("tool_schemas", "durable", "", ""),
    HarnessPart("context_reset", "compensating",
                "stops early near context limit", "long_task_completion"),
    HarnessPart("edit_apply_model", "compensating",
                "unreliable code edits", "patch_apply_rate"),
    HarnessPart("persona_prompt", "compensating",
                "unclear task framing", "task_accuracy"),
]

def removal_candidates(parts, passed_without_part):
    """Return compensating parts whose removal eval passed on the new model."""
    return [
        p.name for p in parts
        if p.kind == "compensating"
        and passed_without_part.get(p.removal_eval, False)
    ]

# Results from running each removal eval with the part disabled
print(removal_candidates(PARTS, {"long_task_completion": True,
                                 "patch_apply_rate": False}))
# ['context_reset']

This list also helps with model changes that you do not choose. Our model deprecation runbook uses the same evals when a provider retires a model.

When This Advice Is the Wrong Choice

The Bitter Lesson describes long-run trends. Some situations make it a poor guide for the next quarter.

• You must ship now. If a compensating part makes the product work today, build it. Sutton agrees that human knowledge "always helps in the short term." Make it easy to remove.
• Your model is pinned. If you run a fixed open-weight model or a small specialized model, upgrades do not arrive on their own. Compensating parts can last for years.
• The fixed path is a business rule. Approval gates, audit steps, and compliance checks are not model patches. Keep them deterministic. See keep deterministic steps deterministic.
• You have no evals. Without an eval, you cannot tell whether a new model removed a weakness. Deleting scaffolding without evidence is a guess. Build the eval first.

Make Your Agent Stack Get Better With Each Model Release

The right harness design lets each new model reduce your cost and improve your results, without a rebuild. OpenNash puts forward-deployed engineers on your team and works with Fortune 2000 companies to build agents that work this way.

1. Map the workflow and the systems of record, and get the data ready.
2. Pick the right model for each task, and record which harness parts patch which model limits.
3. Build evals first, including a removal eval for each compensating part.
4. Ship to production with audit trails, permissions, and cost controls.
5. Train your team to run the upgrade procedure when the next model ships.

At the end, you own the agent code, the evals, the harness registry, and the runbooks.

Book a call with an OpenNash forward-deployed engineer or email us at hello@opennash.com. In the first session, we review your top AI use cases and data. We show which ones give the highest business value and the fastest ROI.