How to Reduce AI Agent Cost With Plan Caching

An invoice-reconciliation agent does not need to rediscover accounting from first principles for every invoice. The vendor, amount, and purchase order change, but the work often follows the same shape: retrieve the records, compare them, apply tolerance rules, request missing approval, and write the result. Paying a large model to invent that sequence thousands of times is an expensive habit.

Caching the previous answer would be reckless because yesterday's vendor and amounts are wrong today. Caching the validated procedure can be useful. That is the core idea behind plan caching: remember a proven way of working, bind it to current data and policy, and fall back to fresh planning when the fit is weak.

The opportunity is real, but "50 percent cheaper" is not a property of the technique. It is a measured result under particular workloads. Enterprise teams should treat plan reuse as a systems optimization with acceptance tests, invalidation rules, and unit economics.

Three Kinds of Caching Solve Three Different Problems

Teams use the word caching for several mechanisms that behave differently.

Prompt caching reuses model-side computation for an identical prefix. Stable system instructions, long reference documents, examples, and tool schemas stay at the beginning of the request. Variable user content comes afterward. The model still produces a fresh response, but the provider can process the repeated prefix faster or at a lower price.

Google Cloud's context-caching guide describes reusing large amounts of repeated prompt context, including documents and extensive system instructions. Anthropic's prompt-caching documentation similarly lets developers mark reusable prompt prefixes and reports cache creation and read tokens. This is an inference optimization, not workflow memory.

Semantic caching matches a new query to an earlier query and may return the stored answer. It works when similarity implies that the answer remains valid. "What is the return window?" and "How long do I have to return an item?" may share an answer if policy has not changed. "What is the balance on account 4831?" must not reuse the answer for a similar sentence about account 4832.

Plan caching stores a structured procedure and adapts it to the new request. The agent still fetches current data and acts in the current environment. It avoids paying for the most expensive planning work when the task belongs to a known class.

The Agentic Plan Caching paper reported average cost reductions of 50.31 percent across five workloads while retaining 96.61 percent of the performance of the accuracy-optimal baseline. Its reported 27.28 percent latency reduction came from a separate 100-query FinanceBench microbenchmark with a 46 percent cache-hit rate. In a FinanceBench accuracy comparison, the plan approach reached 85.5 percent versus 72 percent for a full-history cache, while also costing less. Filtering a trace down to a concise procedure mattered more than dumping the whole previous run into context.

That result should shape the experiment, not the sales forecast. Cache matching is a precision problem: in the paper's FinanceBench test, relaxing the similarity threshold raised the hit rate from 46 to 64 percent but reduced accuracy from 85.5 to 77 percent. Measure false plan matches, not just hits. The average cache overhead was 1.04 percent, so a low-hit workload may add machinery without creating meaningful savings.

A Reusable Plan Is a Typed Procedure, Not a Transcript

Raw agent traces are poor cache entries. They contain customer names, record IDs, dates, failed attempts, temporary errors, model chatter, and tool outputs that should not cross into another request. A reusable plan keeps invariant structure and replaces request-specific values with typed slots.

A trace for an invoice exception might contain:

1. Search for invoice INV-8842 from Acme Parts.
2. Retrieve purchase order PO-1093.
3. Compare the invoiced quantity and total with the receipt.
4. Apply the current tolerance policy.
5. Ask controller Mia Chen to approve the variance.
6. Update the case and notify the requester.

The reusable template should look more like this:

Names and IDs are not embedded. Tool calls are typed. Each step has a completion assertion. The plan refers to current policy rather than copying yesterday's threshold. This makes reuse safer and easier to test.

Store metadata beside each plan:

• Task class and a plain-language scope statement.
• Required and optional inputs.
• Tool names and schema versions.
• Policy, prompt, and model versions used during validation.
• Tenant and data-class restrictions.
• Preconditions and completion assertions.
• Cost, latency, and accepted-task result from prior runs.
• Known failure modes and the date of last review.

Do not make one global plan library for every customer or business unit. Plans can reveal process knowledge, and their tool bindings can encode sensitive structure. Apply the same tenant separation and retention rules used for workflow data.

The useful unit is not "a similar prompt." It is a task class with compatible inputs, tools, policies, and success criteria.

Build the Plan-Reuse Loop in Seven Steps

A safe implementation has seven stages.

1. Capture successful executions. Instrument model calls, tool calls, approvals, retries, errors, final state, latency, and cost. Label success using a business acceptance rule. The agent saying "done" is not evidence of completion.

2. Extract the invariant plan. Remove entity values, transient results, and failed branches. Keep the ordered operations, dependencies, required inputs, policy lookups, and completion assertions. Review the first plan set with the operators who perform the work.

3. Index by task class. Use workflow labels, structured features, keywords, or embeddings to retrieve candidate plans. Similarity is only candidate generation. It is not permission to execute.

4. Check compatibility. Verify tenant, tool versions, policy version, required inputs, data sensitivity, and action scope. A plan built for a read-only support workflow cannot be promoted into a refund workflow because the tickets look alike.

5. Adapt with current context. Bind current record IDs and user scope. A smaller model may adapt the template, but deterministic code should handle identifiers, authorization, limits, and policy calculations.

6. Validate before execution. Confirm that every tool exists, every required input is available, and each proposed action falls inside the user's permissions. Route high-impact writes through approval. If validation fails, invoke the normal planner.

7. Learn from the result. Record whether the plan was accepted, edited, abandoned, or rolled back. Promote plans after repeated success. Demote them after incompatible matches or policy changes.

This loop should have a full-planning fallback by design. A cache miss is not an incident. It is the correct outcome for unfamiliar work.

Do not hide plan selection inside one opaque model call. Emit a trace showing the matched task class, candidate plan, similarity score, compatibility checks, adaptations, and reason for reuse or fallback. That trace becomes the evidence for debugging and cost attribution.

Invalidation and Guardrails Decide Whether Savings Are Real

Every cache eventually becomes stale. Agent plans are especially sensitive because they refer to changing tools and business rules.

Invalidate or revalidate a plan when:

• A tool name, parameter, response schema, or side effect changes.
• A policy, approval threshold, or compliance rule changes.
• The model or system instructions change enough to alter execution behavior.
• A credential, role mapping, tenant boundary, or data classification changes.
• The authoritative source becomes stale or unavailable.
• Completion criteria change.
• The plan causes repeated edits, failures, or escalations.

Version dependencies explicitly. A plan with policy_version: AP-2026-04 and tool_schema: invoices-v3 is inspectable. A plan labeled "invoice flow" is a future incident.

Put hard constraints around the adapted plan. Set maximum tool calls, retries, records affected, financial value, external recipients, and runtime. Require approval when an action is hard to reverse. The controls from our CRM writeback production guide apply here too: validate, authorize, execute with idempotency, verify downstream state, and preserve rollback evidence.

Plan caches also create poisoning risk. If the system promotes any apparently successful execution, a malicious or merely lucky trace can influence future work. Promotion should require trusted outcome evidence, scoped provenance, and enough repeated successes for the action's risk. High-impact plans need human review before entering the reusable library.

Data retention deserves a procurement check. Provider-side prompt caching, application-side semantic caching, and a company-owned plan store have different storage behavior. For example, OpenAI's API data-control table notes that extended prompt caching stores key-value tensors as application state and is not eligible for Zero Data Retention. The point is not that one caching method is unsafe. It is that "cached" must trigger questions about what is stored, where, for how long, under which tenant key, and who can delete it.

Measure Cost per Accepted Workflow

Token savings can look impressive while business cost rises. A reused plan that needs more correction, retries, or review is not cheaper.

Create a baseline before adding the cache:

Then measure the cache itself:

• Candidate retrieval rate: how often at least one plan is found.
• Valid cache hit rate: how often a compatible plan is reused.
• Hit precision: how often reused plans belong to the correct task class.
• Adaptation success: how often current inputs are bound without full replanning.
• Regression rate: accepted-workflow loss compared with the baseline.
• Net savings: model, tool, storage, review, and failure cost avoided.

The denominator should be accepted workflows, not requests. Use:

cost per accepted workflow = total model + tool + cache + review + retry cost / accepted workflows

Run an A/B or interleaved test on a held-out set of real cases. Compare full planning with plan reuse under the same model, tools, and policies. Segment results by task class because a high-volume easy flow can hide failures in a smaller, higher-risk flow.

The FinOps Foundation's unit-economics guidance recommends connecting technology cost to business value through a unit metric. For agents, accepted invoice exceptions, resolved tickets, completed account briefs, or approved reviews are better units than tokens.

The cheapest planning call is not always the best target. First remove loops, duplicate retrieval, oversized tool results, and unnecessary context. Use provider prompt caching for stable prefixes. Route simple substeps to smaller models. Then add plan reuse where repeated planning remains a material share of cost. Our model-routing strategy shows how to qualify cheaper routes against workflow evals rather than model reputation. If the cost comes from agents considering too many irrelevant capabilities, start with the tool-selection playbook before caching plans.

Choose the Right First Workflow

Good candidates have high volume, repeated structure, current data, measurable completion, and an expensive planning phase. Poor candidates are rare, ambiguous, policy-volatile, or dominated by cheap deterministic code.

Score candidates from 1 to 5:

Start with workflows scoring high on the first four and low on risk. Invoice matching, standard support resolution, account research, renewal preparation, and recurring compliance evidence collection can fit. Executive strategy, novel incident response, negotiation, and one-off research usually do not.

Set a 30-day decision gate. The pilot should prove accurate matches, compatible adaptation, no material acceptance regression, and lower cost per accepted workflow. If hit precision is weak, improve task classification before expanding. If planning is only a small part of total cost, stop. The optimization has found its ceiling.

How OpenNash Can Help

OpenNash helps teams find where agent cost actually comes from before adding another caching layer. We trace the workflow, separate planning from execution and review, establish cost per accepted task, and identify task classes where a reusable procedure can remove meaningful work.

For a plan-caching pilot, we build the trace pipeline, extraction schema, scoped plan store, matcher, compatibility checks, fallback path, and evaluation suite. Tool, policy, prompt, model, and plan versions remain visible so the team can invalidate, audit, and roll back safely.

A managed platform is sensible when its caching controls match the workflow and data requirements. Custom plan reuse earns its cost when procedures, permissions, and completion checks are company-specific. Teams with low volume or unstable processes should fix the workflow first.

Book a call to map this to one workflow. Bring a week of traces, the current model bill, and the business rule that decides whether the work was accepted.