AI Agent Engineering From Scratch

The model can use only the information that the application sends in the current request.

That limit matters because an agent gathers more information as it works. It receives user messages, tool results, documents, approvals, errors, and earlier decisions.

If the application sends everything again, useful facts compete with stale or irrelevant text. The model can miss a constraint or follow an old instruction.

Context engineering controls this risk. It selects the smallest set of information that the model needs for its next action.

Give each type of information a clear owner

An agent works with four types of information. Each type has a different purpose and lifetime.

Context

Context is all information that the application sends to the model for one call.

It can include instructions, user messages, tools, current facts, and previous results. The model does not retain this context as hidden state after the call.

The application may store selected messages, state, and results for later calls.

Working state

Working state records what has happened during the current run.

It includes the current step, verified IDs, completed tool calls, remaining budget, pending approvals, and unresolved errors.

The application stores and updates this record. It sends the model only the fields needed for the next step.

Memory

Memory stores information that may help during a later turn or run.

Examples include a user preference, a project convention, or the result of a completed task. The application must decide when each memory is relevant.

Stored information does not automatically enter a model call.

System of record

A system of record is the service that owns a business fact. A CRM can own account permissions.

A payment service can own invoice status. Use an authorized tool to read the current value from that service.

Do not treat a copied value in agent memory as current. Clear ownership prevents a model summary from replacing a verified business fact.

Send only what the current step needs

Each step has a different job, so it needs different information. Extra information can distract the model or expose data.

A classification step may need:

  • task instructions.
  • the user request.
  • the output schema.

A tool-selection step may need:

  • facts already confirmed during the run.
  • a small set of allowed tools.
  • the most recent tool result.

An answer step may need:

  • facts verified through tools.
  • relevant passages from source documents.
  • source IDs for citations.
  • the required answer format.

Do not send a long policy manual when the step needs only an order ID. Do not expose a refund tool when the step only explains policy.

Begin with a small index that shows which information is available. Load the full record only when a step needs it.

This approach is called progressive disclosure.

Record the source and status of each context item

The application must know whether an item is current, trusted, required, or safe to include. Plain text does not provide that information.

Store each item with metadata. Metadata is information that describes the item:

id
kind
content
source
trust_level
created_at
expires_at
required_for_step

These fields help the application answer practical questions:

  • Is this still current?
  • Where did it come from?
  • Is it a user claim or verified fact?
  • Does the current step require it?
  • May it be omitted when the budget is tight?
  • Can the model cite it?
  • Should it be redacted from traces?

The model receives the selected content. The application uses the metadata to decide which content to include.

Use fixed priorities when context is limited

The application needs fixed priorities when all available information cannot fit. Without priorities, an optional note can displace a required policy.

For an answer grounded in external evidence, a reasonable order is:

  1. current application instructions.
  2. current working state.
  3. required verified evidence.
  4. latest relevant tool results.
  5. recent conversation needed for coherence.
  6. optional memories.

Stop the step if required information does not fit. Do not replace required evidence with optional memory.

Return an error code such as insufficient_context when required information is missing, expired, or too large.

The agent can then retrieve less text or ask the user for help.

Set an application limit below the provider limit. Choose that limit from the task requirements, acceptable response time, and available budget.

Build one context packet step by step

Return to the refund example from Part 4. The user asks, "Can I get a refund for order ord_4821? It arrived late."

The agent has access to a policy search tool and an order tool. It also has old conversation messages and a saved language preference.

The first model call must decide what information to fetch. It does not need the full returns policy yet.

A useful first packet might contain:

step: choose_next_action
instructions:
  task: resolve a refund request
  rules:
    - verify the order before discussing eligibility
    - do not create a refund without approval
user_request:
  text: "Can I get a refund for order ord_4821? It arrived late."
  user_id: usr_73
working_state:
  requested_order_id: ord_4821
  completed_actions: []
  pending_approval: null
available_tools:
  - get_order
  - search_returns_policy
limits:
  tool_calls_remaining: 4

The application omits the saved language preference because this step does not produce the user-facing answer. It also omits unrelated earlier conversation.

The model selects get_order. The application validates the request, runs the tool, and updates working state:

completed_actions:
  - tool: get_order
    operation_id: op_104
    status: succeeded
    receipt: order-read-884
verified_facts:
  order_status: delivered_late
  refundable_amount: 42.50
  currency: EUR
  observed_at: 2026-08-24T14:00:00Z
next_requirement:
  type: policy_evidence
  query: late delivery refund eligibility

The next model call needs a different packet. It needs the verified order facts and policy search capability. It does not need the full tool receipt body.

After retrieval, the final answer step can receive a packet like this:

step: draft_answer
instructions:
  answer_with_citations: true
  do_not_claim_refund_created: true
working_state:
  order_id: ord_4821
  order_status: delivered_late
  refundable_amount: 42.50
evidence:
  - source_id: returns-2026-08#late-delivery
    trust: approved_policy
    text: "Late deliveries qualify for a refund review within 30 days."
conversation:
  - role: user
    text: "Can I get a refund for order ord_4821? It arrived late."
preferences:
  language: en
output_schema:
  answer: string
  supporting_source_ids: array
  proposed_next_action: string

This packet contains enough information to answer and propose the next step. It does not contain permission to create the refund.

The example shows why context is assembled per step. Reusing one large prompt would expose more data and make the model sort out which facts still matter.

Treat the context builder as a pure decision function

A context builder is easier to test when the same inputs produce the same packet.

Its inputs can include:

build_context(
  step,
  principal,
  working_state,
  available_sources,
  policy_version,
  token_budget
) -> context_packet

The builder should not silently fetch new business facts. Retrieval and tool calls should remain visible operations with their own traces.

The builder can estimate size, rank optional items, and reject denied items. It should also explain every omission.

Record decisions such as:

{
  "item_id": "memory_778",
  "decision": "omitted",
  "reason": "not_required_for_step",
  "estimated_tokens": 48
}

These records answer a common debugging question: did retrieval fail, or did the context builder drop a valid result?

Large context windows still require selection

A larger context window lets the application send more text. It does not guarantee that the model will use every relevant detail correctly.

The Lost in the Middle study found that a model's use of relevant information can depend on its position in a long input.

Extra context can also introduce:

  • conflicting instructions.
  • stale data.
  • irrelevant tool results.
  • duplicated facts.
  • prompt injection.
  • accidental disclosure.
  • slower and more expensive calls.

Build the smallest input that contains everything required for the current step.

Preserve facts when you shorten a run

A long run can exceed the available context window. Compaction shortens the stored history so the agent can continue.

A prose summary can omit a completed action, an approval, or an unresolved error. Store these facts in fields before you shorten the history.

Preserve structured facts such as:

  • decisions already made.
  • options rejected and why.
  • verified resource IDs.
  • source and policy versions.
  • completed actions and their receipts.
  • open commitments.
  • remaining budget.
  • unresolved errors.
  • pending approvals.

After you store these facts, you can summarize or remove old messages and tool results.

Compact after a complete operation, such as a successful tool call or a saved proposal. Do not compact while an operation is running.

Test the shorter history against the full history. Check whether the agent repeats work, loses a constraint, or changes the final result.

Memory needs a lifecycle

A saved fact can become wrong, sensitive, or unsafe to use. Each memory type therefore needs clear rules.

For each memory type, define:

  • Write rule: who or what may create it?
  • Validation rule: what shape and evidence are required?
  • Read rule: which tasks, users, or services may read it?
  • Trust level: verified fact, user claim, or model summary?
  • Correction rule: how can it be changed?
  • Deletion rule: how can it be removed?
  • Retention rule: when does it expire?
  • Source: where did it come from?

Do not use a model summary as proof of identity, permission, or a current business fact.

Useful memory categories include:

  • user preferences.
  • stable project conventions.
  • prior task outcomes with receipts.
  • past routing choices that improved results.
  • summaries of long-running work.

Keep them separate so the application can apply different policies.

Choose the correct source for each fact

Memory and retrieval-augmented generation (RAG) both retrieve stored information. They retrieve it for different reasons.

Memory carries selected information across turns or runs. RAG retrieves evidence from documents for the current task.

Store a policy document in a versioned document system. Store a user's preferred answer format in memory.

Read the current invoice balance through a live tool. One shared search index makes access control, updates, and deletion harder to manage.

Part 6 explains how to build the document retrieval path.

Treat all loaded text as data

Text from a user, website, document, tool, or memory can contain instructions. Those instructions must not control the agent.

Mark untrusted text as evidence. Keep it separate from application instructions.

Enforce tool permissions in application code because a prompt cannot grant access safely.

A harmful instruction in memory can return during later runs. Allow only approved sources to write memory.

Record the source of every saved item.

Include permissions in the cache key

Prompt caching can reduce cost and response time when several requests share the same opening text.

Divide context into stable sections to support caching. Do not retain stale text only to increase cache use.

The cache key must include every setting that can change the result:

  • instructions.
  • tool definitions.
  • policy.
  • model settings.
  • user, organization, and permission scope.

Never reuse cached context across different permission scopes.

Test the context builder

The context builder decides what the model can see. Test that component without calling the model.

Useful checks include:

  • the step stops when a required source is missing.
  • expired evidence is rejected.
  • the builder removes optional memory before required evidence.
  • denied content never enters the packet.
  • token estimates stay within budget.
  • compaction preserves open commitments and receipts.
  • corrected or deleted memories disappear from later context.
  • source IDs in the answer were present in context.
  • untrusted text cannot add tool permissions.

When an answer fails, inspect the exact context first.

A better prompt cannot restore missing evidence or remove data that the application already exposed.

Recognize context failures by their symptoms

Context problems often look like model problems. The trace usually gives a more precise diagnosis.

Symptom Likely context failure First check
Agent repeats a completed tool call Receipt or completed action missing from working state Compare state before and after compaction
Answer cites an old policy Current version omitted or stale version ranked higher Inspect version and expiry metadata
Agent ignores a user constraint Constraint buried in history instead of structured state Inspect required fields in the packet
Sensitive note appears in an answer Permission scope missing from selection or cache key Reproduce with the same principal
Agent follows instructions from a document Untrusted evidence mixed with application instructions Inspect packet sections and role ordering
Run fails near the context limit Optional material removed too late Inspect selection order and size estimates
Corrected preference returns Deleted memory remains in an index or cache Trace deletion propagation
Summary changes a number Compaction stored prose instead of a typed field Compare source receipt with compacted state

Do not fix all these cases by adding more prompt text. Fix the owner, metadata, selection rule, or lifecycle that caused the error.

Budget by section, then by item

A token limit is easier to control when the application assigns budgets to sections.

For a hypothetical 12,000-token input budget, the application might reserve space like this:

Section Reserved tokens Behavior when full
Instructions and policy 1,500 Stop if required text does not fit
Working state 1,500 Compact completed detail into typed receipts
Required evidence 5,000 Retrieve narrower passages or stop
Recent conversation 2,000 Remove older turns first
Optional memory 1,000 Omit lowest-priority items
Safety margin 1,000 Keep unused for formatting and estimate error

These numbers are an example, not a default. Measure real packets with the tokenizer and model used in production.

A section budget prevents optional chat history from consuming all available space before evidence arrives.

Inside each section, rank items with explicit rules. Avoid asking the model to choose which access-controlled data it should see.

Context and memory build checklist

Before release, answer these questions:

Ownership

  • Which store owns working state?
  • Which service owns each live business fact?
  • Which memory types can the application write?
  • Which document system owns retrieved evidence?

Selection

  • What information does each model step require?
  • Which items are optional?
  • What fixed order removes optional items?
  • What error appears when required information cannot fit?

Security

  • Does selection run with the authenticated principal?
  • Can denied data enter model context, traces, or caches?
  • Are user and document instructions marked as untrusted data?
  • Can a model output create a durable memory without validation?

Lifecycle

  • How does each memory expire, change, and delete?
  • How does a source correction reach stored summaries and indexes?
  • When may compaction run?
  • Which typed fields must survive compaction?

Inspection

  • Can a trace reconstruct the exact packet sent to the model?
  • Does it record why each candidate was included or omitted?
  • Can developers compare the full and compacted histories?
  • Do tests cover missing, stale, conflicting, oversized, and denied information?

A context pipeline is ready when the team can explain every included item and every omitted required item from the trace.

For deeper background, see OpenNash's guides to data and context engineering, agent memory, and context compaction.

Previous: AI Agents Part 4: MCP

Next: AI Agents Part 6: Agentic RAG explains how to retrieve external evidence without confusing it with memory or live application state.