Context Engineering
Writing system prompts for agents, budgeting the context window, ordering information, and compacting long runs
Context Engineering
TL;DR
The context window is the agent's entire world — it knows nothing else. Context engineering is deciding what goes in it, in what order, and what to remove when it fills. It is the highest-leverage tuning available on an agent, and it is mostly writing and arithmetic rather than machine learning.
| Property | Value |
|---|---|
| Level | Intermediate → Advanced |
| Reading time | ~20 minutes |
| Prerequisites | Memory |
| You will understand | System prompts, context budgets, ordering, and compaction |
What Is Actually in the Window
Every request in an agent loop looks roughly like this:
┌─ SYSTEM PROMPT ──────────────── ~400 tokens (fixed)
│ Role, rules, constraints, output format
├─ TOOL DEFINITIONS ───────────── ~800 tokens (fixed, EVERY call)
│ Every tool: name, description, parameters
├─ LONG-TERM MEMORY ───────────── ~200 tokens (retrieved)
│ Facts about this user, if relevant
├─ THE TASK ───────────────────── ~100 tokens
│ What the user actually asked
├─ RUNNING STATE ──────────────── ~250 tokens (updated each step)
│ Goal, plan, progress, findings so far
├─ TRACE SO FAR ──────────────── ~3,500 tokens (GROWS)
│ Every thought, action, observation
└─ GOAL RESTATEMENT ───────────── ~50 tokens
The goal again, at the end where attention is reliableNotice which block grows: the trace. Everything else is roughly constant. So context management is almost entirely about controlling the trace, and the fixed blocks matter mainly because you pay for them on every single iteration.
The System Prompt
The standing instructions sent on every call. For agents it has jobs a chatbot prompt does not.
| Section | Purpose |
|---|---|
| Role | What this agent is and who it serves |
| Goal behaviour | What success looks like — and what "done" means |
| Tool policy | When to use tools, and that it must not guess facts it can look up |
| Boundaries | What it must never do, and when to stop and ask |
| Output format | The exact shape expected |
| Escalation | When to hand off to a human, and how |
You are a supplier research assistant for the procurement team.
Using tools:
- Use tools for any fact about a company. Never answer from memory.
- If a tool returns an error, read it and correct your call.
- If a tool fails twice, stop and report what you could not check.
Finishing:
- You are done when all items in your plan are complete.
- Say clearly what you could NOT verify. An incomplete answer that is
honest is correct; a complete answer that is guessed is not.
Never:
- Never contact a supplier directly.
- Never state a compliance conclusion without a source.The most valuable lines in an agent system prompt are usually about stopping and admitting failure. Models are strongly pulled toward producing a complete-looking answer. Unless honest partial results are explicitly made correct, you get confident fabrication on exactly the hard cases where you needed the truth.
Ordering Matters
Models read the beginning and end of a long context most reliably, and skim the middle. This is the same lost in the middle effect that appears in RAG, and agents hit it harder because their context grows every step.
| Position | Put here |
|---|---|
| Beginning | System prompt, tool definitions, the task |
| Middle | The older trace — the part you can afford to have skimmed |
| End | Running state, the goal restated, the immediate question |
Restate the goal near the end of every request. It costs a handful of tokens and it is the cheapest fix available for agents that drift on long tasks. By iteration 12 the original task sits far back in a long context; a restatement puts it back where attention is reliable.
The Context Budget
Treat the window as a budget you allocate, not a limit you hit.
| Block | Suggested share | Notes |
|---|---|---|
| System prompt | 5% | Fixed. Keep it tight — you pay every call. |
| Tool definitions | 10% | Fixed. Fewer tools directly cuts this. |
| Retrieved memory | 5% | Only what is relevant now |
| Running state | 5% | Compact and structured |
| Trace | 60% | The working area |
| Headroom | 15% | For the response, and for one large surprise result |
Keep real headroom. A single unexpectedly large tool result should not be able to overflow the window. Truncate or summarise tool output before it enters the context, and enforce a maximum size per observation.
Compaction
When the trace grows too large, compress it. This is the standard mechanism for long-running agents.
Compaction
Work normally
Think, act, observe — the trace grows
Check the budget
Past ~70% of the window?
Summarise the old part
One model call turns older steps into a compact summary of findings
Rebuild the context
System + tools + state + summary + the last few steps in full
Continue
Same task, much smaller context
| What to preserve | What to compress |
|---|---|
| The goal, exactly | Reasoning that reached a conclusion already recorded |
| The plan and what remains | Raw tool payloads |
| Conclusions reached | Steps that led nowhere |
| Failed approaches, one line each | Full text of long documents |
| The last 2–3 steps in full | Everything older |
Always keep a note of what failed. The most common compaction bug is summarising away the failures, after which the agent cheerfully retries an approach that already did not work — and can do so indefinitely.
Reducing the Fixed Cost
The fixed blocks are paid on every iteration, so trimming them multiplies out.
| Lever | Effect |
|---|---|
| Fewer tools | Directly cuts the largest fixed block |
| Shorter tool descriptions | Keep them specific but not verbose |
| Tighter system prompt | Remove restated rules and dead instructions |
| Route tool sets by task type | Show only the tools this kind of task needs |
| Prompt caching, where available | The fixed prefix can often be cached across calls, cutting its cost substantially |
Prompt caching pairs especially well with agents, because the system prompt and tool definitions are byte-identical on every iteration of every task. Keeping that prefix stable — same order, same content — is what makes caching effective, so avoid inserting anything variable near the start.
Common Mistakes
What goes wrong
Raw tool output straight into context
The default path to context overflow. Cap and summarise every observation before it enters.
No goal restatement
The task ends up buried in the middle of a long context, and the agent drifts.
No headroom
One large result overflows the window and the whole run fails.
Compaction that drops failures
The agent retries what already failed, sometimes forever.
Variable content at the start of the prompt
Breaks prompt caching on every call, quietly multiplying cost.
A system prompt that never says how to fail
Produces confident fabrication precisely on the hard cases.
Not measuring context size
You discover the ceiling by hitting it in production.
Concept Checks
Check yourself
Why does trimming tool descriptions matter more in an agent than in a single call?
Because the definitions are re-sent on every iteration. In a one-shot call you pay for them once; in a twelve-step agent run you pay twelve times, on every task. A block that looks like a minor 800-token overhead becomes nearly ten thousand tokens per task. The same multiplication applies to the system prompt, which is why the fixed blocks deserve far more editing attention than their size alone suggests.
Why restate the goal at the end rather than trusting the original task message?
Because the original message sits at the start, and by iteration twelve it is separated from the decision point by thousands of tokens of trace. Models attend most reliably to the beginning and end, so material in the middle gets skimmed — and the original task has effectively drifted into that region. A short restatement at the end costs almost nothing and puts the goal back where it will actually be read.
Your compaction summarises the trace and the agent starts repeating failed approaches. What happened?
The summary preserved conclusions but dropped the failures. Unsuccessful attempts feel like noise worth compressing away, but they carry essential information: this route does not work. Once removed, the agent has no record of having tried, so it re-derives the same plausible next step and fails identically — potentially in a loop. Keep a one-line note for every failed approach through every compaction.
Why keep 15% of the window free rather than filling it?
Because you must fit the model's response, and because tool results are not fully predictable in size. A query that normally returns three rows may return three thousand. Without headroom, a single oversized observation overflows the window and the run fails outright rather than degrading. The headroom, combined with a hard per-observation size cap, turns a fatal error into a truncated result the agent can still work with.
Key Concepts Recap
| Concept | One-line summary |
|---|---|
| The window is the whole world | The agent knows nothing that is not in it |
| Only the trace grows | Fixed blocks are constant but paid every iteration |
| System prompt must cover failing | Say what "done" means and when to give up |
| Beginning and end are read best | Middle gets skimmed |
| Restate the goal at the end | Cheapest fix for drift |
| Budget the window | Allocate shares; keep 15% headroom |
| Cap observation size | Before it enters the context, not after |
| Compaction | Summarise old steps past ~70% full |
| Never compact away failures | Or the agent repeats them |
| Stable prefix | Keeps prompt caching working |
Next
One agent, well built. Now what happens when you use several: Multi-Agent Systems.
Memory
Working, short-term, and long-term memory — what to keep, what to forget, and how agents remember anything at all between calls
Multi-Agent Systems
Orchestrator-workers, handoffs, debate, and parallel specialists — the coordination patterns, what they cost, and the honest case for using one agent