Context Assembly: The Window Never Overflowed, and the Agent Still Forgot

Most agent failures get blamed on a full context window. The real culprit is quieter: nobody decided what the model should see on the turn that mattered. Here is how to treat the window as RAM and the harness as its memory manager, built twice.

Your agent loses the thread mid-task even though the token count never came close to the limit, and you cannot figure out why. Most agent failures get blamed on a full context window.

In this article: You will learn why an AI agent can forget critical facts while the token count sits comfortably under the limit, and what to do about it. We cover the four levers of context window management (select, compress, isolate, write), build three of them in both the Claude Agent SDK and LangChain Deep Agents, and end with the one judgment no framework can make for you. By the end you will assemble the window on purpose instead of hoping it holds together.

Maria needs to be in Chicago by Thursday for a medical appointment. Over six turns she tells the rebooking agent the essentials: the Thursday deadline, no connection through Denver, a preference for United, and a $400 credit from the cancelled flight. The agent retrieves options, confirms pricing, and gets three steps from done. Then on turn fourteen it recommends a Denver connection. On turn seventeen it forgets the $400 credit and quotes full fare. On turn twenty it stops and asks her to repeat the constraints she gave it twenty minutes ago. Maria closes the tab and calls the airline.

Here is the detail that matters: the window never overflowed. The token count sat comfortably under the limit the whole time. The agent did not run out of room. It lost the ability to see the facts that mattered, because nothing in the harness was protecting them. That is not a model defect, and it is not bad luck. It is what happens when nobody decides what lives in the context window and what gets thrown away.

There is a useful distinction hiding in this failure. One job is what the agent remembers: the durable store of everything it has been told. A different job is what the agent sees: the slice of that store that actually reaches the model on a given call. Conflating the two is how you get an agent with a perfect memory store and amnesia at the moment of decision. This article is about the second job, context window management, the discipline of assembling the model's working memory deliberately on every single call.

The window is RAM, and you are the memory manager

The inference layer holds nothing between calls. No memory, no state, no continuity. The moment a call completes, everything the model knew is gone, and the next call reassembles the whole context from scratch: system prompt, tool definitions, history, retrieved facts, and tool results. That property makes one thing true that reorganizes how you build: the harness is the only memory manager the model has.

Andrej Karpathy framed it cleanly: the model is the CPU, and the context window is the RAM. What an operating system does for a process, the harness must do for the model. Like RAM, the window has limited usable capacity, less than the spec sheet implies. Flagship models advertise around a million tokens, but self-attention scales quadratically, and measured degradation begins well short of the ceiling, often in the 128,000 to 200,000 token range. The practical window is smaller than the advertised one. Anthropic's own framing is that the model has an attention budget, and every token you admit depletes it.

So the question on every call is not "does this fit" but "is this worth attention." There are exactly four levers for answering it, and each prevents a different failure:

• Select keeps noise out before it enters. Prevents distraction.
• Compress reclaims headroom from tokens already present. Prevents rot and panic.
• Isolate keeps one agent's context out of another's. Prevents clash.
• Write pins the facts that must not be lost. Prevents Maria's failure.

We build select, compress, and write here, in both frameworks. Isolate is really an orchestration decision, so it gets its own treatment elsewhere.

Select: keep the noise out

The cheapest token is the one you never admit. The most common self-inflicted bloat is loading every tool definition up front: fifty MCP tool schemas run roughly 72,000 tokens before the agent does any work, while on-demand discovery of the same tools costs about 8,700, a 95% cut in that one component. The harness should admit tools, instructions, and tool results on a need-to-know basis.

In the Claude Agent SDK, on-demand tool discovery is the default, so the win is mostly about not undoing it: keep the tool surface lean, restrict which configuration sources load, and return narrow projections from tools instead of dumping raw payloads.

This listing defines a tool that returns a one-line projection rather than the raw record, then wires it into options that keep both the tool surface and the configuration sources deliberately small. Selection happens in two places: inside the tool body, and in the options.

from claude_agent_sdk import ClaudeAgentOptions, tool, create_sdk_mcp_server

@tool("get_itinerary", "Fetch one itinerary by confirmation code", {"code": str})
async def get_itinerary(args):
    it = itinerary_api.get(args["code"])
    summary = f"{it['route']} | {it['date']} | status={it['status']}"  # ① projection
    return {"content": [{"type": "text", "text": summary}]}            # not the raw blob

server = create_sdk_mcp_server("travel", tools=[get_itinerary])

options = ClaudeAgentOptions(
    mcp_servers={"travel": server},
    allowed_tools=["mcp__travel__get_itinerary"],   # ② small, scoped tool surface
    setting_sources=[],                             # ③ no implicit .claude/ config bloat
    system_prompt="You rebook flights. Fetch only the itinerary you are acting on.",
)

① The tool hands back a one-line projection, not the full record, so a verbose API does not flood the window. ② A scoped allowed_tools list keeps the schema cost low: only the one tool the agent needs is admitted. ③ Empty setting_sources stops stray CLAUDE.md and .claude/ files from loading silently into context.

Note: The full extracted listing at code/harness_engineering/part-4-context-assembly/listings/01-select-sdk.py shows the imports and surrounding wiring.

Deep Agents expresses selection through skills: the agent reads each SKILL.md frontmatter at startup and loads the full body only when it judges the skill relevant. The capability is present without its tokens being resident.

from deepagents import create_deep_agent

agent = create_deep_agent(
    model="anthropic:claude-sonnet-4-6",
    tools=[get_itinerary],                          # ① narrow tool set
    skills=["/skills/"],                            # ② frontmatter loaded; body on demand
    system_prompt="You rebook flights. Load a skill only when the task calls for it.",
)
# Only the frontmatter of each skill sits in context until the agent opens one.

① The tool surface stays small. ② Skills are progressive disclosure: their headers cost a few tokens; their full instructions cost nothing until used. Selection is the same discipline in both: decide what earns a place before it takes one.

Compress: reclaim headroom without losing structure

Selection bounds what enters. Compression reclaims what has already accumulated. The trap is blind compaction: summarize the whole history into prose and you lose the structured facts, the open subtasks, the exact figures. Structure-preserving compression keeps a progress record intact while collapsing the chatter around it.

This is where the two frameworks differ most, and it is worth seeing plainly. Deep Agents compresses for you. Offloading fires when a tool input or result crosses 20,000 tokens, moving the payload to the filesystem and leaving a pointer plus a short preview. Summarization fires as the window crosses about 85% of the model's input limit, condensing older messages while keeping the recent ones verbatim. You get both by constructing the agent, and you can tune or trigger them.

from deepagents import create_deep_agent

agent = create_deep_agent(
    model="anthropic:claude-sonnet-4-6",
    tools=[get_itinerary],
)
# ① Offloading: tool inputs/results over 20k tokens move to disk, replaced by a pointer.
# ② Summarization: near 85% of the input window, older turns are summarized,
#    recent turns kept intact. Both are built in.
# ③ Optional: a summarization tool lets the agent compress between tasks, at a clean
#    seam, instead of mid-step at a fixed threshold.

The Claude Agent SDK gives you the primitives and expects you to set the policy. The production move is a harness-driven compaction that measures the budget with the token counter and rewrites old turns into a structured progress block you control.

The two functions below split the job: one decides whether the window is over budget by counting tokens for free, the other rewrites the older history into a structured progress record while leaving the most recent turns untouched.

import anthropic

client = anthropic.Anthropic()

def over_budget(messages, model="claude-sonnet-4-6", limit=150_000) -> bool:
    count = client.messages.count_tokens(model=model, messages=messages)  # ① free check
    return count.input_tokens > limit

def compact(messages: list[dict]) -> list[dict]:
    head, tail = messages[:-6], messages[-6:]          # ② keep recent turns verbatim
    summary = client.messages.create(
        model="claude-sonnet-4-6", max_tokens=512,
        messages=head + [{"role": "user", "content":
            "Summarize the above as a PROGRESS RECORD: open tasks, decisions, and "    # ③
            "every concrete figure (prices, dates, codes). Preserve structure."}],
    ).content[0].text
    return [{"role": "user", "content": f"PROGRESS RECORD:\n{summary}"}] + tail  # ④

① count_tokens measures the window without an inference call, at no cost, so the policy is data-driven, not guessed. ② The history is split into an older head and the most recent six turns; the tail stays verbatim and only the head is compressed. ③ The summarization prompt demands a structured progress record, forcing prices, dates, and codes to be preserved rather than paraphrased away. ④ The compacted history is the summary block followed by the verbatim tail, so recent context survives intact alongside the structured record.

Note: The full extracted listing at code/harness_engineering/part-4-context-assembly/listings/02-compact-sdk.py shows the helpers as a runnable module.

Automatic or hand-rolled, the rule is the same: compress chatter, preserve structure.

Write: pin the facts that must not rot

Now the heart of Maria's failure. Compression manages volume; it does not understand saliency. A summarizer that is told to be concise will happily paraphrase "$400 credit from cancelled flight UA482" into "has some credit," and three turns later the agent quotes full fare. The fix is a persistent fact block: a small set of load-bearing facts the harness re-injects verbatim on every call, exempt from compaction, so they cannot rot no matter how long the conversation runs.

This is a different goal from durable cross-session memory. There, the aim is survival across sessions, loaded by identity. Here, the aim is survival across compaction, within a session, unparaphrased. Same instinct, different boundary.

The helper renders a dictionary of facts into a fixed, labeled block, and the MARIA constant is the concrete fact set the rest of the section pins. The block's wording is the contract: it tells the model not to drop or paraphrase what follows.

def build_fact_block(facts: dict) -> str:
    """Render load-bearing facts verbatim for re-injection every turn."""
    lines = [f"- {k}: {v}" for k, v in facts.items()]                       # ①
    return "PINNED FACTS (do not drop, do not paraphrase):\n" + "\n".join(lines)  # ②

MARIA = {
    "deadline": "Must arrive Chicago by Thursday",
    "constraint": "No connection through Denver (DEN)",      # ③
    "credit": "$400 credit from cancelled flight UA482",
    "preference": "Prefers United",
}

① Each fact becomes one verbatim bullet line, so the exact wording of the source dict survives into the rendered block. ② The block opens with an explicit instruction not to drop or paraphrase, which is the guardrail the summarizer must respect. ③ The fact set carries the two load-bearing items from Maria's failure, the Denver constraint and the $400 credit, stored as exact strings.

In the Claude Agent SDK the fact block goes into the system prompt, which is reassembled on every call, so the facts ride along with each request regardless of how the history is compacted.

from claude_agent_sdk import ClaudeAgentOptions

options = ClaudeAgentOptions(
    system_prompt=(
        "You rebook flights for this customer.\n\n"
        + build_fact_block(MARIA)                    # ① pinned, present every call
    ),
    setting_sources=[],
)
# Turn 14 cannot recommend Denver: the constraint is in front of the model every turn.
# Turn 17 cannot forget the $400 credit: it is pinned, not summarized.

In Deep Agents the same facts live in a memory file the agent always loads, so they persist in context across the summarization that would otherwise paraphrase them away.

This listing writes the rendered fact block to a memory file in a store, then constructs an agent that always loads that file and routes the /memories/ path to the store-backed backend so the pinned file is durable while ordinary state stays in the state backend.

from deepagents import create_deep_agent
from deepagents.backends import CompositeBackend, StateBackend, StoreBackend
from deepagents.backends.utils import create_file_data
from langgraph.store.memory import InMemoryStore

store = InMemoryStore()
store.put(("maria",), "/memories/pinned.md",
          create_file_data(build_fact_block(MARIA)))   # ① pinned facts as a memory file

agent = create_deep_agent(
    model="anthropic:claude-sonnet-4-6",
    memory=["/memories/pinned.md"],                    # ② loaded into context every run
    backend=lambda rt: CompositeBackend(
        default=StateBackend(rt),                      # ③ ordinary working state
        routes={"/memories/": StoreBackend(rt, namespace=lambda rt: ("maria",))},  # ④
    ),
    store=store,
)

① The rendered fact block is written verbatim into a store under the /memories/ path, so it lives outside the volatile turn-by-turn history. ② memory= names that file as always-loaded, so the constraint and the credit are in context on every run. ③ The default StateBackend holds ordinary working state that compaction is allowed to touch. ④ The route sends /memories/ reads and writes to the store-backed backend under Maria's namespace, keeping the pinned file durable and isolated from the summarized history.

Note: The full extracted listing at code/harness_engineering/part-4-context-assembly/listings/03-pin-deepagents.py shows the imports and the fact-block helper it depends on.

The honest contrast

Deep Agents automates the volume problem: offloading and summarization keep you under the ceiling without your code. The Claude Agent SDK hands you the measurement and the controls and asks you to write the policy. Both are reasonable designs. Neither solves Maria by itself, because automation manages size, not importance. Deciding that the $400 credit and the Denver constraint are load-bearing, and pinning them, is a judgment the harness author makes in either framework. The window is RAM; you are still the one who decides what stays resident.

Do this today

• Run a twenty-turn rebooking with no fact block and watch a concrete figure (a price, a credit) get paraphrased away by compaction. Add the pinned block and rerun, and confirm the figure survives.
• Instrument count_tokens before and after compact. Confirm the structured progress record reclaims headroom while the pinned facts stay byte-for-byte identical.
• Preload fifty tool schemas, then switch to on-demand discovery, and measure the token delta on the first call. You should see most of the 72k vanish.
• Audit one real agent for what it admits silently: stray config files, raw tool payloads, every tool defined up front. Each is a token you could keep out.
• Write down the three or four facts in your domain that must never rot, and pin them verbatim before you tune anything else.

Assemble the window on purpose

Maria's agent did not fail because the model was weak or the window was full. It failed because no one decided what the model should see on turn seventeen. Select keeps noise out, compress reclaims room without shredding structure, and write pins the facts that must never rot. Assemble the window on purpose, in either framework, and the agent stops losing the thread under the limit.

The lesson generalizes past travel rebooking. Any agent that runs long enough will face the same quiet failure: a fact that mattered, paraphrased into vagueness while the token count looked fine. The frameworks give you measurement, automatic compaction, and durable storage. What they cannot give you is the judgment about which facts are load-bearing. That judgment is the job. Treat the context window as RAM, treat yourself as its memory manager, and decide on purpose what stays resident.

---