Where the tokens go: steering the context of AI models

In working with AI agents, control over the context is a key skill. On it depends how good the result gets and how much time and how many tokens, that is license limits, a task costs you.

Roughly put, within a session a modern language model reads all the text written so far every time: the dialogue, the system and the user prompt, the loaded tools and so on. The context window is the maximum amount of text the model can read. It is measured in tokens. A token corresponds to about 0.75 words in English and around 0.5 words in German, because umlauts and longer word forms are split by the tokenizer into more parts than plain Latin letters.

For free models the context is sometimes still 32K, in coding agents it is larger: 200K for Claude Sonnet 4.6, 1M for Claude Opus 4.8, 400K for GPT-5.5 in Codex. You might think a decent codebase fits into a million, so no cause for concern. But the more tokens lie in the window, the harder the model finds handling them. The reasoning gets weaker, answers come slower, and every move consumes more tokens and limits. That is why it pays to make the context visible, know its working zone and place the task exactly there.

What fills the context before the first question

The command /context in Claude Code shows the state. Even in the empty state, without a single message, 15,000 to 30,000 tokens are occupied by system internals. On Sonnet’s 200K window that is around 10 percent, plus about 16.5 percent reserved for auto-compaction. On Opus 4.8’s 1M window the same system parts occupy less than 3 percent.

The main categories in /context:

System prompt and tools: the developers’ system prompt and the description of the tools. A startup tax there is no way around.
MCP: used to cost a lot of context, but after an optimization by Anthropic it has gotten better. The descriptions load on demand, not all at once at startup. If MCP already occupies a noticeable share on the start screen, check whether a heavy server is connected that you have long stopped using.
Memory: the current project’s CLAUDE.md plus the first 200 lines of your MEMORY.md. Grows with auto-memory; if it becomes too much, it is worth a look at what can be offloaded or shortened.
Skills: only names and descriptions. The actual instruction loads only on call and stays in the context as long as the associated work runs.
Messages: the history, that is your messages, Claude’s answers, tool results and the content of read files. Grows faster than all other categories combined, and this is exactly where the main consumption sits.
Free space: what remains free. It results from the window minus everything occupied minus the buffer for auto-compaction.

The developers have optimized the startup sentence well in recent months, which is why I do not advise beginners to additionally offload something there. You cut consumption elsewhere.

First, Opus 4.8 consumes more tokens than Sonnet 4.6. The larger window offsets that, but token consumption is also the API bill or the melting license limit. On top of that, the model’s entire reasoning goes into the context. The higher its level, the higher the consumption. On the Max x5 plan I almost always use Opus 4.8 on “xhigh.” On Pro I recommend starting with Sonnet 4.6 on “high” and switching to higher levels or Opus only when you know what you want from the model. In Codex I stay on the current version (GPT-5.5) and medium or high reasoning level; for getting started or simple tasks, GPT-5.4 Mini suffices.

There is no universal recipe. A model on “high” consumes fewer tokens than on “max.” But if it solves the task only on the second or third attempt while the higher level manages it right away, the higher level consumes less in the end.

/context is worth it at four points:

Right after starting a new session, to see the startup tax of your configuration, that is how heavily MCP and memory are occupied.
Before a big task, to check whether both the task and the subsequent discussion with corrections fit into the remaining window.
After a failed step, to decide whether /clear or /rewind is due, before the model sinks into the garbage.
When you have worked long or been distracted. The feeling of intuitively knowing how much context remains is deceptive; the ten seconds for the exact number are well invested.

Context rot and the real working zone

On every request the model reads all the text. As the volume grows, a problem begins that researchers call context rot. Human memory immediately highlights the important and throws away the incidental, sometimes so thoroughly that we no longer know what we talked about five minutes ago. With the model the problem is reversed: it keeps everything and, on long contexts, starts to confuse what is important and what is not. Especially when several different tasks, faulty results and interim questions already lie in the context.

The intuition “a million context, so the model works well right up to its edge” is therefore wrong. On which size you should not exceed day to day, there are various opinions. I stick to these values. With Sonnet 4.6 (200K) I stay under about 120K to 140K. With GPT-5.5 in Codex (400K, a limit of Codex itself, in the API the model has a million) I stay at around 200K, because I found no solid recommendations. With Opus 4.8 (1M), the predecessor Opus 4.7 held the context well up to about 400K; with 4.8 the number should rise a bit.

The window can be limited via the environment variable CLAUDE_CODE_AUTO_COMPACT_WINDOW. Set, say, 140000 or 400000, and the client treats this value as the effective window and starts auto-compaction closer to this threshold. But I prefer to watch by hand, because after 140K or 400K the model does not get worse all at once. I would rather a big task runs slightly over the limit than have auto-compaction start in the middle of it.

Besides quality, a long context hits two further points. First, the prefill rises with the fill level, that is the time from the request to the start of the answer. Near a million this takes two minutes and more on Opus 4.7; even a short “please be more precise” then starts with a long pause. Second, token and limit consumption shoots up on long contexts, because the agent reads the whole history, and prompt caching does not fully solve this either.

Tasks that need the full window

There are cases where the full million makes sense.

The first is a document that cannot be cut apart: a contract, a scientific report, the full transcript of an interview. When the conclusions have to come from the whole, for instance to compare phrasings at the beginning and end or to connect names across the text, splitting means losing the connection.

The second is a one-off, pinpoint question to a large base. You put the entire documentation, an old repository or a long collection into the session and ask where a function is mentioned or whether a certain condition occurs. This search for the needle in the haystack via keywords works usably on long windows and usually suffices for a one-off task.

The third is an ongoing session with a large goal, where splitting into subagents or restarting would be more expensive than the quality loss toward the middle of the window. Mostly these are deeply specialized tasks. But a good example is also the very first sessions with Claude Code, in which it is easier to run the whole task through a dialogue than to dive straight into commands and subagents.

How to place the task into the working zone

Most tasks, even big ones, fit easily into 400K or even 140K.

The first means is to cut large files into pieces. An eight-hundred-page report or a repository of hundreds of files is not read in one go but section by section, and after each you put an interim summary into a separate document or into memory. The main session then holds not the whole text but the excerpt.

The second means is pinpoint queries instead of “read everything.” In a codebase you search via grep and glob, by function name, by string, by path. The model sees only the necessary pieces instead of the whole project.

The third means is subagents. A subagent is a separate Claude instance with its own context window that completes a task and returns only the result to your session. The busywork, search lists and pages-long diffs do not land in your main context. Claude Code sometimes starts subagents itself; if you want to trigger it explicitly, write something in the chat like “hand the fact-check to a subagent.” Subagents also speed up the work: if Claude finds, say, five errors in a text and is unsure at one spot, the main instance can work in the corrections while a subagent takes over the deep search in parallel.

The fourth means is to put lasting knowledge into memory instead of the context. Fixed rules, agreements and preferences live in two places. First in the CLAUDE.md files: the project-related CLAUDE.md in the root directory (shared with the team via Git) or the global ~/.claude/CLAUDE.md (applies to all your projects). Second in auto-memory, that is the notes Claude keeps itself; they live under ~/.claude/projects/<project>/memory/ and are tied to the respective repository. Into the startup context of every session, all CLAUDE.md load in full and the first 200 lines of the MEMORY.md index, the rest Claude fetches on demand. The rule is simple: explain the same thing a second or third time, and it goes into memory, and the context is freed immediately.

Commands for session control

/resume: return to an old session. Unlike in ChatGPT, the central units in Claude Code are the project folders, not the chats. Via /resume you reach the history anyway, for instance when you had to break off mid-work and do not want to explain everything again next time.
/clear (also /new and /reset): empty everything. Starts a completely new chat, fitting when you switch to an independent task. The old session stays reachable via /resume and can be output to a file or the clipboard with /export.
/rewind (also double Esc): rewinds the session by one or more steps and removes them completely from the context. You see the list of steps and can reset only the conversation, only the files, both, or “summarize from here,” that is a targeted /compact from a chosen point.
/compact: condense the history into a short summary. You keep the thread but lose details like literal phrasings and interim states. With an argument you can prioritize, for example /compact keep the API markup and the test results. If you do not press it yourself, Claude Code starts compaction on its own at the default threshold (around 95 percent of the window), that is auto-compaction.
/btw <question>: ask a quick follow-up without taking it into the main conversation. Question and answer stay outside the session context, handy for a small thing like “how is this command’s flag spelled.”
/branch [name]: a branch from the current conversation. The context up to the branch is preserved, after it everything runs separately. Basically a Git branch for the dialogue, when you want to try another approach without losing the ongoing work.

I best run compaction when the task has reached an interim state. That is why I do not wait for auto-compaction; it comes at the most inconvenient moment and can lose important details.

Back to /rewind. If the model did something wrong, I choose from three options. If the result is weak and useless, I reset with /rewind, set the task differently, raise the reasoning level and try again without garbage in the model’s head. If I want the model to improve a mediocre result, or deliberately show it as a bad example, I leave the context standing. If it is a general instruction, it belongs in CLAUDE.md or auto-memory, for instance “write in German, without em-dashes,” “do not use library X, we switched to Y” or “never touch folder Z.” If you catch yourself typing the same instruction three or four times, it belongs in CLAUDE.md.

In closing

This is best summed up in one image: an AI agent is a diligent junior you have to guide. A big task it accepts but does not manage in one piece, so split it. Out of exhaustion it falls over at the wrong moment by triggering auto-compaction, so keep an eye on the session and steer it yourself. Sometimes it asks for help, sometimes it does not dare, so start subagents when needed. Whoever has the context under control gets noticeably more reliable work out of the same model, at lower cost.