Managing Your AI Coding Context Window | Coding Capybaras

Photo by Daniel Jerez on Unsplash

Your AI coding assistant gets worse three hours into a session because its context window fills with old messages, tool output, and half-finished detours, and the model's attention gets stretched too thin to reliably use any of it. The fix isn't a bigger context window. It's managing what goes into the one you have: starting fresh sessions before quality craters, writing important state to a file the assistant re-reads, and keeping each session pointed at one job. This is the single habit that separated my good coding days from my frustrating ones, and it has nothing to do with how good the underlying model is.

TL;DR

• The "context window" is everything your AI assistant can see at once: your instructions, the conversation, and every file it read. It's finite.
• Quality degrades as it fills — a pattern researchers call "context rot" — well before you hit the advertised limit.
• The three levers that work: start fresh sessions often, persist state to a file (like CLAUDE.md or NOTES.md), and scope each session to one task.
• You don't need to read code to do this. You need to notice the warning signs and reset before things go sideways.

What a context window actually is

A context window is the total amount of text an AI model can hold in view at one moment — measured in tokens, which are chunks of words. Everything the model "knows" during your session lives there: the system instructions, your prompts, its own replies, and the full contents of every file it opened or command it ran.

Think of it as the assistant's working memory, not its long-term memory. When you start a session in Claude Code, Cursor, or Cowork, that memory is nearly empty. As you work, it fills. Each question you ask, each file the assistant reads, each error message it inspects — all of it accumulates in the same window.

The number attached to a model — 200,000 tokens, a million tokens — is the ceiling, not the comfortable working range. A million-token window sounds like it should hold your entire project with room to spare. In practice, the useful zone is much smaller, and the gap between "technically fits" and "the model can actually use it well" is where most non-technical founders get burned without knowing why.

Here's the part that trips people up: the assistant doesn't tell you when its memory is getting crowded. It just quietly gets less reliable. It starts forgetting a decision you made an hour ago. It re-suggests a fix it already tried. It edits the wrong file. From the outside it looks like the model got dumber. It didn't. Its working memory got full, and full memory is degraded memory.

If you're new to working with these tools, my starter guide to Claude Code for non-developers covers the basics of a session. This post is the next layer: what happens to that session as it runs long.

Why quality degrades as the window fills

The intuitive assumption — that a model reads the 10,000th token as carefully as the 100th — is wrong, and it's wrong in a way that has been measured directly. The AI research team at Chroma ran a study across 18 frontier models, including GPT-4.1, Claude Opus 4, and Gemini 2.5, and found that every model's accuracy degrades non-uniformly as input length grows — sometimes dropping sharply well before the documented context limit. They named the effect "context rot."

Anthropic's engineering team describes the same phenomenon and explains the mechanism in its guide to effective context engineering for AI agents. The short version: a model has a finite "attention budget." Every token you add draws it down. The underlying architecture forces each token to relate to every other token, so as the window grows, the model's attention gets spread thinner and thinner across more and more material. Anthropic's framing is worth quoting plainly: context should be treated "as a finite resource with diminishing marginal returns."

Two practical consequences fall out of this.

First, stuffing more into the window doesn't help past a point — it hurts. Pasting your entire codebase into the chat feels thorough. It actually buries the relevant three files under thousands of irrelevant tokens and makes the model less likely to focus on what matters.

Second, information in the middle of a long context gets the least attention. Models tend to attend well to the beginning and end of what they're given and poorly to the middle — so the decision you made an hour ago, now sitting halfway up a long transcript, is exactly the thing the assistant is most likely to lose track of.

None of this is a defect you can prompt your way out of with the right magic words. It's a property of how the models work. Which means the solution is operational, not verbal: you manage the window instead of fighting it. This is closely related to why your AI keeps inventing file paths — a crowded, stale context is fertile ground for confident mistakes.

The three levers that actually work

You have three tools for keeping a session sharp. Anthropic's team calls versions of these compaction, structured note-taking, and sub-agent architectures. Translated into a non-technical founder's daily workflow, they are: reset, persist, and scope.

| Lever | What you do | When it helps most | | --- | --- | --- | | Reset | Start a fresh session (or compact the current one) before quality craters | Long back-and-forth sessions that have wandered across many topics | | Persist | Keep durable state in a file the assistant re-reads (CLAUDE.md, NOTES.md) | Multi-day work where decisions and progress must survive a reset | | Scope | Point each session at one clearly-bounded task | Any time the job is big enough to tempt you into "let's also just…" |

Reset is the bluntest and most effective. When a session has been running for hours and the assistant starts making the mistakes described above, don't push harder — start a new session. Most tools also offer a "compact" command that summarizes the conversation so far and continues with the summary instead of the full transcript. Anthropic describes how Claude Code does exactly this: it preserves the architectural decisions, unresolved bugs, and implementation details while discarding redundant tool output, then continues with that compressed summary plus the handful of most recently touched files. You get continuity without dragging the full, bloated history along.

Persist solves the problem reset creates: if you start fresh, how does the new session know what the old one figured out? The answer is to write it down somewhere the assistant reads automatically. That's what a CLAUDE.md file is for — standing orders and durable project facts the assistant loads at the start of every session. For in-progress work, a running NOTES.md ("we decided X, we're mid-way through Y, don't touch Z") does the same job for transient state. The file is your long-term memory; the context window is just working memory.

Scope prevents the window from filling with unrelated material in the first place. A session that starts as "fix the login bug" and drifts into "also redesign the dashboard and rename these files" ends up holding three unrelated jobs' worth of context, none of them cleanly. One session, one task. When the task is done, reset.

A simple session lifecycle for non-technical founders

Here's the loop I run now, drawn as a decision flow. It's nothing more than the three levers in sequence, but having it explicit is what keeps me from grinding away in a degraded session out of sunk-cost stubbornness.

flowchart TD
    A[Start fresh session] --> B[State the one task]
    B --> C[Work: assistant reads files, makes edits]
    C --> D{Quality still sharp?}
    D -->|Yes| E{Task done?}
    D -->|No| F[Save state to NOTES.md / commit]
    E -->|No| C
    E -->|Yes| G[Commit work, note decisions]
    F --> H[Start a new session]
    G --> H
    H --> B

The decision point that matters is the diamond: is quality still sharp? You don't need to read code to answer it. The warning signs are behavioral, and once you've seen them a few times they're unmistakable:

• The assistant re-suggests a fix it already tried and you already rejected.
• It references a file or variable that doesn't exist, or the wrong version of one.
• It "forgets" a decision you made earlier in the same session.
• Its edits get sloppier — touching files you didn't ask about, or undoing earlier work.
• You find yourself re-explaining context you already gave it.

Any two of those together mean the window is crowded. Save your state, reset, and come back fresh. It feels like giving up progress. It's the opposite — you're trading a degraded three-hour session for a sharp ten-minute one. When a session gets truly stuck in a loop, my debugging playbook for Claude Code covers how to break out of it, and a reset is almost always step one.

How this changes the way you plan work

Once you internalize that context is finite, you start planning around it, and the planning is where the real gains show up.

Break big jobs into session-sized pieces. Before you start, ask: can this be done in one focused session, or is it really three? "Add user accounts" is three sessions — the database change, the sign-up flow, the settings page — not one marathon. Planning the seams in advance means each session stays scoped and sharp. This pairs naturally with writing a clear spec up front, which I get into in prompt engineering for non-developers.

Front-load the durable context. Anything the assistant must never forget — your architecture, your conventions, the mistakes it keeps repeating — belongs in CLAUDE.md, not in a chat message it'll lose on the next reset. Chat is for the task at hand. The file is for the standing truth.

Commit your work often. Saving your progress in version control gives you a clean checkpoint every time you reset. It's the difference between "start fresh from a known-good state" and "start fresh and hope I remember what was working." Even if you never read the code, the habit of committing before a reset is worth building.

Stop treating a fresh session as a failure. This was the mindset shift for me. Early on, starting over felt like I'd done something wrong. Now I see a reset the way a writer sees a new page: not lost work, just a clean surface. The work that mattered is committed or written to a file. Everything else was scaffolding, and scaffolding is supposed to come down.

The founders I see struggle with AI coding tools are almost never limited by the model. They're limited by fighting a full context window for hours instead of resetting it in seconds. This is a workflow habit, and workflow habits are learnable regardless of whether you can write a line of code. It's the same principle behind the whole vibe coding workflow: you don't need to understand every line, but you do need to manage the process well.

What this looks like in a real session

Let me make it concrete, because the abstract version ("manage your context") is easy to nod along to and hard to actually do.

Say I'm adding a password-reset flow. I start a fresh session and state exactly one thing: "Add a password-reset flow — request form, email with a token, reset page." The assistant reads the relevant auth files, and for the first hour it's excellent. It writes the request form, wires the email, builds the reset page. Clean edits, sensible questions, no drift.

Then I get greedy. While it's "in there anyway," I ask it to also tidy up the sign-up page, then to rename a few things, then to chase a styling bug I noticed. Each of those pulls more files into the window. Forty minutes later the assistant suggests a fix I explicitly rejected earlier, and edits a file I never mentioned. That's not a worse model than I had at the start of the hour. It's the same model with a crowded window — I did that to it by piling three jobs into one session.

The disciplined version looks boring by comparison, and that's the point. When the password-reset flow works, I commit it. I jot two lines in NOTES.md: "reset flow done; sign-up page cleanup still TODO; don't touch the email templates, they're final." Then I start a new session for the sign-up cleanup. The assistant re-reads CLAUDE.md, picks up my note, and begins with a nearly empty window pointed at one job. Ten focused minutes instead of another degraded hour.

The trap is always the same shape: a session going well tempts you to keep loading it up, right up until it quietly stops going well. The founders who ship consistently aren't the ones with the best prompts. They're the ones who commit, note, and reset while the session is still sharp — treating the fresh start as routine hygiene, not an admission of defeat.

Frequently asked questions

What is a context window in AI coding tools?

It's the total amount of text an AI model can consider at once during a session — your instructions, the conversation, and every file or command output it has seen. It's measured in tokens and it's finite. When it fills up, the model's reliability drops even though it doesn't warn you.

Why does my AI assistant get worse the longer I work?

Because its context window fills with accumulated messages and file contents, and the model's attention gets spread too thin to use any of it reliably. Researchers call this "context rot," and it has been measured across every major model. The fix is to reset the session, not to push harder.

Does a bigger context window solve the problem?

Not really. A larger window raises the ceiling but doesn't remove the degradation — quality still drops as you fill it, often well before the limit. Anthropic recommends treating context as a finite resource with diminishing returns regardless of the advertised size.

How do I keep important information across a session reset?

Write it to a file the assistant reads automatically. A CLAUDE.md file holds standing project facts and conventions; a running NOTES.md holds in-progress state like decisions made and steps remaining. The file is durable memory; the chat window is just working memory that gets wiped on reset.

How do I know when to start a fresh session?

Watch for behavioral warning signs: the assistant repeats rejected fixes, references files that don't exist, forgets earlier decisions, or makes sloppier edits. Any two together mean the window is crowded. Save your state and reset — you'll get a sharper result in less time.

The habit that pays for itself

Managing your AI coding context window is the highest-leverage habit a non-technical founder can build, and it costs nothing but the discipline to reset before you're desperate. Keep sessions scoped to one task, persist the durable stuff to a file, and start fresh the moment quality slips. Do that and the same model that felt flaky at hour three feels sharp all day.

If you're shipping a SaaS with AI coding tools and want a codebase that's already organized for this — clear regions, a CLAUDE.md in every part of the project so your assistant always has the right standing context — Coding Capybaras is the free boilerplate I built for exactly this workflow.