What Is a Context Window? Why Bigger Is Not Always Better

A context window is the working memory of a single model request. Anthropic defines it as "all the text a language model can reference when generating a response, including the response itself" (Anthropic context windows docs). It is not the model's training data, and it is not unlimited. The size of that window, and what you put in it, decides both the bill and the quality of the answer.

TL;DR

• The context window holds everything the model can reference, including its own output.
• System prompts, messages, tool results, images, documents and tool definitions all consume it.
• Current Claude models such as Opus 5.5, Sonnet 5.5 and Haiku 5.5 have a 1M-token window; other models such as Sonnet 4.5 have 200k.
• More context is not automatically better: accuracy and recall degrade as the input grows.
• Caching changes the price, not the space, so plan for compaction and trimming.

Disclosure: ZeroRaised publishes this guide and is a launch site for bootstrapped products. Facts come from the linked sources as checked on 2 Nov 2026; ZeroRaised has no commercial relationship with the companies named.

Context window is the total amount of text a language model can reference when generating a response, including the response itself. Context rot is the drop in accuracy and recall that appears as the token count of the input grows.

What counts toward the context window?#

Everything in the request counts. Anthropic lists the system prompt, the messages including tool results, images and documents, the tool definitions, and the output including extended thinking (Anthropic context windows docs).

That list is longer than most builders assume. A tool schema you added months ago is still there on every call. An image is not free, and extended thinking is part of the output budget. If you only count the user message, your estimate of the window is far too low.

How large is a context window in 2026?#

Anthropic says current Claude models, including Opus 5.5, Sonnet 5.5 and Haiku 5.5, have a 1M-token context window. Other models, such as Sonnet 4.5, have 200k. A single request can generate up to 128k output tokens (Anthropic context windows docs).

Those numbers describe capacity, not a target. A 1M-token window means the model will accept the request, not that it will use the material well. The same page warns that "more context isn't automatically better" and that accuracy and recall degrade as token count grows (Anthropic context windows docs).

There are also media limits: up to 600 images or PDF pages for the large-window models, and 100 for the 200k models (Anthropic context windows docs).

What is context rot?#

Context rot is the quality loss that comes with longer input. Anthropic warns that accuracy and recall degrade as token count grows (Anthropic context windows docs).

Chroma's research put a number on the pattern. It evaluated 18 models and found performance declines as input length grows, even on simple tasks. Degradation was worse when the question and the relevant text had lower similarity, and worse again when distracting content was present (Chroma context rot research).

The similarity detail matters: in practice, that argues for sending material close to the question rather than padding the prompt with loosely related text.

Why does more context hurt accuracy?#

Our read: more tokens give the model more chances to attend to the wrong thing, and a competing or outdated passage can pull the answer off course.

The cost problem runs alongside it: every token is billed as input.

This is why "just send everything" is a bad default. Retrieve and rank, then send a smaller set. The RAG guide covers that pipeline.

Does prompt caching free up space?#

No. Anthropic is explicit: "prompt caching changes what you pay for those tokens, not whether they count" (Anthropic context windows docs).

This is the most common misconception. Caching lowers the cost of a repeated prefix, but the prefix still occupies the window. A cache hit does not make a long prompt shorter.

The prompt caching guide covers how to structure a prompt for hits. Use caching to make a stable prefix cheaper, not to pretend it is absent.

What are output limits and why do they matter?#

The output is part of the context, and Anthropic caps a single request at 128k output tokens on the current Claude models (Anthropic context windows docs). Extended thinking counts toward that output budget.

For a product, the output limit sets the maximum size of a generated report or code change, and a large output leaves less room for input. If your feature needs a long response, keep the input side lean.

Long context is billed at standard pricing, except on Haiku 5.5, where prompts over 100,000 tokens cost more (Anthropic context windows docs).

How do compaction and context editing help?#

When a conversation outgrows the window, you have two practical options. Server-side compaction summarises earlier parts of the conversation so it can continue past the limit, and it is in beta. Context editing lets you clear old tool results and thinking blocks (Anthropic context windows docs).

Compaction is lossy: it replaces detail with a summary, right for a long chat and wrong for an audit trail. Context editing is more surgical: drop old tool output, keep the decision it supported.

How do you stay under the limit?#

Start with measurement. Anthropic provides a token counting API to estimate a request before sending it, and an input that alone exceeds the window returns a 400 error saying the prompt is too long (Anthropic context windows docs).

Then apply a few habits:

1. Trim the system prompt. Long policy text is paid for on every call.
2. Send fewer retrieved chunks. Rank first, then include the best few.
3. Prune tool definitions. Definitions count against the limit, so remove tools a step does not need.
4. Cap the output. A shorter response leaves more room and costs less.
5. Summarise old turns. Use compaction or your own summary when a conversation gets long.

What this means for bootstrapped founders#

The context window is a cost and quality control, not a feature to advertise. A product that sends the right few thousand tokens can beat one that stuffs in a million.

Treat context as a design decision: what the model must see, what it can retrieve on demand, and what to summarise away. Measure quality on your cases, because context rot is task-specific. The LLM evaluation guide covers a small test set, and the hallucination guide covers grounding and verification.

If you price an AI feature, a large window does not make cost predictable. Long prompts cost real money on every call, and Haiku 5.5 charges more above 100,000 tokens. The LLM API comparison helps you weigh window size against price.

Where ZeroRaised fits#

ZeroRaised is a launch and discovery site for products built without institutional venture capital or accelerator equity. If you have an AI product people can try, submit it here and state what it sends to the model and what it costs.

The ZeroRaised newsletter carries new analysis like this. We make no promise of traffic or results.

Sources#

• Anthropic: Context windows
• Chroma: Context rot research