Every AI system has a context budget, whether it keeps one or not. Each request carries a window of finite size, and everything competing for it spends from the same account: instructions, documents, history and tool output. Most teams never look at the ledger. They notice the balance only once it is gone. Answers get vaguer, latency climbs and the bill arrives.
The invisible expense
Unbudgeted context fails quietly. A retrieval step that returns forty snippets instead of six does not throw an error. It simply buries the two lines that mattered under thirty-eight that did not. The model then does what models do with excess: it averages. The answer is not wrong enough to alarm anyone, but it is worse than it had to be and costs several times as much.
A longer window is not free
The cost of a long prompt does not sit only on the invoice. A transformer relates every token to every other one, so the compute grows quadratically with the length of the input. Implementations such as FlashAttention cut memory traffic considerably but do not change that underlying quantity. Then there is the KV cache holding the intermediate states of the tokens so far. It grows linearly with length and occupies server memory that other requests then lack. A generously filled window therefore raises the cost of your own request and lowers how many requests the same machine can serve at once.
More context does not automatically make answers better
The common reflex is that extra context does no harm as long as the window can hold it. Measurements say otherwise. The much-cited study "Lost in the Middle" by Liu and colleagues from 2023 systematically varied where in a long input the needed information sat. The result is a U-shaped curve: models find the information reliably when it stands at the beginning or the end, and considerably less reliably when it sits in the middle. That held even for models built explicitly for long contexts.
An uncomfortable consequence follows. Embedding the two decisive lines among thirty-eight others lowers the chance that they get used at all. Additional context is not neutral, it dilutes what matters. Position within the prompt is therefore a design decision and not a detail.
What a system reads is a decision. Failing to make it is still a decision.
What a budget changes
A budget is not a cap. It is a question asked before every expense: what does this task actually need? Anyone who asks it selects more deliberately, because material has to earn its tokens, and gives that material a shape which carries the same meaning in less space. Provenance comes free, because a system that decides what it reads can also say what it read.
Why no savings figure appears here
A number would sit well at this point, and we deliberately give none. We withdrew our earlier LeanCTX benchmark figures because they did not meet our own requirements, and they are explicitly no longer available for product copy. A gain only counts here when the same workload has a known baseline and treatment, a declared quality threshold and visible methodology. A task that gets cheaper while failing is not a gain.
What the tool contains instead is the bookkeeping. Every saving is recorded per run, in a hash-chained record whose integrity can be verified. The number that ends up on the table is therefore yours rather than ours. That is less catchy than a percentage on a landing page and considerably harder to dispute.
The objection
Models get cheaper and windows get longer. Why budget a resource whose price falls every quarter? Because the budget was never mainly about money. A window is attention, and attention does not scale with its container. The more a model is given, the less any single line weighs, and the U-shaped curve does not disappear because the window grew.
This is the argument LeanCTX is built on, and the bar our own numbers are measured against.