When a team talks about AI cost, it almost always means the supplier invoice. That is the most visible of the three quantities, but it is the same number that governs the other two. A prompt ten times larger than it needs to be does not only cost ten times more money. It takes longer, and it occupies compute in a data centre that draws power for it.
Prefill and decode are two different bills
A request falls into two phases. During prefill the model reads the entire prompt; during decode it produces the answer token by token. The two behave differently. Prefill parallelises well and scales with the length of the input. Decode runs sequentially and scales with the length of the answer. Halve your prompt and you halve the prefill work while leaving decode untouched.
In practice this means the question of whether a system is slow stays too coarse. It becomes useful once you separate time to first token from time to a finished answer. Context cuts move the first number a great deal and the second one hardly at all. We measure them separately because they hang on different screws.
Tokens are not words, and German costs more of them
A token is a fragment of text, usually a few characters long. Common tokenizers are trained predominantly on English, so German words break into more pieces, and with compound nouns the gap widens. The same statement costs noticeably more tokens in German than in English. For a Swiss company working with German documents that is not a detail but a surcharge on every single request. Anyone estimating cost should count on their own texts rather than on an English rule of thumb.
The column missing almost everywhere
There are few reliable figures for energy. Suppliers publish little about what a single request consumes, and the estimates in circulation differ by orders of magnitude because they assume different models, hardware and utilisation. We keep the column anyway, but as an estimate with the assumption written next to it.
In LeanCTX that estimate lives in the code rather than on a slide. We reckon with 0.4 joules per saved token, a deliberately conservative midpoint of measured inference on current hardware, and derive watt-hours from it. The CO₂ figure follows at 475 grams per kilowatt-hour, the global average grid intensity. Anyone on a cleaner mix can override that value so their own number stays honest. For scale: 9,000 saved tokens come to exactly one watt-hour.
Two things keep the estimate honest. It is set low, because what gets removed is mostly cheaper prefill tokens, and it never enters the tamper-evident record that holds the saved tokens. Energy is a derived quantity there rather than a measurement, and an estimate has no business inside a chain of evidence.
A quantity nobody reports is a quantity nobody optimises.
What we can state is relative rather than absolute. If a run needs half as many input tokens, the prefill work falls roughly in proportion. That is not a figure in watts, but it is a direction, and a direction is enough to decide with. We do not claim absolute values, and we think suppliers are the only ones who could publish them credibly.
What we do not claim
A context budget is not a climate programme. Measured against what the industry consumes, the share a single team saves is small. On top of that comes an effect familiar from other fields: what gets cheaper gets used more often, and part of the saving disappears into additional use. Leaving that rebound out would be convenient and dishonest.
Why the number counts anyway
The saving decides who gets to take part. A process that costs one franc per run is a different decision for a local association or a municipal office than for a corporation. Bring the same run down to ten cents and the threshold moves for who can afford this technology at all. The same holds for model size: a system that manages on a fraction of the context can move work onto smaller models, including ones that run on your own hardware.
It is the one quantity in this field that makes work cheaper, faster and less wasteful at the same time, and it costs nothing beyond care. Levers like that are rare enough to be worth pulling.