In conversations with enterprise and token factory technology leaders, I keep watching the same moment repeat. Ask how their AI infrastructure is performing, and the answer comes back in GPUs: cluster size, availability, and spend. Ask what each token costs, how many tokens are delivered within service-level objectives (SLOs), or how much GPU investment becomes billable output, and the room goes quiet.
Access to compute resources got AI into production. The harder challenge now is converting those compute resources into token output efficiently, predictably, and economically. A high GPU utilisation rate can sit right alongside slow responses, missed SLOs, and expensive tokens. It tells you the GPU is occupied; it does not tell you whether the system is turning that time into the output it is supposed to deliver.
A question that has quietly changed
A few years ago, the questions were about acquisition: How many GPUs do we need, and how fast can we get them? Today the conversation centres on production economics. Enterprise AI teams ask how to reduce the cost of running AI on infrastructure they already own or rent.
Token factories ask how to turn the same cluster into more dependable, billable tokens. One is optimising cost; the other is optimising billable output. Both have arrived at the same discipline: token economics. GPU clusters are no longer just scarce assets to acquire; they are production systems whose economics have to be managed every day.
Why token economics matters
The shift follows AI’s move into continuous inference. Cost per token and tokens per GPU-hour reveal what GPU counts cannot: how efficiently infrastructure turns investment into production output.
Token economics does not try to judge the value of an AI application. It shows how economically and predictably the infrastructure delivers that application’s output.
Three levels of AI infrastructure measurement:
- Level 1: Scale, or “How much do we have?” Measured in accelerators, memory, and theoretical performance.
- Level 2: Utilisation, or “Is it busy?” Measured in utilisation and availability.
- Level 3: Infrastructure yield, or “How efficiently does paid GPU capacity convert into delivered token output?” Measured in cost per token, tokens per GPU-hour, and tokens delivered within SLOs.
Scale shows how much capacity exists. Utilisation shows whether the cluster is busy.
Infrastructure yield shows how efficiently that activity converts into delivered token output within cost and SLOs. A GPU can be highly utilised while waiting on memory, repeating work, or processing an inefficient workload mix. Yield connects activity to output, SLOs, and cost.
Where infrastructure yield is won or lost
Identical accelerators can produce very different economics because efficiency emerges from the whole serving system. Inference cluster architecture, batching, scheduling, memory, cache placement, and concurrency all shape how much token output a GPU-hour actually yields.
In large-model inference, during the prefill phase that processes a large amount of context, subsequent prefills can theoretically reuse the key-value pairs generated by previous prefill requests. Re-computation should only be performed when the key-value pairs are not saved in the cache or when the cache misses. Otherwise, a GPU can be busy repeating work while requests wait in line. As a result, it delivers fewer tokens than the hardware is capable of producing.
Picture two clusters running the same model on the same accelerators. One keeps data flowing, batches and schedules requests well, and minimises repeated work. The other loses GPU time to data stalls, poor scheduling, and re-computation.
Both can report the same utilisation, yet one delivers far more tokens per GPU-hour and, therefore, a lower cost per token. Utilisation shows that the GPUs are busy; infrastructure yield shows what that activity delivers.
Infrastructure yield measures production efficiency.
Improving yield is rarely one optimisation. It might mean removing a redundant transfer, orchestrating resources, or scheduling work so resources are not stranded.
The goal is not to make a single tier faster in isolation; it is to raise the token output delivered within cost and SLOs. Yield is a system property. It cannot be read off one benchmark or one infrastructure tier; it has to be observed across the whole path, from request admission to token delivery.
Different objectives, one infrastructure discipline
For enterprises, higher yield means a lower cost per token and more from every owned or rented GPU-hour. For token factories, it means more dependable, billable tokens from the same GPU cluster.
One side measures the saving, the other the output, but both depend on converting more paid GPU capacity into tokens delivered within cost and SLOs. For an enterprise, that can lower unit costs, defer new purchases, and support more internal users without more investment. For a token factory, the same gain increases saleable tokens, revenue per GPU-hour, and confidence in service commitments.
A shift in the questions leaders ask
The practical change is to tie infrastructure reviews to conversion. Ask: What does each token cost? How many tokens does each GPU-hour deliver within SLOs? Instrument output, performance, and cost together.
More hardware fixes a compute shortage; it does not correct inefficiency elsewhere in the system. A rigorous review sets paid GPU-hours against the tokens actually delivered inside the agreed service envelope.
GPU utilisation tells leaders whether a GPU is active. Infrastructure yield tells them what that activity produces and what it costs. The chips set the ceiling; yield determines how close each organisation gets to it.
