What AI Actually Runs On: sand, power, and heat
Descend from silicon die to power grid: why GPUs, what memory bandwidth means, how a training run works, and the…
The Assembly upstairs shows what a model is. This station descends through what it is made of: sand, gold, water, and roughly a nation's worth of electricity. Five levels down. Mind the heat.
Why this chip and not your laptop's
A CPU is a brilliant generalist: a few powerful cores doing complicated things one after another. A neural network needs the opposite — one simple thing (multiply, add) done billions of times simultaneously, because a model's "thinking" is just enormous grids of numbers being multiplied together. GPUs, born for videogame pixels, happen to be exactly that: thousands of small cores in parallel. That accident of history is why one company's green silicon became the picks-and-shovels of the entire gold rush.
The die's partner is memory bandwidth — how fast weights can be fed to those cores. Modern accelerators stack high-bandwidth memory physically next to the die because the bottleneck is usually not computing the numbers; it is carrying them. When the syllabus tells you VRAM decides what runs on your machine, this level is why.
Density is the whole story
Accelerators are ganged eight-plus to a board, boards stacked into racks, racks laced together with interconnect fabric fast enough that thousands of chips can behave like one giant machine — a training run is exactly that: one calculation smeared across a warehouse for weeks. The consequence is heat density with no precedent in computing: the IEA measured AI rack power density rising 11-fold from 2020 to 2025, projecting a further fourfold jump by 2027 — at which point a single refrigerator-sized rack draws power equivalent to about 65 households. Air cannot carry that heat away anymore; liquid cooling — plates and pipes against the silicon — became standard equipment, not exotica.
Where the pour happens
Gather the racks and you get the hall: a large data centre can consume as much electricity as 100,000 households — the IEA’s own phrasing — and the agency states that the largest currently under construction could consume as much as 2 million households. Training happens here (the weeks-long pour The Assembly describes), and then inference happens here forever after — every chat, every image, every agent, each sip small on its own: the IEA's 2026 estimates put a standard assistant request around 1.1 Wh and a heavy reasoning-agent request near 50 Wh, while a disclosed median short text prompt measured 0.24 Wh and 0.26 ml of water (Google technical paper, 2025). Small sips; unimaginable frequency.
The part everyone forgets is thermodynamics
Every watt in becomes heat out; the only questions are how efficiently (the industry metric PUE — total facility power divided by computing power — with modern halls pushing toward 1.1) and what carries it away. Often: water, through evaporative cooling — which is why hyperscaler water disclosures rose sharply through 2024–25 and why closed-loop liquid designs, which cut direct water use dramatically at higher capital cost, are spreading. Cooling is now its own line on the world's bill: Gartner forecasts cooling electricity alone climbing 22.6% in 2026 to 195 TWh.
Re-verified against the issuing organisation this month: the IEA's projection of ~945 TWh of datacentre electricity demand by 2030 (April 2025 Energy and AI report; the agency has since updated its central projection to about 950 TWh, from 485 TWh in 2025), and the comparison that a large datacentre can consume as much electricity as 100,000 households, with the largest under construction reaching 2 million. The remaining figures on this page — rack power density, the 2026 cooling forecast, and the year-on-year growth rates — carry their original source and date but were not re-checked in this cycle. They are marked here rather than quietly presented as equally fresh.
The numbers, dated and sourced
Two honest framings, held together: an individual query costs less electricity than boiling water for tea — and the buildout in aggregate is the largest new load the grid has met in a generation, concentrated in a handful of regions whose bills and build-outs are already reshaped by it. Both sentences are true. Any page that gives you only one of them is selling something.
Ascend
You have seen the body. The mind it runs is poured upstairs at THE ASSEMBLY; what it costs the internet is the museum above that; and whether it should live in these halls or on your desk is weighed, without sides, in Local or Cloud.