HOW AN AI IS BUILT

It starts as sand. Purified into silicon, sliced into wafers, printed with circuitry at a scale measured in billionths of a metre, then packaged into the specialised processors that do one thing extraordinarily fast: multiply enormous grids of numbers.

A frontier training run occupies tens of thousands of these chips continuously for months. They are not general computers in any everyday sense — they are arithmetic engines, and everything above this layer is that arithmetic wearing a costume.

This layer also explains the energy question. Each chip generates heat that must be removed, which is why AI is a buildings-and-power-grid problem as much as a software one. The International Energy Agency projected global datacentre electricity demand roughly doubling by 2030 to around 945 terawatt-hours — slightly more than Japan's entire electricity consumption — in its April 2025 Energy and AI report. The agency has since updated that central projection to about 950 TWh, from 485 TWh in 2025.Source: IEA, Energy and AI (World Energy Outlook Special Report), April 2025 — Base Case 945 TWh by 2030, ~3% of global electricity demand; central projection subsequently updated to ~950 TWh. Datacentres consumed ~415 TWh in 2024 (~1.5% of world electricity).

Trillions of words, images and lines of code — scraped from the open web, licensed from publishers, bought, and increasingly generated by other models. Then filtered: duplicates removed, low-quality pages dropped, some categories excluded entirely.

This layer decides what the finished system can possibly know, and it carries most of the field's unresolved fights: who owns the text, whether consent was given, what happens when models begin training on their own output. What goes in decides what can come out — and unlike the layers above it, this one cannot be fixed later.

And the quarry is changing composition. A growing share of what is available to scrape was itself machine-generated — roughly half of newly published articles, by the figures in the quarterly record. Research published in Nature in 2024 showed that models trained repeatedly on their own outputs degrade: rare cases vanish first, then variety collapses, and the failure is easy to miss because performance on common cases can look fine or even improve while the tails rot away. One of the authors put the danger precisely — the surprise is how fast it starts and how elusive it is.

This is not a prediction of doom. Labs filter aggressively, license human-written corpora, and use generated data deliberately where it helps. But it does mean the economics of the open web are now a training-data question: if human pages stop being viable to produce, the quarry that every layer above depends on gets worse.Shumailov et al., "AI models collapse when trained on recursively generated data", Nature, 2024. The economic and legal half — including the copyright rulings that now turn on how training data was obtained — on AI and the open web.

Here is the whole mechanism, honestly: the system is shown a piece of text with the next part hidden, guesses what comes next, and has billions of internal numbers nudged slightly whenever it guesses wrong. Repeat for months.

Nobody writes the rules. Grammar, arithmetic, the shape of an argument, the conventions of a recipe — all of it emerges from correction at scale. Which is also why nobody can point to where a specific skill lives inside the finished model, and why testing it is genuinely hard rather than merely tedious.

The result is a set of frozen numbers — the weights. That file is the model. Everything you have ever seen an AI do came out of arithmetic performed on numbers that were settled during this stage and have not changed since.

A freshly pretrained model is capable and unusable: it will continue any text at all, helpfully or otherwise. Alignment is the work of shaping behaviour afterwards — people ranking outputs, written principles the model is trained against, red teams paid to make it fail, and evaluations run before anything ships.

Two honest caveats belong here. First, this shapes behaviour, not beliefs — the model has none. Second, it is demonstrably incomplete: every frontier system has been talked out of its guidelines by someone, which is why alignment is an active research field rather than a solved checkbox.

The part that gets left out: this layer runs on people. Ranking outputs, writing demonstrations, labelling harmful material and probing for failures are human jobs, done at scale, frequently through outsourced contracts and often by workers in lower-income countries. Some of it involves sustained exposure to the worst material a model might be asked to reproduce.

It is worth naming for two reasons. Practically: the behaviour you experience as a model's "values" was substantially decided by the judgement of people whose names you will never see. Honestly: a technology routinely described as automating work is, at this layer, an employer of a workforce whose conditions are rarely discussed by the people describing it.

Chat windows, APIs, copilots inside other software. Almost nothing here is the model itself — it is packaging: what the system is told before you type, how much of your conversation it can still see, which tools it may reach for, what it refuses.

Worth knowing when a product feels dramatically better or worse than another using the same underlying model: you are usually comparing the wrapping, not the mind. ChatGPT's launch invented nothing fundamental — the architecture was five years old. What changed was a text box anyone could type into.

Systems that do not merely answer but act: browse, write and run code, call other software, take multi-step tasks and pursue them. Structurally this is the same predictor from Layer 03, given tools and a loop.

The reason this layer gets its own name is that the failure modes change. A wrong answer is a wrong answer; a wrong action books the flight, sends the message, deletes the file. Everything unresolved in Layer 04 becomes more consequential here, which is precisely why this floor is still under inspection.