What is the map actually a map of?

01 / WHAT IS MOVING?

A transformer starts with tokens: pieces of the words in a prompt. It gives each position a vector of numbers, then repeatedly updates those vectors. This changing working state is the residual stream. In the Gemma 3 4B demonstration, each position has 2,560 numbers and passes through 34 layers. The learned weights stay fixed while the state changes. Attention lets a position read information from earlier positions; the feed-forward network, or FFN, applies a learned transformation at that position.

A VISUAL STUDY / DIRECTIONS ADDED TOGETHER

In this constructed animation from The Mechanism, capital, currency and language directions add into one packed vector. It makes a basic operation visible: several contributions can coexist in the same list of numbers. These are hand-built six-dimensional vectors, rather than the measured Gemma trajectory in the opening film. The next question is what a particular reader can recover from a combined state.

02 / HOW TO READ THE PICTURE

The dot follows the residual at the last token position, where the next-token prediction is read. Tokyo, Paris and the other landmarks come from the model’s output directions for selected tokens. The plotting code chooses three axes from those directions and projects the much larger state onto them. It also normalises the projected points onto a sphere. That gives us a useful view of a route, but leaves most dimensions out of sight. Apparent distance in this picture is not the model’s answer probability.

LAYERS AND TOKENS ARE TWO DIFFERENT CLOCKS

For “The capital of Japan is”, the picture moves towards Tokyo as we advance through the layers of one prediction. After Tokyo is generated, the input has changed: the model now has another token to work with. It runs through the layers again to predict what follows, perhaps punctuation. The dot can therefore move away from Tokyo without the model having forgotten Japan’s capital. We are watching successive computations, each with its own trajectory.

03 / WHY SYDNEY APPEARS BEFORE CANBERRA

The terminal view asks a related question: if we tried to read an answer here, before the model finished, what would it be? It applies the final normalisation and output reader to intermediate residuals—a logit lens. In the Australia example, Sydney appears around layer 24; Canberra appears by layer 26 and remains the final answer. The model has not printed Sydney and then corrected its sentence. These are provisional readouts from inside a single prediction.

04 / CHANGE THE STATE, CHANGE THE ANSWER

The baseline prompt is “The capital of Australia is”. A separate pass computes the state for “The capital of France is”. The demonstration places that France state into the Australia computation and lets the remaining layers run. The answer becomes Paris. An early swap and a later swap near layer 26 both make the point: the remaining computation responds to the state it receives, even when the prompt displayed above it still says Australia.

THE SIZE OF THE INTERVENTION MATTERS

The script replaces all token positions at the chosen boundary, not just the final position’s 2,560 numbers. That is essential context for the claim. Later attention can still read other positions, so changing one vector and changing the whole sequence are different experiments. The follow-up full-sequence transplant records the donor’s next-token distribution exactly in three tested conditions at layers 14 and 26. This supports a statement about those forward passes. It does not establish that one small vector retains an entire conversation through future generation.

05 / WHAT WROTE THE ANSWER INTO THE STATE?

The map shows a route. KV Anatomist lets us inspect contributions to it. Its layer-by-head heatmap uses direct logit attribution: it scores how an attention head’s output points towards a chosen answer token. Selecting a head exposes its output projections, and the injection view compares a modified run with the original. This is more specific than asking where attention looked. A large attribution is a clue to investigate; changing or removing the head is a separate intervention.

KNOWLEDGE IN THE WEIGHTS; INFORMATION IN THE PROMPT

A familiar capital-city question can draw on associations learned during training. The film then supplies an invented fact in a document: Zarkov Industries was founded in Voltara. Now the answer can come from the supplied text. The demonstration traces a strong answer contribution to a late attention head. This gives us two concrete situations to compare—recalling a familiar fact and using new context—without assuming every fact always belongs to one isolated head or pathway.

06 / WHAT DOES THE EIGHT-BYTE INJECTION CONTAIN?

The injection script makes the intervention unusually small: an answer-token ID and a coefficient. It already specifies “Volt”, derives a signal from the document-conditioned run and strengthens it for the injection. The model can then be steered towards that answer without receiving the whole document in the recipient prompt. The interesting result is that a small, correctly placed signal can change the readout. It is not an eight-byte encoding from which arbitrary facts in an unknown document have been recovered.

AND THE 370,000-TOKEN DOCUMENT?

Later in the film, the Apollo example asks who won a porridge-eating contest. The system selects a stored window, loads its retained state and tokens, reads the relevant passage, and answers John Coyle with 23 bowls. The useful sequence is route → load → read → answer. The passage is still part of the mechanism. This connects the map to a practical memory system: how do we find the right place to resume reading, and what must that place retain?

07 / DOES A COORDINATE HAVE A MEANING OF ITS OWN?

MAP-1 tests a simple way of getting this wrong. Reorder the hidden coordinates but leave the FFN weights untouched, and the calculation changes. Reorder the coordinates and the matching weights together, and the standalone output returns, within numerical precision. Think of rearranging spreadsheet columns while updating every formula that reads them. The useful relationship survives the relabelling. A numbered coordinate does not acquire a universal meaning independently of the operations that consume it.

CAN A NEIGHBOURING LAYER READ THE SAME MAP?

That is a harder question than renaming coordinates. MAP-2b substitutes a neighbouring layer’s FFN at the real model boundary. First it fixes the earlier baseline so putting the original FFN back is bit-identical to the unmodified run. Across eight prompts, matched random perturbation performs better than the neighbouring-layer swap on six, worse on one, and ties on one. The sample is small (two-sided sign test p = 0.125). It gives us no dependable rule that nearby layers are interchangeable readers.

08 / FROM A PICTURE TO A MECHANISM

The Mechanism continues the investigation with something we can build and inspect. Its small FFN plants six address–value pairs by hand: a query activates key directions, and their activations combine value directions into an output. The saved example recovers all six answers. This constructed memory makes the operation tangible. The repository also contains separate native-model experiments; the toy’s clean addresses are not themselves evidence that a trained model organises every fact this way.