Ask
the work.

CONCEPT / DISCOVERY ASSOCIATIONS

4 records tagged ffn. These are discovery associations from editorial tags or film metadata, not evidence that the records agree.

EDITORIAL DRAFT / QUESTION · OPEN · V0.1

What if the FFN is a graph? The question is about whether the representation makes something useful possible: addressing, traversing or explaining the computation.

EDITORIAL DRAFT / OBSERVATION · OPEN · V0.1

01 / WHAT IS MOVING?. A transformer starts with tokens: pieces of the words in a prompt. It gives each position a vector of numbers, then repeatedly updates those vectors. This changing working state is the residual stream. In the Gemma 3 4B demonstration, each position has 2,560 numbers and passes through 34 layers. The learned weights stay fixed while the state changes. Attention lets a position read information from earlier positions; the feed-forward network, or FFN, applies a learned transformation at that position.

EDITORIAL DRAFT / CONNECTION · OPEN · V0.1

Try a memory you can inspect yourself. This small, constructed FFN lets you choose an address, follow its key matches and change the calculation by switching off a neuron.. Try the mechanism — interactive memory study

EDITORIAL DRAFT / OBSERVATION · OPEN · V0.2

01 / WHAT IS AN ADDRESS?. Ask for the capital of France. France identifies the entity; capital identifies the relation you want to read. Currency would ask something different about the same place. In the opening animation, that question arrives at an FFN and a Paris direction is written into the residual stream—the model’s changing working state. The conveyor is an explanatory drawing. The address in a model is a vector of numbers, and matching it need not isolate a single neuron or a single fact.

EDITORIAL DRAFT / OBSERVATION · OPEN · V0.2

02 / BUILD A MEMORY WITH SIX FACTS. The Mechanism’s ffn.py plants capital, currency and language facts for two invented places. Atlantis has Paris, Euro and Latin; Zerivia has Cairo, Rand and Tamil. Each full question receives its own random, normalised 24-number key. Each answer receives a value direction. We put the six keys into the rows of an input matrix and the corresponding values into an output matrix. There is no training: we choose what this little memory contains.

EDITORIAL DRAFT / OBSERVATION · OPEN · V0.1

THE THREAD. Start with the representation. Ask what can be addressed. Specify a traversal, then compare it with the original computation. Keep the distinction between an explanatory picture and an operational result.

AUTOMATIC CAPTIONS / 7:26

geographic neighbors as well. So, let's go a little bit raw here. That was a pretty clean view. Let's look at the feedforward network underneath the hood there. Again, everything we're looking at is the FFN, which is really the knowledge store where all of the data stored. Attention is slightly different. We'll talk about that later. But if I do something like show features, and again we'll limit it to layer 26. Feature two is really

AUTOMATIC CAPTIONS / 8:44

stores that within the weight? So, basically, a feature is a single column in the FFN. It's one gate vector that decides when it fires, and one down vector that decides what it outputs. Gate times down. That is basically the edge. So, it's an edge in the graph. And the gate is a direction in the residual stream. I'm not going to go into too much detail of what a residual stream is. I've

INTERACTIVE STUDY / CONSTRUCTED EXAMPLE

Choose a planted address, inspect its key matches and suppress a neuron in a constructed six-fact FFN. The browser study makes the calculation inspectable; it does not run a trained language model.

CURATED THREAD / WORKING EDITION · V0.1

What can a model’s changing state tell us about the way it reads? Follow the map film through the visual notebooks, studies you can work yourself, and the questions behind LARQL and VINDEX3. A curated reading order, rather than a chronology of discovery. Films keep their original dates. Notes and dossiers remain working drafts. Interactive studies are labelled as either constructed examples or replays of recorded results. WATCH THE STATE MOVE. Begin with the capital of Japan. The film follows the changing residual state, then asks what happens when a France state enters an Australia computation. ASK WHAT THE PICTURE MEANS. Which token position is moving? What are the axes? The notebook reads the demonstration alongside its projection code and distinguishes a useful picture from an answer probability. CHANGE THE STATE. Replacing one position and replacing the full sequence are different interventions. Follow the transplant evidence into the question of what must persist after the next word. EXAMINE THE READER. A state becomes useful when an operation can read it. Follow key matches, activations and value contributions, then compare the constructed memory with the native-model experiments. MAKE AN INTERVENTION. Choose capital of Atlantis. Inspect the competing activations, remove a neuron and watch the answer scores change. Every operation in this little memory is visible. ASK WHICH ONE ANSWERS. A stored memory eventually holds two things that disagree. Follow the recorded arc from a record that replaces a retired source, to one that is completely inert against a source still being read. RETIRE A READ. Eight attention layers can see the whole context. Switch one off and the newer record takes the answer. The study replays what was measured, and stays silent about what was not. WATCH THE ADDRESS TAKE SHAPE. Return to native facts. A final-position transplant and an independent reader locate the entity transition between L24 and L28. Explore the early relation frame and changing coordinates, with the answer-token endpoint marked explicitly. TURN THE QUESTION TOWARDS SOFTWARE. LARQL explores how learned systems can be queried. Return from the small memory to the larger engineering question: which operations can a model reliably expose? ASK HOW TO REPRESENT IT. If a model has parts we can address and relationships we can inspect, how should we represent them? VINDEX3 gives that question its own system and specification.