THE CONNECTED RECORD
Find the question. Follow the work. Return to the source.
4 THREAD · 4 SYSTEMS · 31 NOTEBOOK ENTRIES · 2 QUESTIONS · 242 FILMS · 239 CHAPTERS
Search the authored text, its questions and refusals, and the films around it. Drafts keep their status. Chapters identify a topic and a time; only the 2 indexed transcripts provide spoken passages, and their automatic captions are unreviewed. This edition retrieves sources rather than generating an answer. Explore what is connected ↗
4 RESULTS FOR “FFN graph”
EDITORIAL DRAFT / QUESTION · OPEN · V0.1
What if the FFN is a graph? The question is about whether the representation makes something useful possible: addressing, traversing or explaining the computation.
READ THIS DRAFT ↗EDITORIAL DRAFT / OBSERVATION · OPEN · V0.1
THE THREAD. Start with the representation. Ask what can be addressed. Specify a traversal, then compare it with the original computation. Keep the distinction between an explanatory picture and an operational result.
READ THIS DRAFT ↗AUTOMATIC CAPTIONS / 7:26
geographic neighbors as well. So, let's go a little bit raw here. That was a pretty clean view. Let's look at the feedforward network underneath the hood there. Again, everything we're looking at is the FFN, which is really the knowledge store where all of the data stored. Attention is slightly different. We'll talk about that later. But if I do something like show features, and again we'll limit it to layer 26. Feature two is really
ORIGINAL SOURCE ↗AUTOMATIC CAPTIONS / 8:44
stores that within the weight? So, basically, a feature is a single column in the FFN. It's one gate vector that decides when it fires, and one down vector that decides what it outputs. Gate times down. That is basically the edge. So, it's an edge in the graph. And the gate is a direction in the residual stream. I'm not going to go into too much detail of what a residual stream is. I've
ORIGINAL SOURCE ↗