II · THE IDEA · ARTIFICIAL INTELLIGENCE
The Feedforward Block
▶ Listen · narrated
If you want to know where a model stores facts, look past the attention mechanism. The feedforward block is larger, simpler in structure, and increasingly understood as a kind of memory.
At a glance
- Shape
- Narrow → wide → narrow, applied identically at every sequence position
- Parameter share
- Roughly two thirds of a transformer's total parameters
- Typical expansion
- Four times the model dimension, sometimes more
- What it stores
- Evidence suggests factual associations and key-value patterns
Think of a library where every book is stored as a pattern of shelves and a set of retrieval instructions. You walk in with a question — the narrow input vector. The library expands your question into a wide space where thousands of shelves light up in parallel, each one recognising some aspect of what you are asking. Some shelves activate for the word "France", others for "capital", others for the grammatical structure. Then a second set of instructions reads the pattern of lit shelves and assembles an answer, narrowing it back down to the compact form the next room expects. The feedforward block is that library. The width is what allows many associations to be checked at once. The two projections are the instructions for expanding the question and assembling the answer.
The feedforward block applies two learned linear transformations with a nonlinearity between them, independently at each sequence position. For input dimension d and expansion factor m (typically 4), the first projection is a d×md matrix, the second is md×d. The intermediate activations are md-dimensional. Early transformers used ReLU; more recent architectures use GELU or gating mechanisms. Gated variants such as SwiGLU split the intermediate layer into two md-dimensional pathways, one modulating the other, which roughly doubles the parameter count at that stage but improves performance. Mor Geva and colleagues showed that the feedforward block can be understood as a key-value memory: the first projection maps input vectors to a space where certain neurons activate for certain patterns (keys), and the second projection maps those activations to output adjustments (values). This interpretation is supported by probing studies and by the success of model-editing techniques that locate and modify feedforward weights to change factual outputs. The block is applied identically at every position, so it does not directly model dependencies between tokens — that is the job of attention. However, the input to the feedforward block has already been shaped by preceding attention layers, so it carries contextual information. The block's independence across positions makes it highly parallelisable. In a typical transformer, the feedforward block accounts for approximately two thirds of the total parameters, because the two weight matrices scale as d×md and md×d, while attention scales as d×d per head.
Look closer
The expansion ratio is a design choice, not a consequence
A model with dimension 768 will typically widen to 3072 in the feedforward block, then project back to 768. That factor of four is conventional rather than derived. Some architectures go wider; some use gating variants that effectively double the parameter count at the wide layer. The ratio trades off expressiveness against memory and speed, and it is chosen before training starts.
Each position is processed independently
Attention mixes information across the sequence. The feedforward block does not. It applies the same two-layer network to every token position separately, in parallel. The only communication between positions happens in the attention layers. This independence is part of why the feedforward block is sometimes described as a bank of memories: each input vector queries the same learned associations, without regard to its neighbours.
The middle layer activations are where the patterns live
Mor Geva and colleagues showed that the feedforward block behaves like a key-value memory. The first projection maps the input to a high-dimensional space where certain neurons activate strongly for certain patterns — the keys. The second projection reads out associated information — the values. A neuron in the wide layer might activate for mentions of countries and contribute a continent name to the output, or activate for plural nouns and suppress a singular verb. The width is what allows many such patterns to be stored and retrieved in parallel.
The story
Every transformer layer has two main components. Attention is the first, and it moves information between positions in the sequence. The feedforward block is the second, and it transforms each position independently. The asymmetry in fame does not match the asymmetry in size. In a typical architecture, the feedforward block accounts for roughly two thirds of the parameters.
The structure is simple. Take the representation at one position — a vector of dimension d. Multiply it by a learned matrix to produce a wider vector, often 4d. Apply a nonlinearity. Multiply by a second learned matrix to return to dimension d. Add the result back to the original vector and move on. The same two matrices are used at every position in the sequence, and the same operation is repeated in every layer of the model, alternating with attention.
The width is the key. A narrow vector passes through a wide intermediate space, then narrows again. The expansion gives the layer room to compute many independent features in parallel — different neurons in the wide layer can specialise for different patterns in the input. The contraction forces the layer to summarise what it found into the same compact representation the next layer expects.
For years this block was treated as a black box that added expressiveness. More recently, research by Mor Geva and colleagues reframed it as a key-value memory. The first matrix projects the input into a space where certain dimensions light up for certain patterns. Those are the keys. The second matrix maps those activations to adjustments in the output space. Those are the values. A single feedforward block in a large model might encode thousands of such associations, all retrieved in one parallel operation.
The nonlinearity between the two projections is what makes this work. Early transformers used ReLU, which zeroes out negative values and leaves positive ones unchanged. That created a sharp distinction: a neuron either fires or it does not. More recent architectures use smooth activations like GELU, or gating mechanisms where one pathway modulates another. Noam Shazeer showed that gated variants — where the wide layer is split into two paths, one gating the other — improve performance, though they roughly double the parameter count at that stage.
The feedforward block does not know what came before or after the current position. It sees only the vector it is given. But that vector has already been shaped by earlier attention layers, so it carries context. The feedforward block's job is to recognise patterns in that context-enriched representation and apply learned transformations. Store France, retrieve Europe. See plural subject, inhibit singular verb. The width is what allows many such rules to fire at once without interfering.
Because the block is applied identically at every position, it is also easy to parallelise. Modern hardware is built for exactly this kind of operation: the same matrix multiplication, repeated across a batch of independent inputs. Attention requires more coordination, because every position must look at every other. The feedforward block is faster.
Why it mattered then
The feedforward block appeared in the original transformer architecture in 2017, where it was described simply as a position-wise fully connected layer with one hidden layer. The paper noted that it could be thought of as two convolutions with kernel size one, which was a nod to the convolutional networks that dominated vision at the time. The expansion ratio of four was chosen empirically, and it worked well enough that most subsequent architectures kept it. The block was understood to add capacity — more parameters meant more expressiveness — but there was little theory about what it was doing. It was there because removing it made the model worse.
Why it matters now
The reinterpretation of the feedforward block as a memory has changed how researchers think about model editing, interpretability, and scaling. If factual knowledge is stored in feedforward weights, then locating and updating a specific fact becomes a tractable problem. Techniques like ROME, which edit model behaviour by modifying feedforward parameters, rely on this understanding. The block's size also explains why scaling laws are so expensive: doubling the model dimension means quadrupling the feedforward width, and that is where most of the parameters live. Gating variants and sparse activations are active areas of research, trying to get the benefits of width without the full cost. The feedforward block is no longer a black box. It is the part of the model where knowledge is thought to live, and that makes it a target for both understanding and intervention.
The surprising detail
The feedforward block processes every token in parallel, but the two weight matrices are shared across all positions and all examples. That means the same set of key-value associations is queried billions of times during training, and the weights learn to encode the patterns that are most useful on average. A single neuron in the wide layer might activate for dozens of unrelated surface patterns that happen to benefit from the same output adjustment. The network does not learn tidy, human-interpretable rules. It learns whatever statistical regularities reduce loss, and the width is what allows it to pack thousands of overlapping, approximate associations into the same set of weights. Interpretability research has found neurons that correspond to clean concepts — one that fires for code, one for legal language — but most neurons are polysemantic, firing for a strange mixture of contexts that make sense only in the high-dimensional geometry of the model's learned representations.
Remember this
Two thirds of the parameters are here, not in attention. The widen-then-narrow shape is where models store learned associations.
Test yourself
A researcher wants to edit a factual error in a model — changing the capital it associates with a country. Why is the feedforward block a more promising target than the attention weights?
Attention moves and combines information that is already present in the residual stream, but it does not typically store factual associations itself. The feedforward block, by contrast, is understood to encode key-value memories: the first matrix recognises a pattern, the second retrieves associated information. A fact like "France → Paris" is more likely to be stored as a learned transformation in a feedforward layer than as a relational pattern in attention. Empirically, techniques that locate and modify specific feedforward weights have succeeded in changing a model's factual outputs without catastrophic side effects, which supports the memory interpretation. Attention weights are also shared across many different factual relationships simultaneously, so editing them is more likely to have unintended consequences.
Go deeper
- Transformer Feed-Forward Layers Are Key-Value Memories · arXiv · Mor Geva et al. · 2020-12-29
- GLU Variants Improve Transformer · arXiv · Noam Shazeer et al. · 2020-02-12
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.