II · THE IDEA · ARTIFICIAL INTELLIGENCE
Architectural Limits
▶ Listen · narrated
Scale has delivered remarkable capabilities, but some problems require a number of reasoning steps that exceeds the number of layers. No amount of training fixes that.
At a glance
- The constraint
- Each layer can perform roughly one step of serial reasoning
- What breaks first
- Tasks requiring more composition steps than the model has layers
- Training cannot help
- This is an architectural ceiling, not a data problem
- Known since
- Theoretical work from 2019 onward, empirical confirmation ongoing
Imagine a relay race where each runner can only pass the baton once. If the course requires thirteen handoffs and you only have twelve runners, the race cannot finish, no matter how fast each runner is. A transformer is similar: each layer processes information and passes it to the next layer, and if a problem requires more steps than there are layers, the model runs out of room. You cannot make the runners faster to compensate for a missing runner, and you cannot make the layers wider to compensate for a missing layer. The number of steps is the constraint.
A transformer of depth L processes information through L sequential layers, each applying self-attention and a feedforward network. For tasks requiring more than L steps of serial computation, the model cannot represent the full solution, because it has no mechanism to revisit earlier layers with updated information. Merrill et al. proved that certain formal languages and arithmetic tasks require depth proportional to the problem size, and that finite-depth transformers cannot solve them in the general case. Dziri et al. confirmed this empirically: accuracy on compositional reasoning tasks remained stable as chain length approached layer count, then collapsed sharply when chain length exceeded it. Width, parameter count, and training time do not compensate. The depth is the budget for sequential reasoning steps, and it is fixed at architecture time. Low-precision arithmetic compounds the issue, as rounding errors accumulate across layers, making exact intermediate values unattainable even when depth is theoretically sufficient.
Look closer
Depth sets a step budget
Each transformer layer processes its input in parallel across all positions, then passes the result to the next layer. If a task requires ten sequential logical steps, a model with fewer than ten layers cannot reliably solve it, because it has nowhere to perform the intermediate steps. The model can approximate, guess, memorise common cases, but it cannot execute the full chain. This is not about training time or dataset size. It is about the number of times information can flow vertically through the stack.
Compositionality is the test case
Dziri and colleagues tested transformers on tasks that required chaining functions: given a set of simple rules, apply them in sequence to reach an answer. They found that models performed well when the chain length stayed within their depth, but accuracy collapsed when the required steps exceeded the number of layers. The failure was not gradual. It was a cliff. Importantly, the models had seen similar problems during training; this was not a matter of unfamiliar structure. The architecture itself was the bottleneck.
Precision compounds the problem
Merrill and co-authors showed that even when depth is sufficient in principle, the use of low-precision arithmetic, common in large models for efficiency, introduces another hard limit. Each layer's computation incurs rounding error, and those errors accumulate as information moves through the stack. For tasks requiring exact intermediate values, such as certain arithmetic or formal reasoning problems, the cumulative noise can make a correct answer unattainable, regardless of how well the model has been trained. The parallelism that makes transformers fast also makes them bad at anything that must happen step by step.
The story
A transformer is a stack of layers, and each layer operates on all positions in the sequence at once. This parallelism is why transformers are fast and why they scale well to long contexts. But it also means that anything requiring a sequence of dependent steps, where step three cannot begin until step two finishes, must somehow distribute those steps across the layers. Each layer gets one chance to move the computation forward before handing off to the next.
If the task requires more steps than there are layers, the model runs out of room. It has no recursion, no loop, no way to revisit a layer with updated information from further down the chain. The depth of the network is the depth of reasoning it can perform.
This is not speculation. Merrill and others have constructed formal proofs showing that certain classes of problem cannot be solved by a transformer of fixed depth, no matter how wide the layers are or how much data the model sees. The limitation is structural. Dziri's empirical work confirmed it: when they gave models compositional reasoning tasks that required chaining more steps than the model had layers, performance did not degrade gracefully. It fell off a cliff. A twelve-layer model could handle twelve-step problems reasonably well. Thirteen-step problems broke it.
The width of the layers does not help. You can make each layer enormously wide, giving it millions of parameters to work with, but that does not create an additional reasoning step. It just makes each step more expressive. If the task requires a thirteenth step and you only have twelve layers, no amount of width compensates.
This matters because many real reasoning tasks are inherently compositional. Understanding a sentence with nested clauses, following a chain of logical implications, executing a multi-step plan: these all require threading information through multiple dependent operations. A model that can only take twelve steps will struggle with the thirteenth, and training longer will not fix it.
Why it mattered then
The theoretical work emerged as transformers began to dominate natural language processing, and researchers wanted to understand not just what they could do but what they could not. The proofs mattered because they separated architectural limits from training limits. If a model fails at a task, is it because we have not trained it enough, or because the architecture itself cannot represent the solution? Merrill's work on log-precision transformers and Dziri's experiments on compositionality both appeared as the field was scaling models aggressively, and both offered a caution: depth is not fungible with width or data. Some problems require a minimum number of layers, and no other resource substitutes for it.
Why it matters now
The limitation has become practical. As models are deployed for reasoning tasks, multi-step planning, and code generation, the depth ceiling shows up in production. A model may handle most cases well but fail reliably on problems that require one more step than it has layers. This is why some research groups are exploring architectures that allow recurrence or iterative refinement: ways to let a model take more steps without building a prohibitively deep network. It also shapes how we think about chain-of-thought prompting. Asking a model to show its work, step by step, may help not because it makes the model think harder, but because it spreads a multi-step problem across multiple forward passes, effectively giving the model more layers to work with. The architectural limit is still there, but the prompting strategy works around it.
The surprising detail
The failure is not smooth. Dziri's experiments showed that models performed well on tasks requiring up to their layer count in reasoning steps, then accuracy dropped sharply, not gradually, when the step count exceeded depth. This suggests the models are not learning to approximate long chains; they are learning to execute chains up to a hard limit, then failing categorically. It also means that a model's performance on a benchmark may tell you little about its performance on a slightly harder variant of the same task, if that variant crosses the depth threshold.
Remember this
Each layer is roughly one reasoning step. When the problem requires more steps than the model has layers, training cannot bridge the gap.
Test yourself
A 24-layer model solves a reasoning task that requires exactly 24 steps. You scale it up to twice the width but keep the same depth. Does it solve a 25-step version of the task?
No. Width does not substitute for depth. The model still has only 24 layers, so it still has only 24 opportunities to advance the reasoning chain. Making each layer wider gives it more expressive power per step, but does not create an additional step. The 25-step task remains out of reach. This is why some problems require deeper models, not just larger ones, and why architectural choices about depth are not fungible with choices about parameter count.
Go deeper
- The Parallelism Tradeoff: Limitations of Log-Precision Transformers · arXiv · William Merrill et al. · 2022-07-02
- Faith and Fate: Limits of Transformers on Compositionality · arXiv · Nouha Dziri et al. · 2023-05-29
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.