II · THE IDEA · ARTIFICIAL INTELLIGENCE
Progressive Growing of GANs
▶ Listen · narrated
A full-resolution adversarial game is hard to stabilise. Starting tiny, then fading in new layers as resolution doubles, changes how that race unfolds from the first iteration.
At a glance
- Core idea
- Train G and D from low resolution, then progressively add layers
- Start scale
- Typically 4×4, doubling toward the target resolution
- Transition
- New layers are faded in rather than switched on abruptly
- Claimed gains
- Improved quality, training stability, and output variation
- Metric caution
- Inception Score has known limits as a sole quality measure
Think of teaching someone to paint a portrait. You do not hand them a vast blank canvas and demand every eyelash on day one. You start with a tiny sketch of the head’s outline, then a slightly larger study of light and shadow, then bigger surfaces for skin and hair, and only late on the fine lines. Progressive growing does something similar for two competing networks: a generator that invents images and a discriminator that tries to spot fakes.
Both begin with images only a few pixels across. At that scale they can only argue about broad shapes and colours. When that argument settles, each network gains a new block of layers that doubles the image size, and the new block is mixed in gently so the old skills are not erased. The process repeats until the pictures are large and detailed. Early layers stay in the model the whole time; fine detail is stacked on top of coarse structure rather than learned in isolation.
Progressive growing schedules the depth and output resolution of a GAN. Initialise G and D for N×N output (often N=4). Train to convergence at that scale. To move to 2N×2N, append a new conv block to G (with upsampling) and a new conv block to D (with downsampling). During a transition, the new block’s output is combined with a projection of the previous stage via a residual mix: output = (1−α)·old + α·new, with α annealed from 0 to 1. After α reaches 1, freeze the topology at 2N and continue. Repeat until the target resolution.
All prior layers remain trainable. Stabilising practices reported alongside the schedule include per-layer equalised learning rates, pixelwise normalisation of generator feature vectors, and a minibatch standard-deviation feature at the discriminator to penalise low-variance batches. The method targets improved sample fidelity, optimisation stability, and variation relative to training the full-resolution pair from scratch. Evaluation used Inception Score and multi-scale similarity statistics; a subsequent note on Inception Score documents its blind spots, so those numbers should not be read in isolation. Limitations include longer pipeline complexity, the need to choose stage durations and fade schedules, and the fact that the curriculum is resolution-based rather than semantics-based—it does not by itself fix every mode-collapse mechanism.
Look closer
Both networks grow together
The generator and the discriminator are enlarged in lockstep. When a new block is introduced on the generator side to emit a higher-resolution image, a matching block appears on the discriminator so that the adversarial task remains balanced. The early, low-resolution layers stay in place and keep receiving gradient updates throughout; they are not discarded once finer scales appear.
Fade-in, not a hard cut
A newly added layer does not instantly dominate. Its contribution is blended with a residual path from the previous resolution, with a mixing weight that rises smoothly from zero to one. During that interval the network can treat the higher-resolution path as a residual refinement rather than a sudden change of architecture, which reduces the shock to already-learned weights.
Coarse structure before fine detail
At the lowest resolutions the networks only have to agree on large-scale layout—rough face shape, pose, colour masses. Finer layers later specialise in texture, edges and small features. That curriculum is a structural consequence of the growth schedule, not an extra loss term: early training literally cannot represent high-frequency detail because those pixels do not yet exist.
The story
Generative adversarial networks pit a generator against a discriminator. When both are asked to operate at high resolution from the first step, the discriminator can often reject fakes on the basis of local statistics long before the generator has learned a coherent global layout. Gradients become uninformative or unstable, and many runs collapse or produce limited variety.
Progressive growing changes the schedule rather than the basic adversarial objective. Training begins with a generator that outputs a very small image—commonly four pixels on a side—and a discriminator that scores images at that same scale. Once that stage has stabilised, both networks receive an additional convolutional block that doubles spatial resolution. The process repeats until the desired output size is reached.
New capacity is not dropped in as a hard architectural switch. During a transition phase the higher-resolution path is mixed with an upsampled (or downsampled) version of the previous stage, and the mix weight is increased gradually. After the fade completes, the residual path is removed and training continues at the new fixed resolution. Layers introduced earlier remain active and continue to be trained; the network is growing, not being replaced.
Because early stages only model coarse structure, the generator is forced to find large-scale modes of the data distribution before it is allowed to spend capacity on fine texture. The discriminator, operating at the same scale, cannot rely on high-frequency artefacts that would be unavailable at low resolution. Practitioners using this schedule reported faster wall-clock convergence to a given quality level as well as more stable optimisation, relative to training the full-resolution stack from scratch.
The original work also described several accompanying practices intended to keep signal scales well behaved as depth increased—among them equalised learning rates across layers, pixelwise feature normalisation in the generator, and a minibatch standard-deviation statistic fed to the discriminator to discourage collapse to a few modes. Those details sit alongside the progressive schedule; the schedule itself is the central organisational idea.
Evaluation in the paper relied in part on the Inception Score and on multi-scale statistical comparisons between real and generated sets. A separate note on the Inception Score later stressed that the metric can be insensitive to important failure modes and can be gamed, so numerical gains on that score alone should be read with caution. Progressive growing was influential less as a final architecture than as evidence that curriculum over resolution is a practical lever for high-fidelity synthesis.
Why it mattered then
Around the time of the work, unconditional and conditional image GANs were producing compelling samples at modest resolutions but struggled as output size increased. Training collapses, mode dropping, and long, fragile hyperparameter searches were common complaints. A method that made 1024×1024 face synthesis practically attainable—and that framed stability as a consequence of growing capacity with resolution—offered a concrete alternative to simply scaling networks and batch sizes. It also fitted the hardware of the period: early stages were cheap, so much of the optimisation happened when each step was still inexpensive.
Why it matters now
Later generators moved on—style-based architectures, diffusion models, and transformer hybrids shifted the default toolkit. The underlying lesson remains useful wherever adversarial or multi-scale generators are still trained: do not force the full difficulty of the task on day one. Resolution curricula, staged training, and smooth architectural transitions reappear in cascade models, multi-scale diffusions, and progressive fine-tuning recipes. Anyone debugging an unstable high-resolution run still benefits from asking whether the networks were asked to invent fine detail before coarse structure had anywhere to live.
The surprising detail
The same early layers that learn a blurry 4×4 layout are still present, and still updated, when the network is emitting a megapixel image. High-resolution skill is literally built on top of the low-resolution representation rather than trained as a separate stack. That continuity is easy to miss if one only looks at the final samples.
What is disputed
Reported gains in quality, stability and variation come from the original paper’s experiments and metrics. The Inception Score, used in that evaluation, is known to be an incomplete proxy for sample quality and diversity; treat score tables as supportive evidence, not as proof that every failure mode was solved.
Remember this
Progressive growing stabilises high-resolution GANs by training coarse structure first and fading in finer layers only once both networks are ready.
Test yourself
If new higher-resolution layers were added to the generator alone, without a matching change to the discriminator and without a fade-in period, what two distinct problems would you expect during the transition?
First, the discriminator would still be sized and trained for the old resolution, so it could not fairly score the new high-resolution output; the adversarial signal would be misaligned. Second, an abrupt switch dumps randomly initialised high-resolution filters into a generator whose earlier layers had already converged, shocking the learned representation and often collapsing training. Progressive growing avoids both by growing G and D together and by blending the new path in gradually.
Go deeper
- [1710.10196] Progressive Growing of GANs for Improved Quality, Stability, and Variation · arxiv.org
- [1801.01973] A Note on the Inception Score · arxiv.org
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.