Synthetic Data for Data-Scarce Domains

In fields where every labelled example must be found, checked and annotated by hand, training data is the bottleneck. Hence the appeal of a generative model — a network trained to produce new samples that resemble the data it was shown. If its output truly behaves like the real thing, a training set can grow without another round of collection.

A generative adversarial network, or GAN, is one family of models built for this job, and its training is a contest between two networks. The generator starts from a latent code — a list of random numbers — and transforms it into a candidate sample, say an image. The discriminator is shown a mix of real samples and the generator's fakes and must say which is which. After each round, the discriminator is updated to catch more fakes, and the generator is updated to slip more fakes past it. Neither network is ever told what the data means; the only training signal is real versus fake.

The hoped-for ending is that the generator's distribution — the full range of samples it can produce, and how often it produces each — comes to match the distribution of the real data. Matching the distribution, not producing a few good samples, is the actual goal, because a downstream model trained on the synthetic set will only learn about the cases the generator emits.

Theoretical work on generalisation and equilibrium in GANs asked whether the contest reliably reaches that ending. The answer is no, and the reasons are concrete. The training set is finite: the discriminator can only compare fakes against the examples it has, not against the true underlying variety. And the discriminator itself has bounded capacity — it can only detect so many kinds of difference. Under those two conditions, the analysis shows that a pure equilibrium — a stable state in which neither network can improve against the other — need not exist at all. The game can keep shifting, or settle somewhere that is not a match to the data.

The failure this licenses has a name: mode collapse. The mechanism is worth walking through slowly. The generator is rewarded only for fooling the discriminator. Suppose it finds one region of the data — one mode — that it can imitate convincingly. Producing endless variations on that region keeps the discriminator fooled. Nothing in the training signal punishes the generator for never producing the other regions, because the discriminator judges each sample on its own; it has no memory of what the generator has failed to attempt. So whole stretches of the real variation vanish from the output while every individual sample still looks credible.

For a data-scarce domain, this failure lands in the worst place. The rare, awkward cases are usually why the real data was thin in the first place. Those are the modes a collapsing generator drops first, because they were least represented in training and hardest to fake. You can multiply your training rows tenfold and, at the same time, narrow the range of things the downstream model ever sees.

A second line of work changed how the generator is steered, which matters when synthetic samples must carry meaning. In a plain design, the latent code enters once, at the network's first layer, and every property of the output is tangled into that single injection. A style-based generator reroutes it. A small mapping network first converts the random input into an intermediate code. From that code, per-layer settings called styles are derived, and each style adjusts one layer of the generator through a step called adaptive instance normalisation — in effect, rescaling and shifting that layer's activity. Early layers work at coarse spatial scale and late layers at fine scale, so coarse styles tend to set large attributes, such as overall pose or shape, while fine styles affect small structure. Separate random noise is injected at each layer to supply incidental detail — texture, grain — so the styles do not have to carry every random fluctuation.

The result is a generator whose attributes are less tangled: hold most styles fixed, vary one, and you get a family of samples that differ in one respect. That is control, and control is a step towards labels. But only a step. Nowhere in the adversarial loop does a label exist; the generator was never shown one. A trustworthy label on a synthetic example must come from a condition you fed the generator, from a latent factor you can measure and have verified, or from a person or second model tagging the output afterwards. The papers establish realism and structure in the sample stream; they do not print annotations.

What is not known matters too. The equilibrium analysis says when matching can fail; it is not a test you can run on one trained model to certify that matching succeeded. And how cleanly the styles separate varies by dataset and by training run; it is observed, not guaranteed. The joint lesson is sober. Generative models can multiply examples, and style structure makes the multiplication easier to steer. Neither sharp pictures nor a large folder of files is evidence that the generator covers the rare modes that made the real data scarce. Trust the coverage you can demonstrate, not the number of files you can sample.