II · THE IDEA · ARTIFICIAL INTELLIGENCE
Active Learning: Choosing What to Label
▶ Listen · narrated
Every label a human writes costs time and money. Labelling a random sample spends that budget blindly. The useful question is which examples to label next, and why those.
At a glance
- Goal
- Maximise model improvement per human label
- Mechanism
- Rank unlabelled points with an acquisition function
- Bayesian route
- Uncertainty via approximate posterior predictive
- Batch problem
- Top-scoring points often duplicate one another
- BADGE idea
- Gradient embeddings plus diverse sampling
You have a huge stack of unmarked exam papers and only time to mark a few. Picking at random is wasteful. You would look for the scripts that reveal most about where the class is confused, and you would skip twenty papers with near-identical answers.
Active learning does this for a model. An acquisition function gives every unlabelled example a score, high when the model is unsure about it, or when labelling it would shift the model usefully. The top-scoring examples go to a person for labels, then the model is retrained on the larger set.
Scores such as BALD measure uncertainty by running the model several times with small random changes inside it and seeing how much the answers differ. A wide spread means the model has not made up its mind. Batch methods such as BADGE add one more requirement. For each example they work out the nudge it would give the model during training, then pick a batch whose nudges point in different directions — so the round of labelling buys fifty different lessons rather than fifty copies of one hard case.
Pool-based deep active learning iterates: train on labelled set L, score every candidate in unlabelled pool U with an acquisition function a(x; θ), query an oracle for labels on a batch B ⊂ U, then set L ← L ∪ B and retrain.
Uncertainty acquisitions include predictive entropy H[y|x,L], which is high when the predictive distribution is flat; variation ratios, the fraction of stochastic forward passes in which the modal class is not chosen; and BALD, I[y,ω|x,L] = H[y|x,L] − E_p(ω|L)[H[y|x,ω]], the mutual information between the label and the weights ω. BALD is large when individual sampled models are each confident but disagree with each other — that is, epistemic uncertainty rather than noise in the label itself. The posterior over ω is approximated by Monte Carlo dropout: dropout stays active at test time, and several forward passes act as samples from that approximate posterior, as used in deep Bayesian active learning on image data.
Single-point acquisition ignores batch structure. Taking the top k of a(x) tends to return neighbours in representation space, so after the first label the rest are close to redundant. BADGE instead builds, for each x ∈ U, the gradient embedding g_x = ∇_θ_last ℓ(f_θ(x), ŷ) — the loss gradient with respect to the final-layer parameters, with ŷ = arg max f_θ(x) substituted for the unobserved label. The norm ||g_x|| correlates with uncertainty; the direction of g_x encodes the update the example would propose. Seeding a batch with k-means++ over {g_x} favours both large norms and angular spread, so uncertainty and diversity are traded off without a hand-tuned weight.
Limits apply to all of it. These scores depend on the quality of the posterior approximation or the final-layer linearisation, they degrade when the model is miscalibrated, and they remain heuristics: they rank expected informativeness under the current model, and certify nothing about generalisation after the update.
Look closer
Acquisition is a ranking, not a guarantee
An acquisition function assigns each unlabelled candidate a scalar score. Higher scores are treated as more informative under a chosen criterion — predictive entropy, mutual information between labels and parameters, variation ratios, or a gradient-based proxy. The score is a heuristic conditioned on the current model and the pool; it does not prove that labelling that point will help more than labelling another. What it does is concentrate annotation budget where the model is currently least settled or where the loss landscape is expected to move.
Uncertainty alone clusters
In pool-based active learning the highest-scoring points often sit near one another in representation space. A batch built only from top ranks can therefore request many near-duplicates: once one is labelled, the others become far less useful. Deep Bayesian work with image data made this pattern visible when acquisition favoured ambiguous regions of the input space, and batch methods later treated diversity as a first-class constraint rather than an afterthought.
Gradients as a joint signal
BADGE embeds each candidate by the gradient of the loss with respect to the final-layer parameters, using the model’s own predicted label as a stand-in where the true label is unknown. Magnitude of that embedding tracks a form of uncertainty; direction tracks how the update would point. Sampling a batch with a diverse seeding procedure over those embeddings therefore pulls points that are both uncertain and dissimilar in the update they would induce — a practical answer to redundant batches without a separate hand-tuned diversity weight.
The story
Active learning starts from an imbalance. Unlabelled examples — raw inputs with no correct answer attached — are cheap and plentiful. Labels are not, because a person has to supply each one. So the model is trained on a small labelled set, then allowed to ask a human, called the oracle, for labels on a few examples drawn from a large unlabelled pool. This is the pool-based setting. The bet is that a well-chosen handful of labels buys more accuracy than the same number of labels picked at random.
The choosing is done by an acquisition function: a rule that takes the current model and one unlabelled input, and returns a single number saying how useful that input would be if labelled. Rank the pool by that number, label the top of the list, retrain, repeat.
Several such rules measure how unsure the model is. Predictive entropy asks how spread out the model's guess is across the possible classes; a confident guess scores low, a near-tie scores high. Variation ratios run the model several times with small random changes inside it and count how often the winning class changes. BALD estimates the mutual information between the unknown label and the model's own parameters — in plain terms, how much learning that one label would settle the model's internal disagreement about itself. All three need a spread of predictions rather than a single one. Monte Carlo dropout supplies that cheaply: dropout, which switches off random units during training, is left switched on at prediction time, so several passes over the same input produce several slightly different answers. Deep Bayesian active learning on image data showed this works without training a fresh ensemble of models.
These scores behave well when one example is requested at a time. They fail when a batch is requested, and a batch is what real pipelines need, because retraining after every single label is far too slow. The trouble is concrete: the highest-scoring examples usually sit close together, because they come from the same confusing region of the data. Label one of them and the rest teach almost nothing new. But a plain top-k selection has already committed to all of them. The oracle spends a full round of effort on near-duplicates.
BADGE fixes that without throwing away the uncertainty signal. For each unlabelled example it computes a gradient embedding — the gradient of the loss with respect to the final layer's parameters, which is the direction and size of the nudge that example would give the model if it were labelled. The true label is missing, so the method substitutes the model's own predicted class and computes the gradient anyway. Two useful things fall out. The length of that vector tends to be large where the model is unsure. The direction of it says which way the decision boundary would move. A batch is then picked by k-means++ seeding over these vectors, a procedure that deliberately spreads choices apart, so the selected examples are both uncertain and pulling the model in different directions.
Neither method removes the human. Both assume a fixed pool, a model that can be scored repeatedly, and an oracle whose labels are taken as correct for the next round of training. What they change is where that human effort lands: not on random examples, but on the ones the current model has not resolved — and, for batches, on a set whose members do not repeat each other.
Why it mattered then
Deep models made random labelling expensive. Accuracy tended to rise with the volume of labelled data, but annotation budgets did not rise with it. Two obstacles stood in the way of choosing labels smartly. First, uncertainty scores needed a range of predictions from the model, and computing a full spread over a deep network's weights was impractical; Monte Carlo dropout gave a cheap stand-in by running the same input through the network several times with random units switched off. Second, teams could not afford to retrain after every single label, so they had to request labels in batches — which exposed the flaw in ranking by uncertainty alone, since the top of the ranking is usually one confusing region counted many times over. Work on BALD-style acquisition with image data and on gradient-based batch selection answered these two pressures: how to tell what the model does not know, and how to ask for several things at once without asking the same thing repeatedly.
Why it matters now
Annotation is still among the largest costs in applied machine learning, whether the person supplying labels is a specialist radiologist, a legal reviewer, or a crowd worker. Foundation models shrink the labelled set needed to fine-tune a system, but they do not remove the need to choose that set with care when the domain is narrow or each label is slow and costly to obtain. An acquisition function is simply the rule deciding which of the current model's mistakes are worth paying a human to correct. And batch diversity is still the difference between a labelling round that covers the errors that remain and one that buys the same confusion fifty times.
The surprising detail
At selection time BADGE does not know the true label, so it assumes the model's own prediction is right and computes the training gradient from that assumption. The resulting vector is longest exactly where the assumption is shakiest — the method uses a target it knows may be wrong to find the examples most worth correcting.
What is disputed
Reported gains for BALD, entropy, and BADGE depend on architecture, pool composition, and how the posterior or gradient is approximated. Neither paper establishes a universal ranking of acquisition functions; when the pool is already diverse or the model is badly miscalibrated, random sampling can remain competitive.
Remember this
Acquisition functions spend the labelling budget where the model is unresolved. Good batch methods add a second demand: the chosen examples must not repeat one another.
Test yourself
You score an unlabelled pool with predictive entropy and take the top 50 as a batch. After labelling and retraining, performance barely moves. What failure mode of uncertainty-only batch selection is the most likely explanation, and how does a gradient-embedding method such as BADGE try to avoid it?
The top-entropy points were probably near-duplicates in representation space — one ambiguous cluster counted many times — so most labels repeated the same information. BADGE maps each candidate to a final-layer loss gradient (using the predicted label) and picks a diverse subset in that embedding space, so the batch mixes uncertainty with distinct proposed updates rather than fifty copies of one confusion.
Go deeper
- [1703.02910] Deep Bayesian Active Learning with Image Data · arxiv.org
- [1906.03671] Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds · arxiv.org
Image: Original diagram, The Daily Triptych. Licence: Original work. Source.