Understand the technology
Follow the inputs, the target and the update.
For May and November 2027. This introductory lesson is shared SL and HL content.
01 | HOW CAN A MODEL LEARN TO REMOVE NOISE?
A noisy image contains disturbed numerical patterns. A denoiser needs more than an instruction to “make it clearer”: it needs a learned way to interpret those patterns.
A neural denoiser is a trained neural network that estimates information used to reduce noise. The booklet describes it as typically a CNN. Its ability comes from training on many deliberately constructed examples.
This lesson uses the common DDPM task of predicting added noise. Some diffusion formulations predict clean data or another related quantity instead, so noise prediction is an important example rather than a rule for every model.
02 | CREATE A TASK WITH A KNOWN ANSWER
Start with a clean training image or representation. The training process samples a noise level and random Gaussian noise, then combines them using the forward noising rule.
Because the process created the corruption, it knows the particular sampled noise used. That noise can serve as the target : the answer against which a noise prediction is compared.
This avoids asking a person to annotate every noisy pixel manually. However, constructing a known target does not make the prediction easy; the model still has to learn relationships across many examples and noise levels.
03 | INPUT AND TARGET HAVE DIFFERENT ROLES
In the noise-prediction setup, the model receives the noisy sample and information identifying its timestep or noise level. A conditional model may also receive text or other supported guidance.
The sampled noise is the target used by the training calculation. It is not normally supplied to the model as the answer it should copy. The clean source is used to construct the task, not normally handed directly to the denoiser as an extra input.
Noisy sample, noise-level information and any conditioning.
An estimate of the noise in this parameterisation.
The actual noise sampled when creating the example.
Keeping these roles separate is essential to understanding how learning happens.
04 | WHY PROVIDE THE NOISE LEVEL?
A slightly noisy sample and a heavily noisy sample pose different problems. With mild corruption, visible edges may provide strong clues. With heavy corruption, the model must rely more on learned patterns and any guidance.
The timestep tells the model which stage of the forward process the sample represents. The system encodes this information numerically so that it can influence the network.
A timestep is not a label identifying the correct object. It also is not the count of optimizer updates already performed.
05 | WHAT IS THE PREDICTED NOISE?
The prediction is numerical data shaped to match the relevant sample representation. For a pixel-space model, it can contain estimates for image channels and positions. For a latent model, it relates to the latent representation instead.
The network does not simply return the word “noise”. It estimates a pattern of values that a later calculation can use.
An individual prediction can be imperfect. Learning aims to improve performance across the chosen training objective, rather than guarantee the exact random noise for every heavily corrupted sample.
06 | MEASURE THE PREDICTION ERROR
A loss function compares the prediction with the known target. A common noise-prediction objective uses mean squared error: calculate the difference at each component, square the differences and average them.
Squaring prevents positive and negative errors from cancelling and gives larger errors a greater contribution. The average summarises error across the selected components.
This is a learning measurement. A lower training loss does not by itself prove that finished images meet a campaign brief or that the model is free of bias.
07 | A SMALL WORKED LOSS CALCULATION
Consider an invented four-component example . The true noise is [1, −1, 0, 2] and the predicted noise is [0, −1, 1, 0]. These small vectors stand in for much larger arrays.
| Component | Target | Prediction | Prediction − target | Squared error |
|---|---|---|---|---|
| 1 | 1 | 0 | −1 | 1 |
| 2 | −1 | −1 | 0 | 0 |
| 3 | 0 | 1 | 1 | 1 |
| 4 | 2 | 0 | −2 | 4 |
The sum is 6. Dividing by four gives MSE = 1.5 . This checks arithmetic for the learning task; it does not simulate a complete diffusion model.
08 | THE OPTIMIZER CHANGES PARAMETERS
After the loss is calculated, backpropagation computes gradients: information about how the loss changes locally with the learned parameters. An optimizer uses that information to update parameters.
Those changes affect later predictions. Training repeats prediction, comparison and updating over many examples.
The process does not simply replace every predicted value with the known answer and declare the model trained. It adjusts shared numerical parameters so that the network can respond usefully to other inputs too.
09 | A LEARNING RATE CONTROLS UPDATE SCALE
A learning rate influences how strongly optimizer updates change parameters. If updates are too aggressive, learning may become unstable. If they are too small, progress may be slow.
There is no universal best value. Behaviour also depends on the optimizer, data, model and training setup. Training stability therefore requires monitoring rather than assuming that more updates must always improve the result.
This is supporting vocabulary for interpreting training, not a requirement to calculate gradients or choose a particular optimizer in this introductory lesson.
10 | LEARN ACROSS EXAMPLES AND NOISE LEVELS
A useful denoiser needs experience beyond one clean image and one noise pattern. Training samples different examples, timesteps and noise realisations.
This variation helps the model learn a range of denoising tasks. Training commonly processes batches and averages an objective across examples before applying an update.
Many implementations can construct a selected noisy level directly from a clean sample. They do not have to execute every earlier forward timestep just to create each training example.
11 | TRY THE PREDICTION CHECK
Estimate the four target noise values shown below, then calculate the loss. Begin with the supplied prediction, inspect which components contribute most, and try a closer prediction.
The target is visible so you can practise the comparison. In actual training it belongs to the loss calculation, not an answer field supplied to the denoiser. This activity contains no trained network or optimizer.
12 | WHY A CNN CAN HELP
A convolutional neural network applies learned filters to local groups of values. Neighbouring image values often contain useful relationships, such as edges and textures.
By combining information across layers, a network can use increasingly broad context rather than treating every component as unrelated. This can support useful noise estimates.
The booklet describes a CNN as typical; not every modern diffusion denoiser has the same architecture. Later lessons examine CNNs and DDPM more closely.
13 | GENERALISATION MATTERS
A model that performs well only on training examples is not enough. Generalisation concerns useful behaviour on new examples relevant to the task.
Assess development choices using separate validation data. Watch for overfitting, where training performance improves without corresponding improvement on appropriate held-out examples.
For generative systems, also inspect completed samples. A denoising objective and an output-quality review answer different questions, and both can be relevant to Visionary Studios.
14 | A NOISE ESTIMATE IS NOT AN EXACT UNDO BUTTON
Strong corruption can remove much of the evidence distinguishing possible clean sources. Several clean samples can be compatible with a noisy observation.
Training learns estimates from statistical relationships. It does not guarantee recovery of the exact original image or exact noise for every observation.
The known training target supplies a learning signal. Its availability to the training process does not mean that the model can always infer it perfectly from the inputs.
15 | USING A PREDICTION REQUIRES THE SAMPLING RULE
It is tempting to say that generation simply subtracts predicted noise once. A real diffusion sampler uses the model’s prediction with schedule-dependent coefficients and a defined update procedure.
Depending on the sampler, an update may also include a stochastic contribution. Several model evaluations can progressively refine a sample.
Do not treat raw subtraction as a complete DDPM sampling formula. The important connection here is that training produces a predictor whose outputs can be used by a later sampling process.
16 | TRAINING AND GENERATION REVISITED
During training, examples and known targets support a loss calculation and optimizer updates. During ordinary generation, the trained model makes predictions without a clean ground-truth target for the requested new image.
Generation can use the same learned parameters repeatedly across denoising steps. Repeated inference is different from repeated parameter learning.
17 | APPLY IT TO VISIONARY STUDIOS
Training quality can affect output quality and useful variation. Unsuitable data, unstable optimization or weak generalisation can undermine the studio’s results.
The company should distinguish using an already trained model from training or fine-tuning one. Training requires data preparation, repeated computations and evaluation; using a model still has generation and review costs.
The booklet does not state that Visionary Studios will build a denoiser from scratch. Describe this as technology the company needs to understand when evaluating options.
18 | EXPLAIN THE TRAINING LOOP
In your book, draw a training loop showing a clean sample, sampled noise level, noisy input, predicted noise, target, loss and optimizer update. Use separate arrows for model inputs and the loss target.
Then explain why the clean source is needed by the task-construction process but is not normally supplied directly as the denoiser’s answer.
Reveal a possible explanation
The training process combines a clean example with known sampled noise at a chosen level. The model receives the noisy sample and timestep, with any conditioning, and predicts noise. A loss compares prediction and actual noise; backpropagation and an optimizer update learned parameters. This repeats across examples.
Supplying the target as an answer to copy would not teach the intended inference task. During generation, the model must provide useful predictions without access to the exact noise or clean target of a new requested sample.
Read alongside the diffusion section of the official 2027 case study. The vectors and activity are independent teaching illustrations.
CHECK YOUR UNDERSTANDING | INPUT, TARGET, PREDICTION OR UPDATE?