14 · Project: ANN regression
Swap BCE for MSE and remove the sigmoid output — the same architecture now regresses. The pattern generalises across loss/task pairs.
One idea, one try. At your own pace.
LEARN BY DOINGPut your intuition to work3 steps
14 · Project: ANN regression
Use the lesson’s examples and explanation to check your idea.
Personal reflection. Nothing is sent to an AI service.Same ANN, swap BCE for MSE and remove the sigmoid output — now it regresses. The pattern generalizes across loss/task pairs.
Without this:
Without the regression variant, you'd assume neural nets are only for classification.
The previous lesson built a binary classifier. Now we change exactly two things and get a regression model:
- Remove the sigmoid output activation — regression targets are unbounded, so the raw linear output is correct.
- Swap BCE loss for MSE loss — instead of measuring log-likelihood of class labels, we measure mean squared distance from the target values.
Everything else — the hidden layer, ReLU activation, He init, SGD with momentum, the backprop equations — stays exactly the same. This pattern is one of the most powerful lessons in deep learning: the architecture (layer topology, activations, init, optimiser) is almost completely separable from the task (output activation + loss function).
The dataset uses a non-linear target: y = sin(x1) + 0.5*x2^2 + noise. A linear model can't capture this; our ANN can.
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
Python (in browser)
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
PyTorch read-along: regression — MSELoss replacing BCELoss, no sigmoid on output
Regression output activation checklist — which activation for which target range
To convert an ANN binary classifier to a regression model, what changes?
- ANN regressor = ANN classifier with sigmoid removed from output and BCE replaced by MSE. Hidden layers are unchanged.
- Output activation selection: none (raw linear) for unconstrained targets; sigmoid for [0,1]; tanh for [-1,1]; relu/softplus for non-negative.
- Linear models fail on non-linear targets (sin, x^2) — the hidden layer's non-linearity is the difference.
- Same pattern for multi-output regression: widen the final nn.Linear layer to n_out > 1 and use MSELoss — PyTorch handles it automatically.
Same pattern works for any single-output regression. For multi-output, just widen the final layer; PyTorch handles it automatically.
If you remove it: You'd assume neural nets are classification-only and miss the unified view that makes transfer learning and multi-task architectures intuitive.