CHAPTER 13 / 40
Deep Learning Foundations
4 lessons · Explore, experiment and understand.
Let’s figure it out.
Ask me about this course or where to start.
Composing linear layers with non-linearities and chain-rule gradients gives you a neural network — every concept in this course is reused inside the simplest MLP.
- MLPs
- activation functions
- backpropagation
- Step 54~20m2Code · 15⚡Challenge
Softmax and Cross-Entropy
Every classifier in modern deep learning ends with the same two-step ritual: turn raw scores into probabilities with softmax, then measure t…
Open lesson - Step 55~20m3D · 12Code · 15★Most cinematic⚡Challenge
Regularization: L1, L2, and Elastic Net
Models that fit training data perfectly often fail in production. The cause is overfitting: when a model has more capacity than the data war…
Open lesson - Step 56~20m3D · 12Code · 15★Most cinematic⚡Challenge
Backpropagation Through Layers
Training a neural network is just gradient descent on a very deeply nested function. The miracle is that the gradient — through possibly hun…
Open lesson - Step 57~20m3D · 12Code · 25★Most cinematic⚡Challenge
Attention: Q, K, V and Scaled Dot-Product
Attention is the operation that powers every modern transformer — the architecture behind GPT, Claude, BERT, and essentially every state-of-…
Open lesson