CHAPTER 07 / 40
Continuous Optimization
3 lessons · Explore, experiment and understand.
DOJO PETYour little coding companion
DOJO / YOUR GUIDE
Let’s figure it out.
Ask me about this course or where to start.
Chapter lessons
3 lessons
Why this matters in ML
Gradient descent, momentum, Adam and constrained optimization are how parameters actually move — the training loop is an optimizer dressed up.
- SGD
- Adam
- Lagrangian methods
- Step 39~25m3D · 145★Most cinematic⚡Challenge
Gradient Descent
Gradient descent is the simplest and most-used continuous optimization algorithm: to minimize a differentiable , start at some and iterative…
Open lesson - Step 40~20m3D · 115★Most cinematic⚡Challenge
Constrained Optimization and Lagrange Multipliers
Real optimization problems often come with constraints: minimize subject to (equality) or (inequality). Constraints restrict the feasible re…
Open lesson - Step 41~20m3D · 125★Most cinematic⚡Challenge
Convex Optimization
A set is convex if for any and , the convex combination is also in — the line segment between any two points stays inside. A function is con…
Open lesson