CHAPTER 29 / 40
Divergences, Line Search & Bayesian Approximation
3 lessons · Explore, experiment and understand.
DOJO PETYour little coding companion
DOJO / YOUR GUIDE
Let’s figure it out.
Ask me about this course or where to start.
Chapter lessons
3 lessons
- Step 122~15m3D · 11Code · 13★Most cinematic
Jensen-Shannon and f-Divergences
KL divergence is asymmetric and can blow up to infinity. The Jensen-Shannon divergence symmetrizes it and stays bounded by log 2 — one membe…
Open lesson - Step 123~15m3D · 11Code · 13★Most cinematic
Line Search and the Wolfe Conditions
Gradient descent picks a direction, but how big a step? Exact line search is expensive; backtracking with the Armijo sufficient-decrease con…
Open lesson - Step 124~15m3D · 11Code · 13★Most cinematic
Empirical Bayes and the Laplace Approximation
Empirical Bayes estimates the prior's hyperparameters from the data instead of fixing them by hand. The Laplace approximation replaces an in…
Open lesson