CHAPTER 22 / 40
Markov Chains and Monte Carlo
3 lessons · Explore, experiment and understand.
DOJO PETYour little coding companion
DOJO / YOUR GUIDE
Let’s figure it out.
Ask me about this course or where to start.
Chapter lessons
3 lessons
Why this matters in ML
Markov chains, Monte Carlo and Metropolis–Hastings are how Stan, PyMC and modern RL value-iteration extract answers from posteriors and state-action distributions you cannot solve analytically.
- MCMC
- HMC / NUTS
- particle filters
- Q-learning
- Step 95~15m3D · 113★Most cinematic
Markov Chains and Stationary Distributions
A Markov chain is a memoryless random walk on a state space — and its stationary distribution is the universal target of MCMC, the convergen…
Open lesson - Step 96~10m13
Monte Carlo Estimation and Importance Sampling
Monte Carlo turns intractable expectations into sample averages with a 1/√N convergence rate that does not depend on dimension — and importa…
Open lesson - Step 97~15m3D · 113★Most cinematic
Metropolis–Hastings (MCMC)
Metropolis–Hastings is a recipe for building a Markov chain whose stationary distribution is anything you want — even an unnormalized target…
Open lesson