The Optimiser Race: Loss after N Steps, Time to Target & Loss Curves
This lesson is part of the course
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Loss & Gradient: Which Way Is Downhill? Pro
- Plain Gradient Descent: Step Tables, Learning Rates & Divergence Pro
- Momentum: A Velocity Term, Overshoot & Faster Progress on a Stretched Bowl Pro
- RMSProp: Running Averages of Squared Gradients & Per-Weight Step Sizes Pro
- Adam: First & Second Moments, Bias Correction and the Adam Step Pro
- Noisy Gradients: Mini-Batch Noise, Batch Size & the Smoothing Effect Pro
- Learning-Rate Schedules: Step Decay, Cosine, Linear Warm-Up & Weight Decay Pro
- The Optimiser Race: Loss after N Steps, Time to Target & Loss Curves Pro