Experiment Grids: Learning Rates × Batch Sizes × Seeds and Early Stopping
This lesson is part of the course
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Dataset Tokens: Documents, Words, Deduplication & the Validation Split Pro
- Training Compute: The 6 × Parameters × Tokens Rule & Scientific Notation Pro
- GPU-Hours, Utilisation & Cluster Size for a Deadline Pro
- On-Demand, Reserved or Interruptible: Pricing the GPU-Hours Pro
- Fine-Tuning Budgets: Epochs, Low-Rank Adapters & Memory per Parameter Pro
- Experiment Grids: Learning Rates × Batch Sizes × Seeds and Early Stopping Pro
- Scaling Curves: Fitting a Power Law and Predicting Loss at a Bigger Budget Pro
- The Project Budget: Checkpoints, Storage, Evaluations, Contingency & the Chart Pro