Reinforcement Learning by Hand: Bandits, Q-Learning & Value Iteration
Build the core RL algorithms cell by cell with given random numbers: bandit estimates, ε-greedy and UCB, discounted returns, the Bellman equations, value iteration, Q-learning updates and policy extraction, every cell checked live.
The first lesson is free — no account needed.
About this course
A reinforcement learning course that builds the core algorithms cell by cell with given random numbers, so every result can be checked, for data scientists, computer science students and engineers who want to see what the update rules actually do. You estimate the value of three app offers with sample averages and the incremental update, run an ε-greedy agent visit by visit, score offers with the UCB1 exploration bonus, calculate discounted returns forwards and backwards, evaluate a random policy for a small warehouse robot with the Bellman expectation equation sweep by sweep, run value iteration on a slippery floor until it converges, replay eight Q-learning updates with the TD target and TD error and extract the optimal policy to compare it with a coin flip, cell by cell. The first lab is free.
Lessons
- Multi-Armed Bandits: Sample Averages & the Incremental UpdateFree
- Epsilon-Greedy: Explore or Exploit, Visit by Visit Pro
- Upper Confidence Bound (UCB): Optimism with a Bonus Pro
- Returns & Discounting: G = R + γR + γ²R + … Pro
- Bellman Expectation: Evaluate a Random Policy Sweep by Sweep Pro
- Value Iteration: Take the Best Move, Sweep to Convergence Pro
- Q-Learning by Hand: Eight Updates of a Q-Table Pro
- Policy Extraction: From V* to the Best Action, and Random vs Optimal Pro