Recurrent Networks by Hand: Hidden States, Unrolling Through Time, Vanishing Gradients, LSTM Gates, GRU Cells & Sequence Prediction
See how a network remembers. You will run a tiny recurrent cell step by step, unroll it over a toy sequence, measure how gradients shrink across time, compute forget, input and output gates and the cell state of an LSTM, compare it with a GRU, count parameters and predict the next value of a toy series, with given weights and every cell checked as you go.
One-time payment · lifetime access · lessons, labs, exam and certificate included · no subscription.
About this course
This course opens up recurrent networks with nothing but a spreadsheet and given toy weights. You lay out sequences one time step per row, turn words into numbers and see why order matters. You run a simple recurrent cell with tanh written as EXP, unroll it over four-word toy reviews and score them from the final hidden state with a sigmoid. You multiply step factors to watch gradients vanish or explode, compute the forget, input and output gates and the cell state of an LSTM, compare it with a GRU, count the weights of each cell type and finish by forecasting a toy series and comparing the MAE with a naive forecast.
Lessons
- Sequences as Rows: Time Steps, Scaled Inputs, Words as Numbers & Why Order Matters Pro
- The Simple Recurrent Cell: Weighted Input, Recurrent Weight, Bias & tanh, Step by Step Pro
- Unrolling Through Time & Many-to-One: The Final Hidden State as a Summary and a Toy Sentiment Score Pro
- Vanishing & Exploding Gradients: Multiplying Step Factors Back Through Time Pro
- LSTM Gates: Forget, Input & Output Gates, the Candidate Value and the Cell State over Six Steps Pro
- GRU Cells: Update & Reset Gates, the New Hidden State and a Comparison with the LSTM Pro
- Parameter Counts & Cost: Weights per Gate, Simple vs GRU vs LSTM and a Size Table Pro
- Capstone: Next-Value Prediction with Given Trained Weights, MAE vs a Naive Forecast & a Chart Pro