Positional Signals: Sine & Cosine Waves Added to Embeddings
This lesson is part of the course
Buy the course once for $9.99 to unlock every lesson and lab, the final exam and the certificate – lifetime access, no subscription.
Lessons in this course
- Tokens: Splitting a Prompt, Vocabulary Ids & Token Budgets Pro
- Embeddings: Looking Up Token Vectors & Dot Products Pro
- Positional Signals: Sine & Cosine Waves Added to Embeddings Pro
- Queries, Keys & Values: Weight Tables and SUMPRODUCT Pro
- Attention Scores: Scaling by √d, the Causal Mask & Softmax per Row Pro
- Weighted Values: The Attention Output & Which Token Looks Where Pro
- Next-Token Probabilities: Temperature, Greedy Choice, Top-k & Sampling Pro
- Context Length: Re-Sent History, Turn Costs & Why Long Prompts Weigh More Pro