19 · 8 lessons

AI: building blocks

Eight hands-on lessons in attention, decoding, tokenization, retrieval and normalization.

Progress and your code are kept in this browser only.

  1. 1Stable Softmax & Temperature
  2. 2Scaled Dot-product Attention
  3. 3Causal Attention Mask
  4. 4Nucleus Sampling Candidates
  5. 5Beam Search
  6. 6Byte-pair Merge Step
  7. 7Cosine Retrieval
  8. 8Layer Normalization