AI: language models from the inside

Tokens, embeddings, retrieval, sampling, attention and transformer blocks, then a tiny GPT trained on the device, LoRA fine-tuning and perplexity.

Progress and your code are kept in this browser only.

1. 1Tokens
2. 2Embeddings
3. 3Retrieval
4. 4Sampling
5. 5A self-attention layer
6. 6The transformer block
7. 7A tiny GPT
8. 8Fine-tuning with LoRA
9. 9Measuring a language model