The mathematics of neural networks

Vectors, matrices, derivatives, backprop, softmax, probability, eigenvalues, optimizers and initialisation in NumPy.

Progress and your code are kept in this browser only.

1. 1Vectors and norms
2. 2Dot products and projections
3. 3Matrices as maps
4. 4Batches of vectors
5. 5Derivatives
6. 6The chain rule
7. 7Gradients and gradient checking
8. 8Activations and their slopes
9. 9Backpropagation by hand
10. 10Expectation, variance and likelihood
11. 11Softmax and cross-entropy
12. 12Eigenvectors and the SVD
13. 13Learning rate and curvature
14. 14Momentum and Adam
15. 15Initialisation and variance