PythiloLesson 3 of 18 · progress kept in this browser

Gradient descent

With slopes in hand, learning is a loop: compute the loss, get the gradients, step every parameter a little against its gradient, clear the gradients, repeat.

import torch

x = torch.linspace(-1, 1, 20)
y = 3 * x
w = torch.zeros(1, requires_grad=True)
for step in range(200):
    loss = ((w * x - y) ** 2).mean()
    loss.backward()
    with torch.no_grad():
        w -= 0.1 * w.grad
    w.grad.zero_()
print(w.item())  # close to 3

Two details matter. The update happens inside torch.no_grad(), so it is not recorded as part of the model. And gradients add up across backward() calls, so they must be cleared every step.

The step size, the learning rate, decides everything: too small and nothing moves, too large and the loss explodes.

Your task

Write fit_line(x, y, lr=0.1, steps=500) that learns w and b for y ≈ w * x + b by gradient descent on the mean squared error, starting from zero, and returns them as floats (w, b).

This lesson uses torch, which Python in this browser does not include. Read it here, and run and check it in the Pythilo app on iPhone or iPad.