A two-layer network: h = tanh(W1 @ x + b1), then out = w2 @ h + b2, with loss L = 0.5 (out - y) ** 2. Backpropagation walks back through it, one link at a time, reusing what the forward pass stored.

import numpy as np

rng = np.random.default_rng(0)
W1 = rng.normal(size=(3, 2))
b1 = np.zeros(3)
w2 = rng.normal(size=3)
x, y = np.array([1.0, -1.0]), 0.5
h = np.tanh(W1 @ x + b1)
out = w2 @ h
d_out = out - y          # dL/dout
d_w2 = d_out * h
d_h = d_out * w2
d_pre = d_h * (1 - h ** 2)
d_W1 = np.outer(d_pre, x)
print(d_W1.shape, d_w2.round(3))

Each gradient has the shape of its parameter. A weight matrix's gradient is the outer product of the gradient arriving at its output and the input it saw, np.outer(d_pre, x). A bias's gradient is just the gradient arriving at its output.

Write backward(params, x, y) for the network above, where params is a dict with W1, b1, w2 and b2 (a float). Return the dict of gradients of L with the same keys and shapes (b2's as a float). forward(params, x) is given.

The explanation opens here once every check passes.