The dot product multiplies two vectors element by element and adds the results: a @ b. It is the most common operation in a neural network, because every unit of a layer computes one dot product of its weights with its input.

Geometrically, a @ b equals |a| |b| cos(angle). Dividing by both lengths leaves the cosine: 1 for the same direction, 0 for perpendicular vectors, -1 for opposite ones. Embedding search ranks results by this cosine similarity.

import numpy as np

a = np.array([2.0, 1.0])
b = np.array([1.0, 0.0])
print(a @ b)  # 2.0
cos = a @ b / (np.linalg.norm(a)
               * np.linalg.norm(b))
print(round(cos, 3))  # 0.894
p = (a @ b) / (b @ b) * b
print(p, a - p)  # [2. 0.] [0. 1.]

The projection of a onto b is the part of a that points along b: (a @ b) / (b @ b) times b. What is left, a minus the projection, is perpendicular to b.

Write cosine(a, b) returning the cosine similarity as a float, project(a, b) returning the projection of a onto b as an array, and orthogonal_part(a, b) returning a minus that projection.

The explanation opens here once every check passes.