A language model does not read letters or words. It reads tokens: numbers for pieces of text that appear often. Byte pair encoding builds them. Start from the bytes of the text, find the pair of neighbours that occurs most, give that pair a new number, replace it everywhere, and repeat.

text = "aaab"
ids = list(text.encode("utf-8"))
print(ids)        # [97, 97, 97, 98]
# (97, 97) is the commonest pair:
# merge it into 256
print([256, 97, 98])

Each merge makes the sequence shorter, so the model sees more text in the same window. Common words end up as a single token; rare words stay split into pieces, so nothing is ever unknown.

Write pair_counts(ids) returning a dict from each adjacent pair to how often it occurs. Write merge(ids, pair, new_id) replacing every occurrence of pair, left to right without overlaps. Write train_bpe(text, merges): start from the UTF-8 bytes, and merges times merge the most frequent pair (the first one found on a tie) into the next id from 256. Return (ids, learned), where learned lists the merged pairs in order.

The explanation opens here once every check passes.