A thread per job does not scale to a thousand jobs. A ThreadPoolExecutor keeps a fixed number of worker threads and feeds them your jobs.

from concurrent.futures import (
    ThreadPoolExecutor,
)

def square(n):
    return n * n

pool = ThreadPoolExecutor(max_workers=4)
with pool:
    future = pool.submit(square, 7)
    print(future.result())  # 49
    squares = pool.map(square, [1, 2, 3])
    print(list(squares))  # [1, 4, 9]

submit returns a Future, a placeholder for a result that may not exist yet. result() waits for it. If the function raised, result() raises the same exception in your thread. Leaving the with block waits for every job and shuts the pool down.

map returns results in input order, but it raises the first exception it meets, and the results after it are lost. To keep going past failures, submit each job and inspect each future yourself. future.exception() returns the error instead of raising it, and as_completed(futures) yields futures in the order they finish.

Write fetch_all(keys, fetch, workers=4) that calls fetch(key) for every key on a pool of workers threads and returns two dicts, (results, errors): each key that succeeded mapped to its value, and each key that failed mapped to the exception it raised.

The explanation opens here once every check passes.