Async generators and async for
Some data arrives in pieces: pages from an API, lines from a stream. An async generator is an async def that uses yield. It can await between items, and the caller reads it with async for.
import asyncio
async def countdown(n):
while n > 0:
await asyncio.sleep(0.01)
yield n
n -= 1
async def main():
async for n in countdown(3):
print(n)
evens = [n async for n in countdown(6)
if n % 2 == 0]
print(evens) # [6, 4, 2]
asyncio.run(main())Like an ordinary generator it is lazy: it runs only when the loop asks for the next item. If the loop stops early, the rest is never fetched.
A plain for cannot read it, and neither can list(). Use async for, or an async comprehension as above, inside a coroutine.
Your task
Write an async generator paginate(get_page) that awaits get_page(1), get_page(2) and so on, yields every item of every page, and stops at the first empty page. Write async def take(items, n) that returns a list of the first n items of an async iterable, without reading any further.
The explanation opens here once every check passes.