Generators and coroutines

A generator is a function whose execution can be paused and resumed. You write yield instead of return, and the function becomes a state machine that produces values one at a time.

def counter(n):
    for i in range(n):
        yield i
 
for x in counter(3):
    print(x)
# 0, 1, 2

How they work

Calling a generator function returns a generator object - the function body has not run yet. Each call to next(gen) advances the function to the next yield, returns the yielded value, and suspends. The function's locals, instruction pointer, and call stack are preserved between calls.

When the function returns (or raises StopIteration), iteration ends.

Why they matter

• Memory. A generator produces values lazily. Reading a 10GB file line by line uses O(1) memory; reading it into a list uses 10GB.
• Composition. Generators chain together as pipelines without intermediate storage. (x*x for x in lines if x).
• Infinite sequences. itertools.count() is a generator that never ends. You take what you need.

From generators to coroutines

PEP 342 (Python 2.5) added send, throw, and close to generators, turning them into two-way channels. You could send a value into a generator; yield would return it.

def echo():
    while True:
        received = yield
        print(received)
 
g = echo()
next(g)         # Prime it
g.send("hi")    # Prints "hi"
g.send("there") # Prints "there"

This was the foundation for asyncio. Coroutines were generators that yielded to the event loop instead of producing values. PEP 380 (3.3) added yield from to delegate to another generator. PEP 492 (3.5) gave us async def and await as cleaner syntax over the same machinery.

Today async def foo(): await bar() is sugar over the generator coroutine model. You can think of await as yield from for coroutines.

yield from

yield from sub_gen delegates iteration to another generator. Anything sub_gen yields, the outer generator yields too. Values sent in pass through. Exceptions propagate.

def inner():
    yield 1
    yield 2
 
def outer():
    yield 0
    yield from inner()
    yield 3
 
list(outer())  # [0, 1, 2, 3]

The mental model: a generator is a function with a save point. Yield to pause. Resume on next. Memory stays bounded, composition stays clean, and the same machinery powers everything from CSV streaming to asyncio.