Python: Iterators, Generators, and Coroutines

Iterator

An iterator is an object that contains a countable number of values and consists of the methods __iter__ and __next__.

• __iter__: This method is called when an iterator is required for a container. Must always return an iterator
• __next__: Return the next item from the container.

class MyIterator:
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.end:
            raise StopIteration

        result = self.current
        self.current += 1
        return result

"""For Loop through the MyIterator"""
for num in MyIterator(1, 5):
    print(num)

Generator

Generator functions allow you to declare a function that behaves like an iterator (can be used in a for loop – without having to store values all at once in memory).

The next time next() is called on the generator iterator (i.e. in the next step in a for loop, for example), the generator resumes execution from where it called yield.

def MyGenerator(top):
    current = 0
    
    while current < top:
        yield current
        current += 1


counter = MyGenerator(5)
for it in counter:
    print(it)

A function with “yield” – generator. Both Iterator and Generator – to generate the sequence:

• The Iterator keeps value in self.
• The Generator uses local variables

Generators are more for concurrency code.

• generators are data producers
• coroutines are data consumers

Coroutine

def grep(pattern):
    print("start grep")

    try:
        while True:
            line = yield
            if pattern in line:
                print(line)
    except GeneratorExit:
        print("stop grep")

# calling coroutine, nothing will happen
g = grep("lol")

# g.send(None) => start grep
next(g)

# sending inputs
g.send("lol is in the line")
g.send("no pattern in the line")

# Exceptions can be thrown inside a coroutine at the point where the generator was paused
g.throw("RuntimeError", "Something is wrong")

# Close the coroutine - exception GeneratorExit 
g.close()

line 6 – yield is to the right – coroutine just waiting for data that should be sent to the coroutine (g.send command), main code continue to execute => g.send to send data to the coroutine.

Coroutine waits for data to work with from the main code by using .send(). to stop the coroutine execution: g.close() – it will generate GeneratorExit exception that could be used inside the coroutine:

Use yield from to execute one coroutine from another:

def grep(pattern):
    print("start grep")

    try:
        while True:
            line = yield
            if pattern in line:
                print(line)
    except GeneratorExit:
        print("stop grep")


def grep_coroutine():
    g = grep("lol")
    yield from g

g = grep_coroutine()
next(g) # g.send(None) => start grep
g.send("lol is in the line")
g.send("no pattern in the line")
g.close()

How coroutine is different from threads, both seem to do the same job:

• in case of threads, it’s an operating system that switches between threads according to the scheduler
• in case of a coroutine, it’s the programmer and programming language which decides when to switch coroutines