Python3 Iterators and Generators

Iterators

Iteration is one of Python’s most powerful features and is a way of accessing elements in a collection.

An iterator is an object that remembers the position of traversal.

An iterator object starts accessing from the first element of the collection until all elements are accessed. Iterators can only go forward and cannot go backward.

Iterators have two basic methods: iter() and next().

Strings, lists, or tuple objects can all be used to create iterators:

Example (Python 3.0+)

>>> list=[1,2,3,4]
>>> it = iter(list)    # Create an iterator object
>>> print (next(it))   # Output the next element of the iterator
1
>>> print (next(it))
2
>>>

Iterator objects can be traversed using conventional for statements:

Example (Python 3.0+)

#!/usr/bin/python3
 
list=[1,2,3,4]
it = iter(list)    # Create an iterator object
for x in it:
    print (x, end=" ")

Executing the above program produces the following output:

1 2 3 4

You can also use the next() function:

Example (Python 3.0+)

#!/usr/bin/python3
 
import sys         # Import the sys module
 
list=[1,2,3,4]
it = iter(list)    # Create an iterator object
 
while True:
    try:
        print (next(it))
    except StopIteration:
        sys.exit()

Executing the above program produces the following output:

Creating an Iterator

To use a class as an iterator, you need to implement two methods in the class: __iter__() and __next__().

If you already understand object-oriented programming, you know that every class has a constructor. Python’s constructor is __init__(), which is executed when the object is initialized.

For more, see: Python3 Object-Oriented Programming

The __iter__() method returns a special iterator object that implements the __next__() method and identifies the completion of iteration through the StopIteration exception.

The __next__() method (called next() in Python 2) returns the next iterator object.

Create an iterator that returns numbers, starting with an initial value of 1 and incrementing by 1:

Example (Python 3.0+)

class MyNumbers:
  def __iter__(self):
    self.a = 1
    return self
 
  def __next__(self):
    x = self.a
    self.a += 1
    return x
 
myclass = MyNumbers()
myiter = iter(myclass)
 
print(next(myiter))
print(next(myiter))
print(next(myiter))
print(next(myiter))
print(next(myiter))

Execution output:

StopIteration

The StopIteration exception is used to identify the completion of iteration, preventing infinite loops. In the __next__() method, we can set it to raise a StopIteration exception after completing the specified number of iterations to end the iteration.

Stop after 20 iterations:

Example (Python 3.0+)

class MyNumbers:
  def __iter__(self):
    self.a = 1
    return self
 
  def __next__(self):
    if self.a <= 20:
      x = self.a
      self.a += 1
      return x
    else:
      raise StopIteration
 
myclass = MyNumbers()
myiter = iter(myclass)
 
for x in myiter:
  print(x)

Execution output:

Generators

In Python, a function that uses yield is called a generator.

yield is a keyword used to define generator functions. A generator function is a special type of function that can produce values gradually during iteration rather than returning all results at once.

Unlike regular functions, a generator is a function that returns an iterator, which can only be used for iteration operations. To put it more simply, a generator is an iterator.

When a yield statement is used in a generator function, the function’s execution is paused, and the value of the expression after yield is returned as the current iteration value.

Then, each time the generator’s next() method is called or a for loop is used for iteration, the function resumes execution from where it last paused until it encounters the yield statement again. In this way, the generator function can produce values gradually without needing to compute and return all results at once.

Calling a generator function returns an iterator object.

Here is a simple example demonstrating the use of generator functions:

Example

def countdown(n):
    while n > 0:
        yield n
        n -= 1
 
# Create a generator object
generator = countdown(5)
 
# Get values by iterating the generator
print(next(generator))  # Output: 5
print(next(generator))  # Output: 4
print(next(generator))  # Output: 3
 
# Use a for loop to iterate the generator
for value in generator:
    print(value)  # Output: 2 1

In the above example, the countdown function is a generator function. It uses the yield statement to gradually produce countdown numbers from n to 1. Each time the yield statement is called, the function returns the current countdown value and resumes from where it last paused on the next call.

By creating a generator object and using the next() function or a for loop to iterate the generator, we can gradually obtain the values produced by the generator function. In this example, we first use the next() function to get the first three countdown values, and then use a for loop to get the remaining two countdown values.

The advantage of generator functions is that they can generate values on demand, avoiding the need to generate large amounts of data at once and consume large amounts of memory. Additionally, generators can seamlessly work with other iteration tools (such as for loops), providing a concise and efficient iteration approach.

Executing the above program produces the following output:

The following example uses yield to implement the Fibonacci sequence:

Example (Python 3.0+)

#!/usr/bin/python3
 
import sys
 
def fibonacci(n): # Generator function - Fibonacci
    a, b, counter = 0, 1, 0
    while True:
        if (counter > n): 
            return
        yield a
        a, b = b, a + b
        counter += 1
f = fibonacci(10) # f is an iterator, returned by the generator
 
while True:
    try:
        print (next(f), end=" ")
    except StopIteration:
        sys.exit()

Executing the above program produces the following output:

0 1 1 2 3 5 8 13 21 34 55