Python Iterators: Part Four

February 20, 2015 by Leave a Comment

Python iteratorsTo see a practical application of the iteration protocol, let’s consider the case of the list comprehension. You can use range to change a list as you step across it; e.g.:

>>> L = [1, 2, 3, 4, 5]
>>> for i in range(len(L)):
	L[i] += 1
>>> L
[2, 3, 4, 5, 6]

This works, but there is an easier way. We can replace the loop with a single expression that produces the desired list:

>>> L = [x + 1 for x in L]
>>> L
[2, 3, 4, 5, 6]

The final result is the same, but it requires less coding on our part and is substantially faster to boot. The list comprehension is not exactly the same as the for loop statement version in that it makes a new list object.

The list comprehension’s syntax is derived from a construct in set theory notation that applies an operation to each item in set. Alternatively, you can think for it as a backwards for loop.

List comprehensions are written in square brackets because they are ultimately a way to construct a new list. They begin with an arbitrary expression that we make up, which uses a loop variable that we make up (x + 1). That is followed by what you should recognize as the head of the for loop, which names the loop variable, and an iterable object (for x in L).

To run the expression, Python executes an iteration across L inside the interpreter, assigning x to each item in turn, and collects the results of running the items through the expression on the left side. The result list we get back is exactly what the last comprehension says: a new list containing x + 1 for every x in L.

Technically, we do not need list comprehensions, because we can always build up a list of expression results manually with for loops. List comprehensions, however, are more concise to write, and because this code pattern of building up result lists is so common in Python work, they turn out to be very handy in many contexts. Moreover, list comprehensions can run much faster than manual for loop statements because their iterations are performed in C language speed inside the interpreter (rather than with manual Python code). Especially for larger data sets, there is a large performance advantage.

External Links:

Python Iterators at Python Wiki

Python Iterator tutorial at bogotobogo.com

Filed Under: Guides, Tutorials Tagged With: , , ,

Python Iterators: Part Two (The Next Function)

February 13, 2015 by Leave a Comment

Python iteratorsIn the first article in this series, we introduced Python iterators and how they can be used to streamline Python code. In this article, we will continue our look at iterators, beginning with the next function.

To support manual iteration code, Python 3.0 also provides a built-in function, next, that automatically calls an object’s __next__ method. Given an iterable object X, the call next(X) is the same as X.__next__(). With files, for example, either form could be used:

>>> f = open('simple.py')
>>> f.__next__()

>>> f = open('simple.py')
>>> next(f)

Technically, there is one more piece to the iteration protocol. When the for loop begins, it obtains an iterator from the iterable object by passing it to the iter built-in function; the object returned by iter has the required next method. We can illustrate this with the following code:

>>> LS = [1, 2, 3, 4, 5]
>>> myIter = iter(LS)
>>> myIter.next()
1
>>> myIter.next()
2

This initial step is not required for files, because a file object is its own iterator: files have their own __next__ method and so do not need to return a different object that does.

Lists and many other built-in object, are not their own iterators because they support multiple open operations. For such objects, we must call iter to start iterating. For example:

>>> LS = [1, 2, 3, 4, 5]
>>> iter(LS) is LS
False
>>> LS.__next__()
Traceback (most recent call last):
  File "<pyshell#50>", line 1, in 
    LS.__next__()
AttributeError: 'list' object has no attribute '__next__'

>>> myIter = iter(LS)
>>> myIter.__next__()
1
>>> next(myIter)
2

Although Python iteration tools call these functions automatically, we can use them to apply the iteration protocol manually, too. The following demonstrates the equivalence between automatic and manual iteration:

>>> LS = [1, 2, 3, 4, 5]
>>> for X in LS:
	print(X ** 2, end=' ')

1 4 9 16 25

>>> myIter = iter(LS)
>>> while True:
	try:
		X = next(I)
	except StopIteration:
		break
	print(X ** 2, end=' ')

1 4 9 16 25

To understand this code, you need to know that try statements run an action and catch exceptions (we covered that in the series of articles on exceptions). Also, for loops and other iteration contexts can sometimes work differently for user-defined classes, repeatedly indexing an object instead of running the iteration protocol.

External Links:

Python Iterators at Python Wiki

Python Iterator tutorial at bogotobogo.com

Filed Under: Guides, Tutorials Tagged With: , , , ,