Python Iterators: Part Three

February 17, 2015 by Leave a Comment

Python iteratorsBesides files and physical sequences such as lists, other types also have useful iterators. In older versions of Python, for example, one would step through the keys of a dictionary by requesting the keys list explicitly:

>>> K = {'alpha':1, 'bravo':2, 'charlie':3}
>>> for key in K.keys():
	print(key,K[key])

alpha 1
bravo 2
charlie 3

In more recent versions of Python, however, dictionaries have an iterator that automatically, returns one key at a time in an iteration context:

>>> I = iter(K)
>>> next(I)
'alpha'
>>> next(I)
'bravo'
>>> next(I)
'charlie'
>>> next(I)
Traceback (most recent call last):
  File "<pyshell#88>", line 1, in 
    next(I)
StopIteration

The effect here as that we no longer need to call the keys method to step through dictionary keys. The for loop will use the iteration protocol to grab one key each time through.

Other Python object types also support the iterator protocol and therefore may be used in for loops as well. For example, a shelf is a persistent, dictionary-like object in which the values can be arbitrary Python objects, such as class instances, recursive data types, and objects. Shelves support the iterator protocol. So does os.popen, a tool for reading the output of shell commands:

>>> import os
>>> I = os.popen('dir')
>>> I.__next__()
' Volume in drive C has no label.\n'
>>> I.__next__()
' Volume Serial Number is 9664-E470\n'
>>> next(I)
Traceback (most recent call last):
  File "<pyshell#93>", line 1, in 
    next(I)
TypeError: '_wrap_close' object is not an iterator

Note that the popen objects support a P.next() method in Python 2.6. In 3.0 and later, they support the P.__next__() method, but not the next(P) built-in. It is not clear if this behavior will continue in future releases, but as of Python 3.4.1, it is still the case. This is only an issue for manual iteration; if you iterate over these objects automatically with for loops and other iteration contexts, they return successive lines in either Python version.

The iteration protocol is also the reason we have had to wrap some results in a list call to see their values all at once. Objects that are iterable return results one at a time, not in a physical list:

>>> RG = range(5)
>>> RG 
range(0, 5)
>>> I = iter(RG)
>>> next(I)
0
>>> next(I)
1
>>> list(range(5))
[0, 1, 2, 3, 4]

Now that you have a better understanding of this protocol, you should be able to see how it explains why the enumerate tool introduced in the prior chapter works the way it does:

>>> EN = enumerate('quick')
>>> EN

>>> I = iter(EN)
>>> next(I)
(0, 'q')
>>> next(I)
(1, 'u')
>>> list(enumerate('quick'))
[(0, 'q'), (1, 'u'), (2, 'i'), (3, 'c'), (4, 'k')]

We don’t normally see what is going on under the hood because for loops run it for us automatically to step through results. In face, everything that scans left-to-right in Python employes the iteration protocol.

External Links:

Python Iterators at Python Wiki

Python Iterator tutorial at bogotobogo.com

Filed Under: Guides, Tutorials Tagged With: , , , , , , ,

Python Iterators: Part Two (The Next Function)

February 13, 2015 by Leave a Comment

Python iteratorsIn the first article in this series, we introduced Python iterators and how they can be used to streamline Python code. In this article, we will continue our look at iterators, beginning with the next function.

To support manual iteration code, Python 3.0 also provides a built-in function, next, that automatically calls an object’s __next__ method. Given an iterable object X, the call next(X) is the same as X.__next__(). With files, for example, either form could be used:

>>> f = open('simple.py')
>>> f.__next__()

>>> f = open('simple.py')
>>> next(f)

Technically, there is one more piece to the iteration protocol. When the for loop begins, it obtains an iterator from the iterable object by passing it to the iter built-in function; the object returned by iter has the required next method. We can illustrate this with the following code:

>>> LS = [1, 2, 3, 4, 5]
>>> myIter = iter(LS)
>>> myIter.next()
1
>>> myIter.next()
2

This initial step is not required for files, because a file object is its own iterator: files have their own __next__ method and so do not need to return a different object that does.

Lists and many other built-in object, are not their own iterators because they support multiple open operations. For such objects, we must call iter to start iterating. For example:

>>> LS = [1, 2, 3, 4, 5]
>>> iter(LS) is LS
False
>>> LS.__next__()
Traceback (most recent call last):
  File "<pyshell#50>", line 1, in 
    LS.__next__()
AttributeError: 'list' object has no attribute '__next__'

>>> myIter = iter(LS)
>>> myIter.__next__()
1
>>> next(myIter)
2

Although Python iteration tools call these functions automatically, we can use them to apply the iteration protocol manually, too. The following demonstrates the equivalence between automatic and manual iteration:

>>> LS = [1, 2, 3, 4, 5]
>>> for X in LS:
	print(X ** 2, end=' ')

1 4 9 16 25

>>> myIter = iter(LS)
>>> while True:
	try:
		X = next(I)
	except StopIteration:
		break
	print(X ** 2, end=' ')

1 4 9 16 25

To understand this code, you need to know that try statements run an action and catch exceptions (we covered that in the series of articles on exceptions). Also, for loops and other iteration contexts can sometimes work differently for user-defined classes, repeatedly indexing an object instead of running the iteration protocol.

External Links:

Python Iterators at Python Wiki

Python Iterator tutorial at bogotobogo.com

Filed Under: Guides, Tutorials Tagged With: , , , ,

Python Programming: Part Three (Conditional Statements; Lists, Tuples and Dictionaries)

October 28, 2014 by Leave a Comment

Python listIn the previous article, we covered variables, and how to save and run Python modules. In this article, we will introduce some more basic concepts including statements that alter the control flow of Python, and lists, tuples and dictionaries.

Up to this point, we have been executing Python statements using a linear control flow. A programming language is of minimal value, however, without conditional statements. Python provides three such statements: if, elif, and else. For example:

>>> if x < 0:
                print(‘x is a negative number’)

In this example, we used the comparison operator “<” (less than) to test to see if x is less than 0. If x is less than zero, we will print a statement that tells us x is a negative number. We might also want to print something out if x is zero or positive:

>>> if x < 0:
               print(‘x is a negative number’)
         elif x == 0:
                print(‘x equals zero’)
         else:
                print(‘x is a positive number’)

Here, the first two lines are the same, but if x is not less than zero, we test it to see if it is equal to zero and if it is, we print a message. If x is not less than zero or zero, we assume it is positive and print another message. Another useful control flow statement is the for statement. The for statement in Python differs a bit from the for statement in C and Pascal. Rather than always iterating over an arithmetic progression of numbers, as in Pascal, or giving the user the ability to define both the iteration step and halting condition as in C, Python’s for statement iterates over the items of any sequence in the order that they appear in the sequence. For example:

>>> word = ‘coffee’
>>> for c in word:
                 print(c)

This will print every letter in the string word on a separate line. If you need to iterate over a sequence of numbers, the built in function range() comes in handy. It generates arithmetic progressions:

>>> for i in range(10):
                 print(i,’ ‘,i**2)

This code will print out two columns: one containing integers 0 through 9, the other containing their squares. If you specify only a single parameter, 0 will be the lower bound and the specified parameter will be the upper bound. But you can specify a lower bound:

>>> for i in range(5,10):
                print(i,’ ‘,i**2)

This code will print out integers 5 through 9 and their squares. We can also specify a step value:

>>> for i in range(1,10,2):
                 print(i,’ ‘,i**2)

This code will print out all odd-numbered integers from 1 through 9 and their squares.

Lists, Tuples and Dictionaries

In C, there are arrays, which act as a compound data type. In Python, there are two sequence types not yet covered: lists and tuples. There are also dictionaries, which is not a sequence type, but is nonetheless very useful.

Lists are collections of some data type. Creating a list can be done quite easily:

>>> mylist = [5, 10, 15]

This will create a list with three items in it. We can iterate through the list as well:

>>> for i in mylist:
                 print(i)

This will print each item in mylist on a separate line. You can use the assignment operator on a list:

>>> otherlist = mylist

Now we will be able to access the list, but it is important to note that otherlist is not a separate copy of mylist. It also points to mylist. So this statement:

>>> otherlist[0] = 100

will alter the contents of mylist:

>>> print(mylist)
[100, 10, 15]

You can also nest a list within a list. For example:

>>> secondlist = [1, 2, 3, 4]
>>> mylist[0] = secondlist

nests secondlist inside mylist as the first element.

You can also create an empty list, by specifying an empty set of brackets in the assignment, like this:

mylist = []

Lists have a built-in method called append that can be used to append an item to the end of a list:

>>> mylist.append(125)

appends 125 to the end of the list.

Whereas a list is a mutable sequence data type, a tuple is an immutable sequence data type. A tuple consists of a number of values separated by commas. For example:

>>> tu = 1, 2, 3

A tuple need not contain all the same data type. The following statement is valid:

>>> tu = 5, 10, 15, ‘twenty’

To confirm that tuples are immutable, we can try to change one of the items:

>>> tu[0] = 121
Traceback (most recent call last):
File “”, line 1, in
TypeError: ‘tuple’ object does not support item assignment

On output, tuples are always enclosed in parentheses, so that nested tuples are interpreted correctly; they may be input with or without surrounding parenthesis, although often parenthesis are necessary anyway if the tuple is part of a larger expression. It is not possible to make an assignment to an individual item of a tuple; however, it is possible to create tuples which contain mutable objects, such as lists.

Another useful data type built into Python is the dictionary. Unlike sequences, which are indexed by a range of numbers, dictionaries are indexed by keys, which can be any immutable type. Strings and numbers can always be keys. Tuples can be used as keys if they contain only strings, numbers, or tuples. If a tuple contains any mutable object either directly or indirectly, it cannot be used as a key. You cannot use lists as keys, since lists can be modified.

You can think of dictionaries as unordered sets of key: value pairs, with the requirement that the keys are unique within one dictionary. A pair of braces creates an empty dictionary. Placing a comma-separated list of key:value pairs within the braces adds initial key:value pairs to the dictionary.

Here is an example of a dictionary:

>>> generals = { ‘Germany’: ‘Hindenburg’, ‘Russia’: ‘Samsonov’, ‘France’: ‘Nivelle’, ‘United Kingdom’: ‘Haig’ }
>>> print(generals[‘Germany’])
‘Hindenburg’
>>> del generals[‘Russia’]
>>> generals[‘France’] = ‘Foch’
>>> print(generals)
{‘Germany’: ‘Hindenburg’, ‘France’: ‘Foch’, ‘United Kingdom’: ‘Haig’}

You can use for to iterate through a dictionary. For example:

>>> for c in generals.keys()
print(generals[c])

Hindenburg
Foch
Haig

In the next article, we will introduce the concept of modules, and write our first function.

External Links:

Python documentation from the official Python website

Filed Under: Guides Tagged With: , , , , , ,