{"id":137,"date":"2015-02-09T17:00:24","date_gmt":"2015-02-09T22:00:24","guid":{"rendered":"http:\/\/pfsensesetup.com\/pythonscript.net\/?p=137"},"modified":"2015-02-09T11:59:00","modified_gmt":"2015-02-09T16:59:00","slug":"python-iterators-part-one","status":"publish","type":"post","link":"http:\/\/pfsensesetup.com\/pythonscript.net\/python-iterators-part-one\/","title":{"rendered":"Python Iterators: Part One"},"content":{"rendered":"

\"Python<\/a>Now that we have completed our look at strings, we will begin to look at another useful object: Python iterators.<\/p>\n

The for loop is often used when we have to iterate over a series of numbers; for example:<\/p>\n

>>> for x in range(1,5):\r\n\tprint(x)\r\n\r\n1\r\n2\r\n3\r\n4\r\n<\/strong><\/pre>\n
The for loop can also work on any sequence type in Python, including lists, tuples and strings. For example:<\/p>\n
>>> for x in [1, 2, 3, 4]:\r\n\tprint(x)\r\n\r\n1\r\n2\r\n3\r\n4<\/strong>\r\n\r\n>>> for x in 'quick':\r\n\tprint(x * 2)\r\n\r\nqq\r\nuu\r\nii\r\ncc\r\nkk\r\n<\/strong><\/pre>\n
Actually, the for loop turns out to be even more generic than this: it works on any iterable object. In fact, this is true of all iteration tools that scan objects from left to right in Python, including for loops, list comprehensions, membership tests, the map built-in function, and more.<\/p>\n
Introduction to Python Iterators<\/strong><\/h3>\n
The concept of iterable objects is relatively recent in Python, but it has come to permeate the language’s design.<\/p>\n
We can get a better understanding of Python iterators and how they have become pervasive in the language’s design by looking at a built-in type such as the file. Open file objects have a method called readline, which reads one line of text from a file at a time. Each time we call the readline method, we advance to the next line. At the send of the file, an empty string is returned, which we can detect to break out of the loop:<\/p>\n
>>> f = open('simple.py')\r\n>>> f.readline()\r\n'a = [1, 2, 3, 4]\\n'\r\n>>> f.readline()\r\n'for x in a:\\n'\r\n>>> f.readline()\r\n'\tprint(x)\\n'\r\n>>> f.readline()\r\n'\tprint(x*2)\\n'\r\n>>> f.readline()\r\n''\r\n<\/strong><\/pre>\n
However, files also have a method name __next__ that has a nearly identical effect: it returns the next line from a file each time it is called. The only noticeable difference is that __next__ raises a built-in StopIteration exception at the end of file instead of returning an empty string:<\/p>\n
>>> f = open('simple.py')\r\n>>> f.__next__()\r\n'a = [1, 2, 3, 4,]\\n'\r\n>>> f.__next__()\r\n'for x in a:\\n'\r\n>>> f.__next__()\r\n'\\tprint(x)\\n'\r\n>>> f.__next__()\r\n'\\tprint(x*2)\\n'\r\n>>> f.__next__()\r\nTraceback (most recent call last):\r\n  File \"<pyshell#31>\", line 1, in \r\n    f.__next__()\r\nStopIteration\r\n<\/strong><\/pre>\n
This interface is exactly what we call the iteration protocol in Python. Any object with a __next__ method to advance to a next result, which raises StopIteration at the end of the series of results, is considered a Python iterator. Any such object may also be stepped through with a for loop or other iteration tool, because all Python iterators tools normally work internally by calling __next__ on each iteration and catching the StopIteration exception to determine when to exit.<\/p>\n
Thus, the best way to reach a text file line by line is to allow the for loop to automatically call __next__ to advance to the next line on each iteration. The file object’s Python iterator will do the work of automatically loading lines as you go. For example, if we want to read in simple.py line by line, we can code it as follows:<\/p>\n
>>> for line in open('simple.py'):\r\n\tprint(line, end='')\r\n\r\na = [1, 2, 3, 4]\r\nfor x in a:\r\n\tprint(x)\r\n\tprint(x*2)\r\n<\/strong><\/pre>\n
Almost as easily, we can iterate through the file and print it out in uppercase:<\/p>\n
>>> for line in open('simple.py'):\r\n\tprint(line.upper(), end='')\r\n\r\nA = [1, 2, 3, 4]\r\nFOR X IN A:\r\n\tPRINT(X)\r\n\tPRINT(X*2)\r\n<\/strong><\/pre>\n
Notice that the print uses end=” here to suppress adding a \\n, because line strings already have one. This is considered the best way to read text files line by line today for several reasons: it is the simplest to code, it might be the quickest to run, and is the best in terms of memory and usage. The older, original way to achieve the same effect with a for loop was to call the file readlines method to load the file’s content into memory as a list of line strings:<\/p>\n
>>> for line in open('simple.py').readlines():\r\n\tprint(line.upper(), end='')\r\n<\/strong><\/pre>\n
This readlines technique still works, but it is not considered the best practice today and performs poorly in terms of memory usage. In fact, because this version really does load the entire file into memory all at once, it will not even work for files too big to fit into the memoery space available on your computer. Rather, because it reads one line at a time, the Python iterator-based version is not prone to such memory issues.<\/p>\n
The iterator version might run quicker as well, though it can vary by release. It is also possible to read a file line by line with a while loop:<\/p>\n
>>> f = open('simple.py')\r\n>>> while True:\r\n\tline = f.readline()\r\n\tif not line: break\r\n\tprint(line.upper(), end='')\r\n\r\n\t\r\nA = [1, 2, 3, 4]\r\nFOR X IN A:\r\n\tPRINT(X)\r\n\tPRINT(X*2)\r\n<\/strong><\/pre>\n
However, this may run slower than the iterator-based for loop version, because Python iterators run at C language speed inside Python, whereas the while loop version run Python byte code through the Python virtual machine. Any time you swap Python code, for C code, speed tends to increase, though not in all cases.<\/p>\n
External Links:<\/strong><\/span><\/h3>\n
Python Iterators at Python Wiki<\/a><\/p>\n
Python Iterator tutorial at bogotobogo.com<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Now that we have completed our look at strings, we will begin to look at another useful object: Python iterators. The for loop is often used when we have to iterate over a series of numbers; for example: >>> for x in range(1,5): print(x) 1 2 3 4 The for loop can also work on […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,3],"tags":[],"class_list":["post-137","post","type-post","status-publish","format-standard","hentry","category-guides","category-tutorials","entry"],"_links":{"self":[{"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/posts\/137","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/comments?post=137"}],"version-history":[{"count":3,"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/posts\/137\/revisions"}],"predecessor-version":[{"id":141,"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/posts\/137\/revisions\/141"}],"wp:attachment":[{"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/media?parent=137"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/categories?post=137"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/pfsensesetup.com\/pythonscript.net\/wp-json\/wp\/v2\/tags?post=137"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}