Python - Iterator vs Iterable

There are many built-in functions and methods that return iterables and iterators. Here are a few examples:

range() returns an iterable

dict.keys() returns an iterable

dict.items() returns an iterable

dict.values() returns an iterable

enumerate() returns an iterator

zip() returns an iterator

reversed() returns an iterator

open() returns an iterator

map() returns an iterator

filter() returns an iterator

Some of the functions and methods return iterables, and some return iterators. We need to know the type of object that we are dealing with (iterable or iterator), otherwise we will get unexpected results. Let us see why it is so.

We know that after an iterator is exhausted, it raises a StopIteration exception each time next is called on it. There is no way to reset or restart an iterator. If you need to iterate over the same data stream again, then you will have to create a new fresh iterator by calling iter function over the iterable from which you got the iterator.

So, an iterator becomes a useless throw-away object once it is exhausted. It is not possible to reset or restart an iterator. You need to get a fresh iterator if you need to iterate again. The iteration tools that we have seen in the previous section, work internally by calling iter on the iterable and so they get a fresh iterator automatically.

Suppose we have an iterable x and we use it in these 3 iteration contexts:

for i in x:

pass

L = list(x)

n = max(x)

All these iteration tools will call iter on the iterable, get fresh iterators, and will use those iterators. This code will work as expected, and there is no problem here.

Now suppose we have an iterator y. You know that iterators can respond to the iter function, which can be used in any iteration context. So, we use this iterator y in the same three places:

for i in y:

pass

L = list(y)

n = max(y)

The for loop will work as expected, but the next two statements will not work as expected and they will not show any error also. This is because when an iterator is passed through the iter() function, you get the same iterator back. The for loop exhausted the iterator y, and when the list function called iter function on y, it got the same exhausted iterator back. When it called the next function on the exhausted iterator, a Stopiteration error was raised, which was caught. Similarly, in the last statement, when iter will be called on y, y will be only returned which is already exhausted. An exhausted iterator is like an empty container.

From these three iteration contexts, whichever is written first will work, and the other two will not work. If you write the max function before the for loop and list function, then the max function will work but the other two will not work, because in that case the max function will consume the iterator and the other two will get the exhausted iterator.

So, you can iterate many times over an iterable, but you can iterate only once over an iterator.

This is because when an iterable is passed to iter function, it returns a fresh iterator while when an iterator is passed to iter function it returns the same iterator.

If a function returns an iterable object, then that object supports multiple iterations, while the iterator objects support just one iteration. This is because an iterable can be used to get a fresh iterator every time iter is used on it. But for an iterator, the same iterator will be returned every time iter is used on it.