What exactly are iterator iterable and iteration

Question

What is the most basic definition of  iterable    iterator  and  iteration  in Python   I have read multiple definitions but I am unable to identify the exact meaning as it still won t sink in   Can someone please help me with the 3 definitions in layman terms

User · Answer

Other people already explained comprehensively, what is iterable and iterator, so I will try to do the same thing with generators.

IMHO the main problem for understanding generators is a confusing use of the word “generator”, because this word is used in 2 different meanings:

as a tool for creating (generating) iterators,
- in the form of a function returning an iterator (i.e. with the yield statement(s) in its body),
- in the form of a generator expression
as a result of the use of that tool, i.e. the resulting iterator.
(In this meaning a generator is a special form of an iterator — the word “generator” points out how this iterator was created.)

Generator as a tool of the 1^st type:

In[2]: def my_generator():
  ...:     yield 100
  ...:     yield 200

In[3]: my_generator

Out[3]: <function __main__.my_generator()>

In[4]: type(my_generator)

Out[4]: function

Generator as a result (i.e. an iterator) of the use of this tool:

In[5]: my_iterator = my_generator()
In[6]: my_iterator

Out[6]: <generator object my_generator at 0x00000000053EAE48>

In[7]: type(my_iterator)

Out[7]: generator

Generator as a tool of the 2^nd type — indistinguishable from the resulting iterator of this tool:

In[8]: my_gen_expression = (2 * i for i in (10, 20))
In[9]: my_gen_expression

Out[9]: <generator object <genexpr> at 0x000000000542C048>

In[10]: type(my_gen_expression)

Out[10]: generator

User · Answer

Iterables have a   iter   method that instantiates a new iterator every time  Iterators implement a   next   method that returns individual items  and a   iter   method that returns self   Therefore  iterators are also iterable  but iterables are not iterators   Luciano Ramalho  Fluent Python

User · Answer

An iterable is a object which has a   iter     method  It can possibly iterated over several times  such as list  s and tuple  s   An iterator is the object which iterates  It is returned by an   iter     method  returns itself via its own   iter     method and has a next   method    next     in 3 x    Iteration is the process of calling this next   resp    next     until it raises StopIteration   Example    gt  gt  gt  a    1  2  3    iterable  gt  gt  gt  b1   iter a    iterator 1  gt  gt  gt  b2   iter a    iterator 2  independent of b1  gt  gt  gt  next b1  1  gt  gt  gt  next b1  2  gt  gt  gt  next b2    start over  as it is the first call to b2 1  gt  gt  gt  next b1  3  gt  gt  gt  next b1  Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  StopIteration  gt  gt  gt  b1   iter a    new one  start over  gt  gt  gt  next b1  1

User · Answer

The above answers are great  but as most of what I ve seen  don t stress the distinction enough for people like me   Also  people tend to get  too Pythonic  by putting definitions like  X is an object that has   foo     method  before   Such definitions are correct--they are based on duck-typing philosophy  but the focus on methods tends to get between when trying to understand the concept in its simplicity   So I add my version     In natural language    iteration is the process of taking one element at a time in a row of elements    In Python    iterable is an object that is  well  iterable  which simply put  means that it can be used in iteration  e g  with a for loop   How   By using iterator  I ll explain below      while iterator is an object that defines how to actually do the iteration--specifically what is the next element   That s why it must have next   method     Iterators are themselves also iterable  with the distinction that their   iter     method returns the same object  self   regardless of whether or not its items have been consumed by previous calls to next       So what does Python interpreter think when it sees for x in obj  statement      Look  a for loop    Looks like a job for an iterator     Let s get one        There s this obj guy  so let s ask him        Mr  obj  do you have your iterator        calls iter obj   which calls   obj   iter      which happily hands out a shiny new iterator  i         OK  that was easy    Let s start iterating then   x    i next       x    i next         Since Mr  obj succeeded in this test  by having certain method returning a valid iterator   we reward him with adjective  you can now call him  iterable Mr  obj    However  in simple cases  you don t normally benefit from having iterator and iterable separately   So you define only one object  which is also its own iterator    Python does not really care that  i handed out by obj wasn t all that shiny  but just the obj itself     This is why in most examples I ve seen  and what had been confusing me over and over   you can see   class IterableExample object        def   iter   self           return self      def next self           pass   instead of  class Iterator object       def next self           pass  class Iterable object       def   iter   self           return Iterator     There are cases  though  when you can benefit from having iterator separated from the iterable  such as when you want to have one row of items  but more  cursors    For example when you want to work with  current  and  forthcoming  elements  you can have separate iterators for both   Or multiple threads pulling from a huge list  each can have its own iterator to traverse over all items  See  Raymond s and  glglgl s answers above   Imagine what you could do   class SmartIterableExample object        def create iterator self             An amazingly powerful yet simple way to create arbitrary           iterator  utilizing object state  or not  if you are fan           of functional   magic and nuclear waste--no kittens hurt          pass      don t forget to add the next   method      def   iter   self           return self create iterator     Notes     I ll repeat again  iterator is not iterable   Iterator cannot be used as a  source  in for loop   What for loop primarily needs is   iter      that returns something with next     Of course  for is not the only iteration loop  so above applies to some other constructs as well  while      Iterator s next   can throw StopIteration to stop iteration   Does not have to  though  it can iterate forever or use other means  In the above  thought process    i does not really exist   I ve made up that name  There s a small change in Python 3 x  next   method  not the built-in  now must be called   next       Yes  it should have been like that all along  You can also think of it like this  iterable has the data  iterator pulls the next item   Disclaimer  I m not a developer of any Python interpreter  so I don t really know what the interpreter  thinks    The musings above are solely demonstration of how I understand the topic from other explanations  experiments and real-life experience of a Python newbie

User · Answer

iterable    1  2    iterator   iter iterable   print iterator   next          print iterator   next           so    iterable is an object that can be looped over  e g  list   string   tuple etc  using the iter function on our iterable object will return an iterator object  now this iterator object has method named   next    in Python 3  or just next in Python 2  by which you can access each element of iterable    so       OUTPUT OF ABOVE CODE WILL BE   1  2

User · Answer

For me  Python s glossery was most helpful for these questions  e g  for iterable it says   An object capable of returning its members one at a time  Examples of iterables include all sequence types  such as list  str  and tuple  and some non-sequence types like dict  file objects  and objects of any classes you define with an iter   method or with a getitem   method that implements Sequence semantics    Iterables can be used in a for loop and in many other places where a sequence is needed  zip    map          When an iterable object is passed as an argument to the built-in function iter    it returns an iterator for the object  This iterator is good for one pass over the set of values  When using iterables  it is usually not necessary to call iter   or deal with iterator objects yourself  The for statement does that automatically for you  creating a temporary unnamed variable to hold the iterator for the duration of the loop  See also iterator  sequence  and generator

User · Answer

Before dealing with the  iterables and iterator the major factor that decide the iterable and iterator is sequence  Sequence  Sequence is the collection of data  Iterable  Iterable are the sequence type object that support   iter   method   Iter method  Iter method take sequence as an input and create an object which is known as iterator  Iterator  Iterator are the object which call next method and transverse through the sequence  On calling the next method it returns the object that it traversed currently   example   x  1 2 3 4    x is a sequence which consists of collection of data  y iter x    On calling iter x  it returns a iterator only when the x object has iter method otherwise it raise an exception If it returns iterator then y is assign like this   y  1 2 3 4    As y is a iterator hence it support next   method  On calling next method it returns the individual elements of the list one by one   After returning the last element of the sequence if we again call the next method it raise an StopIteration error  example    gt  gt  gt  y next   1  gt  gt  gt  y next   2  gt  gt  gt  y next   3  gt  gt  gt  y next   4  gt  gt  gt  y next   StopIteration

User · Answer

I don   t know if it helps anybody but I always like to visualize concepts in my head to better understand them  So as I have a little son I visualize iterable iterator concept with bricks and white paper   Suppose we are in the dark room and on the floor we have bricks for my son  Bricks of different size  color  does not matter now  Suppose we have 5 bricks like those  Those 5 bricks can be described as an object     let   s say bricks kit  We can do many things with this bricks kit     can take one and then take second and then third  can change places of bricks  put first brick above the second  We can do many sorts of things with those  Therefore this bricks kit is an iterable object or sequence as we can go through each brick and do something with it  We can only do it like my little son     we can play with one brick at a time  So again I imagine myself this bricks kit to be an iterable   Now remember that we are in the dark room  Or almost dark  The thing is that we don   t clearly see those bricks  what color they are  what shape etc  So even if we want to do something with them     aka iterate through them     we don   t really know what and how because it is too dark   What we can do is near to first brick     as element of a bricks kit     we can put a piece of white fluorescent paper in order for us to see where the first brick-element is  And each time we take a brick from a kit  we replace the white piece of paper to a next brick in order to be able to see that in the dark room  This white piece of paper is nothing more than an iterator  It is an object as well  But an object with what we can work and play with elements of our iterable object     bricks kit   That by the way explains my early mistake when I tried the following in an IDLE and got a TypeError     gt  gt  gt  X    1 2 3 4 5    gt  gt  gt  next X   Traceback  most recent call last       File   lt pyshell 19 gt    line 1  in  lt module gt        next X   TypeError   list  object is not an iterator   List X here was our bricks kit but NOT a white piece of paper  I needed to find an iterator first    gt  gt  gt  X    1 2 3 4 5   gt  gt  gt  bricks kit    1 2 3 4 5   gt  gt  gt  white piece of paper   iter bricks kit   gt  gt  gt  next white piece of paper  1  gt  gt  gt  next white piece of paper  2  gt  gt  gt    Don   t know if it helps  but it helped me  If someone could confirm correct visualization of the concept  I would be grateful  It would help me to learn more

User · Answer

Iterable - something that is iterable is iterable  like sequences like lists  strings etc    Also it has either the   getitem   method or an   iter   method  Now if we use iter   function on that object  we ll get an iterator     Iterator - When we get the iterator object from the iter   function  we call   next     method  in python3  or simply next    in python2  to get elements one by one  This class or instance of this class is called an iterator   From docs -  The use of iterators pervades and unifies Python  Behind the scenes  the  for  statement calls  iter    on the container object  The function returns an iterator object that defines the method    next      which accesses elements in the container one at a time  When there are no more elements     next      raises a  StopIteration  exception which tells the  for  loop to terminate  You can call the    next      method using the  next    built-in function  this example shows how it all works    gt  gt  gt  s    abc   gt  gt  gt  it   iter s   gt  gt  gt  it  lt iterator object at 0x00A1DB50 gt   gt  gt  gt  next it   a   gt  gt  gt  next it   b   gt  gt  gt  next it   c   gt  gt  gt  next it  Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt      next it  StopIteration   Ex of a class -   class Reverse         Iterator for looping over a sequence backwards         def   init   self  data           self data   data         self index   len data      def   iter   self           return self     def   next   self           if self index    0              raise StopIteration         self index   self index - 1         return self data self index     gt  gt  gt  rev   Reverse  spam    gt  gt  gt  iter rev   lt   main   Reverse object at 0x00A1DB50 gt   gt  gt  gt  for char in rev          print char      m a p s

User · Answer

I don t think that you can get it much simpler than the documentation  however I ll try    Iterable is something that can be iterated over  In practice it usually means a sequence e g  something that has a beginning and an end and some way to go through all the items  in it  You can think Iterator as a helper pseudo-method  or pseudo-attribute  that gives  or holds  the next  or first  item in the iterable   In practice it is just an object that defines the method next    Iteration is probably best explained by the Merriam-Webster definition of the word        b   the repetition of a sequence of computer instructions a specified   number of times or until a condition is met     compare recursion

User · Answer

Iterators are objects that implement the iter and next methods  If those methods are defined  we can use for loop or comprehensions  class Squares      def   init   self  length           self length   length         self i   0              def   iter   self           print  calling   iter       this will be called first and only once         return self          def   next   self            print  calling   next       this will be called for each iteration         if self i  gt   self length              raise StopIteration         else              result   self i    2             self i    1             return result  Iterators get exhausted  It means after you iterate over items  you cannot reiterate  you have to create a new object  Let s say you have a class  which holds the cities properties and you want to iterate over  class Cities      def   init   self           self  cities     Brooklyn    Manhattan    Prag    Madrid    London           self  index   0          def   iter   self           return self          def   next   self           if self  index  gt   len self  cities               raise StopIteration         else              item   self  cities self  index              self  index    1             return item  Instance of class Cities is an iterator  However if you want to reiterate over cities  you have to create a new object which is an expensive operation  You can separate the class into 2 classes  one returns cities and second returns an iterator which gets the cities as init param  class Cities      def   init   self           self  cities     New York    Newark    Istanbul    London               def   len   self           return len self  cities     class CityIterator      def   init   self  city obj             cities is an instance of Cities         self  city obj   city obj         self  index   0              def   iter   self           return self          def   next   self           if self  index  gt   len self  city obj               raise StopIteration         else              item   self  city obj  cities self  index              self  index    1             return item  Now if we need to create a new iterator  we do not have to create the data again  which is cities  We creates cities object and pass it to the iterator  But we are still doing extra work  We could implement this by creating only one class  Iterable is a Python object that implements the iterable protocol  It requires only   iter     that returns a new instance of iterator object  class Cities      def   init   self           self  cities     New York    Newark    Istanbul    Paris                def   len   self           return len self  cities           def   iter   self           return self CityIterator self           class CityIterator          def   init   self  city obj               self  city obj   city obj             self  index   0          def   iter   self               return self          def   next   self               if self  index  gt   len self  city obj                   raise StopIteration             else                  item   self  city obj  cities self  index                  self  index    1                 return item  Iterators has   iter   and   next    iterables have   iter    so we can say Iterators are also iterables but they are iterables that get exhausted  Iterables on the other hand never become exhausted because they always return a new iterator that is then used to iterate You notice that the main part of the iterable code is in the iterator  and the iterable itself is nothing more than an extra layer that allows us to create and access the iterator  Iterating over an iterable Python has a built function iter   which calls the   iter      When we iterate over an iterable  Python calls the iter   which returns an iterator  then it starts using   next     of iterator to iterate over the data  NOte that in the above example  Cities creates an iterable but it is not a sequence type  it means we cannot get a city by an index  To fix this we should just add   get item   to the Cities class  class Cities      def   init   self           self  cities     New York    Newark    Budapest    Newcastle                def   len   self           return len self  cities           def   getitem   self  s     now a sequence type         return self  cities s           def   iter   self           return self CityIterator self           class CityIterator          def   init   self  city obj               self  city obj   city obj             self  index   0          def   iter   self               return self          def   next   self               if self  index  gt   len self  city obj                   raise StopIteration             else                  item   self  city obj  cities self  index                  self  index    1                 return item

User · Answer

Here s my cheat sheet    sequence           v    def   getitem   self  index  int                     raise IndexError                          def   iter   self                                                   return  lt iterator gt                                         -- gt  or  lt -----         def   next   self                                                                 raise StopIteration        v                      iterable                                                                v                  ---- gt  and  ------- gt  iterator                                                         v                                    iter  lt iterable gt    ----------------------                                                  def generator                                    yield 1                                                   generator expression  -                                                  - gt  generator    - gt  generator iterator  -    Quiz  Do you see how      every iterator is an iterable  a container object s   iter     method can be implemented as a generator  an iterable that has a   next   method is not necessarily an iterator    Answers    Every iterator must have an   iter   method  Having   iter   is enough to be an iterable  Therefore every iterator is an iterable  When   iter   is called it should return an iterator  return  lt iterator gt  in the diagram above   Calling a generator returns a generator iterator which is a type of iterator   class Iterable1      def   iter   self             a method  which is a function defined inside a class body            calling iter   converts iterable  tuple  to iterator         return iter  1 2 3    class Iterable2      def   iter   self             a generator         for i in  1  2  3               yield i  class Iterable3      def   iter   self             with PEP 380 syntax         yield from  1  2  3     passes assert list Iterable1       list Iterable2       list Iterable3        1  2  3   Here is an example   class MyIterable       def   init   self           self n   0      def   getitem   self  index  int           return  1  2  3  index       def   next   self           n   self n   self n   1         if n  gt  3              raise StopIteration         return n    if you can iter it without raising a TypeError  then it s an iterable  iter MyIterable       but obviously  MyIterable    is not an iterator since it does not have   an    iter    method  from collections abc import Iterator assert isinstance MyIterable    Iterator     AssertionError

User · Answer

Here s the explanation I use in teaching Python classes   An ITERABLE is    anything that can be looped over  i e  you can loop over a string or file  or anything that can appear on the right-side of a for-loop    for x in iterable      or anything you can call with iter   that will return an ITERATOR   iter obj  or an object that defines   iter   that returns a fresh ITERATOR  or it may have a   getitem   method suitable for indexed lookup    An ITERATOR is an object    with state that remembers where it is during iteration  with a   next   method that    returns the next value in the iteration updates the state to point at the next value signals when it is done by raising StopIteration  and that is self-iterable  meaning that it has an   iter   method that returns self     Notes    The   next   method in Python 3 is spelt next in Python 2  and The builtin function next   calls that method on the object passed to it    For example    gt  gt  gt  s    cat         s is an ITERABLE                      s is a str object that is immutable                      s has no state                      s has a   getitem     method    gt  gt  gt  t   iter s       t is an ITERATOR                      t has state  it starts by pointing at the  c                       t has a next   method and an   iter     method   gt  gt  gt  next t           the next   function returns the next value and advances the state  c   gt  gt  gt  next t           the next   function returns the next value and advances  a   gt  gt  gt  next t           the next   function returns the next value and advances  t   gt  gt  gt  next t           next   raises StopIteration to signal that iteration is complete Traceback  most recent call last       StopIteration   gt  gt  gt  iter t  is t      the iterator is self-iterable

User · Answer

Iteration is a general term for taking each item of something  one after another  Any time you use a loop  explicit or implicit  to go over a group of items  that is iteration   In Python  iterable and iterator have specific meanings   An iterable is an object that has an   iter   method which returns an iterator  or which defines a   getitem   method that can take sequential indexes starting from zero  and raises an IndexError when the indexes are no longer valid   So an iterable is an object that you can get an iterator from   An iterator is an object with a next  Python 2  or   next    Python 3  method    Whenever you use a for loop  or map  or a list comprehension  etc  in Python  the next method is called automatically to get each item from the iterator  thus going through the process of iteration   A good place to start learning would be the iterators section of the tutorial and the iterator types section of the standard types page  After you understand the basics  try the iterators section of the Functional Programming HOWTO

User · Answer

In Python everything is an object  When an object is said to be iterable  it means that you can step through  i e  iterate  the object as a collection   Arrays for example are iterable  You can step through them with a for loop  and go from index 0 to index n  n being the length of the array object minus 1   Dictionaries  pairs of key value  also called associative arrays  are also iterable  You can step through their keys   Obviously the objects which are not collections are not iterable  A bool object for example only have one value  True or False  It is not iterable  it wouldn t make sense that it s an iterable object    Read more  http   www lepus org uk ref companion Iterator xml

User · Answer

Here s another view using collections abc   This view may be useful the second time around or later  From collections abc we can see the following hierarchy  builtins object     Iterable         Iterator             Generator  i e  Generator is derived from Iterator is derived from Iterable is derived from the base object  Hence   Every iterator is an iterable  but not every iterable is an iterator   For example   1  2  3  and range 10  are iterables  but not iterators   x   iter  1  2  3   is an iterator and an iterable  A similar relationship exists between Iterator and Generator  Calling iter   on an iterator or a generator returns itself   Thus  if it is an iterator  then iter it  is it is True  Under the hood  a list comprehension like  2   x for x in nums  or a for loop like for x in nums   acts as though iter   is called on the iterable  nums  and then iterates over nums using that iterator   Hence  all of the following are functionally equivalent  with  say  nums  1  2  3     for x in nums  for x in iter nums   for x in iter iter nums    for x in iter iter iter iter iter nums

[python] What exactly are iterator, iterable, and iteration?

Examples related to python

Examples related to iterator

Examples related to iteration

Examples related to terminology

Examples related to iterable