What does the yield keyword do

Question

What is the use of the yield keyword in Python  and what does it do   For example  I m trying to understand this code1   def  get child candidates self  distance  min dist  max dist       if self  leftchild and distance - max dist  lt  self  median          yield self  leftchild     if self  rightchild and distance   max dist  gt   self  median          yield self  rightchild     And this is the caller   result  candidates        self  while candidates      node   candidates pop       distance   node  get dist obj      if distance  lt   max dist and distance  gt   min dist          result extend node  values      candidates extend node  get child candidates distance  min dist  max dist   return result   What happens when the method  get child candidates is called  Is a list returned  A single element  Is it called again  When will subsequent calls stop      1  This piece of code was written by Jochen Schulz  jrschulz   who made a great Python library for metric spaces  This is the link to the complete source  Module mspace

User · Answer

I was going to post  read page 19 of Beazley s  Python  Essential Reference  for a quick description of generators   but so many others have posted good descriptions already   Also  note that yield can be used in coroutines as the dual of their use in generator functions   Although it isn t the same use as your code snippet   yield  can be used as an expression in a function   When a caller sends a value to the method using the send   method  then the coroutine will execute until the next  yield  statement is encountered   Generators and coroutines are a cool way to set up data-flow type applications   I thought it would be worthwhile knowing about the other use of the yield statement in functions

User · Answer

yield is similar to return  The difference is    yield makes a function iterable  in the following example primes n   1  function becomes iterable   What it essentially means is the next time the function is called  it will continue from where it left  which is after the line of yield expression    def isprime n       if n    1          return False     for x in range 2  n           if n   x    0              return False     else          return True  def primes n   1      while True          if isprime n   yield n        n    1   for n in primes        if n  gt  100  break     print n    In the above example if isprime n  is true it will return the prime number  In the next iteration it will continue from the next line   n    1

User · Answer

The yield keyword is reduced to two simple facts    If the compiler detects the yield keyword anywhere inside a function  that function no longer returns via the return statement  Instead  it immediately returns a lazy  pending list  object called a generator A generator is iterable  What is an iterable  It s anything like a list or set or range or dict-view  with a built-in protocol for visiting each element in a certain order    In a nutshell  a generator is a lazy  incrementally-pending list  and yield statements allow you to use function notation to program the list values the generator should incrementally spit out   generator   myYieldingFunction      x   list generator      generator        v  x 0                       generator              v  x 0   x 1                             generator                    v  x 0   x 1   x 2                                     StopIteration exception  x 0   x 1   x 2       done  list   x 0   x 1   x 2       Example  Let s define a function makeRange that s just like Python s range  Calling makeRange n  RETURNS A GENERATOR   def makeRange n         return 0 1 2     n-1     i   0     while i  lt  n          yield i         i    1   gt  gt  gt  makeRange 5   lt generator object makeRange at 0x19e4aa0 gt    To force the generator to immediately return its pending values  you can pass it into list    just like you could any iterable     gt  gt  gt  list makeRange 5    0  1  2  3  4      Comparing example to  just returning a list   The above example can be thought of as merely creating a list which you append to and return     list-version                        generator-version def makeRange n                     def makeRange n          return  0 1 2     n-1               return 0 1 2     n-1        TO RETURN                      gt      i   0                               i   0     while i  lt  n                         while i  lt  n          TO RETURN     i                     yield i         i    1                              i    1     indented     return TO RETURN               gt    gt  gt  gt  makeRange 5   0  1  2  3  4    There is one major difference  though  see the last section     How you might use generators  An iterable is the last part of a list comprehension  and all generators are iterable  so they re often used like so                        ITERABLE   gt  gt  gt   x 10 for x in makeRange 5    10  11  12  13  14    To get a better feel for generators  you can play around with the itertools module  be sure to use chain from iterable rather than chain when warranted   For example  you might even use generators to implement infinitely-long lazy lists like itertools count    You could implement your own def enumerate iterable   zip count    iterable   or alternatively do so with the yield keyword in a while-loop   Please note  generators can actually be used for many more things  such as implementing coroutines or non-deterministic programming or other elegant things  However  the  lazy lists  viewpoint I present here is the most common use you will find     Behind the scenes  This is how the  Python iteration protocol  works  That is  what is going on when you do list makeRange 5    This is what I describe earlier as a  lazy  incremental list     gt  gt  gt  x iter range 5    gt  gt  gt  next x  0  gt  gt  gt  next x  1  gt  gt  gt  next x  2  gt  gt  gt  next x  3  gt  gt  gt  next x  4  gt  gt  gt  next x  Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  StopIteration   The built-in function next   just calls the objects  next   function  which is a part of the  iteration protocol  and is found on all iterators  You can manually use the next   function  and other parts of the iteration protocol  to implement fancy things  usually at the expense of readability  so try to avoid doing that       Minutiae  Normally  most people would not care about the following distinctions and probably want to stop reading here   In Python-speak  an iterable is any object which  understands the concept of a for-loop  like a list  1 2 3   and an iterator is a specific instance of the requested for-loop like  1 2 3    iter      A generator is exactly the same as any iterator  except for the way it was written  with function syntax    When you request an iterator from a list  it creates a new iterator  However  when you request an iterator from an iterator  which you would rarely do   it just gives you a copy of itself   Thus  in the unlikely event that you are failing to do something like this      gt  x   myRange 5   gt  list x   0  1  2  3  4   gt  list x           then remember that a generator is an iterator  that is  it is one-time-use  If you want to reuse it  you should call myRange      again  If you need to use the result twice  convert the result to a list and store it in a variable x   list myRange 5    Those who absolutely need to clone a generator  for example  who are doing terrifyingly hackish metaprogramming  can use itertools tee if absolutely necessary  since the copyable iterator Python PEP standards proposal has been deferred

User · Answer

All of the answers here are great  but only one of them  the most voted one  relates to how your code works  Others are relating to generators in general  and how they work   So I won t repeat what generators are or what yields do  I think these are covered by great existing answers  However  after spending few hours trying to understand a similar code to yours  I ll break it down how it works   Your code traverse a binary tree structure  Let s take this tree for example       5          3   6          1   4   8   And another simpler implementation of a binary-search tree traversal   class Node object      def   iter   self       if self has left child            for child in self left              yield child      yield self val      if self has right child            for child in self right              yield child   The execution code is on the Tree object  which implements   iter   as this   def   iter   self        class EmptyIter            def next self               raise StopIteration      if self root          return self root   iter         return EmptyIter     The while candidates statement can be replaced with for element in tree  Python translate this to  it   iter TreeObj     returns iter self root  which calls self root   iter     for element in it          process element       Because Node   iter   function is a generator  the code inside it is executed per iteration  So the execution would look like this    root element is first  check if it has left childs and for iterate them  let s call it it1 because its the first iterator object  it has a child so the for is executed  The for child in self left creates a new iterator from self left  which is a Node object itself  it2  Same logic as 2  and a new iterator is created  it3  Now we reached the left end of the tree  it3 has no left childs so it continues and yield self value On the next call to next it3  it raises StopIteration and exists since it has no right childs  it reaches to the end of the function without yield anything  it1 and it2 are still active - they are not exhausted and calling next it2  would yield values  not raise StopIteration Now we are back to it2 context  and call next it2  which continues where it stopped  right after the yield child statement  Since it has no more left childs it continues and yields it s self val    The catch here is that every iteration creates sub-iterators to traverse the tree  and holds the state of the current iterator  Once it reaches the end it traverse back the stack  and values are returned in the correct order  smallest yields value first    Your code example did something similar in a different technique  it populated a one-element list for every child  then on the next iteration it pops it and run the function code on the current object  hence the self    I hope this contributed a little to this legendary topic  I spent several good hours drawing this process to understand it

User · Answer

yield is similar to return  The difference is    yield makes a function iterable  in the following example primes n   1  function becomes iterable   What it essentially means is the next time the function is called  it will continue from where it left  which is after the line of yield expression    def isprime n       if n    1          return False     for x in range 2  n           if n   x    0              return False     else          return True  def primes n   1      while True          if isprime n   yield n        n    1   for n in primes        if n  gt  100  break     print n    In the above example if isprime n  is true it will return the prime number  In the next iteration it will continue from the next line   n    1

User · Answer

A simple generator function  def my gen        n   1     print  This is printed first         Generator function contains yield statements     yield n      n    1     print  This is printed second       yield n      n    1     print  This is printed at last       yield n   yield statement pauses the function saving all its states and later continues from there on successive calls   https   www programiz com python-programming generator

User · Answer

Here is a mental image of what yield does   I like to think of a thread as having a stack  even when it s not implemented that way    When a normal function is called  it puts its local variables on the stack  does some computation  then clears the stack and returns  The values of its local variables are never seen again   With a yield function  when its code begins to run  i e  after the function is called  returning a generator object  whose next   method is then invoked   it similarly puts its local variables onto the stack and computes for a while  But then  when it hits the yield statement  before clearing its part of the stack and returning  it takes a snapshot of its local variables and stores them in the generator object  It also writes down the place where it s currently up to in its code  i e  the particular yield statement    So it s a kind of a frozen function that the generator is hanging onto   When next   is called subsequently  it retrieves the function s belongings onto the stack and re-animates it  The function continues to compute from where it left off  oblivious to the fact that it had just spent an eternity in cold storage   Compare the following examples   def normalFunction        return     if False          pass  def yielderFunction        return     if False          yield 12   When we call the second function  it behaves very differently to the first  The yield statement might be unreachable  but if it s present anywhere  it changes the nature of what we re dealing with    gt  gt  gt  yielderFunction    lt generator object yielderFunction at 0x07742D28 gt    Calling yielderFunction   doesn t run its code  but makes a generator out of the code   Maybe it s a good idea to name such things with the yielder prefix for readability     gt  gt  gt  gen   yielderFunction    gt  gt  gt  dir gen      class              iter         Returns gen itself  to make it work uniformly with containers                  when given to a for loop   Containers return an iterator instead     close     gi code     gi frame     gi running     next           The method that runs the function s body    send     throw     The gi code and gi frame fields are where the frozen state is stored  Exploring them with dir      we can confirm that our mental model above is credible

User · Answer

The function containing the yield keyword is a generator  When you call it  its formal parameters are bound to actual arguments  but its body isn t actually evaluated  Instead  a generator-iterator is returned  Each call to the generator-iterator s next   method performs another pass through the iterative algorithm  Each step s value is the value specified by the yield keyword  Think of yield as the generator-iterator version of return  indicating the boundary between each iteration of the algorithm  Each time you call next    the generator code resumes from the statement following the yield

User · Answer

For those who prefer a minimal working example  meditate on this interactive Python session    gt  gt  gt  def f          yield 1       yield 2       yield 3       gt  gt  gt  g   f    gt  gt  gt  for i in g        print i       1 2 3  gt  gt  gt  for i in g        print i        gt  gt  gt    Note that this time nothing was printed

User · Answer

An analogy could help to grasp the idea here     Imagine that you have created an amazing machine that is capable of generating thousands and thousands of lightbulbs per day  The machine generates these lightbulbs in boxes with a unique serial number  You don t have enough space to store all these lightbulbs at the same time  i e   you cannot keep up with the speed of the machine due to storage limitation   so you would like to adjust this machine to generate lightbulbs on demand   Python generators don t differ much from this concept   Imagine that you have a function x that generates unique serial numbers for the boxes  Obviously  you can have a very large number of such barcodes generated by the function  A wiser  and space efficient  option is to generate those serial numbers on-demand   Machine s code   def barcode generator        serial number   10000    Initial barcode     while True          yield serial number         serial number    1   barcode   barcode generator   while True      number of lightbulbs to generate   int input  How many lightbulbs to generate          barcodes    next barcode  for   in range number of lightbulbs to generate       print barcodes         function to create the next batch of lightbulbs barcodes       produce more   input  Produce more   Y n          if produce more     n           break   As you can see we have a self-contained  function  to generate the next unique serial number each time  This function returns back a generator  As you can see we are not calling the function each time we need a new serial number  but we are using next   given the generator to obtain the next serial number   Output   How many lightbulbs to generate  5  10000  10001  10002  10003  10004  Produce more   Y n   y How many lightbulbs to generate  6  10005  10006  10007  10008  10009  10010  Produce more   Y n   y How many lightbulbs to generate  7  10011  10012  10013  10014  10015  10016  10017  Produce more   Y n   n

User · Answer

yield is just like return - it returns whatever you tell it to  as a generator   The difference is that the next time you call the generator  execution starts from the last call to the yield statement  Unlike return  the stack frame is not cleaned up when a yield occurs  however control is transferred back to the caller  so its state will resume the next time the function is called   In the case of your code  the function get child candidates is acting like an iterator so that when you extend your list  it adds one element at a time to the new list   list extend calls an iterator until it s exhausted  In the case of the code sample you posted  it would be much clearer to just return a tuple and append that to the list

User · Answer

There s one extra thing to mention  a function that yields doesn t actually have to terminate  I ve written code like this   def fib        last  cur   0  1     while True           yield cur         last  cur   cur  last   cur   Then I can use it in other code like this   for f in fib        if some condition  break     coolfuncs f     It really helps simplify some problems  and makes some things easier to work with

User · Answer

All great answers  however a bit difficult for newbies   I assume you have learned the return statement   As an analogy  return and yield are twins  return means  return and stop  whereas  yield  means  return  but continue         Try to get a num list with return       def num list n       for i in range n           return i   Run it   In  5   num list 3  Out 5   0   See  you get only a single number rather than a list of them  return never allows you prevail happily  just implements once and quit         There comes yield      Replace return with yield   In  10   def num list n                for i in range n                    yield i           In  11   num list 3  Out 11    lt generator object num list at 0x10327c990 gt   In  12   list num list 3   Out 12    0  1  2    Now  you win to get all the numbers   Comparing to return which runs once and stops  yield runs times you planed  You can interpret return as return one of them  and yield as return all of them  This is called iterable         One more step we can rewrite yield statement with return      In  15   def num list n                result                   for i in range n                    result append i               return result  In  16   num list 3  Out 16    0  1  2    It s the core about yield   The difference between a list return outputs and the object yield output is   You will always get  0  1  2  from a list object but only could retrieve them from  the object yield output  once  So  it has a new name generator object as displayed in Out 11    lt generator object num list at 0x10327c990 gt    In conclusion  as a metaphor to grok it    return and yield are twins list and generator are twins

User · Answer

It s returning a generator  I m not particularly familiar with Python  but I believe it s the same kind of thing as C  s iterator blocks if you re familiar with those   The key idea is that the compiler interpreter whatever does some trickery so that as far as the caller is concerned  they can keep calling next   and it will keep returning values - as if the generator method was paused  Now obviously you can t really  pause  a method  so the compiler builds a state machine for you to remember where you currently are and what the local variables etc look like  This is much easier than writing an iterator yourself

User · Answer

Here s a simple yield based approach  to compute the fibonacci series  explained   def fib limit 50       a  b   0  1     for i in range limit          yield b        a  b   b  a b   When you enter this into your REPL and then try and call it  you ll get a mystifying result    gt  gt  gt  fib    lt generator object fib at 0x7fa38394e3b8 gt    This is because the presence of yield signaled to Python that you want to create a generator  that is  an object that generates values on demand   So  how do you generate these values  This can either be done directly by using the built-in function next  or  indirectly by feeding it to a construct that consumes values    Using the built-in next   function  you directly invoke  next   next    forcing the generator to produce a value    gt  gt  gt  g   fib    gt  gt  gt  next g  1  gt  gt  gt  next g  1  gt  gt  gt  next g  2  gt  gt  gt  next g  3  gt  gt  gt  next g  5   Indirectly  if you provide fib to a for loop  a list initializer  a tuple initializer  or anything else that expects an object that generates produces values  you ll  consume  the generator until no more values can be produced by it  and it returns    results      for i in fib 30           consumes fib     results append i     can also be accomplished with results   list fib 30     consumes fib   Similarly  with a tuple initializer     gt  gt  gt  tuple fib 5           consumes fib  1  1  2  3  5    A generator differs from a function in the sense that it is lazy  It accomplishes this by maintaining it s local state and allowing you to resume whenever you need to    When you first invoke fib by calling it   f   fib     Python compiles the function  encounters the yield keyword and simply returns a generator object back at you  Not very helpful it seems    When you then request it generates the first value  directly or indirectly  it executes all statements that it finds  until it encounters a yield  it then yields back the value you supplied to yield and pauses  For an example that better demonstrates this  let s use some print calls  replace with print  text  if on Python 2    def yielder value           This is an infinite generator  Only use next on it          while 1          print  I m going to generate the value for you           print  Then I ll pause for a while           yield value         print  Let s go through it again      Now  enter in the REPL    gt  gt  gt  gen   yielder  Hello  yield      you have a generator object now waiting for a command for it to generate a value  Use next and see what get s printed    gt  gt  gt  next gen    runs until it finds a yield I m going to generate the value for you Then I ll pause for a while  Hello  yield     The unquoted results are what s printed  The quoted result is what is returned from yield  Call next again now    gt  gt  gt  next gen    continues from yield and runs again Let s go through it again  I m going to generate the value for you Then I ll pause for a while  Hello  yield     The generator remembers it was paused at yield value and resumes from there  The next message is printed and the search for the yield statement to pause at it performed again  due to the while loop

User · Answer

To understand what yield does  you must understand what generators are  And before you can understand generators  you must understand iterables  Iterables When you create a list  you can read its items one by one  Reading its items one by one is called iteration   gt  gt  gt  mylist    1  2  3   gt  gt  gt  for i in mylist         print i  1 2 3  mylist is an iterable  When you use a list comprehension  you create a list  and so an iterable   gt  gt  gt  mylist    x x for x in range 3    gt  gt  gt  for i in mylist         print i  0 1 4  Everything you can use  quot for    in    quot  on is an iterable  lists  strings  files    These iterables are handy because you can read them as much as you wish  but you store all the values in memory and this is not always what you want when you have a lot of values  Generators Generators are iterators  a kind of iterable you can only iterate over once  Generators do not store all the values in memory  they generate the values on the fly   gt  gt  gt  mygenerator    x x for x in range 3    gt  gt  gt  for i in mygenerator         print i  0 1 4  It is just the same except you used    instead of     BUT  you cannot perform for i in mygenerator a second time since generators can only be used once  they calculate 0  then forget about it and calculate 1  and end calculating 4  one by one  Yield yield is a keyword that is used like return  except the function will return a generator   gt  gt  gt  def createGenerator           mylist   range 3         for i in mylist             yield i i      gt  gt  gt  mygenerator   createGenerator     create a generator  gt  gt  gt  print mygenerator    mygenerator is an object   lt generator object createGenerator at 0xb7555c34 gt   gt  gt  gt  for i in mygenerator          print i  0 1 4  Here it s a useless example  but it s handy when you know your function will return a huge set of values that you will only need to read once  To master yield  you must understand that when you call the function  the code you have written in the function body does not run  The function only returns the generator object  this is a bit tricky  -  Then  your code will continue from where it left off each time for uses the generator  Now the hard part  The first time the for calls the generator object created from your function  it will run the code in your function from the beginning until it hits yield  then it ll return the first value of the loop  Then  each subsequent call will run another iteration of the loop you have written in the function and return the next value  This will continue until the generator is considered empty  which happens when the function runs without hitting yield  That can be because the loop has come to an end  or because you no longer satisfy an  quot if else quot    Your code explained Generator    Here you create the method of the node object that will return the generator def  get child candidates self  distance  min dist  max dist          Here is the code that will be called each time you use the generator object         If there is still a child of the node object on its left       AND if the distance is ok  return the next child     if self  leftchild and distance - max dist  lt  self  median          yield self  leftchild        If there is still a child of the node object on its right       AND if the distance is ok  return the next child     if self  rightchild and distance   max dist  gt   self  median          yield self  rightchild        If the function arrives here  the generator will be considered empty       there is no more than two values  the left and the right children  Caller    Create an empty list and a list with the current object reference result  candidates   list     self     Loop on candidates  they contain only one element at the beginning  while candidates         Get the last candidate and remove it from the list     node   candidates pop          Get the distance between obj and the candidate     distance   node  get dist obj         If distance is ok  then you can fill the result     if distance  lt   max dist and distance  gt   min dist          result extend node  values         Add the children of the candidate in the candidate s list       so the loop will keep running until it will have looked       at all the children of the children of the children  etc  of the candidate     candidates extend node  get child candidates distance  min dist  max dist    return result  This code contains several smart parts   The loop iterates on a list  but the list expands while the loop is being iterated  -  It s a concise way to go through all these nested data even if it s a bit dangerous since you can end up with an infinite loop  In this case  candidates extend node  get child candidates distance  min dist  max dist   exhaust all the values of the generator  but while keeps creating new generator objects which will produce different values from the previous ones since it s not applied on the same node   The extend   method is a list object method that expects an iterable and adds its values to the list    Usually we pass a list to it   gt  gt  gt  a    1  2   gt  gt  gt  b    3  4   gt  gt  gt  a extend b   gt  gt  gt  print a   1  2  3  4   But in your code  it gets a generator  which is good because   You don t need to read the values twice  You may have a lot of children and you don t want them all stored in memory   And it works because Python does not care if the argument of a method is a list or not  Python expects iterables so it will work with strings  lists  tuples  and generators  This is called duck typing and is one of the reasons why Python is so cool  But this is another story  for another question    You can stop here  or read a little bit to see an advanced use of a generator  Controlling a generator exhaustion  gt  gt  gt  class Bank      Let s create a bank  building ATMs        crisis   False        def create atm self              while not self crisis                 yield  quot  100 quot   gt  gt  gt  hsbc   Bank     When everything s ok the ATM gives you as much as you want  gt  gt  gt  corner street atm   hsbc create atm    gt  gt  gt  print corner street atm next     100  gt  gt  gt  print corner street atm next     100  gt  gt  gt  print  corner street atm next   for cash in range 5       100     100     100     100     100    gt  gt  gt  hsbc crisis   True   Crisis is coming  no more money   gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  wall street atm   hsbc create atm     It s even true for new ATMs  gt  gt  gt  print wall street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  hsbc crisis   False   The trouble is  even post-crisis the ATM remains empty  gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  brand new atm   hsbc create atm     Build a new one to get back in business  gt  gt  gt  for cash in brand new atm         print cash  100  100  100  100  100  100  100  100  100      Note  For Python 3  useprint corner street atm   next      or print next corner street atm   It can be useful for various things like controlling access to a resource  Itertools  your best friend The itertools module contains special functions to manipulate iterables  Ever wish to duplicate a generator  Chain two generators  Group values in a nested list with a one-liner  Map   Zip without creating another list  Then just import itertools  An example  Let s see the possible orders of arrival for a four-horse race   gt  gt  gt  horses    1  2  3  4   gt  gt  gt  races   itertools permutations horses   gt  gt  gt  print races   lt itertools permutations object at 0xb754f1dc gt   gt  gt  gt  print list itertools permutations horses      1  2  3  4     1  2  4  3     1  3  2  4     1  3  4  2     1  4  2  3     1  4  3  2     2  1  3  4     2  1  4  3     2  3  1  4     2  3  4  1     2  4  1  3     2  4  3  1     3  1  2  4     3  1  4  2     3  2  1  4     3  2  4  1     3  4  1  2     3  4  2  1     4  1  2  3     4  1  3  2     4  2  1  3     4  2  3  1     4  3  1  2     4  3  2  1    Understanding the inner mechanisms of iteration Iteration is a process implying iterables  implementing the   iter     method  and iterators  implementing the   next     method   Iterables are any objects you can get an iterator from  Iterators are objects that let you iterate on iterables  There is more about it in this article about how for loops work

User · Answer

Shortcut to understanding yield  When you see a function with yield statements  apply this easy trick to understand what will happen    Insert a line result      at the start of the function  Replace each yield expr with result append expr   Insert a line return result at the bottom of the function  Yay - no more yield statements  Read and figure out code  Compare function to the original definition    This trick may give you an idea of the logic behind the function  but what actually happens with yield is significantly different than what happens in the list based approach  In many cases  the yield approach will be a lot more memory efficient and faster too  In other cases  this trick will get you stuck in an infinite loop  even though the original function works just fine  Read on to learn more     Don t confuse your Iterables  Iterators  and Generators  First  the iterator protocol - when you write  for x in mylist         loop body      Python performs the following two steps    Gets an iterator for mylist   Call iter mylist  -  this returns an object with a next   method  or   next     in Python 3     This is the step most people forget to tell you about  Uses the iterator to loop over items   Keep calling the next   method on the iterator returned from step 1  The return value from next   is assigned to x and the loop body is executed  If an exception StopIteration is raised from within next    it means there are no more values in the iterator and the loop is exited    The truth is Python performs the above two steps anytime it wants to loop over the contents of an object - so it could be a for loop  but it could also be code like otherlist extend mylist   where otherlist is a Python list    Here mylist is an iterable because it implements the iterator protocol  In a user-defined class  you can implement the   iter     method to make instances of your class iterable  This method should return an iterator  An iterator is an object with a next   method  It is possible to implement both   iter     and next   on the same class  and have   iter     return self  This will work for simple cases  but not when you want two iterators looping over the same object at the same time   So that s the iterator protocol  many objects implement this protocol    Built-in lists  dictionaries  tuples  sets  files  User-defined classes that implement   iter      Generators    Note that a for loop doesn t know what kind of object it s dealing with - it just follows the iterator protocol  and is happy to get item after item as it calls next    Built-in lists return their items one by one  dictionaries return the keys one by one  files return the lines one by one  etc  And generators return    well that s where yield comes in   def f123        yield 1     yield 2     yield 3  for item in f123        print item   Instead of yield statements  if you had three return statements in f123   only the first would get executed  and the function would exit  But f123   is no ordinary function  When f123   is called  it does not return any of the values in the yield statements  It returns a generator object  Also  the function does not really exit - it goes into a suspended state  When the for loop tries to loop over the generator object  the function resumes from its suspended state at the very next line after the yield it previously returned from  executes the next line of code  in this case  a yield statement  and returns that as the next item  This happens until the function exits  at which point the generator raises StopIteration  and the loop exits    So the generator object is sort of like an adapter - at one end it exhibits the iterator protocol  by exposing   iter     and next   methods to keep the for loop happy  At the other end  however  it runs the function just enough to get the next value out of it  and puts it back in suspended mode   Why Use Generators   Usually  you can write code that doesn t use generators but implements the same logic  One option is to use the temporary list  trick  I mentioned before  That will not work in all cases  for e g  if you have infinite loops  or it may make inefficient use of memory when you have a really long list  The other approach is to implement a new iterable class SomethingIter that keeps the state in instance members and performs the next logical step in it s next    or   next     in Python 3  method  Depending on the logic  the code inside the next   method may end up looking very complex and be prone to bugs  Here generators provide a clean and easy solution

User · Answer

It s returning a generator  I m not particularly familiar with Python  but I believe it s the same kind of thing as C  s iterator blocks if you re familiar with those   The key idea is that the compiler interpreter whatever does some trickery so that as far as the caller is concerned  they can keep calling next   and it will keep returning values - as if the generator method was paused  Now obviously you can t really  pause  a method  so the compiler builds a state machine for you to remember where you currently are and what the local variables etc look like  This is much easier than writing an iterator yourself

User · Answer

An easy example to understand what it is  yield  def f123        for   in range 4           yield 1         yield 2   for i in f123        print  i    The output is    1 2 1 2 1 2 1 2

User · Answer

yield yields something  It s like somebody asks you to make 5 cupcakes  If you are done with at least one cupcake  you can give it to them to eat while you make other cakes   In  4   def make cake numbers               for i in range numbers                   yield  Cake     format i           In  5   factory   make cake 5    Here factory is called a generator  which makes you cakes  If you call make function  you get a generator instead of running that function  It is because when yield keyword is present in a function  it becomes a generator   In  7   next factory  Out 7    Cake 0   In  8   next factory  Out 8    Cake 1   In  9   next factory  Out 9    Cake 2   In  10   next factory  Out 10    Cake 3   In  11   next factory  Out 11    Cake 4    They consumed all the cakes  but they ask for one again   In  12   next factory  --------------------------------------------------------------------------- StopIteration                             Traceback  most recent call last   lt ipython-input-12-0f5c45da9774 gt  in  lt module gt  ---- gt  1 next factory   StopIteration    and they are being told to stop asking more  So once you consumed a generator you are done with it  You need to call make cake again if you want more cakes  It is like placing another order for cupcakes   In  13   factory   make cake 3   In  14   for cake in factory               print cake           Cake 0 Cake 1 Cake 2   You can also use for loop with a generator like the one above   One more example  Lets say you want a random password whenever you ask for it   In  22   import random  In  23   import string  In  24   def random password generator                 while True                   yield    join  random choice string ascii letters  for   in range 8              In  25   rpg   random password generator    In  26   for i in range 3                print next rpg            FXpUBhhH DdUDHoHn dvtebEqG  In  27   next rpg  Out 27    mJbYRMNo    Here rpg is a generator  which can generate an infinite number of random passwords  So we can also say that generators are useful when we don t know the length of the sequence  unlike list which has a finite number of elements

User · Answer

There is another yield use and meaning  since Python 3 3   yield from  lt expr gt   From PEP 380 -- Syntax for Delegating to a Subgenerator   A syntax is proposed for a generator to delegate part of its operations to another generator  This allows a section of code containing  yield  to be factored out and placed in another generator  Additionally  the subgenerator is allowed to return with a value  and the value is made available to the delegating generator  The new syntax also opens up some opportunities for optimisation when one generator re-yields values produced by another   Moreover this will introduce  since Python 3 5   async def new coroutine data             await blocking action    to avoid coroutines being confused with a regular generator  today yield is used in both

User · Answer

Here is a simple example   def isPrimeNumber n       print  isPrimeNumber     call  format n      if n  1          return False     for x in range 2 n           if n   x    0              return False     return True  def primes  n 1       while True           print  loop step ----------------     format n          if isPrimeNumber n   yield n         n    1  for n in primes        if n gt  10 break     print  wiriting result     format n    Output   loop step ---------------- 1 isPrimeNumber 1  call loop step ---------------- 2 isPrimeNumber 2  call loop step ---------------- 3 isPrimeNumber 3  call wiriting result 3 loop step ---------------- 4 isPrimeNumber 4  call loop step ---------------- 5 isPrimeNumber 5  call wiriting result 5 loop step ---------------- 6 isPrimeNumber 6  call loop step ---------------- 7 isPrimeNumber 7  call wiriting result 7 loop step ---------------- 8 isPrimeNumber 8  call loop step ---------------- 9 isPrimeNumber 9  call loop step ---------------- 10 isPrimeNumber 10  call loop step ---------------- 11 isPrimeNumber 11  call   I am not a Python developer  but it looks to me yield holds the position of program flow and the next loop start from  yield  position  It seems like it is waiting at that position  and just before that  returning a value outside  and next time continues to work   It seems to be an interesting and nice ability  D

User · Answer

Can also send data back to the generator  Indeed  as many answers here explain  using yield creates a generator  You can use the yield keyword to send data back to a  quot live quot  generator  Example  Let s say we have a method which translates from english to some other language  And in the beginning of it  it does something which is heavy and should be done once  We want this method run forever  don t really know why        and receive words words to be translated  def translator          load all the words in English language and the translation to  other lang      my words dict     hello    hello in other language    dog    dog in other language        while True          word    yield          yield my words dict get word   Unknown word       Running  my words translator   translator    next my words translator  print my words translator send  dog     next my words translator  print my words translator send  cat     will print  dog in other language Unknown word     To summarise  use send method inside a generator to send data back to the generator  To allow that  a  yield  is used

User · Answer

What does the yield keyword do in Python   Answer Outline Summary  A function with yield  when called  returns a Generator  Generators are iterators because they implement the iterator protocol  so you can iterate over them  A generator can also be sent information  making it conceptually a coroutine  In Python 3  you can delegate from one generator to another in both directions with yield from   Appendix critiques a couple of answers  including the top one  and discusses the use of return in a generator    Generators  yield is only legal inside of a function definition  and the inclusion of yield in a function definition makes it return a generator  The idea for generators comes from other languages  see footnote 1  with varying implementations  In Python s Generators  the execution of the code is frozen at the point of the yield  When the generator is called  methods are discussed below  execution resumes and then freezes at the next yield  yield provides an easy way of implementing the iterator protocol  defined by the following two methods    iter   and next  Python 2  or   next    Python 3    Both of those methods make an object an iterator that you could type-check with the Iterator Abstract Base Class from the collections module   gt  gt  gt  def func            yield  I am          yield  a generator         gt  gt  gt  type func                    A function with yield is still a function  lt type  function  gt   gt  gt  gt  gen   func    gt  gt  gt  type gen                     but it returns a generator  lt type  generator  gt   gt  gt  gt  hasattr gen     iter         that s an iterable True  gt  gt  gt  hasattr gen   next           and with  next     next   in Python 3  True                             implements the iterator protocol   The generator type is a sub-type of iterator   gt  gt  gt  import collections  types  gt  gt  gt  issubclass types GeneratorType  collections Iterator  True  And if necessary  we can type-check like this   gt  gt  gt  isinstance gen  types GeneratorType  True  gt  gt  gt  isinstance gen  collections Iterator  True  A feature of an Iterator is that once exhausted  you can t reuse or reset it   gt  gt  gt  list gen    I am    a generator     gt  gt  gt  list gen      You ll have to make another if you want to use its functionality again  see footnote 2    gt  gt  gt  list func      I am    a generator     One can yield data programmatically  for example  def func an iterable       for item in an iterable          yield item  The above simple generator is also equivalent to the below - as of Python 3 3  and not available in Python 2   you can use yield from  def func an iterable       yield from an iterable  However  yield from also allows for delegation to subgenerators  which will be explained in the following section on cooperative delegation with sub-coroutines  Coroutines  yield forms an expression that allows data to be sent into the generator  see footnote 3  Here is an example  take note of the received variable  which will point to the data that is sent to the generator  def bank account deposited  interest rate       while True          calculated interest   interest rate   deposited          received   yield calculated interest         if received              deposited    received    gt  gt  gt  my account   bank account 1000   05   First  we must queue up the generator with the builtin function  next  It will call the appropriate next or   next   method  depending on the version of Python you are using   gt  gt  gt  first year interest   next my account   gt  gt  gt  first year interest 50 0  And now we can send data into the generator   Sending None is the same as calling next      gt  gt  gt  next year interest   my account send first year interest   1000   gt  gt  gt  next year interest 102 5  Cooperative Delegation to Sub-Coroutine with yield from Now  recall that yield from is available in Python 3  This allows us to delegate coroutines to a subcoroutine   def money manager expected rate         must receive deposited value from  send        under management   yield                     yield None to start      while True          try              additional investment   yield expected rate   under management              if additional investment                  under management    additional investment         except GeneratorExit                 TODO  write function to send unclaimed funds to state                raise         finally                 TODO  write function to mail tax info to client              def investment account deposited  manager          very simple model of an investment account that delegates to a manager          must queue up manager      next manager          lt - same as manager send None        This is where we send the initial deposit to the manager      manager send deposited      try          yield from manager     except GeneratorExit          return manager close      delegate   And now we can delegate functionality to a sub-generator and it can be used by a generator just as above  my manager   money manager  06  my account   investment account 1000  my manager  first year return   next my account    - gt  60 0  Now simulate adding another 1 000 to the account plus the return on the account  60 0   next year return   my account send first year return   1000  next year return   123 6  You can read more about the precise semantics of yield from in PEP 380  Other Methods  close and throw The close method raises GeneratorExit at the point the function execution was frozen  This will also be called by   del   so you can put any cleanup code where you handle the GeneratorExit  my account close    You can also throw an exception which can be handled in the generator or propagated back to the user  import sys try      raise ValueError except      my manager throw  sys exc info     Raises  Traceback  most recent call last     File  quot  lt stdin gt  quot   line 4  in  lt module gt    File  quot  lt stdin gt  quot   line 6  in money manager   File  quot  lt stdin gt  quot   line 2  in  lt module gt  ValueError  Conclusion I believe I have covered all aspects of the following question   What does the yield keyword do in Python   It turns out that yield does a lot  I m sure I could add even more thorough examples to this  If you want more or have some constructive criticism  let me know by commenting below   Appendix  Critique of the Top Accepted Answer    It is confused on what makes an iterable  just using a list as an example  See my references above  but in summary  an iterable has an   iter   method returning an iterator  An iterator provides a  next  Python 2 or    next    Python 3  method  which is implicitly called by for loops until it raises StopIteration  and once it does  it will continue to do so  It then uses a generator expression to describe what a generator is  Since a generator is simply a convenient way to create an iterator  it only confuses the matter  and we still have not yet gotten to the yield part  In Controlling a generator exhaustion he calls the  next method  when instead he should use the builtin function  next  It would be an appropriate layer of indirection  because his code does not work in Python 3  Itertools  This was not relevant to what yield does at all  No discussion of the methods that yield provides along with the new functionality yield from in Python 3  The top accepted answer is a very incomplete answer   Critique of answer suggesting yield in a generator expression or comprehension  The grammar currently allows any expression in a list comprehension  expr stmt  testlist star expr  annassign   augassign  yield expr testlist                               yield expr testlist star expr         yield expr   yield   yield arg  yield arg   from  test   testlist  Since yield is an expression  it has been touted by some as interesting to use it in comprehensions or generator expression - in spite of citing no particularly good use-case  The CPython core developers are discussing deprecating its allowance  Here s a relevant post from the mailing list   On 30 January 2017 at 19 05  Brett Cannon  wrote   On Sun  29 Jan 2017 at 16 39 Craig Rodrigues  wrote   I m OK with either approach   Leaving things the way they are in Python 3 is no good  IMHO   My vote is it be a SyntaxError since you re not getting what you expect from the syntax   I d agree that s a sensible place for us to end up  as any code relying on the current behaviour is really too clever to be maintainable  In terms of getting there  we ll likely want   SyntaxWarning or DeprecationWarning in 3 7 Py3k warning in 2 7 x SyntaxError in 3 8  Cheers  Nick  --  Nick Coghlan       ncoghlan at gmail com       Brisbane  Australia  Further  there is an outstanding issue  10544  which seems to be pointing in the direction of this never being a good idea  PyPy  a Python implementation written in Python  is already raising syntax warnings   Bottom line  until the developers of CPython tell us otherwise  Don t put yield in a generator expression or comprehension  The return statement in a generator In Python 2   In a generator function  the return statement is not allowed to include an expression list  In that context  a bare return indicates that the generator is done and will cause StopIteration to be raised   An expression list is basically any number of expressions separated by commas - essentially  in Python 2  you can stop the generator with return  but you can t return a value  In Python 3   In a generator function  the return statement indicates that the generator is done and will cause StopIteration to be raised  The returned value  if any  is used as an argument to construct StopIteration and becomes the StopIteration value attribute   Footnotes  The languages CLU  Sather  and Icon were referenced in the proposal to introduce the concept of generators to Python  The general idea is that a function can maintain internal state and yield intermediate data points on demand by the user  This promised to be superior in performance to other approaches  including Python threading  which isn t even available on some systems    This means  for example  that range objects aren t Iterators  even though they are iterable  because they can be reused  Like lists  their   iter   methods return iterator objects       yield was originally introduced as a statement  meaning that it could only appear at the beginning of a line in a code block  Now yield creates a yield expression  https   docs python org 2 reference simple stmts html grammar-token-yield stmt This change was proposed to allow a user to send data into the generator just as one might receive it  To send data  one must be able to assign it to something  and for that  a statement just won t work

User · Answer

yield in python is in a way similar to the return statement  except for some differences  If multiple values have to be returned from a function  return statement will return all the values as a list and it has to be stored in the memory in the caller block  But what if we don t want to use extra memory  Instead  we want to get the value from the function when we need it  This is where yield comes in  Consider the following function  -  def fun       yield 1    yield 2    yield 3   And the caller is  -  def caller       print   First value printing      print  fun       print   Second value printing      print  fun       print   Third value printing      print  fun      The above code segment  caller function  when called  outputs  -  First value printing 1 Second value printing 2 Third value printing 3   As can be seen from above  yield returns a value to its caller  but when the function is called again  it doesn t start from the first statement  but from the statement right after the yield  In the above example   First value printing  was printed and the function was called  1 was returned and printed  Then  Second value printing  was printed and again fun   was called  Instead of printing 1  the first statement   it returned 2  i e   the statement just after yield 1  The same process is repeated further

User · Answer

I was going to post  read page 19 of Beazley s  Python  Essential Reference  for a quick description of generators   but so many others have posted good descriptions already   Also  note that yield can be used in coroutines as the dual of their use in generator functions   Although it isn t the same use as your code snippet   yield  can be used as an expression in a function   When a caller sends a value to the method using the send   method  then the coroutine will execute until the next  yield  statement is encountered   Generators and coroutines are a cool way to set up data-flow type applications   I thought it would be worthwhile knowing about the other use of the yield statement in functions

User · Answer

In Python generators  a special type of iterators  are used to generate series of values and yield keyword is just like the return keyword of generator functions    The other fascinating thing yield keyword  does is saving the state of a generator function    So  we can set a number to a different value each time the generator yields    Here s an instance   def getPrimes number       while True          if isPrime number               number   yield number       a miracle occurs here         number    1  def printSuccessivePrimes iterations  base 10       primeGenerator   getPrimes base      primeGenerator send None      for power in range iterations           print primeGenerator send base    power

User · Answer

Like every answer suggests  yield is used for creating a sequence generator  It s used for generating some sequence dynamically  For example  while reading a file line by line on a network  you can use the yield function as follows   def getNextLines       while con isOpen           yield con read     You can use it in your code as follows   for line in getNextLines        doSomeThing line    Execution Control Transfer gotcha  The execution control will be transferred from getNextLines   to the for loop when yield is executed  Thus  every time getNextLines   is invoked  execution begins from the point where it was paused last time   Thus in short  a function with the following code  def simpleYield        yield  first time      yield  second time      yield  third time      yield  Now some useful value     format 12   for i in simpleYield        print i   will print   first time   second time   third time   Now some useful value 12

User · Answer

It s returning a generator  I m not particularly familiar with Python  but I believe it s the same kind of thing as C  s iterator blocks if you re familiar with those   The key idea is that the compiler interpreter whatever does some trickery so that as far as the caller is concerned  they can keep calling next   and it will keep returning values - as if the generator method was paused  Now obviously you can t really  pause  a method  so the compiler builds a state machine for you to remember where you currently are and what the local variables etc look like  This is much easier than writing an iterator yourself

User · Answer

Think of it this way   An iterator is just a fancy sounding term for an object that has a next   method   So a yield-ed function ends up being something like this   Original version   def some function        for i in xrange 4           yield i  for i in some function        print i   This is basically what the Python interpreter does with the above code   class it      def   init   self             Start at -1 so that we get 0 when we add 1 below          self count   -1        The   iter   method will be called once by the  for  loop        The rest of the magic happens on the object returned by this method        In this case it is the object itself      def   iter   self           return self        The next method will be called repeatedly by the  for  loop       until it raises StopIteration      def next self           self count    1         if self count  lt  4              return self count         else                A StopIteration exception is raised               to signal that the iterator is done                This is caught implicitly by the  for  loop              raise StopIteration  def some func        return it    for i in some func        print i   For more insight as to what s happening behind the scenes  the for loop can be rewritten to this   iterator   some func   try      while 1          print iterator next   except StopIteration      pass   Does that make more sense or just confuse you more       I should note that this is an oversimplification for illustrative purposes

User · Answer

In Python generators  a special type of iterators  are used to generate series of values and yield keyword is just like the return keyword of generator functions    The other fascinating thing yield keyword  does is saving the state of a generator function    So  we can set a number to a different value each time the generator yields    Here s an instance   def getPrimes number       while True          if isPrime number               number   yield number       a miracle occurs here         number    1  def printSuccessivePrimes iterations  base 10       primeGenerator   getPrimes base      primeGenerator send None      for power in range iterations           print primeGenerator send base    power

User · Answer

yield is like a return element for a function  The difference is  that the yield element turns a function into a generator  A generator behaves just like a function until something is  yielded   The generator stops until it is next called  and continues from exactly the same point as it started  You can get a sequence of all the  yielded  values in one  by calling list generator

User · Answer

yield is just like return - it returns whatever you tell it to  as a generator   The difference is that the next time you call the generator  execution starts from the last call to the yield statement  Unlike return  the stack frame is not cleaned up when a yield occurs  however control is transferred back to the caller  so its state will resume the next time the function is called   In the case of your code  the function get child candidates is acting like an iterator so that when you extend your list  it adds one element at a time to the new list   list extend calls an iterator until it s exhausted  In the case of the code sample you posted  it would be much clearer to just return a tuple and append that to the list

User · Answer

In summary  the yield statement transforms your function into a factory that produces a special object called a generator which wraps around the body of your original function  When the generator is iterated  it executes your function  until it reaches the next yield then suspends execution and evaluates to the value passed to yield  It repeats this process on each iteration until the path of execution exits the function  For instance   def simple generator        yield  one      yield  two      yield  three   for i in simple generator        print i   simply outputs  one two three   The power comes from using the generator with a loop that calculates a sequence  the generator executes the loop stopping each time to  yield  the next result of the calculation  in this way it calculates a list on the fly  the benefit being the memory saved for especially large calculations  Say you wanted to create a your own range function that produces an iterable range of numbers  you could do it like so   def myRangeNaive i       n   0     range          while n  lt  i          range append n          n   n   1     return range   and use it like this   for i in myRangeNaive 10       print i   But this is inefficient because   You create an array that you only use once  this wastes memory  This code actually loops over that array twice       Luckily Guido and his team were generous enough to develop generators so we could just do this   def myRangeSmart i       n   0     while n  lt  i         yield n        n   n   1     return  for i in myRangeSmart 10       print i   Now upon each iteration a function on the generator called next   executes the function until it either reaches a  yield  statement in which it stops and   yields  the value or reaches the end of the function  In this case on the first call  next   executes up to the yield statement and yield  n   on the next call it will execute the  increment statement  jump back to the  while   evaluate it  and if true  it will stop and yield  n  again  it will continue that way until the while condition returns false and the generator jumps to the end of the function

User · Answer

Shortcut to understanding yield  When you see a function with yield statements  apply this easy trick to understand what will happen    Insert a line result      at the start of the function  Replace each yield expr with result append expr   Insert a line return result at the bottom of the function  Yay - no more yield statements  Read and figure out code  Compare function to the original definition    This trick may give you an idea of the logic behind the function  but what actually happens with yield is significantly different than what happens in the list based approach  In many cases  the yield approach will be a lot more memory efficient and faster too  In other cases  this trick will get you stuck in an infinite loop  even though the original function works just fine  Read on to learn more     Don t confuse your Iterables  Iterators  and Generators  First  the iterator protocol - when you write  for x in mylist         loop body      Python performs the following two steps    Gets an iterator for mylist   Call iter mylist  -  this returns an object with a next   method  or   next     in Python 3     This is the step most people forget to tell you about  Uses the iterator to loop over items   Keep calling the next   method on the iterator returned from step 1  The return value from next   is assigned to x and the loop body is executed  If an exception StopIteration is raised from within next    it means there are no more values in the iterator and the loop is exited    The truth is Python performs the above two steps anytime it wants to loop over the contents of an object - so it could be a for loop  but it could also be code like otherlist extend mylist   where otherlist is a Python list    Here mylist is an iterable because it implements the iterator protocol  In a user-defined class  you can implement the   iter     method to make instances of your class iterable  This method should return an iterator  An iterator is an object with a next   method  It is possible to implement both   iter     and next   on the same class  and have   iter     return self  This will work for simple cases  but not when you want two iterators looping over the same object at the same time   So that s the iterator protocol  many objects implement this protocol    Built-in lists  dictionaries  tuples  sets  files  User-defined classes that implement   iter      Generators    Note that a for loop doesn t know what kind of object it s dealing with - it just follows the iterator protocol  and is happy to get item after item as it calls next    Built-in lists return their items one by one  dictionaries return the keys one by one  files return the lines one by one  etc  And generators return    well that s where yield comes in   def f123        yield 1     yield 2     yield 3  for item in f123        print item   Instead of yield statements  if you had three return statements in f123   only the first would get executed  and the function would exit  But f123   is no ordinary function  When f123   is called  it does not return any of the values in the yield statements  It returns a generator object  Also  the function does not really exit - it goes into a suspended state  When the for loop tries to loop over the generator object  the function resumes from its suspended state at the very next line after the yield it previously returned from  executes the next line of code  in this case  a yield statement  and returns that as the next item  This happens until the function exits  at which point the generator raises StopIteration  and the loop exits    So the generator object is sort of like an adapter - at one end it exhibits the iterator protocol  by exposing   iter     and next   methods to keep the for loop happy  At the other end  however  it runs the function just enough to get the next value out of it  and puts it back in suspended mode   Why Use Generators   Usually  you can write code that doesn t use generators but implements the same logic  One option is to use the temporary list  trick  I mentioned before  That will not work in all cases  for e g  if you have infinite loops  or it may make inefficient use of memory when you have a really long list  The other approach is to implement a new iterable class SomethingIter that keeps the state in instance members and performs the next logical step in it s next    or   next     in Python 3  method  Depending on the logic  the code inside the next   method may end up looking very complex and be prone to bugs  Here generators provide a clean and easy solution

User · Answer

Here is an example in plain language  I will provide a correspondence between high-level human concepts to low-level Python concepts   I want to operate on a sequence of numbers  but I don t want to bother my self with the creation of that sequence  I want only to focus on the operation I want to do  So  I do the following    I call you and tell you that I want a sequence of numbers which is produced in a specific way  and I let you know what the algorithm is   This step corresponds to defining the generator function  i e  the function containing a yield  Sometime later  I tell you   OK  get ready to tell me the sequence of numbers    This step corresponds to calling the generator function which returns a generator object  Note that you don t tell me any numbers yet  you just grab your paper and pencil  I ask you   tell me the next number   and you tell me the first number  after that  you wait for me to ask you for the next number  It s your job to remember where you were  what numbers you have already said  and what is the next number  I don t care about the details   This step corresponds to calling  next   on the generator object      repeat previous step  until    eventually  you might come to an end  You don t tell me a number  you just shout   hold your horses  I m done  No more numbers    This step corresponds to the generator object ending its job  and raising a StopIteration exception The generator function does not need to raise the exception  It s raised automatically when the function ends or issues a return    This is what a generator does  a function that contains a yield   it starts executing  pauses whenever it does a yield  and when asked for a  next   value it continues from the point it was last  It fits perfectly by design with the iterator protocol of Python  which describes how to sequentially request values   The most famous user of the iterator protocol is the for command in Python  So  whenever you do a   for item in sequence    it doesn t matter if sequence is a list  a string  a dictionary or a generator object like described above  the result is the same  you read items off a sequence one by one   Note that defining a function which contains a yield keyword is not the only way to create a generator  it s just the easiest way to create one   For more accurate information  read about iterator types  the yield statement and generators in the Python documentation

User · Answer

Here is an example in plain language  I will provide a correspondence between high-level human concepts to low-level Python concepts   I want to operate on a sequence of numbers  but I don t want to bother my self with the creation of that sequence  I want only to focus on the operation I want to do  So  I do the following    I call you and tell you that I want a sequence of numbers which is produced in a specific way  and I let you know what the algorithm is   This step corresponds to defining the generator function  i e  the function containing a yield  Sometime later  I tell you   OK  get ready to tell me the sequence of numbers    This step corresponds to calling the generator function which returns a generator object  Note that you don t tell me any numbers yet  you just grab your paper and pencil  I ask you   tell me the next number   and you tell me the first number  after that  you wait for me to ask you for the next number  It s your job to remember where you were  what numbers you have already said  and what is the next number  I don t care about the details   This step corresponds to calling  next   on the generator object      repeat previous step  until    eventually  you might come to an end  You don t tell me a number  you just shout   hold your horses  I m done  No more numbers    This step corresponds to the generator object ending its job  and raising a StopIteration exception The generator function does not need to raise the exception  It s raised automatically when the function ends or issues a return    This is what a generator does  a function that contains a yield   it starts executing  pauses whenever it does a yield  and when asked for a  next   value it continues from the point it was last  It fits perfectly by design with the iterator protocol of Python  which describes how to sequentially request values   The most famous user of the iterator protocol is the for command in Python  So  whenever you do a   for item in sequence    it doesn t matter if sequence is a list  a string  a dictionary or a generator object like described above  the result is the same  you read items off a sequence one by one   Note that defining a function which contains a yield keyword is not the only way to create a generator  it s just the easiest way to create one   For more accurate information  read about iterator types  the yield statement and generators in the Python documentation

User · Answer

Think of it this way   An iterator is just a fancy sounding term for an object that has a next   method   So a yield-ed function ends up being something like this   Original version   def some function        for i in xrange 4           yield i  for i in some function        print i   This is basically what the Python interpreter does with the above code   class it      def   init   self             Start at -1 so that we get 0 when we add 1 below          self count   -1        The   iter   method will be called once by the  for  loop        The rest of the magic happens on the object returned by this method        In this case it is the object itself      def   iter   self           return self        The next method will be called repeatedly by the  for  loop       until it raises StopIteration      def next self           self count    1         if self count  lt  4              return self count         else                A StopIteration exception is raised               to signal that the iterator is done                This is caught implicitly by the  for  loop              raise StopIteration  def some func        return it    for i in some func        print i   For more insight as to what s happening behind the scenes  the for loop can be rewritten to this   iterator   some func   try      while 1          print iterator next   except StopIteration      pass   Does that make more sense or just confuse you more       I should note that this is an oversimplification for illustrative purposes

User · Answer

Here is an example in plain language  I will provide a correspondence between high-level human concepts to low-level Python concepts   I want to operate on a sequence of numbers  but I don t want to bother my self with the creation of that sequence  I want only to focus on the operation I want to do  So  I do the following    I call you and tell you that I want a sequence of numbers which is produced in a specific way  and I let you know what the algorithm is   This step corresponds to defining the generator function  i e  the function containing a yield  Sometime later  I tell you   OK  get ready to tell me the sequence of numbers    This step corresponds to calling the generator function which returns a generator object  Note that you don t tell me any numbers yet  you just grab your paper and pencil  I ask you   tell me the next number   and you tell me the first number  after that  you wait for me to ask you for the next number  It s your job to remember where you were  what numbers you have already said  and what is the next number  I don t care about the details   This step corresponds to calling  next   on the generator object      repeat previous step  until    eventually  you might come to an end  You don t tell me a number  you just shout   hold your horses  I m done  No more numbers    This step corresponds to the generator object ending its job  and raising a StopIteration exception The generator function does not need to raise the exception  It s raised automatically when the function ends or issues a return    This is what a generator does  a function that contains a yield   it starts executing  pauses whenever it does a yield  and when asked for a  next   value it continues from the point it was last  It fits perfectly by design with the iterator protocol of Python  which describes how to sequentially request values   The most famous user of the iterator protocol is the for command in Python  So  whenever you do a   for item in sequence    it doesn t matter if sequence is a list  a string  a dictionary or a generator object like described above  the result is the same  you read items off a sequence one by one   Note that defining a function which contains a yield keyword is not the only way to create a generator  it s just the easiest way to create one   For more accurate information  read about iterator types  the yield statement and generators in the Python documentation

User · Answer

TL DR  Instead of this   def square list n       the list                                Replace     for x in range n           y   x   x         the list append y                   these     return the list                         lines   do this   def square yield n       for x in range n           y   x   x         yield y                             with this one    Whenever you find yourself building a list from scratch  yield each piece instead    This was my first  aha  moment with yield     yield is a sugary way to say      build a series of stuff   Same behavior    gt  gt  gt  for square in square list 4           print square      0 1 4 9  gt  gt  gt  for square in square yield 4           print square      0 1 4 9   Different behavior   Yield is single-pass  you can only iterate through once  When a function has a yield in it we call it a generator function  And an iterator is what it returns  Those terms are revealing  We lose the convenience of a container  but gain the power of a series that s computed as needed  and arbitrarily long   Yield is lazy  it puts off computation  A function with a yield in it doesn t actually execute at all when you call it  It returns an iterator object that remembers where it left off  Each time you call next   on the iterator  this happens in a for-loop  execution inches forward to the next yield  return raises StopIteration and ends the series  this is the natural end of a for-loop    Yield is versatile  Data doesn t have to be stored all together  it can be made available one at a time  It can be infinite    gt  gt  gt  def squares all of them            x   0         while True              yield x   x             x    1      gt  gt  gt  squares   squares all of them    gt  gt  gt  for   in range 4           print next squares       0 1 4 9     If you need multiple passes and the series isn t too long  just call list   on it    gt  gt  gt  list square yield 4    0  1  4  9      Brilliant choice of the word yield because both meanings apply      yield  mdash  produce or provide  as in agriculture       provide the next data in the series      yield  mdash  give way or relinquish  as in political power       relinquish CPU execution until the iterator advances

User · Answer

yield in python is in a way similar to the return statement  except for some differences  If multiple values have to be returned from a function  return statement will return all the values as a list and it has to be stored in the memory in the caller block  But what if we don t want to use extra memory  Instead  we want to get the value from the function when we need it  This is where yield comes in  Consider the following function  -  def fun       yield 1    yield 2    yield 3   And the caller is  -  def caller       print   First value printing      print  fun       print   Second value printing      print  fun       print   Third value printing      print  fun      The above code segment  caller function  when called  outputs  -  First value printing 1 Second value printing 2 Third value printing 3   As can be seen from above  yield returns a value to its caller  but when the function is called again  it doesn t start from the first statement  but from the statement right after the yield  In the above example   First value printing  was printed and the function was called  1 was returned and printed  Then  Second value printing  was printed and again fun   was called  Instead of printing 1  the first statement   it returned 2  i e   the statement just after yield 1  The same process is repeated further

User · Answer

While a lot of answers show why you d use a yield to create a generator  there are more uses for yield   It s quite easy to make a coroutine  which enables the passing of information between two blocks of code   I won t repeat any of the fine examples that have already been given about using yield to create a generator   To help understand what a yield does in the following code  you can use your finger to trace the cycle through any code that has a yield   Every time your finger hits the yield  you have to wait for a next or a send to be entered   When a next is called  you trace through the code until you hit the yield    the code on the right of the yield is evaluated and returned to the caller    then you wait   When next is called again  you perform another loop through the code   However  you ll note that in a coroutine  yield can also be used with a send    which will send a value from the caller into the yielding function  If a send is given  then yield receives the value sent  and spits it out the left hand side    then the trace through the code progresses until you hit the yield again  returning the value at the end  as if next was called    For example    gt  gt  gt  def coroutine            i   -1         while True              i    1             val    yield i              print  Received  s    val       gt  gt  gt  sequence   coroutine    gt  gt  gt  sequence next   0  gt  gt  gt  sequence next   Received None 1  gt  gt  gt  sequence send  hello   Received hello 2  gt  gt  gt  sequence close

User · Answer

To understand what yield does  you must understand what generators are  And before you can understand generators  you must understand iterables  Iterables When you create a list  you can read its items one by one  Reading its items one by one is called iteration   gt  gt  gt  mylist    1  2  3   gt  gt  gt  for i in mylist         print i  1 2 3  mylist is an iterable  When you use a list comprehension  you create a list  and so an iterable   gt  gt  gt  mylist    x x for x in range 3    gt  gt  gt  for i in mylist         print i  0 1 4  Everything you can use  quot for    in    quot  on is an iterable  lists  strings  files    These iterables are handy because you can read them as much as you wish  but you store all the values in memory and this is not always what you want when you have a lot of values  Generators Generators are iterators  a kind of iterable you can only iterate over once  Generators do not store all the values in memory  they generate the values on the fly   gt  gt  gt  mygenerator    x x for x in range 3    gt  gt  gt  for i in mygenerator         print i  0 1 4  It is just the same except you used    instead of     BUT  you cannot perform for i in mygenerator a second time since generators can only be used once  they calculate 0  then forget about it and calculate 1  and end calculating 4  one by one  Yield yield is a keyword that is used like return  except the function will return a generator   gt  gt  gt  def createGenerator           mylist   range 3         for i in mylist             yield i i      gt  gt  gt  mygenerator   createGenerator     create a generator  gt  gt  gt  print mygenerator    mygenerator is an object   lt generator object createGenerator at 0xb7555c34 gt   gt  gt  gt  for i in mygenerator          print i  0 1 4  Here it s a useless example  but it s handy when you know your function will return a huge set of values that you will only need to read once  To master yield  you must understand that when you call the function  the code you have written in the function body does not run  The function only returns the generator object  this is a bit tricky  -  Then  your code will continue from where it left off each time for uses the generator  Now the hard part  The first time the for calls the generator object created from your function  it will run the code in your function from the beginning until it hits yield  then it ll return the first value of the loop  Then  each subsequent call will run another iteration of the loop you have written in the function and return the next value  This will continue until the generator is considered empty  which happens when the function runs without hitting yield  That can be because the loop has come to an end  or because you no longer satisfy an  quot if else quot    Your code explained Generator    Here you create the method of the node object that will return the generator def  get child candidates self  distance  min dist  max dist          Here is the code that will be called each time you use the generator object         If there is still a child of the node object on its left       AND if the distance is ok  return the next child     if self  leftchild and distance - max dist  lt  self  median          yield self  leftchild        If there is still a child of the node object on its right       AND if the distance is ok  return the next child     if self  rightchild and distance   max dist  gt   self  median          yield self  rightchild        If the function arrives here  the generator will be considered empty       there is no more than two values  the left and the right children  Caller    Create an empty list and a list with the current object reference result  candidates   list     self     Loop on candidates  they contain only one element at the beginning  while candidates         Get the last candidate and remove it from the list     node   candidates pop          Get the distance between obj and the candidate     distance   node  get dist obj         If distance is ok  then you can fill the result     if distance  lt   max dist and distance  gt   min dist          result extend node  values         Add the children of the candidate in the candidate s list       so the loop will keep running until it will have looked       at all the children of the children of the children  etc  of the candidate     candidates extend node  get child candidates distance  min dist  max dist    return result  This code contains several smart parts   The loop iterates on a list  but the list expands while the loop is being iterated  -  It s a concise way to go through all these nested data even if it s a bit dangerous since you can end up with an infinite loop  In this case  candidates extend node  get child candidates distance  min dist  max dist   exhaust all the values of the generator  but while keeps creating new generator objects which will produce different values from the previous ones since it s not applied on the same node   The extend   method is a list object method that expects an iterable and adds its values to the list    Usually we pass a list to it   gt  gt  gt  a    1  2   gt  gt  gt  b    3  4   gt  gt  gt  a extend b   gt  gt  gt  print a   1  2  3  4   But in your code  it gets a generator  which is good because   You don t need to read the values twice  You may have a lot of children and you don t want them all stored in memory   And it works because Python does not care if the argument of a method is a list or not  Python expects iterables so it will work with strings  lists  tuples  and generators  This is called duck typing and is one of the reasons why Python is so cool  But this is another story  for another question    You can stop here  or read a little bit to see an advanced use of a generator  Controlling a generator exhaustion  gt  gt  gt  class Bank      Let s create a bank  building ATMs        crisis   False        def create atm self              while not self crisis                 yield  quot  100 quot   gt  gt  gt  hsbc   Bank     When everything s ok the ATM gives you as much as you want  gt  gt  gt  corner street atm   hsbc create atm    gt  gt  gt  print corner street atm next     100  gt  gt  gt  print corner street atm next     100  gt  gt  gt  print  corner street atm next   for cash in range 5       100     100     100     100     100    gt  gt  gt  hsbc crisis   True   Crisis is coming  no more money   gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  wall street atm   hsbc create atm     It s even true for new ATMs  gt  gt  gt  print wall street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  hsbc crisis   False   The trouble is  even post-crisis the ATM remains empty  gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  brand new atm   hsbc create atm     Build a new one to get back in business  gt  gt  gt  for cash in brand new atm         print cash  100  100  100  100  100  100  100  100  100      Note  For Python 3  useprint corner street atm   next      or print next corner street atm   It can be useful for various things like controlling access to a resource  Itertools  your best friend The itertools module contains special functions to manipulate iterables  Ever wish to duplicate a generator  Chain two generators  Group values in a nested list with a one-liner  Map   Zip without creating another list  Then just import itertools  An example  Let s see the possible orders of arrival for a four-horse race   gt  gt  gt  horses    1  2  3  4   gt  gt  gt  races   itertools permutations horses   gt  gt  gt  print races   lt itertools permutations object at 0xb754f1dc gt   gt  gt  gt  print list itertools permutations horses      1  2  3  4     1  2  4  3     1  3  2  4     1  3  4  2     1  4  2  3     1  4  3  2     2  1  3  4     2  1  4  3     2  3  1  4     2  3  4  1     2  4  1  3     2  4  3  1     3  1  2  4     3  1  4  2     3  2  1  4     3  2  4  1     3  4  1  2     3  4  2  1     4  1  2  3     4  1  3  2     4  2  1  3     4  2  3  1     4  3  1  2     4  3  2  1    Understanding the inner mechanisms of iteration Iteration is a process implying iterables  implementing the   iter     method  and iterators  implementing the   next     method   Iterables are any objects you can get an iterator from  Iterators are objects that let you iterate on iterables  There is more about it in this article about how for loops work

User · Answer

Think of it this way   An iterator is just a fancy sounding term for an object that has a next   method   So a yield-ed function ends up being something like this   Original version   def some function        for i in xrange 4           yield i  for i in some function        print i   This is basically what the Python interpreter does with the above code   class it      def   init   self             Start at -1 so that we get 0 when we add 1 below          self count   -1        The   iter   method will be called once by the  for  loop        The rest of the magic happens on the object returned by this method        In this case it is the object itself      def   iter   self           return self        The next method will be called repeatedly by the  for  loop       until it raises StopIteration      def next self           self count    1         if self count  lt  4              return self count         else                A StopIteration exception is raised               to signal that the iterator is done                This is caught implicitly by the  for  loop              raise StopIteration  def some func        return it    for i in some func        print i   For more insight as to what s happening behind the scenes  the for loop can be rewritten to this   iterator   some func   try      while 1          print iterator next   except StopIteration      pass   Does that make more sense or just confuse you more       I should note that this is an oversimplification for illustrative purposes

User · Answer

Here is an example in plain language  I will provide a correspondence between high-level human concepts to low-level Python concepts   I want to operate on a sequence of numbers  but I don t want to bother my self with the creation of that sequence  I want only to focus on the operation I want to do  So  I do the following    I call you and tell you that I want a sequence of numbers which is produced in a specific way  and I let you know what the algorithm is   This step corresponds to defining the generator function  i e  the function containing a yield  Sometime later  I tell you   OK  get ready to tell me the sequence of numbers    This step corresponds to calling the generator function which returns a generator object  Note that you don t tell me any numbers yet  you just grab your paper and pencil  I ask you   tell me the next number   and you tell me the first number  after that  you wait for me to ask you for the next number  It s your job to remember where you were  what numbers you have already said  and what is the next number  I don t care about the details   This step corresponds to calling  next   on the generator object      repeat previous step  until    eventually  you might come to an end  You don t tell me a number  you just shout   hold your horses  I m done  No more numbers    This step corresponds to the generator object ending its job  and raising a StopIteration exception The generator function does not need to raise the exception  It s raised automatically when the function ends or issues a return    This is what a generator does  a function that contains a yield   it starts executing  pauses whenever it does a yield  and when asked for a  next   value it continues from the point it was last  It fits perfectly by design with the iterator protocol of Python  which describes how to sequentially request values   The most famous user of the iterator protocol is the for command in Python  So  whenever you do a   for item in sequence    it doesn t matter if sequence is a list  a string  a dictionary or a generator object like described above  the result is the same  you read items off a sequence one by one   Note that defining a function which contains a yield keyword is not the only way to create a generator  it s just the easiest way to create one   For more accurate information  read about iterator types  the yield statement and generators in the Python documentation

User · Answer

The yield keyword simply collects returning results  Think of yield like return

User · Answer

Here is a simple example   def isPrimeNumber n       print  isPrimeNumber     call  format n      if n  1          return False     for x in range 2 n           if n   x    0              return False     return True  def primes  n 1       while True           print  loop step ----------------     format n          if isPrimeNumber n   yield n         n    1  for n in primes        if n gt  10 break     print  wiriting result     format n    Output   loop step ---------------- 1 isPrimeNumber 1  call loop step ---------------- 2 isPrimeNumber 2  call loop step ---------------- 3 isPrimeNumber 3  call wiriting result 3 loop step ---------------- 4 isPrimeNumber 4  call loop step ---------------- 5 isPrimeNumber 5  call wiriting result 5 loop step ---------------- 6 isPrimeNumber 6  call loop step ---------------- 7 isPrimeNumber 7  call wiriting result 7 loop step ---------------- 8 isPrimeNumber 8  call loop step ---------------- 9 isPrimeNumber 9  call loop step ---------------- 10 isPrimeNumber 10  call loop step ---------------- 11 isPrimeNumber 11  call   I am not a Python developer  but it looks to me yield holds the position of program flow and the next loop start from  yield  position  It seems like it is waiting at that position  and just before that  returning a value outside  and next time continues to work   It seems to be an interesting and nice ability  D

User · Answer

yield is just like return - it returns whatever you tell it to  as a generator   The difference is that the next time you call the generator  execution starts from the last call to the yield statement  Unlike return  the stack frame is not cleaned up when a yield occurs  however control is transferred back to the caller  so its state will resume the next time the function is called   In the case of your code  the function get child candidates is acting like an iterator so that when you extend your list  it adds one element at a time to the new list   list extend calls an iterator until it s exhausted  In the case of the code sample you posted  it would be much clearer to just return a tuple and append that to the list

User · Answer

What does the yield keyword do in Python   Answer Outline Summary  A function with yield  when called  returns a Generator  Generators are iterators because they implement the iterator protocol  so you can iterate over them  A generator can also be sent information  making it conceptually a coroutine  In Python 3  you can delegate from one generator to another in both directions with yield from   Appendix critiques a couple of answers  including the top one  and discusses the use of return in a generator    Generators  yield is only legal inside of a function definition  and the inclusion of yield in a function definition makes it return a generator  The idea for generators comes from other languages  see footnote 1  with varying implementations  In Python s Generators  the execution of the code is frozen at the point of the yield  When the generator is called  methods are discussed below  execution resumes and then freezes at the next yield  yield provides an easy way of implementing the iterator protocol  defined by the following two methods    iter   and next  Python 2  or   next    Python 3    Both of those methods make an object an iterator that you could type-check with the Iterator Abstract Base Class from the collections module   gt  gt  gt  def func            yield  I am          yield  a generator         gt  gt  gt  type func                    A function with yield is still a function  lt type  function  gt   gt  gt  gt  gen   func    gt  gt  gt  type gen                     but it returns a generator  lt type  generator  gt   gt  gt  gt  hasattr gen     iter         that s an iterable True  gt  gt  gt  hasattr gen   next           and with  next     next   in Python 3  True                             implements the iterator protocol   The generator type is a sub-type of iterator   gt  gt  gt  import collections  types  gt  gt  gt  issubclass types GeneratorType  collections Iterator  True  And if necessary  we can type-check like this   gt  gt  gt  isinstance gen  types GeneratorType  True  gt  gt  gt  isinstance gen  collections Iterator  True  A feature of an Iterator is that once exhausted  you can t reuse or reset it   gt  gt  gt  list gen    I am    a generator     gt  gt  gt  list gen      You ll have to make another if you want to use its functionality again  see footnote 2    gt  gt  gt  list func      I am    a generator     One can yield data programmatically  for example  def func an iterable       for item in an iterable          yield item  The above simple generator is also equivalent to the below - as of Python 3 3  and not available in Python 2   you can use yield from  def func an iterable       yield from an iterable  However  yield from also allows for delegation to subgenerators  which will be explained in the following section on cooperative delegation with sub-coroutines  Coroutines  yield forms an expression that allows data to be sent into the generator  see footnote 3  Here is an example  take note of the received variable  which will point to the data that is sent to the generator  def bank account deposited  interest rate       while True          calculated interest   interest rate   deposited          received   yield calculated interest         if received              deposited    received    gt  gt  gt  my account   bank account 1000   05   First  we must queue up the generator with the builtin function  next  It will call the appropriate next or   next   method  depending on the version of Python you are using   gt  gt  gt  first year interest   next my account   gt  gt  gt  first year interest 50 0  And now we can send data into the generator   Sending None is the same as calling next      gt  gt  gt  next year interest   my account send first year interest   1000   gt  gt  gt  next year interest 102 5  Cooperative Delegation to Sub-Coroutine with yield from Now  recall that yield from is available in Python 3  This allows us to delegate coroutines to a subcoroutine   def money manager expected rate         must receive deposited value from  send        under management   yield                     yield None to start      while True          try              additional investment   yield expected rate   under management              if additional investment                  under management    additional investment         except GeneratorExit                 TODO  write function to send unclaimed funds to state                raise         finally                 TODO  write function to mail tax info to client              def investment account deposited  manager          very simple model of an investment account that delegates to a manager          must queue up manager      next manager          lt - same as manager send None        This is where we send the initial deposit to the manager      manager send deposited      try          yield from manager     except GeneratorExit          return manager close      delegate   And now we can delegate functionality to a sub-generator and it can be used by a generator just as above  my manager   money manager  06  my account   investment account 1000  my manager  first year return   next my account    - gt  60 0  Now simulate adding another 1 000 to the account plus the return on the account  60 0   next year return   my account send first year return   1000  next year return   123 6  You can read more about the precise semantics of yield from in PEP 380  Other Methods  close and throw The close method raises GeneratorExit at the point the function execution was frozen  This will also be called by   del   so you can put any cleanup code where you handle the GeneratorExit  my account close    You can also throw an exception which can be handled in the generator or propagated back to the user  import sys try      raise ValueError except      my manager throw  sys exc info     Raises  Traceback  most recent call last     File  quot  lt stdin gt  quot   line 4  in  lt module gt    File  quot  lt stdin gt  quot   line 6  in money manager   File  quot  lt stdin gt  quot   line 2  in  lt module gt  ValueError  Conclusion I believe I have covered all aspects of the following question   What does the yield keyword do in Python   It turns out that yield does a lot  I m sure I could add even more thorough examples to this  If you want more or have some constructive criticism  let me know by commenting below   Appendix  Critique of the Top Accepted Answer    It is confused on what makes an iterable  just using a list as an example  See my references above  but in summary  an iterable has an   iter   method returning an iterator  An iterator provides a  next  Python 2 or    next    Python 3  method  which is implicitly called by for loops until it raises StopIteration  and once it does  it will continue to do so  It then uses a generator expression to describe what a generator is  Since a generator is simply a convenient way to create an iterator  it only confuses the matter  and we still have not yet gotten to the yield part  In Controlling a generator exhaustion he calls the  next method  when instead he should use the builtin function  next  It would be an appropriate layer of indirection  because his code does not work in Python 3  Itertools  This was not relevant to what yield does at all  No discussion of the methods that yield provides along with the new functionality yield from in Python 3  The top accepted answer is a very incomplete answer   Critique of answer suggesting yield in a generator expression or comprehension  The grammar currently allows any expression in a list comprehension  expr stmt  testlist star expr  annassign   augassign  yield expr testlist                               yield expr testlist star expr         yield expr   yield   yield arg  yield arg   from  test   testlist  Since yield is an expression  it has been touted by some as interesting to use it in comprehensions or generator expression - in spite of citing no particularly good use-case  The CPython core developers are discussing deprecating its allowance  Here s a relevant post from the mailing list   On 30 January 2017 at 19 05  Brett Cannon  wrote   On Sun  29 Jan 2017 at 16 39 Craig Rodrigues  wrote   I m OK with either approach   Leaving things the way they are in Python 3 is no good  IMHO   My vote is it be a SyntaxError since you re not getting what you expect from the syntax   I d agree that s a sensible place for us to end up  as any code relying on the current behaviour is really too clever to be maintainable  In terms of getting there  we ll likely want   SyntaxWarning or DeprecationWarning in 3 7 Py3k warning in 2 7 x SyntaxError in 3 8  Cheers  Nick  --  Nick Coghlan       ncoghlan at gmail com       Brisbane  Australia  Further  there is an outstanding issue  10544  which seems to be pointing in the direction of this never being a good idea  PyPy  a Python implementation written in Python  is already raising syntax warnings   Bottom line  until the developers of CPython tell us otherwise  Don t put yield in a generator expression or comprehension  The return statement in a generator In Python 2   In a generator function  the return statement is not allowed to include an expression list  In that context  a bare return indicates that the generator is done and will cause StopIteration to be raised   An expression list is basically any number of expressions separated by commas - essentially  in Python 2  you can stop the generator with return  but you can t return a value  In Python 3   In a generator function  the return statement indicates that the generator is done and will cause StopIteration to be raised  The returned value  if any  is used as an argument to construct StopIteration and becomes the StopIteration value attribute   Footnotes  The languages CLU  Sather  and Icon were referenced in the proposal to introduce the concept of generators to Python  The general idea is that a function can maintain internal state and yield intermediate data points on demand by the user  This promised to be superior in performance to other approaches  including Python threading  which isn t even available on some systems    This means  for example  that range objects aren t Iterators  even though they are iterable  because they can be reused  Like lists  their   iter   methods return iterator objects       yield was originally introduced as a statement  meaning that it could only appear at the beginning of a line in a code block  Now yield creates a yield expression  https   docs python org 2 reference simple stmts html grammar-token-yield stmt This change was proposed to allow a user to send data into the generator just as one might receive it  To send data  one must be able to assign it to something  and for that  a statement just won t work

User · Answer

All of the answers here are great  but only one of them  the most voted one  relates to how your code works  Others are relating to generators in general  and how they work   So I won t repeat what generators are or what yields do  I think these are covered by great existing answers  However  after spending few hours trying to understand a similar code to yours  I ll break it down how it works   Your code traverse a binary tree structure  Let s take this tree for example       5          3   6          1   4   8   And another simpler implementation of a binary-search tree traversal   class Node object      def   iter   self       if self has left child            for child in self left              yield child      yield self val      if self has right child            for child in self right              yield child   The execution code is on the Tree object  which implements   iter   as this   def   iter   self        class EmptyIter            def next self               raise StopIteration      if self root          return self root   iter         return EmptyIter     The while candidates statement can be replaced with for element in tree  Python translate this to  it   iter TreeObj     returns iter self root  which calls self root   iter     for element in it          process element       Because Node   iter   function is a generator  the code inside it is executed per iteration  So the execution would look like this    root element is first  check if it has left childs and for iterate them  let s call it it1 because its the first iterator object  it has a child so the for is executed  The for child in self left creates a new iterator from self left  which is a Node object itself  it2  Same logic as 2  and a new iterator is created  it3  Now we reached the left end of the tree  it3 has no left childs so it continues and yield self value On the next call to next it3  it raises StopIteration and exists since it has no right childs  it reaches to the end of the function without yield anything  it1 and it2 are still active - they are not exhausted and calling next it2  would yield values  not raise StopIteration Now we are back to it2 context  and call next it2  which continues where it stopped  right after the yield child statement  Since it has no more left childs it continues and yields it s self val    The catch here is that every iteration creates sub-iterators to traverse the tree  and holds the state of the current iterator  Once it reaches the end it traverse back the stack  and values are returned in the correct order  smallest yields value first    Your code example did something similar in a different technique  it populated a one-element list for every child  then on the next iteration it pops it and run the function code on the current object  hence the self    I hope this contributed a little to this legendary topic  I spent several good hours drawing this process to understand it

User · Answer

The yield keyword is going to replace return in a function definition to create a generator  def create generator       for i in range 100      yield i myGenerator   create generator   print myGenerator     lt generator object create generator at 0x102dd2480 gt  for i in myGenerator     print i    prints 0-99  When the returned generator is first used   not in the assignment but the for loop   the function definition will execute until it reaches the yield statement  There  it will pause  see why it   s called yield  until used again  Then  it will pick up where it left off  Upon the final iteration of the generator  any code after the yield command will execute  def create generator       print  quot Beginning of generator quot      for i in range 4         yield i    print  quot After yield quot   print  quot Before assignment quot    myGenerator   create generator    print  quot After assignment quot   for i in myGenerator      print i    prints 0-3  quot  quot  quot  Before assignment After assignment Beginning of generator 0 1 2 After yield  The yield keyword modifies a function   s behavior to produce a generator that   s paused at each yield command during iteration  The function isn   t executed except upon iteration  which leads to improved resource management  and subsequently  a better overall performance  Use generators  and yielded functions  for creating large data sets meant for single-use iteration

User · Answer

While a lot of answers show why you d use a yield to create a generator  there are more uses for yield   It s quite easy to make a coroutine  which enables the passing of information between two blocks of code   I won t repeat any of the fine examples that have already been given about using yield to create a generator   To help understand what a yield does in the following code  you can use your finger to trace the cycle through any code that has a yield   Every time your finger hits the yield  you have to wait for a next or a send to be entered   When a next is called  you trace through the code until you hit the yield    the code on the right of the yield is evaluated and returned to the caller    then you wait   When next is called again  you perform another loop through the code   However  you ll note that in a coroutine  yield can also be used with a send    which will send a value from the caller into the yielding function  If a send is given  then yield receives the value sent  and spits it out the left hand side    then the trace through the code progresses until you hit the yield again  returning the value at the end  as if next was called    For example    gt  gt  gt  def coroutine            i   -1         while True              i    1             val    yield i              print  Received  s    val       gt  gt  gt  sequence   coroutine    gt  gt  gt  sequence next   0  gt  gt  gt  sequence next   Received None 1  gt  gt  gt  sequence send  hello   Received hello 2  gt  gt  gt  sequence close

User · Answer

The yield keyword simply collects returning results  Think of yield like return

User · Answer

The yield keyword is going to replace return in a function definition to create a generator  def create generator       for i in range 100      yield i myGenerator   create generator   print myGenerator     lt generator object create generator at 0x102dd2480 gt  for i in myGenerator     print i    prints 0-99  When the returned generator is first used   not in the assignment but the for loop   the function definition will execute until it reaches the yield statement  There  it will pause  see why it   s called yield  until used again  Then  it will pick up where it left off  Upon the final iteration of the generator  any code after the yield command will execute  def create generator       print  quot Beginning of generator quot      for i in range 4         yield i    print  quot After yield quot   print  quot Before assignment quot    myGenerator   create generator    print  quot After assignment quot   for i in myGenerator      print i    prints 0-3  quot  quot  quot  Before assignment After assignment Beginning of generator 0 1 2 After yield  The yield keyword modifies a function   s behavior to produce a generator that   s paused at each yield command during iteration  The function isn   t executed except upon iteration  which leads to improved resource management  and subsequently  a better overall performance  Use generators  and yielded functions  for creating large data sets meant for single-use iteration

User · Answer

Yield   gt  gt  gt  def create generator           my list   range 3         for i in my list             yield i i      gt  gt  gt  my generator   create generator     create a generator  gt  gt  gt  print my generator    my generator is an object   lt generator object create generator at 0xb7555c34 gt   gt  gt  gt  for i in my generator          print i  0 1 4   In short  you can see that the loop does not stop and continues to function even after the object or variable is sent  unlike return where the loop stops after execution

User · Answer

For those who prefer a minimal working example  meditate on this interactive Python session    gt  gt  gt  def f          yield 1       yield 2       yield 3       gt  gt  gt  g   f    gt  gt  gt  for i in g        print i       1 2 3  gt  gt  gt  for i in g        print i        gt  gt  gt    Note that this time nothing was printed

User · Answer

An analogy could help to grasp the idea here     Imagine that you have created an amazing machine that is capable of generating thousands and thousands of lightbulbs per day  The machine generates these lightbulbs in boxes with a unique serial number  You don t have enough space to store all these lightbulbs at the same time  i e   you cannot keep up with the speed of the machine due to storage limitation   so you would like to adjust this machine to generate lightbulbs on demand   Python generators don t differ much from this concept   Imagine that you have a function x that generates unique serial numbers for the boxes  Obviously  you can have a very large number of such barcodes generated by the function  A wiser  and space efficient  option is to generate those serial numbers on-demand   Machine s code   def barcode generator        serial number   10000    Initial barcode     while True          yield serial number         serial number    1   barcode   barcode generator   while True      number of lightbulbs to generate   int input  How many lightbulbs to generate          barcodes    next barcode  for   in range number of lightbulbs to generate       print barcodes         function to create the next batch of lightbulbs barcodes       produce more   input  Produce more   Y n          if produce more     n           break   As you can see we have a self-contained  function  to generate the next unique serial number each time  This function returns back a generator  As you can see we are not calling the function each time we need a new serial number  but we are using next   given the generator to obtain the next serial number   Output   How many lightbulbs to generate  5  10000  10001  10002  10003  10004  Produce more   Y n   y How many lightbulbs to generate  6  10005  10006  10007  10008  10009  10010  Produce more   Y n   y How many lightbulbs to generate  7  10011  10012  10013  10014  10015  10016  10017  Produce more   Y n   n

User · Answer

There is one type of answer that I don t feel has been given yet  among the many great answers that describe how to use generators  Here is the programming language theory answer   The yield statement in Python returns a generator  A generator in Python is a function that returns continuations  and specifically a type of coroutine  but continuations represent the more general mechanism to understand what is going on    Continuations in programming languages theory are a much more fundamental kind of computation  but they are not often used  because they are extremely hard to reason about and also very difficult to implement  But the idea of what a continuation is  is straightforward  it is the state of a computation that has not yet finished  In this state  the current values of variables  the operations that have yet to be performed  and so on  are saved  Then at some point later in the program the continuation can be invoked  such that the program s variables are reset to that state and the operations that were saved are carried out   Continuations  in this more general form  can be implemented in two ways  In the call cc way  the program s stack is literally saved and then when the continuation is invoked  the stack is restored   In continuation passing style  CPS   continuations are just normal functions  only in languages where functions are first class  which the programmer explicitly manages and passes around to subroutines  In this style  program state is represented by closures  and the variables that happen to be encoded in them  rather than variables that reside somewhere on the stack  Functions that manage control flow accept continuation as arguments  in some variations of CPS  functions may accept multiple continuations  and manipulate control flow by invoking them by simply calling them and returning afterwards  A very simple example of continuation passing style is as follows   def save file filename     def write file continuation        write stuff to file filename     check if file exists and user wants to overwrite write file continuation    In this  very simplistic  example  the programmer saves the operation of actually writing the file into a continuation  which can potentially be a very complex operation with many details to write out   and then passes that continuation  i e  as a first-class closure  to another operator which does some more processing  and then calls it if necessary   I use this design pattern a lot in actual GUI programming  either because it saves me lines of code or  more importantly  to manage control flow after GUI events trigger    The rest of this post will  without loss of generality  conceptualize continuations as CPS  because it is a hell of a lot easier to understand and read     Now let s talk about generators in Python  Generators are a specific subtype of continuation  Whereas continuations are able in general to save the state of a computation  i e   the program s call stack   generators are only able to save the state of iteration over an iterator  Although  this definition is slightly misleading for certain use cases of generators  For instance   def f      while True      yield 4   This is clearly a reasonable iterable whose behavior is well defined -- each time the generator iterates over it  it returns 4  and does so forever   But it isn t probably the prototypical type of iterable that comes to mind when thinking of iterators  i e   for x in collection  do something x    This example illustrates the power of generators  if anything is an iterator  a generator can save the state of its iteration   To reiterate  Continuations can save the state of a program s stack and generators can save the state of iteration  This means that continuations are more a lot powerful than generators  but also that generators are a lot  lot easier  They are easier for the language designer to implement  and they are easier for the programmer to use  if you have some time to burn  try to read and understand this page about continuations and call cc    But you could easily implement  and conceptualize  generators as a simple  specific case of continuation passing style   Whenever yield is called  it tells the function to return a continuation   When the function is called again  it starts from wherever it left off  So  in pseudo-pseudocode  i e   not pseudocode  but not code  the generator s next method is basically as follows   class Generator      def   init   self iterable generatorfun       self next continuation   lambda generatorfun iterable     def next self       value  next continuation   self next continuation       self next continuation   next continuation     return value   where the yield keyword is actually syntactic sugar for the real generator function  basically something like   def generatorfun iterable     if len iterable     0      raise StopIteration   else      return  iterable 0   lambda generatorfun iterable 1       Remember that this is just pseudocode and the actual implementation of generators in Python is more complex  But as an exercise to understand what is going on  try to use continuation passing style to implement generator objects without use of the yield keyword

User · Answer

Yet another TL DR  Iterator on list  next   returns the next element of the list  Iterator generator  next   will compute the next element on the fly  execute code   You can see the yield generator as a way to manually run the control flow from outside  like continue loop one step   by calling next  however complex the flow   Note  The generator is NOT a normal function  It remembers the previous state like local variables  stack   See other answers or articles for detailed explanation  The generator can only be iterated on once  You could do without yield  but it would not be as nice  so it can be considered  very nice  language sugar

User · Answer

From a programming viewpoint  the iterators are implemented as thunks  To implement iterators  generators  and thread pools for concurrent execution  etc  as thunks  one uses messages sent to a closure object  which has a dispatcher  and the dispatcher answers to  quot messages quot    quot next quot  is a message sent to a closure  created by the  quot iter quot  call  There are lots of ways to implement this computation  I used mutation  but it is possible to do this kind of computation without mutation  by returning the current value and the next yielder  making it referential transparent    Racket uses a sequence of transformations of the initial program in some intermediary languages  one of such rewriting making the yield operator to be transformed in some language with simpler operators  Here is a demonstration of how yield could be rewritten  which uses the structure of R6RS  but the semantics is identical to Python s  It s the same model of computation  and only a change in syntax is required to rewrite it using yield of Python   Welcome to Racket v6 5 0 3   - gt   define gen       lambda  l          define yield           lambda                if  null  l                  END                 let   v  car l                      set  l  cdr l                    v             lambda m            case m              yield  yield                init   lambda  data                        set  l data                        OK       - gt   define stream  gen   1 2 3    - gt   stream  yield  1 - gt   stream  yield  2 - gt   stream  yield  3 - gt   stream  yield   END - gt    stream  init    a b    OK - gt   stream  yield   a - gt   stream  yield   b - gt   stream  yield   END - gt   stream  yield   END - gt

User · Answer

All great answers  however a bit difficult for newbies   I assume you have learned the return statement   As an analogy  return and yield are twins  return means  return and stop  whereas  yield  means  return  but continue         Try to get a num list with return       def num list n       for i in range n           return i   Run it   In  5   num list 3  Out 5   0   See  you get only a single number rather than a list of them  return never allows you prevail happily  just implements once and quit         There comes yield      Replace return with yield   In  10   def num list n                for i in range n                    yield i           In  11   num list 3  Out 11    lt generator object num list at 0x10327c990 gt   In  12   list num list 3   Out 12    0  1  2    Now  you win to get all the numbers   Comparing to return which runs once and stops  yield runs times you planed  You can interpret return as return one of them  and yield as return all of them  This is called iterable         One more step we can rewrite yield statement with return      In  15   def num list n                result                   for i in range n                    result append i               return result  In  16   num list 3  Out 16    0  1  2    It s the core about yield   The difference between a list return outputs and the object yield output is   You will always get  0  1  2  from a list object but only could retrieve them from  the object yield output  once  So  it has a new name generator object as displayed in Out 11    lt generator object num list at 0x10327c990 gt    In conclusion  as a metaphor to grok it    return and yield are twins list and generator are twins

User · Answer

There is another yield use and meaning  since Python 3 3   yield from  lt expr gt   From PEP 380 -- Syntax for Delegating to a Subgenerator   A syntax is proposed for a generator to delegate part of its operations to another generator  This allows a section of code containing  yield  to be factored out and placed in another generator  Additionally  the subgenerator is allowed to return with a value  and the value is made available to the delegating generator  The new syntax also opens up some opportunities for optimisation when one generator re-yields values produced by another   Moreover this will introduce  since Python 3 5   async def new coroutine data             await blocking action    to avoid coroutines being confused with a regular generator  today yield is used in both

User · Answer

A simple generator function  def my gen        n   1     print  This is printed first         Generator function contains yield statements     yield n      n    1     print  This is printed second       yield n      n    1     print  This is printed at last       yield n   yield statement pauses the function saving all its states and later continues from there on successive calls   https   www programiz com python-programming generator

User · Answer

Here s a simple yield based approach  to compute the fibonacci series  explained   def fib limit 50       a  b   0  1     for i in range limit          yield b        a  b   b  a b   When you enter this into your REPL and then try and call it  you ll get a mystifying result    gt  gt  gt  fib    lt generator object fib at 0x7fa38394e3b8 gt    This is because the presence of yield signaled to Python that you want to create a generator  that is  an object that generates values on demand   So  how do you generate these values  This can either be done directly by using the built-in function next  or  indirectly by feeding it to a construct that consumes values    Using the built-in next   function  you directly invoke  next   next    forcing the generator to produce a value    gt  gt  gt  g   fib    gt  gt  gt  next g  1  gt  gt  gt  next g  1  gt  gt  gt  next g  2  gt  gt  gt  next g  3  gt  gt  gt  next g  5   Indirectly  if you provide fib to a for loop  a list initializer  a tuple initializer  or anything else that expects an object that generates produces values  you ll  consume  the generator until no more values can be produced by it  and it returns    results      for i in fib 30           consumes fib     results append i     can also be accomplished with results   list fib 30     consumes fib   Similarly  with a tuple initializer     gt  gt  gt  tuple fib 5           consumes fib  1  1  2  3  5    A generator differs from a function in the sense that it is lazy  It accomplishes this by maintaining it s local state and allowing you to resume whenever you need to    When you first invoke fib by calling it   f   fib     Python compiles the function  encounters the yield keyword and simply returns a generator object back at you  Not very helpful it seems    When you then request it generates the first value  directly or indirectly  it executes all statements that it finds  until it encounters a yield  it then yields back the value you supplied to yield and pauses  For an example that better demonstrates this  let s use some print calls  replace with print  text  if on Python 2    def yielder value           This is an infinite generator  Only use next on it          while 1          print  I m going to generate the value for you           print  Then I ll pause for a while           yield value         print  Let s go through it again      Now  enter in the REPL    gt  gt  gt  gen   yielder  Hello  yield      you have a generator object now waiting for a command for it to generate a value  Use next and see what get s printed    gt  gt  gt  next gen    runs until it finds a yield I m going to generate the value for you Then I ll pause for a while  Hello  yield     The unquoted results are what s printed  The quoted result is what is returned from yield  Call next again now    gt  gt  gt  next gen    continues from yield and runs again Let s go through it again  I m going to generate the value for you Then I ll pause for a while  Hello  yield     The generator remembers it was paused at yield value and resumes from there  The next message is printed and the search for the yield statement to pause at it performed again  due to the while loop

User · Answer

An easy example to understand what it is  yield  def f123        for   in range 4           yield 1         yield 2   for i in f123        print  i    The output is    1 2 1 2 1 2 1 2

User · Answer

Yet another TL DR  Iterator on list  next   returns the next element of the list  Iterator generator  next   will compute the next element on the fly  execute code   You can see the yield generator as a way to manually run the control flow from outside  like continue loop one step   by calling next  however complex the flow   Note  The generator is NOT a normal function  It remembers the previous state like local variables  stack   See other answers or articles for detailed explanation  The generator can only be iterated on once  You could do without yield  but it would not be as nice  so it can be considered  very nice  language sugar

User · Answer

There s one extra thing to mention  a function that yields doesn t actually have to terminate  I ve written code like this   def fib        last  cur   0  1     while True           yield cur         last  cur   cur  last   cur   Then I can use it in other code like this   for f in fib        if some condition  break     coolfuncs f     It really helps simplify some problems  and makes some things easier to work with

User · Answer

TL DR  Instead of this   def square list n       the list                                Replace     for x in range n           y   x   x         the list append y                   these     return the list                         lines   do this   def square yield n       for x in range n           y   x   x         yield y                             with this one    Whenever you find yourself building a list from scratch  yield each piece instead    This was my first  aha  moment with yield     yield is a sugary way to say      build a series of stuff   Same behavior    gt  gt  gt  for square in square list 4           print square      0 1 4 9  gt  gt  gt  for square in square yield 4           print square      0 1 4 9   Different behavior   Yield is single-pass  you can only iterate through once  When a function has a yield in it we call it a generator function  And an iterator is what it returns  Those terms are revealing  We lose the convenience of a container  but gain the power of a series that s computed as needed  and arbitrarily long   Yield is lazy  it puts off computation  A function with a yield in it doesn t actually execute at all when you call it  It returns an iterator object that remembers where it left off  Each time you call next   on the iterator  this happens in a for-loop  execution inches forward to the next yield  return raises StopIteration and ends the series  this is the natural end of a for-loop    Yield is versatile  Data doesn t have to be stored all together  it can be made available one at a time  It can be infinite    gt  gt  gt  def squares all of them            x   0         while True              yield x   x             x    1      gt  gt  gt  squares   squares all of them    gt  gt  gt  for   in range 4           print next squares       0 1 4 9     If you need multiple passes and the series isn t too long  just call list   on it    gt  gt  gt  list square yield 4    0  1  4  9      Brilliant choice of the word yield because both meanings apply      yield  mdash  produce or provide  as in agriculture       provide the next data in the series      yield  mdash  give way or relinquish  as in political power       relinquish CPU execution until the iterator advances

User · Answer

Here is a mental image of what yield does   I like to think of a thread as having a stack  even when it s not implemented that way    When a normal function is called  it puts its local variables on the stack  does some computation  then clears the stack and returns  The values of its local variables are never seen again   With a yield function  when its code begins to run  i e  after the function is called  returning a generator object  whose next   method is then invoked   it similarly puts its local variables onto the stack and computes for a while  But then  when it hits the yield statement  before clearing its part of the stack and returning  it takes a snapshot of its local variables and stores them in the generator object  It also writes down the place where it s currently up to in its code  i e  the particular yield statement    So it s a kind of a frozen function that the generator is hanging onto   When next   is called subsequently  it retrieves the function s belongings onto the stack and re-animates it  The function continues to compute from where it left off  oblivious to the fact that it had just spent an eternity in cold storage   Compare the following examples   def normalFunction        return     if False          pass  def yielderFunction        return     if False          yield 12   When we call the second function  it behaves very differently to the first  The yield statement might be unreachable  but if it s present anywhere  it changes the nature of what we re dealing with    gt  gt  gt  yielderFunction    lt generator object yielderFunction at 0x07742D28 gt    Calling yielderFunction   doesn t run its code  but makes a generator out of the code   Maybe it s a good idea to name such things with the yielder prefix for readability     gt  gt  gt  gen   yielderFunction    gt  gt  gt  dir gen      class              iter         Returns gen itself  to make it work uniformly with containers                  when given to a for loop   Containers return an iterator instead     close     gi code     gi frame     gi running     next           The method that runs the function s body    send     throw     The gi code and gi frame fields are where the frozen state is stored  Exploring them with dir      we can confirm that our mental model above is credible

User · Answer

To understand what yield does  you must understand what generators are  And before you can understand generators  you must understand iterables  Iterables When you create a list  you can read its items one by one  Reading its items one by one is called iteration   gt  gt  gt  mylist    1  2  3   gt  gt  gt  for i in mylist         print i  1 2 3  mylist is an iterable  When you use a list comprehension  you create a list  and so an iterable   gt  gt  gt  mylist    x x for x in range 3    gt  gt  gt  for i in mylist         print i  0 1 4  Everything you can use  quot for    in    quot  on is an iterable  lists  strings  files    These iterables are handy because you can read them as much as you wish  but you store all the values in memory and this is not always what you want when you have a lot of values  Generators Generators are iterators  a kind of iterable you can only iterate over once  Generators do not store all the values in memory  they generate the values on the fly   gt  gt  gt  mygenerator    x x for x in range 3    gt  gt  gt  for i in mygenerator         print i  0 1 4  It is just the same except you used    instead of     BUT  you cannot perform for i in mygenerator a second time since generators can only be used once  they calculate 0  then forget about it and calculate 1  and end calculating 4  one by one  Yield yield is a keyword that is used like return  except the function will return a generator   gt  gt  gt  def createGenerator           mylist   range 3         for i in mylist             yield i i      gt  gt  gt  mygenerator   createGenerator     create a generator  gt  gt  gt  print mygenerator    mygenerator is an object   lt generator object createGenerator at 0xb7555c34 gt   gt  gt  gt  for i in mygenerator          print i  0 1 4  Here it s a useless example  but it s handy when you know your function will return a huge set of values that you will only need to read once  To master yield  you must understand that when you call the function  the code you have written in the function body does not run  The function only returns the generator object  this is a bit tricky  -  Then  your code will continue from where it left off each time for uses the generator  Now the hard part  The first time the for calls the generator object created from your function  it will run the code in your function from the beginning until it hits yield  then it ll return the first value of the loop  Then  each subsequent call will run another iteration of the loop you have written in the function and return the next value  This will continue until the generator is considered empty  which happens when the function runs without hitting yield  That can be because the loop has come to an end  or because you no longer satisfy an  quot if else quot    Your code explained Generator    Here you create the method of the node object that will return the generator def  get child candidates self  distance  min dist  max dist          Here is the code that will be called each time you use the generator object         If there is still a child of the node object on its left       AND if the distance is ok  return the next child     if self  leftchild and distance - max dist  lt  self  median          yield self  leftchild        If there is still a child of the node object on its right       AND if the distance is ok  return the next child     if self  rightchild and distance   max dist  gt   self  median          yield self  rightchild        If the function arrives here  the generator will be considered empty       there is no more than two values  the left and the right children  Caller    Create an empty list and a list with the current object reference result  candidates   list     self     Loop on candidates  they contain only one element at the beginning  while candidates         Get the last candidate and remove it from the list     node   candidates pop          Get the distance between obj and the candidate     distance   node  get dist obj         If distance is ok  then you can fill the result     if distance  lt   max dist and distance  gt   min dist          result extend node  values         Add the children of the candidate in the candidate s list       so the loop will keep running until it will have looked       at all the children of the children of the children  etc  of the candidate     candidates extend node  get child candidates distance  min dist  max dist    return result  This code contains several smart parts   The loop iterates on a list  but the list expands while the loop is being iterated  -  It s a concise way to go through all these nested data even if it s a bit dangerous since you can end up with an infinite loop  In this case  candidates extend node  get child candidates distance  min dist  max dist   exhaust all the values of the generator  but while keeps creating new generator objects which will produce different values from the previous ones since it s not applied on the same node   The extend   method is a list object method that expects an iterable and adds its values to the list    Usually we pass a list to it   gt  gt  gt  a    1  2   gt  gt  gt  b    3  4   gt  gt  gt  a extend b   gt  gt  gt  print a   1  2  3  4   But in your code  it gets a generator  which is good because   You don t need to read the values twice  You may have a lot of children and you don t want them all stored in memory   And it works because Python does not care if the argument of a method is a list or not  Python expects iterables so it will work with strings  lists  tuples  and generators  This is called duck typing and is one of the reasons why Python is so cool  But this is another story  for another question    You can stop here  or read a little bit to see an advanced use of a generator  Controlling a generator exhaustion  gt  gt  gt  class Bank      Let s create a bank  building ATMs        crisis   False        def create atm self              while not self crisis                 yield  quot  100 quot   gt  gt  gt  hsbc   Bank     When everything s ok the ATM gives you as much as you want  gt  gt  gt  corner street atm   hsbc create atm    gt  gt  gt  print corner street atm next     100  gt  gt  gt  print corner street atm next     100  gt  gt  gt  print  corner street atm next   for cash in range 5       100     100     100     100     100    gt  gt  gt  hsbc crisis   True   Crisis is coming  no more money   gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  wall street atm   hsbc create atm     It s even true for new ATMs  gt  gt  gt  print wall street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  hsbc crisis   False   The trouble is  even post-crisis the ATM remains empty  gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  brand new atm   hsbc create atm     Build a new one to get back in business  gt  gt  gt  for cash in brand new atm         print cash  100  100  100  100  100  100  100  100  100      Note  For Python 3  useprint corner street atm   next      or print next corner street atm   It can be useful for various things like controlling access to a resource  Itertools  your best friend The itertools module contains special functions to manipulate iterables  Ever wish to duplicate a generator  Chain two generators  Group values in a nested list with a one-liner  Map   Zip without creating another list  Then just import itertools  An example  Let s see the possible orders of arrival for a four-horse race   gt  gt  gt  horses    1  2  3  4   gt  gt  gt  races   itertools permutations horses   gt  gt  gt  print races   lt itertools permutations object at 0xb754f1dc gt   gt  gt  gt  print list itertools permutations horses      1  2  3  4     1  2  4  3     1  3  2  4     1  3  4  2     1  4  2  3     1  4  3  2     2  1  3  4     2  1  4  3     2  3  1  4     2  3  4  1     2  4  1  3     2  4  3  1     3  1  2  4     3  1  4  2     3  2  1  4     3  2  4  1     3  4  1  2     3  4  2  1     4  1  2  3     4  1  3  2     4  2  1  3     4  2  3  1     4  3  1  2     4  3  2  1    Understanding the inner mechanisms of iteration Iteration is a process implying iterables  implementing the   iter     method  and iterators  implementing the   next     method   Iterables are any objects you can get an iterator from  Iterators are objects that let you iterate on iterables  There is more about it in this article about how for loops work

User · Answer

First yield program   def countdown gen x         count   x       while count  gt  0             yield count            count -  1  g   countdown gen 5   for item in g       print item    Output  5 4 3 2 1   Understand flow     Note   Return a generator obj Suspend a functon execution and save its status  function can be executed again

User · Answer

Like every answer suggests  yield is used for creating a sequence generator  It s used for generating some sequence dynamically  For example  while reading a file line by line on a network  you can use the yield function as follows   def getNextLines       while con isOpen           yield con read     You can use it in your code as follows   for line in getNextLines        doSomeThing line    Execution Control Transfer gotcha  The execution control will be transferred from getNextLines   to the for loop when yield is executed  Thus  every time getNextLines   is invoked  execution begins from the point where it was paused last time   Thus in short  a function with the following code  def simpleYield        yield  first time      yield  second time      yield  third time      yield  Now some useful value     format 12   for i in simpleYield        print i   will print   first time   second time   third time   Now some useful value 12

User · Answer

In summary  the yield statement transforms your function into a factory that produces a special object called a generator which wraps around the body of your original function  When the generator is iterated  it executes your function  until it reaches the next yield then suspends execution and evaluates to the value passed to yield  It repeats this process on each iteration until the path of execution exits the function  For instance   def simple generator        yield  one      yield  two      yield  three   for i in simple generator        print i   simply outputs  one two three   The power comes from using the generator with a loop that calculates a sequence  the generator executes the loop stopping each time to  yield  the next result of the calculation  in this way it calculates a list on the fly  the benefit being the memory saved for especially large calculations  Say you wanted to create a your own range function that produces an iterable range of numbers  you could do it like so   def myRangeNaive i       n   0     range          while n  lt  i          range append n          n   n   1     return range   and use it like this   for i in myRangeNaive 10       print i   But this is inefficient because   You create an array that you only use once  this wastes memory  This code actually loops over that array twice       Luckily Guido and his team were generous enough to develop generators so we could just do this   def myRangeSmart i       n   0     while n  lt  i         yield n        n   n   1     return  for i in myRangeSmart 10       print i   Now upon each iteration a function on the generator called next   executes the function until it either reaches a  yield  statement in which it stops and   yields  the value or reaches the end of the function  In this case on the first call  next   executes up to the yield statement and yield  n   on the next call it will execute the  increment statement  jump back to the  while   evaluate it  and if true  it will stop and yield  n  again  it will continue that way until the while condition returns false and the generator jumps to the end of the function

User · Answer

yield yields something  It s like somebody asks you to make 5 cupcakes  If you are done with at least one cupcake  you can give it to them to eat while you make other cakes   In  4   def make cake numbers               for i in range numbers                   yield  Cake     format i           In  5   factory   make cake 5    Here factory is called a generator  which makes you cakes  If you call make function  you get a generator instead of running that function  It is because when yield keyword is present in a function  it becomes a generator   In  7   next factory  Out 7    Cake 0   In  8   next factory  Out 8    Cake 1   In  9   next factory  Out 9    Cake 2   In  10   next factory  Out 10    Cake 3   In  11   next factory  Out 11    Cake 4    They consumed all the cakes  but they ask for one again   In  12   next factory  --------------------------------------------------------------------------- StopIteration                             Traceback  most recent call last   lt ipython-input-12-0f5c45da9774 gt  in  lt module gt  ---- gt  1 next factory   StopIteration    and they are being told to stop asking more  So once you consumed a generator you are done with it  You need to call make cake again if you want more cakes  It is like placing another order for cupcakes   In  13   factory   make cake 3   In  14   for cake in factory               print cake           Cake 0 Cake 1 Cake 2   You can also use for loop with a generator like the one above   One more example  Lets say you want a random password whenever you ask for it   In  22   import random  In  23   import string  In  24   def random password generator                 while True                   yield    join  random choice string ascii letters  for   in range 8              In  25   rpg   random password generator    In  26   for i in range 3                print next rpg            FXpUBhhH DdUDHoHn dvtebEqG  In  27   next rpg  Out 27    mJbYRMNo    Here rpg is a generator  which can generate an infinite number of random passwords  So we can also say that generators are useful when we don t know the length of the sequence  unlike list which has a finite number of elements

User · Answer

To understand what yield does  you must understand what generators are  And before you can understand generators  you must understand iterables  Iterables When you create a list  you can read its items one by one  Reading its items one by one is called iteration   gt  gt  gt  mylist    1  2  3   gt  gt  gt  for i in mylist         print i  1 2 3  mylist is an iterable  When you use a list comprehension  you create a list  and so an iterable   gt  gt  gt  mylist    x x for x in range 3    gt  gt  gt  for i in mylist         print i  0 1 4  Everything you can use  quot for    in    quot  on is an iterable  lists  strings  files    These iterables are handy because you can read them as much as you wish  but you store all the values in memory and this is not always what you want when you have a lot of values  Generators Generators are iterators  a kind of iterable you can only iterate over once  Generators do not store all the values in memory  they generate the values on the fly   gt  gt  gt  mygenerator    x x for x in range 3    gt  gt  gt  for i in mygenerator         print i  0 1 4  It is just the same except you used    instead of     BUT  you cannot perform for i in mygenerator a second time since generators can only be used once  they calculate 0  then forget about it and calculate 1  and end calculating 4  one by one  Yield yield is a keyword that is used like return  except the function will return a generator   gt  gt  gt  def createGenerator           mylist   range 3         for i in mylist             yield i i      gt  gt  gt  mygenerator   createGenerator     create a generator  gt  gt  gt  print mygenerator    mygenerator is an object   lt generator object createGenerator at 0xb7555c34 gt   gt  gt  gt  for i in mygenerator          print i  0 1 4  Here it s a useless example  but it s handy when you know your function will return a huge set of values that you will only need to read once  To master yield  you must understand that when you call the function  the code you have written in the function body does not run  The function only returns the generator object  this is a bit tricky  -  Then  your code will continue from where it left off each time for uses the generator  Now the hard part  The first time the for calls the generator object created from your function  it will run the code in your function from the beginning until it hits yield  then it ll return the first value of the loop  Then  each subsequent call will run another iteration of the loop you have written in the function and return the next value  This will continue until the generator is considered empty  which happens when the function runs without hitting yield  That can be because the loop has come to an end  or because you no longer satisfy an  quot if else quot    Your code explained Generator    Here you create the method of the node object that will return the generator def  get child candidates self  distance  min dist  max dist          Here is the code that will be called each time you use the generator object         If there is still a child of the node object on its left       AND if the distance is ok  return the next child     if self  leftchild and distance - max dist  lt  self  median          yield self  leftchild        If there is still a child of the node object on its right       AND if the distance is ok  return the next child     if self  rightchild and distance   max dist  gt   self  median          yield self  rightchild        If the function arrives here  the generator will be considered empty       there is no more than two values  the left and the right children  Caller    Create an empty list and a list with the current object reference result  candidates   list     self     Loop on candidates  they contain only one element at the beginning  while candidates         Get the last candidate and remove it from the list     node   candidates pop          Get the distance between obj and the candidate     distance   node  get dist obj         If distance is ok  then you can fill the result     if distance  lt   max dist and distance  gt   min dist          result extend node  values         Add the children of the candidate in the candidate s list       so the loop will keep running until it will have looked       at all the children of the children of the children  etc  of the candidate     candidates extend node  get child candidates distance  min dist  max dist    return result  This code contains several smart parts   The loop iterates on a list  but the list expands while the loop is being iterated  -  It s a concise way to go through all these nested data even if it s a bit dangerous since you can end up with an infinite loop  In this case  candidates extend node  get child candidates distance  min dist  max dist   exhaust all the values of the generator  but while keeps creating new generator objects which will produce different values from the previous ones since it s not applied on the same node   The extend   method is a list object method that expects an iterable and adds its values to the list    Usually we pass a list to it   gt  gt  gt  a    1  2   gt  gt  gt  b    3  4   gt  gt  gt  a extend b   gt  gt  gt  print a   1  2  3  4   But in your code  it gets a generator  which is good because   You don t need to read the values twice  You may have a lot of children and you don t want them all stored in memory   And it works because Python does not care if the argument of a method is a list or not  Python expects iterables so it will work with strings  lists  tuples  and generators  This is called duck typing and is one of the reasons why Python is so cool  But this is another story  for another question    You can stop here  or read a little bit to see an advanced use of a generator  Controlling a generator exhaustion  gt  gt  gt  class Bank      Let s create a bank  building ATMs        crisis   False        def create atm self              while not self crisis                 yield  quot  100 quot   gt  gt  gt  hsbc   Bank     When everything s ok the ATM gives you as much as you want  gt  gt  gt  corner street atm   hsbc create atm    gt  gt  gt  print corner street atm next     100  gt  gt  gt  print corner street atm next     100  gt  gt  gt  print  corner street atm next   for cash in range 5       100     100     100     100     100    gt  gt  gt  hsbc crisis   True   Crisis is coming  no more money   gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  wall street atm   hsbc create atm     It s even true for new ATMs  gt  gt  gt  print wall street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  hsbc crisis   False   The trouble is  even post-crisis the ATM remains empty  gt  gt  gt  print corner street atm next     lt type  exceptions StopIteration  gt   gt  gt  gt  brand new atm   hsbc create atm     Build a new one to get back in business  gt  gt  gt  for cash in brand new atm         print cash  100  100  100  100  100  100  100  100  100      Note  For Python 3  useprint corner street atm   next      or print next corner street atm   It can be useful for various things like controlling access to a resource  Itertools  your best friend The itertools module contains special functions to manipulate iterables  Ever wish to duplicate a generator  Chain two generators  Group values in a nested list with a one-liner  Map   Zip without creating another list  Then just import itertools  An example  Let s see the possible orders of arrival for a four-horse race   gt  gt  gt  horses    1  2  3  4   gt  gt  gt  races   itertools permutations horses   gt  gt  gt  print races   lt itertools permutations object at 0xb754f1dc gt   gt  gt  gt  print list itertools permutations horses      1  2  3  4     1  2  4  3     1  3  2  4     1  3  4  2     1  4  2  3     1  4  3  2     2  1  3  4     2  1  4  3     2  3  1  4     2  3  4  1     2  4  1  3     2  4  3  1     3  1  2  4     3  1  4  2     3  2  1  4     3  2  4  1     3  4  1  2     3  4  2  1     4  1  2  3     4  1  3  2     4  2  1  3     4  2  3  1     4  3  1  2     4  3  2  1    Understanding the inner mechanisms of iteration Iteration is a process implying iterables  implementing the   iter     method  and iterators  implementing the   next     method   Iterables are any objects you can get an iterator from  Iterators are objects that let you iterate on iterables  There is more about it in this article about how for loops work

User · Answer

Can also send data back to the generator  Indeed  as many answers here explain  using yield creates a generator  You can use the yield keyword to send data back to a  quot live quot  generator  Example  Let s say we have a method which translates from english to some other language  And in the beginning of it  it does something which is heavy and should be done once  We want this method run forever  don t really know why        and receive words words to be translated  def translator          load all the words in English language and the translation to  other lang      my words dict     hello    hello in other language    dog    dog in other language        while True          word    yield          yield my words dict get word   Unknown word       Running  my words translator   translator    next my words translator  print my words translator send  dog     next my words translator  print my words translator send  cat     will print  dog in other language Unknown word     To summarise  use send method inside a generator to send data back to the generator  To allow that  a  yield  is used

User · Answer

Many people use return rather than yield  but in some cases yield can be more efficient and easier to work with   Here is an example which yield is definitely best for      return  in function    import random  def return dates        dates        With  return  you need to create a list then return it     for i in range 5           date   random choice   1st    2nd    3rd    4th    5th    6th    7th    8th    9th    10th            dates append date      return dates      yield  in function    def yield dates        for i in range 5           date   random choice   1st    2nd    3rd    4th    5th    6th    7th    8th    9th    10th            yield date    yield  makes a generator automatically which works                      in a similar way  This is much more efficient       Calling functions   dates list   return dates   print dates list  for i in dates list      print i   dates generator   yield dates   print dates generator  for i in dates generator      print i    Both functions do the same thing  but yield uses three lines instead of five and has one less variable to worry about           This is the result from the code         As you can see both functions do the same thing  The only difference is return dates   gives a list and yield dates   gives a generator   A real life example would be something like reading a file line by line or if you just want to make a generator

User · Answer

From a programming viewpoint  the iterators are implemented as thunks  To implement iterators  generators  and thread pools for concurrent execution  etc  as thunks  one uses messages sent to a closure object  which has a dispatcher  and the dispatcher answers to  quot messages quot    quot next quot  is a message sent to a closure  created by the  quot iter quot  call  There are lots of ways to implement this computation  I used mutation  but it is possible to do this kind of computation without mutation  by returning the current value and the next yielder  making it referential transparent    Racket uses a sequence of transformations of the initial program in some intermediary languages  one of such rewriting making the yield operator to be transformed in some language with simpler operators  Here is a demonstration of how yield could be rewritten  which uses the structure of R6RS  but the semantics is identical to Python s  It s the same model of computation  and only a change in syntax is required to rewrite it using yield of Python   Welcome to Racket v6 5 0 3   - gt   define gen       lambda  l          define yield           lambda                if  null  l                  END                 let   v  car l                      set  l  cdr l                    v             lambda m            case m              yield  yield                init   lambda  data                        set  l data                        OK       - gt   define stream  gen   1 2 3    - gt   stream  yield  1 - gt   stream  yield  2 - gt   stream  yield  3 - gt   stream  yield   END - gt    stream  init    a b    OK - gt   stream  yield   a - gt   stream  yield   b - gt   stream  yield   END - gt   stream  yield   END - gt

User · Answer

There s one extra thing to mention  a function that yields doesn t actually have to terminate  I ve written code like this   def fib        last  cur   0  1     while True           yield cur         last  cur   cur  last   cur   Then I can use it in other code like this   for f in fib        if some condition  break     coolfuncs f     It really helps simplify some problems  and makes some things easier to work with

User · Answer

It s returning a generator  I m not particularly familiar with Python  but I believe it s the same kind of thing as C  s iterator blocks if you re familiar with those   The key idea is that the compiler interpreter whatever does some trickery so that as far as the caller is concerned  they can keep calling next   and it will keep returning values - as if the generator method was paused  Now obviously you can t really  pause  a method  so the compiler builds a state machine for you to remember where you currently are and what the local variables etc look like  This is much easier than writing an iterator yourself

User · Answer

Yield gives you a generator    def get odd numbers i       return range 1  i  2  def yield odd numbers i       for x in range 1  i  2          yield x foo   get odd numbers 10  bar   yield odd numbers 10  foo  1  3  5  7  9  bar  lt generator object yield odd numbers at 0x1029c6f50 gt  bar next   1 bar next   3 bar next   5   As you can see  in the first case foo holds the entire list in memory at once  It s not a big deal for a list with 5 elements  but what if you want a list of 5 million  Not only is this a huge memory eater  it also costs a lot of time to build at the time that the function is called   In the second case  bar just gives you a generator  A generator is an iterable--which means you can use it in a for loop  etc  but each value can only be accessed once  All the values are also not stored in memory at the same time  the generator object  remembers  where it was in the looping the last time you called it--this way  if you re using an iterable to  say  count to 50 billion  you don t have to count to 50 billion all at once and store the 50 billion numbers to count through   Again  this is a pretty contrived example  you probably would use itertools if you really wanted to count to 50 billion      This is the most simple use case of generators  As you said  it can be used to write efficient permutations  using yield to push things up through the call stack instead of using some sort of stack variable  Generators can also be used for specialized tree traversal  and all manner of other things

User · Answer

yield is just like return - it returns whatever you tell it to  as a generator   The difference is that the next time you call the generator  execution starts from the last call to the yield statement  Unlike return  the stack frame is not cleaned up when a yield occurs  however control is transferred back to the caller  so its state will resume the next time the function is called   In the case of your code  the function get child candidates is acting like an iterator so that when you extend your list  it adds one element at a time to the new list   list extend calls an iterator until it s exhausted  In the case of the code sample you posted  it would be much clearer to just return a tuple and append that to the list

User · Answer

Yield is an object  A return in a function will return a single value   If you want a function to return a huge set of values  use yield   More importantly  yield is a barrier      like barrier in the CUDA language  it will not transfer control until it gets   completed    That is  it will run the code in your function from the beginning until it hits yield  Then  it   ll return the first value of the loop   Then  every other call will run the loop you have written in the function one more time  returning the next value until there isn t any value to return

User · Answer

Yield is an object  A return in a function will return a single value   If you want a function to return a huge set of values  use yield   More importantly  yield is a barrier      like barrier in the CUDA language  it will not transfer control until it gets   completed    That is  it will run the code in your function from the beginning until it hits yield  Then  it   ll return the first value of the loop   Then  every other call will run the loop you have written in the function one more time  returning the next value until there isn t any value to return

User · Answer

Shortcut to understanding yield  When you see a function with yield statements  apply this easy trick to understand what will happen    Insert a line result      at the start of the function  Replace each yield expr with result append expr   Insert a line return result at the bottom of the function  Yay - no more yield statements  Read and figure out code  Compare function to the original definition    This trick may give you an idea of the logic behind the function  but what actually happens with yield is significantly different than what happens in the list based approach  In many cases  the yield approach will be a lot more memory efficient and faster too  In other cases  this trick will get you stuck in an infinite loop  even though the original function works just fine  Read on to learn more     Don t confuse your Iterables  Iterators  and Generators  First  the iterator protocol - when you write  for x in mylist         loop body      Python performs the following two steps    Gets an iterator for mylist   Call iter mylist  -  this returns an object with a next   method  or   next     in Python 3     This is the step most people forget to tell you about  Uses the iterator to loop over items   Keep calling the next   method on the iterator returned from step 1  The return value from next   is assigned to x and the loop body is executed  If an exception StopIteration is raised from within next    it means there are no more values in the iterator and the loop is exited    The truth is Python performs the above two steps anytime it wants to loop over the contents of an object - so it could be a for loop  but it could also be code like otherlist extend mylist   where otherlist is a Python list    Here mylist is an iterable because it implements the iterator protocol  In a user-defined class  you can implement the   iter     method to make instances of your class iterable  This method should return an iterator  An iterator is an object with a next   method  It is possible to implement both   iter     and next   on the same class  and have   iter     return self  This will work for simple cases  but not when you want two iterators looping over the same object at the same time   So that s the iterator protocol  many objects implement this protocol    Built-in lists  dictionaries  tuples  sets  files  User-defined classes that implement   iter      Generators    Note that a for loop doesn t know what kind of object it s dealing with - it just follows the iterator protocol  and is happy to get item after item as it calls next    Built-in lists return their items one by one  dictionaries return the keys one by one  files return the lines one by one  etc  And generators return    well that s where yield comes in   def f123        yield 1     yield 2     yield 3  for item in f123        print item   Instead of yield statements  if you had three return statements in f123   only the first would get executed  and the function would exit  But f123   is no ordinary function  When f123   is called  it does not return any of the values in the yield statements  It returns a generator object  Also  the function does not really exit - it goes into a suspended state  When the for loop tries to loop over the generator object  the function resumes from its suspended state at the very next line after the yield it previously returned from  executes the next line of code  in this case  a yield statement  and returns that as the next item  This happens until the function exits  at which point the generator raises StopIteration  and the loop exits    So the generator object is sort of like an adapter - at one end it exhibits the iterator protocol  by exposing   iter     and next   methods to keep the for loop happy  At the other end  however  it runs the function just enough to get the next value out of it  and puts it back in suspended mode   Why Use Generators   Usually  you can write code that doesn t use generators but implements the same logic  One option is to use the temporary list  trick  I mentioned before  That will not work in all cases  for e g  if you have infinite loops  or it may make inefficient use of memory when you have a really long list  The other approach is to implement a new iterable class SomethingIter that keeps the state in instance members and performs the next logical step in it s next    or   next     in Python 3  method  Depending on the logic  the code inside the next   method may end up looking very complex and be prone to bugs  Here generators provide a clean and easy solution

User · Answer

My below answer only speaks from the perspective of using Python generator  not the underlying implementation of generator mechanism  which involves some tricks of stack and heap manipulation    When yield is used instead of a return in a python function  that function is turned into something special called generator function  That function will return an object of generator type  The yield keyword is a flag to notify the python compiler to treat such function specially  Normal functions will terminate once some value is returned from it  But with the help of the compiler  the generator function can be thought of as resumable  That is  the execution context will be restored and the execution will continue from last run  Until you explicitly call return  which will raise a StopIteration exception  which is also part of the iterator protocol   or reach the end of the function  I found a lot of references about generator but this one from the functional programming perspective is the most digestable    Now I want to talk about the rationale behind generator  and the iterator based on my own understanding  I hope this can help you grasp the essential motivation of iterator and generator  Such concept shows up in other languages as well such as C     As I understand  when we want to process a bunch of data  we usually first store the data somewhere and then process it one by one  But this naive approach is problematic  If the data volume is huge  it s expensive to store them as a whole beforehand  So instead of storing the data itself directly  why not store some kind of metadata indirectly  i e  the logic how the data is computed    There are 2 approaches to wrap such metadata    The OO approach  we wrap the metadata as a class  This is the so-called iterator who implements the iterator protocol  i e  the   next      and   iter     methods   This is also the commonly seen iterator design pattern  The functional approach  we wrap the metadata as a function  This is the so-called generator function  But under the hood  the returned generator object still IS-A iterator because it also implements the iterator protocol    Either way  an iterator is created  i e  some object that can give you the data you want  The OO approach may be a bit complex  Anyway  which one to use is up to you

User · Answer

The function containing the yield keyword is a generator  When you call it  its formal parameters are bound to actual arguments  but its body isn t actually evaluated  Instead  a generator-iterator is returned  Each call to the generator-iterator s next   method performs another pass through the iterative algorithm  Each step s value is the value specified by the yield keyword  Think of yield as the generator-iterator version of return  indicating the boundary between each iteration of the algorithm  Each time you call next    the generator code resumes from the statement following the yield

User · Answer

Yield   gt  gt  gt  def create generator           my list   range 3         for i in my list             yield i i      gt  gt  gt  my generator   create generator     create a generator  gt  gt  gt  print my generator    my generator is an object   lt generator object create generator at 0xb7555c34 gt   gt  gt  gt  for i in my generator          print i  0 1 4   In short  you can see that the loop does not stop and continues to function even after the object or variable is sent  unlike return where the loop stops after execution

User · Answer

There s one extra thing to mention  a function that yields doesn t actually have to terminate  I ve written code like this   def fib        last  cur   0  1     while True           yield cur         last  cur   cur  last   cur   Then I can use it in other code like this   for f in fib        if some condition  break     coolfuncs f     It really helps simplify some problems  and makes some things easier to work with

User · Answer

yield is like a return element for a function  The difference is  that the yield element turns a function into a generator  A generator behaves just like a function until something is  yielded   The generator stops until it is next called  and continues from exactly the same point as it started  You can get a sequence of all the  yielded  values in one  by calling list generator

User · Answer

There is one type of answer that I don t feel has been given yet  among the many great answers that describe how to use generators  Here is the programming language theory answer   The yield statement in Python returns a generator  A generator in Python is a function that returns continuations  and specifically a type of coroutine  but continuations represent the more general mechanism to understand what is going on    Continuations in programming languages theory are a much more fundamental kind of computation  but they are not often used  because they are extremely hard to reason about and also very difficult to implement  But the idea of what a continuation is  is straightforward  it is the state of a computation that has not yet finished  In this state  the current values of variables  the operations that have yet to be performed  and so on  are saved  Then at some point later in the program the continuation can be invoked  such that the program s variables are reset to that state and the operations that were saved are carried out   Continuations  in this more general form  can be implemented in two ways  In the call cc way  the program s stack is literally saved and then when the continuation is invoked  the stack is restored   In continuation passing style  CPS   continuations are just normal functions  only in languages where functions are first class  which the programmer explicitly manages and passes around to subroutines  In this style  program state is represented by closures  and the variables that happen to be encoded in them  rather than variables that reside somewhere on the stack  Functions that manage control flow accept continuation as arguments  in some variations of CPS  functions may accept multiple continuations  and manipulate control flow by invoking them by simply calling them and returning afterwards  A very simple example of continuation passing style is as follows   def save file filename     def write file continuation        write stuff to file filename     check if file exists and user wants to overwrite write file continuation    In this  very simplistic  example  the programmer saves the operation of actually writing the file into a continuation  which can potentially be a very complex operation with many details to write out   and then passes that continuation  i e  as a first-class closure  to another operator which does some more processing  and then calls it if necessary   I use this design pattern a lot in actual GUI programming  either because it saves me lines of code or  more importantly  to manage control flow after GUI events trigger    The rest of this post will  without loss of generality  conceptualize continuations as CPS  because it is a hell of a lot easier to understand and read     Now let s talk about generators in Python  Generators are a specific subtype of continuation  Whereas continuations are able in general to save the state of a computation  i e   the program s call stack   generators are only able to save the state of iteration over an iterator  Although  this definition is slightly misleading for certain use cases of generators  For instance   def f      while True      yield 4   This is clearly a reasonable iterable whose behavior is well defined -- each time the generator iterates over it  it returns 4  and does so forever   But it isn t probably the prototypical type of iterable that comes to mind when thinking of iterators  i e   for x in collection  do something x    This example illustrates the power of generators  if anything is an iterator  a generator can save the state of its iteration   To reiterate  Continuations can save the state of a program s stack and generators can save the state of iteration  This means that continuations are more a lot powerful than generators  but also that generators are a lot  lot easier  They are easier for the language designer to implement  and they are easier for the programmer to use  if you have some time to burn  try to read and understand this page about continuations and call cc    But you could easily implement  and conceptualize  generators as a simple  specific case of continuation passing style   Whenever yield is called  it tells the function to return a continuation   When the function is called again  it starts from wherever it left off  So  in pseudo-pseudocode  i e   not pseudocode  but not code  the generator s next method is basically as follows   class Generator      def   init   self iterable generatorfun       self next continuation   lambda generatorfun iterable     def next self       value  next continuation   self next continuation       self next continuation   next continuation     return value   where the yield keyword is actually syntactic sugar for the real generator function  basically something like   def generatorfun iterable     if len iterable     0      raise StopIteration   else      return  iterable 0   lambda generatorfun iterable 1       Remember that this is just pseudocode and the actual implementation of generators in Python is more complex  But as an exercise to understand what is going on  try to use continuation passing style to implement generator objects without use of the yield keyword

User · Answer

Here are some Python examples of how to actually implement generators as if Python did not provide syntactic sugar for them   As a Python generator   from itertools import islice  def fib gen        a  b   1  1     while True          yield a         a  b   b  a   b  assert  1  1  2  3  5     list islice fib gen    5     Using lexical closures instead of generators  def ftake fnext  last       return  fnext   for   in xrange last    def fib gen2         funky scope due to python2 x workaround      for python 3 x use nonlocal     def                a    b     b    a     b         return   a       a    b   0  1     return    assert  1 1 2 3 5     ftake fib gen2    5    Using object closures instead of generators  because ClosuresAndObjectsAreEquivalent   class fib gen3      def   init   self           self a  self b   1  1      def   call   self           r   self a         self a  self b   self b  self a   self b         return r  assert  1 1 2 3 5     ftake fib gen3    5

User · Answer

Here are some Python examples of how to actually implement generators as if Python did not provide syntactic sugar for them   As a Python generator   from itertools import islice  def fib gen        a  b   1  1     while True          yield a         a  b   b  a   b  assert  1  1  2  3  5     list islice fib gen    5     Using lexical closures instead of generators  def ftake fnext  last       return  fnext   for   in xrange last    def fib gen2         funky scope due to python2 x workaround      for python 3 x use nonlocal     def                a    b     b    a     b         return   a       a    b   0  1     return    assert  1 1 2 3 5     ftake fib gen2    5    Using object closures instead of generators  because ClosuresAndObjectsAreEquivalent   class fib gen3      def   init   self           self a  self b   1  1      def   call   self           r   self a         self a  self b   self b  self a   self b         return r  assert  1 1 2 3 5     ftake fib gen3    5

User · Answer

My below answer only speaks from the perspective of using Python generator  not the underlying implementation of generator mechanism  which involves some tricks of stack and heap manipulation    When yield is used instead of a return in a python function  that function is turned into something special called generator function  That function will return an object of generator type  The yield keyword is a flag to notify the python compiler to treat such function specially  Normal functions will terminate once some value is returned from it  But with the help of the compiler  the generator function can be thought of as resumable  That is  the execution context will be restored and the execution will continue from last run  Until you explicitly call return  which will raise a StopIteration exception  which is also part of the iterator protocol   or reach the end of the function  I found a lot of references about generator but this one from the functional programming perspective is the most digestable    Now I want to talk about the rationale behind generator  and the iterator based on my own understanding  I hope this can help you grasp the essential motivation of iterator and generator  Such concept shows up in other languages as well such as C     As I understand  when we want to process a bunch of data  we usually first store the data somewhere and then process it one by one  But this naive approach is problematic  If the data volume is huge  it s expensive to store them as a whole beforehand  So instead of storing the data itself directly  why not store some kind of metadata indirectly  i e  the logic how the data is computed    There are 2 approaches to wrap such metadata    The OO approach  we wrap the metadata as a class  This is the so-called iterator who implements the iterator protocol  i e  the   next      and   iter     methods   This is also the commonly seen iterator design pattern  The functional approach  we wrap the metadata as a function  This is the so-called generator function  But under the hood  the returned generator object still IS-A iterator because it also implements the iterator protocol    Either way  an iterator is created  i e  some object that can give you the data you want  The OO approach may be a bit complex  Anyway  which one to use is up to you

User · Answer

Yield gives you a generator    def get odd numbers i       return range 1  i  2  def yield odd numbers i       for x in range 1  i  2          yield x foo   get odd numbers 10  bar   yield odd numbers 10  foo  1  3  5  7  9  bar  lt generator object yield odd numbers at 0x1029c6f50 gt  bar next   1 bar next   3 bar next   5   As you can see  in the first case foo holds the entire list in memory at once  It s not a big deal for a list with 5 elements  but what if you want a list of 5 million  Not only is this a huge memory eater  it also costs a lot of time to build at the time that the function is called   In the second case  bar just gives you a generator  A generator is an iterable--which means you can use it in a for loop  etc  but each value can only be accessed once  All the values are also not stored in memory at the same time  the generator object  remembers  where it was in the looping the last time you called it--this way  if you re using an iterable to  say  count to 50 billion  you don t have to count to 50 billion all at once and store the 50 billion numbers to count through   Again  this is a pretty contrived example  you probably would use itertools if you really wanted to count to 50 billion      This is the most simple use case of generators  As you said  it can be used to write efficient permutations  using yield to push things up through the call stack instead of using some sort of stack variable  Generators can also be used for specialized tree traversal  and all manner of other things

User · Answer

The yield keyword is reduced to two simple facts    If the compiler detects the yield keyword anywhere inside a function  that function no longer returns via the return statement  Instead  it immediately returns a lazy  pending list  object called a generator A generator is iterable  What is an iterable  It s anything like a list or set or range or dict-view  with a built-in protocol for visiting each element in a certain order    In a nutshell  a generator is a lazy  incrementally-pending list  and yield statements allow you to use function notation to program the list values the generator should incrementally spit out   generator   myYieldingFunction      x   list generator      generator        v  x 0                       generator              v  x 0   x 1                             generator                    v  x 0   x 1   x 2                                     StopIteration exception  x 0   x 1   x 2       done  list   x 0   x 1   x 2       Example  Let s define a function makeRange that s just like Python s range  Calling makeRange n  RETURNS A GENERATOR   def makeRange n         return 0 1 2     n-1     i   0     while i  lt  n          yield i         i    1   gt  gt  gt  makeRange 5   lt generator object makeRange at 0x19e4aa0 gt    To force the generator to immediately return its pending values  you can pass it into list    just like you could any iterable     gt  gt  gt  list makeRange 5    0  1  2  3  4      Comparing example to  just returning a list   The above example can be thought of as merely creating a list which you append to and return     list-version                        generator-version def makeRange n                     def makeRange n          return  0 1 2     n-1               return 0 1 2     n-1        TO RETURN                      gt      i   0                               i   0     while i  lt  n                         while i  lt  n          TO RETURN     i                     yield i         i    1                              i    1     indented     return TO RETURN               gt    gt  gt  gt  makeRange 5   0  1  2  3  4    There is one major difference  though  see the last section     How you might use generators  An iterable is the last part of a list comprehension  and all generators are iterable  so they re often used like so                        ITERABLE   gt  gt  gt   x 10 for x in makeRange 5    10  11  12  13  14    To get a better feel for generators  you can play around with the itertools module  be sure to use chain from iterable rather than chain when warranted   For example  you might even use generators to implement infinitely-long lazy lists like itertools count    You could implement your own def enumerate iterable   zip count    iterable   or alternatively do so with the yield keyword in a while-loop   Please note  generators can actually be used for many more things  such as implementing coroutines or non-deterministic programming or other elegant things  However  the  lazy lists  viewpoint I present here is the most common use you will find     Behind the scenes  This is how the  Python iteration protocol  works  That is  what is going on when you do list makeRange 5    This is what I describe earlier as a  lazy  incremental list     gt  gt  gt  x iter range 5    gt  gt  gt  next x  0  gt  gt  gt  next x  1  gt  gt  gt  next x  2  gt  gt  gt  next x  3  gt  gt  gt  next x  4  gt  gt  gt  next x  Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  StopIteration   The built-in function next   just calls the objects  next   function  which is a part of the  iteration protocol  and is found on all iterators  You can manually use the next   function  and other parts of the iteration protocol  to implement fancy things  usually at the expense of readability  so try to avoid doing that       Minutiae  Normally  most people would not care about the following distinctions and probably want to stop reading here   In Python-speak  an iterable is any object which  understands the concept of a for-loop  like a list  1 2 3   and an iterator is a specific instance of the requested for-loop like  1 2 3    iter      A generator is exactly the same as any iterator  except for the way it was written  with function syntax    When you request an iterator from a list  it creates a new iterator  However  when you request an iterator from an iterator  which you would rarely do   it just gives you a copy of itself   Thus  in the unlikely event that you are failing to do something like this      gt  x   myRange 5   gt  list x   0  1  2  3  4   gt  list x           then remember that a generator is an iterator  that is  it is one-time-use  If you want to reuse it  you should call myRange      again  If you need to use the result twice  convert the result to a list and store it in a variable x   list myRange 5    Those who absolutely need to clone a generator  for example  who are doing terrifyingly hackish metaprogramming  can use itertools tee if absolutely necessary  since the copyable iterator Python PEP standards proposal has been deferred

User · Answer

Think of it this way   An iterator is just a fancy sounding term for an object that has a next   method   So a yield-ed function ends up being something like this   Original version   def some function        for i in xrange 4           yield i  for i in some function        print i   This is basically what the Python interpreter does with the above code   class it      def   init   self             Start at -1 so that we get 0 when we add 1 below          self count   -1        The   iter   method will be called once by the  for  loop        The rest of the magic happens on the object returned by this method        In this case it is the object itself      def   iter   self           return self        The next method will be called repeatedly by the  for  loop       until it raises StopIteration      def next self           self count    1         if self count  lt  4              return self count         else                A StopIteration exception is raised               to signal that the iterator is done                This is caught implicitly by the  for  loop              raise StopIteration  def some func        return it    for i in some func        print i   For more insight as to what s happening behind the scenes  the for loop can be rewritten to this   iterator   some func   try      while 1          print iterator next   except StopIteration      pass   Does that make more sense or just confuse you more       I should note that this is an oversimplification for illustrative purposes

User · Answer

First yield program   def countdown gen x         count   x       while count  gt  0             yield count            count -  1  g   countdown gen 5   for item in g       print item    Output  5 4 3 2 1   Understand flow     Note   Return a generator obj Suspend a functon execution and save its status  function can be executed again

User · Answer

Shortcut to understanding yield  When you see a function with yield statements  apply this easy trick to understand what will happen    Insert a line result      at the start of the function  Replace each yield expr with result append expr   Insert a line return result at the bottom of the function  Yay - no more yield statements  Read and figure out code  Compare function to the original definition    This trick may give you an idea of the logic behind the function  but what actually happens with yield is significantly different than what happens in the list based approach  In many cases  the yield approach will be a lot more memory efficient and faster too  In other cases  this trick will get you stuck in an infinite loop  even though the original function works just fine  Read on to learn more     Don t confuse your Iterables  Iterators  and Generators  First  the iterator protocol - when you write  for x in mylist         loop body      Python performs the following two steps    Gets an iterator for mylist   Call iter mylist  -  this returns an object with a next   method  or   next     in Python 3     This is the step most people forget to tell you about  Uses the iterator to loop over items   Keep calling the next   method on the iterator returned from step 1  The return value from next   is assigned to x and the loop body is executed  If an exception StopIteration is raised from within next    it means there are no more values in the iterator and the loop is exited    The truth is Python performs the above two steps anytime it wants to loop over the contents of an object - so it could be a for loop  but it could also be code like otherlist extend mylist   where otherlist is a Python list    Here mylist is an iterable because it implements the iterator protocol  In a user-defined class  you can implement the   iter     method to make instances of your class iterable  This method should return an iterator  An iterator is an object with a next   method  It is possible to implement both   iter     and next   on the same class  and have   iter     return self  This will work for simple cases  but not when you want two iterators looping over the same object at the same time   So that s the iterator protocol  many objects implement this protocol    Built-in lists  dictionaries  tuples  sets  files  User-defined classes that implement   iter      Generators    Note that a for loop doesn t know what kind of object it s dealing with - it just follows the iterator protocol  and is happy to get item after item as it calls next    Built-in lists return their items one by one  dictionaries return the keys one by one  files return the lines one by one  etc  And generators return    well that s where yield comes in   def f123        yield 1     yield 2     yield 3  for item in f123        print item   Instead of yield statements  if you had three return statements in f123   only the first would get executed  and the function would exit  But f123   is no ordinary function  When f123   is called  it does not return any of the values in the yield statements  It returns a generator object  Also  the function does not really exit - it goes into a suspended state  When the for loop tries to loop over the generator object  the function resumes from its suspended state at the very next line after the yield it previously returned from  executes the next line of code  in this case  a yield statement  and returns that as the next item  This happens until the function exits  at which point the generator raises StopIteration  and the loop exits    So the generator object is sort of like an adapter - at one end it exhibits the iterator protocol  by exposing   iter     and next   methods to keep the for loop happy  At the other end  however  it runs the function just enough to get the next value out of it  and puts it back in suspended mode   Why Use Generators   Usually  you can write code that doesn t use generators but implements the same logic  One option is to use the temporary list  trick  I mentioned before  That will not work in all cases  for e g  if you have infinite loops  or it may make inefficient use of memory when you have a really long list  The other approach is to implement a new iterable class SomethingIter that keeps the state in instance members and performs the next logical step in it s next    or   next     in Python 3  method  Depending on the logic  the code inside the next   method may end up looking very complex and be prone to bugs  Here generators provide a clean and easy solution

User · Answer

Many people use return rather than yield  but in some cases yield can be more efficient and easier to work with   Here is an example which yield is definitely best for      return  in function    import random  def return dates        dates        With  return  you need to create a list then return it     for i in range 5           date   random choice   1st    2nd    3rd    4th    5th    6th    7th    8th    9th    10th            dates append date      return dates      yield  in function    def yield dates        for i in range 5           date   random choice   1st    2nd    3rd    4th    5th    6th    7th    8th    9th    10th            yield date    yield  makes a generator automatically which works                      in a similar way  This is much more efficient       Calling functions   dates list   return dates   print dates list  for i in dates list      print i   dates generator   yield dates   print dates generator  for i in dates generator      print i    Both functions do the same thing  but yield uses three lines instead of five and has one less variable to worry about           This is the result from the code         As you can see both functions do the same thing  The only difference is return dates   gives a list and yield dates   gives a generator   A real life example would be something like reading a file line by line or if you just want to make a generator

[python] What does the "yield" keyword do?

Examples related to python

Examples related to iterator

Examples related to generator

Examples related to yield

Examples related to coroutine