How to get the return value from a thread in python

Question

The function foo below returns a string  foo   How can I get the value  foo  which is returned from the thread s target   from threading import Thread  def foo bar       print  hello     format bar       return  foo   thread   Thread target foo  args   world      thread start   return value   thread join     The  one obvious way to do it   shown above  doesn t work  thread join   returned None

User · Answer

Define your target to 1  take an argument q 2  replace any statements return foo with q put foo   return  so a function  def func a       ans   a   a     return ans   would become  def func a  q       ans   a   a     q put ans      return   and then you would proceed as such  from Queue import Queue from threading import Thread  ans q   Queue   arg tups     i  ans q  for i in xrange 10    threads    Thread target func  args arg tup  for arg tup in arg tups       t start   for t in threads       t join   for t in threads  results    q get   for   in xrange len threads      And you can use function decorators wrappers to make it so you can use your existing functions as target without modifying them  but follow this basic scheme

User · Answer

You can define a mutable above the scope of the threaded function  and add the result to that   I also modified the code to be python3 compatible   returns      def foo bar       print  hello  0   format bar       returns bar     foo   from threading import Thread t   Thread target foo  args   world      t start   t join   print returns    This returns   world     foo    If you use the function input as the key to your results dict  every unique input is guaranteed to give an entry in the results

User · Answer

Jake s answer is good  but if you don t want to use a threadpool  you don t know how many threads you ll need  but create them as needed  then a good way to transmit information between threads is the built-in Queue Queue class  as it offers thread safety   I created the following decorator to make it act in a similar fashion to the threadpool   def threaded f  daemon False       import Queue      def wrapped f q   args    kwargs              this function calls the decorated function and puts the          result in a queue            ret   f  args    kwargs          q put ret       def wrap  args    kwargs              this is the function returned from the decorator  It fires off         wrapped f in a new thread and returns the thread object with         the result queue attached             q   Queue Queue            t   threading Thread target wrapped f  args  q   args  kwargs kwargs          t daemon   daemon         t start           t result queue   q                 return t      return wrap   Then you just use it as    threaded def long task x       import time     x   x   5     time sleep 5      return x    does not block  returns Thread object y   long task 10  print y    this blocks  waiting for the result result   y result queue get   print result   The decorated function creates a new thread each time it s called and returns a Thread object that contains the queue that will receive the result   UPDATE  It s been quite a while since I posted this answer  but it still gets views so I thought I would update it to reflect the way I do this in newer versions of Python   Python 3 2 added in the concurrent futures module which provides a high-level interface for parallel tasks   It provides ThreadPoolExecutor and ProcessPoolExecutor  so you can use a thread or process pool with the same api   One benefit of this api is that submitting a task to an Executor returns a Future object  which will complete with the return value of the callable you submit   This makes attaching a queue object unnecessary  which simplifies the decorator quite a bit    DEFAULT POOL   ThreadPoolExecutor    def threadpool f  executor None        wraps f      def wrap  args    kwargs           return  executor or  DEFAULT POOL  submit f   args    kwargs       return wrap   This will use a default module threadpool executor if one is not passed in   The usage is very similar to before    threadpool def long task x       import time     x   x   5     time sleep 5      return x    does not block  returns Future object y   long task 10  print y    this blocks  waiting for the result result   y result   print result   If you re using Python 3 4   one really nice feature of using this method  and Future objects in general  is that the returned future can be wrapped to turn it into an asyncio Future with asyncio wrap future   This makes it work easily with coroutines   result   await asyncio wrap future long task 10     If you don t need access to the underlying concurrent Future object  you can include the wrap in the decorator    DEFAULT POOL   ThreadPoolExecutor    def threadpool f  executor None        wraps f      def wrap  args    kwargs           return asyncio wrap future  executor or  DEFAULT POOL  submit f   args    kwargs        return wrap   Then  whenever you need to push cpu intensive or blocking code off the event loop thread  you can put it in a decorated function    threadpool def some long calculation               this will suspend while the function is executed on a threadpool result   await some long calculation

User · Answer

One usual solution is to wrap your function foo with a decorator like  result   queue Queue    def task wrapper  args       result put target  args     Then the whole code may looks like that  result   queue Queue    def task wrapper  args       result put target  args    threads    threading Thread target task wrapper  args args  for args in args list   for t in threads      t start       while True           if len threading enumerate     lt  max num               break for t in threads      t join   return result   Note  One important issue is that the return values may be unorderred   In fact  the return value is not necessarily saved to the queue  since you can choose arbitrary thread-safe data structure

User · Answer

Parris   kindall s answer join return answer ported to Python 3   from threading import Thread  def foo bar       print  hello  0   format bar       return  foo   class ThreadWithReturnValue Thread       def   init   self  group None  target None  name None  args     kwargs None     daemon None           Thread   init   self  group  target  name  args  kwargs  daemon daemon           self  return   None      def run self           if self  target is not None              self  return   self  target  self  args    self  kwargs       def join self           Thread join self          return self  return   twrv   ThreadWithReturnValue target foo  args   world       twrv start   print twrv join        prints foo   Note  the Thread class is implemented differently in Python 3

User · Answer

GuySoft s idea is great  but I think the object does not necessarily have to inherit from Thread and start   could be removed from interface   from threading import Thread import queue class ThreadWithReturnValue object       def   init   self  target None  args       kwargs           self  que   queue Queue           self  t   Thread target lambda q arg1 kwargs1  q put target  arg1    kwargs1                     args  self  que  args  kwargs             self  t start        def join self           self  t join           return self  que get     def foo bar       print  hello  0   format bar       return  foo   twrv   ThreadWithReturnValue target foo  args   world       print twrv join        prints foo

User · Answer

As mentioned multiprocessing pool is much slower than basic threading  Using queues as proposeded in some answers here is a very effective alternative  I have use it with dictionaries in order to be able run a lot of small threads and recuperate multiple answers by combining them with dictionaries      usr bin env python3  import threading   use Queue for python2 import queue import random  LETTERS    abcdefghijklmnopqrstuvwxyz  LETTERS     x for x in LETTERS    NUMBERS    0  1  2  3  4  5  6  7  8  9   def randoms k  q       result   dict       result  letter     random choice LETTERS      result  number     random choice NUMBERS      q put  k  result    threads   list   q   queue Queue   results   dict    for name in   alpha    oscar    yankee         threads append  threading Thread target randoms  args  name  q         threads -1  start         t join   for t in threads   while not q empty        results update q get     print results

User · Answer

One way I ve seen is to pass a mutable object  such as a list or a dictionary  to the thread s constructor  along with a an index or other identifier of some sort  The thread can then store its results in its dedicated slot in that object   For example   def foo bar  result  index       print  hello  0   format bar      result index     foo   from threading import Thread  threads    None    10 results    None    10  for i in range len threads        threads i    Thread target foo  args   world    results  i       threads i  start      do some other stuff  for i in range len threads        threads i  join    print     join results     what sound does a metasyntactic locomotive make    If you really want join   to return the return value of the called function  you can do this with a Thread subclass like the following   from threading import Thread  def foo bar       print  hello  0   format bar      return  foo   class ThreadWithReturnValue Thread       def   init   self  group None  target None  name None                   args     kwargs     Verbose None           Thread   init   self  group  target  name  args  kwargs  Verbose          self  return   None     def run self           if self  Thread  target is not None              self  return   self  Thread  target  self  Thread  args                                                    self  Thread  kwargs      def join self           Thread join self          return self  return  twrv   ThreadWithReturnValue target foo  args   world       twrv start   print twrv join       prints foo   That gets a little hairy because of some name mangling  and it accesses  private  data structures that are specific to Thread implementation    but it works   For python3  class ThreadWithReturnValue Thread       def   init   self  group None  target None  name None                   args     kwargs     Verbose None           Thread   init   self  group  target  name  args  kwargs          self  return   None     def run self           print type self  target           if self  target is not None              self  return   self  target  self  args                                                    self  kwargs      def join self   args           Thread join self   args          return self  return

User · Answer

Kindall s answer in Python3  class ThreadWithReturnValue Thread       def   init   self  group None  target None  name None                   args     kwargs        daemon None           Thread   init   self  group  target  name  args  kwargs  daemon          self  return   None       def run self           try              if self  target                  self  return   self  target  self  args    self  kwargs          finally              del self  target  self  args  self  kwargs       def join self timeout None           Thread join self timeout          return self  return

User · Answer

I know this thread is old     but I faced the same problem    If you are willing to use thread join   import threading  class test       def   init   self           self msg  quot  quot       def hello self bar           print  hello     format bar           self msg  quot foo quot        def main self           thread   threading Thread target self hello  args   world              thread start           thread join           print self msg   g test   g main

User · Answer

join always return None  i think you should subclass Thread to handle return codes and so

User · Answer

Another solution that doesn t require changing your existing code  import Queue               Python 2 x  from queue import Queue   Python 3 x  from threading import Thread  def foo bar       print  hello  0   format bar      return  foo   que   Queue Queue          Python 2 x  que   Queue               Python 3 x  t   Thread target lambda q  arg1  q put foo arg1    args  que   world     t start   t join   result   que get   print result  It can be also easily adjusted to a multi-threaded environment  import Queue               Python 2 x  from queue import Queue   Python 3 x from threading import Thread  def foo bar       print  hello  0   format bar      return  foo   que   Queue Queue          Python 2 x  que   Queue               Python 3 x  threads list   list    t   Thread target lambda q  arg1  q put foo arg1    args  que   world     t start   threads list append t     Add more threads here     threads list append t2      threads list append t3         Join all the threads for t in threads list      t join      Check thread s return value while not que empty        result   que get       print result

User · Answer

Based of what kindall mentioned  here s the more generic solution that works with Python3  import threading  class ThreadWithReturnValue threading Thread       def   init   self   init args    init kwargs           threading Thread   init   self   init args    init kwargs          self  return   None     def run self           self  return   self  target  self  args    self  kwargs      def join self           threading Thread join self          return self  return  Usage         th   ThreadWithReturnValue target requests get  args   http   www google com             th start           response   th join           response status code      gt  200

User · Answer

I stole kindall s answer and cleaned it up just a little bit   The key part is adding  args and   kwargs to join   in order to handle the timeout  class threadWithReturn Thread       def   init   self   args    kwargs           super threadWithReturn  self    init    args    kwargs           self  return   None      def run self           if self  Thread  target is not None              self  return   self  Thread  target  self  Thread  args    self  Thread  kwargs       def join self   args    kwargs           super threadWithReturn  self  join  args    kwargs           return self  return   UPDATED ANSWER BELOW  This is my most popularly upvoted answer  so I decided to update with code that will run on both py2 and py3   Additionally  I see many answers to this question that show a lack of comprehension regarding Thread join    Some completely fail to handle the timeout arg  But there is also a corner-case that you should be aware of regarding instances when you have  1  a target function that can return None and  2  you also pass the timeout arg to join     Please see  TEST 4  to understand this corner case   ThreadWithReturn class that works with py2 and py3   import sys from threading import Thread from builtins import super      https   stackoverflow com a 30159479  if sys version info  gt    3  0        thread target key     target       thread args key     args       thread kwargs key     kwargs  else       thread target key     Thread  target       thread args key     Thread  args       thread kwargs key     Thread  kwargs   class ThreadWithReturn Thread       def   init   self   args    kwargs           super     init    args    kwargs          self  return   None      def run self           target   getattr self   thread target key          if not target is None              self  return   target                   getattr self   thread args key                     getattr self   thread kwargs key                     def join self   args    kwargs           super   join  args    kwargs          return self  return   Some sample tests are shown below   import time  random    TEST TARGET FUNCTION def giveMe arg  seconds None       if not seconds is None          time sleep seconds      return arg    TEST 1 my thread   ThreadWithReturn target giveMe  args   stringy     my thread start   returned   my thread join      returned     stringy      TEST 2 my thread   ThreadWithReturn target giveMe  args  None    my thread start   returned   my thread join      returned is None     TEST 3 my thread   ThreadWithReturn target giveMe  args   stringy     kwargs   seconds   5   my thread start   returned   my thread join timeout 2     returned is None    because join   timed out before giveMe   finished    TEST 4 my thread   ThreadWithReturn target giveMe  args  None    kwargs   seconds   5   my thread start   returned   my thread join timeout random randint 1  10     Can you identify the corner-case that we may possibly encounter with TEST 4   The problem is that we expect giveMe   to return None  see TEST 2   but we also expect join   to return None if it times out   returned is None means either    1  that s what giveMe   returned  or   2  join   timed out  This example is trivial since we know that giveMe   will always return None  But in real-world instance  where the target may legitimately return None or something else  we d want to explicitly check for what happened   Below is how to address this corner-case     TEST 4 my thread   ThreadWithReturn target giveMe  args  None    kwargs   seconds   5   my thread start   returned   my thread join timeout random randint 1  10    if my thread isAlive          returned is None because join   timed out       this also means that giveMe   is still running in the background     pass       handle this based on your app s logic else        join   is finished  and so is giveMe         BUT we could also be in a race condition  so we need to update returned  just in case     returned   my thread join

User · Answer

In Python 3 2   stdlib concurrent futures module provides a higher level API to threading  including passing return values or exceptions from a worker thread back to the main thread   import concurrent futures  def foo bar       print  hello     format bar       return  foo   with concurrent futures ThreadPoolExecutor   as executor      future   executor submit foo   world        return value   future result       print return value

User · Answer

Here is the version that I created of  Kindall s answer https   stackoverflow com a 6894023 12900787 This version makes it so that all you have to do is input your command with arguments to create the new thread   I have also included a couple of tests  this was made with python 3 8  from threading import Thread from typing import Any    def threader com      my original Version  Ignore this       try           threader   Thread target   com           threader start        except Exception as e           print e           print  Could not start thread     def test plug  plug2  plug3       print f quot hello  plug  quot       print f I am the second plug    plug2        print plug3      return  I am the return Value     def test2 msg       return f I am from the second test   msg     def test3        print  hello world     def NewThread com  Returning  bool   arguments  - gt  Any       quot  quot  quot      Will create a new thread for a function command        param com  Command to be Executed      param arguments  Arguments to be sent to Command      param Returning  True False Will this command need to return anything      quot  quot  quot           class NewThreadWorker Thread           def   init   self  group   None  target   None  name   None  args       kwargs   None                          daemon   None               Thread   init   self  group  target  name  args  kwargs  daemon   daemon                           self  return   None                  def run self               if self  target is not None                  self  return   self  target  self  args    self  kwargs                   def join self               Thread join self              return self  return          ntw   NewThreadWorker target   com  args     arguments        ntw start       if Returning          return ntw join     if   name       quot   main   quot         threader test  world        print NewThread test  True   hi    test   test2  hi         NewThread test3  True    Hope that this is useful to someone

User · Answer

Taking into consideration  iman comment on  JakeBiesinger answer I have recomposed it to have various number of threads   from multiprocessing pool import ThreadPool  def foo bar  baz       print  hello  0   format bar      return  foo    baz  numOfThreads   3  results       pool   ThreadPool numOfThreads   for i in range 0  numOfThreads       results append pool apply async foo    world    foo       tuple of args for foo     do some other stuff in the main process              results    r get   for r in results  print results  pool close   pool join     Cheers   Guy

User · Answer

Using Queue    import threading  queue  def calc square num  out queue1     l        for x in num      l append x x    out queue1 put l    arr    1 2 3 4 5 6 7 8 9 10  out queue1 queue Queue   t1 threading Thread target calc square  args  arr out queue1   t1 start   t1 join   print  out queue1 get

User · Answer

Most answers I ve found are long and require being familiar with other modules or advanced python features  and will be rather confusing to someone unless they re already familiar with everything the answer talks about  Working code for a simplified approach  import threading  time  random  class ThreadWithResult threading Thread       def   init   self  group None  target None  name None  args     kwargs        daemon None           def function                self result   target  args    kwargs          super     init   group group  target function  name name  daemon daemon   def function to thread n       count   0     while count  lt  3              print f still running thread  n                count   1             time sleep 3      result   random random       print f Return value of thread  n  should be   result        return result   def main        thread1   ThreadWithResult target function to thread  args  1        thread2   ThreadWithResult target function to thread  args  2        thread1 start       thread2 start       thread1 join       thread2 join       print thread1 result      print thread2 result   main    Explanation  I wanted to simplify things significantly  so I created a ThreadWithResult class and had it inherit from threading Thread  The nested function function in   init   calls the threaded function we want to save the value of  and saves the result of that as the instance attribute self result after the thread finishes executing  Creating an instance of this is identical to creating an instance of threading Thread  Pass in the function you want to run on a new thread to the target argument and any arguments that your function might need to the args argument and any keyword arguments to the kwargs argument  e g  my thread   ThreadWithResult target my function  args  arg1  arg2  arg3    I think this is significantly easier to understand than the vast majority of answers  and this approach requires no extra imports  I included the time and random module to simulate the behavior of a thread  but they re not required to achieve the functionality asked in the original question  I know I m answering this looong after the question was asked  but I hope this can help more people in the future   EDIT  I created the save-thread-result PyPI package to allow you to access the same code above and reuse it across projects  GitHub code is here   The PyPI package fully extends the threading Thread class  so you can set any attributes you would set on threading thread on the ThreadWithResult class as well  The original answer above goes over the main idea behind this subclass  but for more information  see the more detailed explanation  from the module docstring  here  Quick usage example  pip3 install -U save-thread-result       MacOS Linux pip  install -U save-thread-result       Windows  python3       MacOS Linux python        Windows  from save thread result import ThreadWithResult    As of Release 0 0 3  you can also specify values for   group    name   and  daemon  if you want to set those   values manually  thread   ThreadWithResult      target   my function      args      my function arg1  my function arg2           kwargs    my function kwarg1 kwarg1 value  my function kwarg2 kwarg2 value          thread start   thread join   if getattr thread   result   None       print thread result  else        thread result attribute not set - something caused       the thread to terminate BEFORE the thread finished       executing the function passed in through the        target  argument     print  ERROR  Something went wrong while executing this thread  and the function you passed in did NOT complete        seeing help about the class and information about the threading Thread super class methods and attributes available  help ThreadWithResult

User · Answer

I m using this wrapper  which comfortably turns any function for running in a Thread - taking care of its return value or exception  It doesn t add Queue overhead    def threading func f          Decorator for running a function in a thread and handling its return     value or exception        def start  args    kw           def run                try                  th ret   f  args    kw              except                  th exc   sys exc info           def get timeout None               th join timeout              if th exc                  raise th exc 0   th exc 1   th exc 2    py2                   raise th exc 1   py3                             return th ret         th   threading Thread None  run          th exc   None         th get   get         th start           return th     return start   Usage Examples  def f x       return 2 5   x th   threading func f  4  print  still running     th is alive    print  result    th get timeout 1 0     threading func def th mul a  b       return a   b th   th mul  text   2 5   try      print th get    except TypeError      print  exception thrown ok      Notes on threading module  Comfortable return value  amp  exception handling of a threaded function is a frequent  Pythonic  need and should indeed already be offered by the threading module - possibly directly in the standard Thread class  ThreadPool has way too much overhead for simple tasks - 3 managing threads  lots of bureaucracy  Unfortunately Thread s layout was copied from Java originally - which you see e g  from the still useless 1st     constructor parameter group

User · Answer

My solution to the problem is to wrap the function and thread in a class  Does not require using pools queues  or c type variable passing  It is also non blocking  You check status instead  See example of how to use it at end of code   import threading  class ThreadWorker                The basic idea is given a function create an object      The object can then run the function in a thread      It provides a wrapper to start it check its status and get data out the function              def   init   self func           self thread   None         self data   None         self func   self save data func       def save data self func              modify function to save its returned data            def new func  args    kwargs               self data func  args    kwargs           return new func      def start self params           self data   None         if self thread is not None              if self thread isAlive                    return  running   could raise exception here           unless thread exists and is alive start or restart it         self thread   threading Thread target self func args params          self thread start           return  started       def status self           if self thread is None              return  not started          else              if self thread isAlive                    return  running              else                  return  finished       def get results self           if self thread is None              return  not started   could return exception         else              if self thread isAlive                    return  running              else                  return self data  def add x y       return x  y  add worker   ThreadWorker add  print add worker start  1 2    print add worker status   print add worker get results

User · Answer

FWIW  the multiprocessing module has a nice interface for this using the Pool class   And if you want to stick with threads rather than processes  you can just use the multiprocessing pool ThreadPool class as a drop-in replacement   def foo bar  baz     print  hello  0   format bar    return  foo    baz  from multiprocessing pool import ThreadPool pool   ThreadPool processes 1   async result   pool apply async foo    world    foo      tuple of args for foo    do some other stuff in the main process  return val   async result get      get the return value from your function

[python] How to get the return value from a thread in python?

Examples related to python

Examples related to multithreading