RuntimeError on windows trying python multiprocessing

Question

I am trying my very first formal python program using Threading and Multiprocessing on a windows machine  I am unable to launch the processes though  with python giving the following message  The thing is  I am not launching my threads in the main module  The threads are handled in a separate module inside a class   EDIT  By the way this code runs fine on ubuntu  Not quite on windows  RuntimeError               Attempt to start a new process before the current process             has finished its bootstrapping phase              This probably means that you are on Windows and you have             forgotten to use the proper idiom in the main module                  if   name         main                         freeze support                                       The  freeze support    line can be omitted if the program             is not going to be frozen to produce a Windows executable    My original code is pretty long  but I was able to reproduce the error in an abridged version of the code  It is split in two files  the first is the main module and does very little other than import the module which handles processes threads and calls a method  The second module is where the meat of the code is     testMain py   import parallelTestModule  extractor   parallelTestModule ParallelExtractor   extractor runInParallel numProcesses 2  numThreads 4      parallelTestModule py   import multiprocessing from multiprocessing import Process import threading  class ThreadRunner threading Thread           This class represents a single instance of a running thread        def   init   self  name           threading Thread   init   self          self name   name     def run self           print self name   n   class ProcessRunner          This class represents a single instance of a running process         def runp self  pid  numThreads           mythreads              for tid in range numThreads               name    Proc-  str pid   -Thread-  str tid              th   ThreadRunner name              mythreads append th           for i in mythreads              i start           for i in mythreads              i join    class ParallelExtractor          def runInParallel self  numProcesses  numThreads           myprocs              prunner   ProcessRunner           for pid in range numProcesses               pr   Process target prunner runp  args  pid  numThreads                myprocs append pr            if   name       parallelTestModule       This didnt work          if   name         main                   This obviously doesnt work          multiprocessing freeze support           added after seeing error to no avail         for i in myprocs              i start            for i in myprocs              i join

User · Answer

In my case it was a simple bug in the code  using a variable before it was created  Worth checking that out before trying the above solutions  Why I got this particular error message  Lord knows

User · Answer

Though the earlier answers are correct  there s a small complication it would help to remark on   In case your main module imports another module in which global variables or class member variables are defined and initialized to  or using  some new objects  you may have to condition that import in the same way   if   name          main       import my module

User · Answer

On Windows the subprocesses will import  i e  execute  the main module at start  You need to insert an if   name         main     guard in the main module to avoid creating subprocesses recursively    Modified testMain py   import parallelTestModule  if   name         main             extractor   parallelTestModule ParallelExtractor       extractor runInParallel numProcesses 2  numThreads 4

User · Answer

hello here is my structure for multi process from multiprocessing import Process import time   start   time perf counter     def do something time for sleep       print f Sleeping  time for sleep  second          time sleep time for sleep      print  Done Sleeping         p1   Process target do something  args  1   p2   Process target do something  args  2     if     name         main         p1 start       p2 start        p1 join       p2 join        finish   time perf counter       print f Finished in  round finish-start 2    second s     you don t have to put imports in the name     main   just running the program you wish to running inside

User · Answer

As  Ofer said  when you are using another libraries or modules  you should import all of them inside the if   name          main       So  in my case  ended like this   if   name         main                import librosa     import os     import pandas as pd     run my program

User · Answer

Try putting your code inside a main function in testMain py  import parallelTestModule  if   name          main       extractor   parallelTestModule ParallelExtractor     extractor runInParallel numProcesses 2  numThreads 4    See the docs    For an explanation of why  on Windows  the if   name         main     part is necessary  see Programming guidelines     which say      Make sure that the main module can be safely imported by a new Python   interpreter without causing unintended side effects  such a starting a   new process          by using if   name         main

[python] RuntimeError on windows trying python multiprocessing

Examples related to python

Examples related to windows

Examples related to multiprocessing