How to troubleshoot an AttributeError exit in multiproccesing in Python

Question

I tried to rewrite some csv-reading code to be able to run it on multiple cores in Python 3 2 2  I tried to use the Pool object of multiprocessing  which I adapted from working examples  and already worked for me for another part of my project   I ran into an error message I found hard to decipher and troubleshoot   The error   Traceback  most recent call last     File  parser5 nodots parallel py   line 256  in  lt module gt      MG ppl   csv2graph r    File  parser5 nodots parallel py   line 245  in csv2graph     node chunks    File   Library Frameworks Python framework Versions 3 2 lib python3 2 multiprocessing pool py   line 251  in map     return self map async func  iterable  chunksize  get     File   Library Frameworks Python framework Versions 3 2 lib python3 2 multiprocessing pool py   line 552  in get     raise self  value AttributeError    exit     The relevant code   import csv import time import datetime import re from operator import itemgetter from multiprocessing import Pool import itertools  def chunks l n          Divide a list of nodes  l  in  n  chunks        l c   iter l      while 1          x   tuple itertools islice l c n           if not x              return         yield x  def csv2nodes r       strptime   time strptime     mktime   time mktime     l          ppl   set       pattern   re compile r    A-Za-z0-9           n           for row in r          with pattern findall row  as f              cell   int f 3               id   int f 2               st   mktime strptime f 0    d  m  Y                ed   mktime strptime f 1    d  m  Y              collect list         l append   id cell  1 st 2  ed               collect separate sets         ppl add id      return  l ppl   def csv2graph source       MG nx MultiGraph         Remember that I use integers for edge attributes  to save space  Dic above        start  1       end  2     p   Pool       node divisor   len p  pool      node chunks   list chunks source int len source  int node divisor         num chunks   len node chunks      pedgelists   p map csv2nodes                         node chunks      ll          ppl   set       for l in pedgelists          ll append l 0           ppl update l 1       MG add edges from ll      return  MG ppl   with open   Users laszlosandor Dropbox peers prisons python codetenus test txt   r   as source      r   source readlines       MG ppl   csv2graph r    What s a good way to troubleshoot this

User · Answer

It is not the asker s problem in this instance but the first troubleshooting step for a generic  AttributeError    exit    should be making sure the brackets are there  e g   with SomeContextManager   as foo       works because a new object is referenced      not  with SomeContextManager as foo       AttributeError because the class is referenced   Catches me out from time to time and I end up here -  -

User · Answer

The reason behind this error is    Flask app is already running  hasn t shut down and in middle of that we try to start another instance by  with app app context          Code Before we use this with statement we need to make sure that scope of the previous running app is closed

User · Answer

The error also happens when trying to use the  with multiprocessing Pool   as pool             with a Python version that is too old  like Python 2 X  and does not support using with together with multiprocessing pools    See this answer https   stackoverflow com a 25968716 1426569 to another question for more details

User · Answer

The problem is in this line   with pattern findall row  as f    You are using the with statement  It requires an object with   enter   and   exit   methods  But pattern findall returns a list  with tries to store the   exit   method  but it can t find it  and raises an error  Just use  f   pattern findall row    instead

[python] How to troubleshoot an "AttributeError: exit" in multiproccesing in Python?

Examples related to python

Examples related to python-3.x

Examples related to multiprocessing

Examples related to pool

[python] How to troubleshoot an "AttributeError: __exit__" in multiproccesing in Python?

Examples related to python

Examples related to python-3.x

Examples related to multiprocessing

Examples related to pool

[python] How to troubleshoot an "AttributeError: exit" in multiproccesing in Python?