In Python try until no error

Question

I have a piece of code in Python that seems to cause an error probabilistically because it is accessing a server and sometimes that server has a 500 internal server error  I want to keep trying until I do not get the error  My solution was   while True      try           code with possible error     except           continue     else            the rest of the code          break   This seems like a hack to me  Is there a more Pythonic way to do this

User · Answer

Here is one that hard fails after 4 attempts, and waits 2 seconds between attempts. Change as you wish to get what you want form this one:

from time import sleep

for x in range(0, 4):  # try 4 times
    try:
        # msg.send()
        # put your logic here
        str_error = None
    except Exception as str_error:
        pass

    if str_error:
        sleep(2)  # wait for 2 seconds before trying to fetch the data again
    else:
        break

Here is an example with backoff:

from time import sleep

sleep_time = 2
num_retries = 4
for x in range(0, num_retries):  
    try:
        # put your logic here
        str_error = None
    except Exception as str_error:
        pass

    if str_error:
        sleep(sleep_time)  # wait before trying to fetch the data again
        sleep_time *= 2  # Implement your backoff algorithm here i.e. exponential backoff
    else:
        break

User · Answer

Like most of the others  I d recommend trying a finite number of times and sleeping between attempts  This way  you don t find yourself in an infinite loop in case something were to actually happen to the remote server   I d also recommend continuing only when you get the specific exception you re expecting  This way  you can still handle exceptions you might not expect   from urllib error import HTTPError import traceback from time import sleep   attempts   10 while attempts  gt  0      try           code with possible error     except HTTPError          attempts -  1         sleep 1          continue     except          print traceback format exc          the rest of the code     break   Also  you don t need an else block  Because of the continue in the except block  you skip the rest of the loop until the try block works  the while condition gets satisfied  or an exception other than HTTPError comes up

User · Answer

Here s an utility function that I wrote to wrap the retry until success into a neater package  It uses the same basic structure  but prevents repetition  It could be modified to catch and rethrow the exception on the final try relatively easily   def try until func  max tries  sleep time       for   in range 0 max tries           try              return func           except              sleep sleep time      raise WellNamedException        could be  return sensibleDefaultValue    Can then be called like this  result   try until my function  100  1000    If you need to pass arguments to my function  you can either do this by having try until forward the arguments  or by wrapping it in a no argument lambda   result   try until lambda   my function x y z   100  1000

User · Answer

It won t get much cleaner  This is not a very clean thing to do  At best  which would be more readable anyway  since the condition for the break is up there with the while   you could create a variable result   None and loop while it is None  You should also adjust the variables and you can replace continue with the semantically perhaps correct pass  you don t care if an error occurs  you just want to ignore it  and drop the break - this also gets the rest of the code  which only executes once  out of the loop  Also note that bare except  clauses are evil for reasons given in the documentation   Example incorporating all of the above   result   None while result is None      try            connect         result   get data          except           pass   other code that uses result but is not involved in getting it

User · Answer

what about the retrying library on pypi  I have been using it for a while and it does exactly what I want and more  retry on error  retry when None  retry with timeout    Below is example from their website   import random from retrying import retry   retry def do something unreliable        if random randint 0  10   gt  1          raise IOError  Broken sauce  everything is hosed   111one       else          return  Awesome sauce    print do something unreliable

User · Answer

Maybe something like this   connected   False  while not connected      try          try connect           connected   True     except              pass

User · Answer

Im attempting this now  this is what i came up with      placeholder   1     while placeholder is not None          try               Code             placeholder   None         except Exception as e              print str datetime time datetime now      8    str e    To log the errors             placeholder   e             time sleep 0 5              continue

User · Answer

Maybe decorator based  You can pass as decorator arguments list of exceptions on which we want to retry and or number of tries   def retry exceptions None  tries None       if exceptions          exceptions   tuple exceptions      def wrapper fun           def retry calls  args    kwargs               if tries                  for   in xrange tries                       try                          fun  args    kwargs                      except exceptions                          pass                     else                          break             else                  while True                      try                          fun  args    kwargs                      except exceptions                          pass                     else                          break         return retry calls     return wrapper   from random import randint   retry  NameError  ValueError   def foo        if randint 0  1           raise NameError  FAIL        print  Success    retry  ValueError   2  def bar        if randint 0  1           raise ValueError  FAIL        print  Success    retry  ValueError   2  def baz        while True          raise ValueError  FAIL     foo   bar   baz     of course the  try  part should be moved to another funcion becouse we using it in both loops but it s just example

User · Answer

e      while e            try          response   ur urlopen  https   https   raw githubusercontent com MrMe42 Joe-Bot-Home-Assistant mac Joe py           e           except          print  Connection refused  Retrying              time sleep 1    This should work  It sets e to    and the while loop checks to see if it is still     If there is an error caught be the try statement  it prints that the connection was refused  waits 1 second and then starts over  It will keep going until there is no error in try  which then sets e to      which kills the while loop

User · Answer

Here is a short piece of code I use to capture the error as a string  Will retry till it succeeds  This catches all exceptions but you can change this as you wish   start   0 str error    Not executed yet   while str error      try            replace line below with your logic   i e  time out  max attempts         start   raw input  enter a number  0 for fail  last was  0     format start           new val   5 int start          str error None     except Exception as str error           pass   WARNING  This code will be stuck in a forever loop until no exception occurs   This is just a simple example and MIGHT require you to break out of the loop sooner or sleep between retries

User · Answer

The itertools iter except recipes encapsulates this idea of  calling a function repeatedly until an exception is raised    It is similar to the accepted answer  but the recipe gives an iterator instead   From the recipes   def iter except func  exception  first None           Call a function repeatedly until an exception is raised         try          if first is not None              yield first                For database APIs needing an initial cast to db first           while True              yield func       except exception          pass   You can certainly implement the latter code directly   For convenience  I use a separate library  more itertools  that implements this recipe for us  optional      Code  import more itertools as mit  list mit iter except  0  1  2  pop  IndexError      2  1  0      Details  Here the pop method  or given function  is called for every iteration of the list object until an IndexError is raised   For your case  given some connect function and expected error  you can make an iterator that calls the function repeatedly until an exception is raised  e g   mit iter except connect function  ConnectionError    At this point  treat it as any other iterator by looping over it or calling next

User · Answer

When retrying due to error  you should always    implement a retry limit  or you may get blocked on an infinite loop implement a delay  or you ll hammer resources too hard  such as your CPU or the already distressed remote server   A simple generic way to solve this problem while covering those concerns would be to use the backoff library   A basic example   import backoff   backoff on exception      backoff expo      MyException      max tries 5   def make request self  data         do the request   This code wraps make request with a decorator which implements the retry logic   We retry whenever our specific error MyException occurs  with a limit of 5 retries   Exponential backoff is a good idea in this context to help minimize the additional burden our retries place on the remote server

User · Answer

Many different answers here but I used a combination of a few of the brilliant answers and threw in my own not-so-secret sauce to come up with the following recipe  def actioin one self       action     define attempted action here     failure   bool action status code    200    example     replace this      self attempt action repeatedly action  5  failure   def action two self       action     define attempted action here     failure   bool action status code    200    example     replace this      self attempt action repeatedly action  5  failure   def attempt action repeatedly self  action  authorized attempts  failure       attempts   0     sleep time   2   seconds     more attempts left   bool attempts  lt  authorized attempts      while failure and more attempts left          action         attempts    1         sleep sleep time       if not more attempts left          print  quot Max number of attempts exceeded  Attempts failed because  quot           data   action json     customize this          print data  detail      customize this

[python] In Python try until no error

Examples related to python

Examples related to error-handling