[python] How do I disable log messages from the Requests library?

By default, the Requests python library writes log messages to the console, along the lines of:

Starting new HTTP connection (1): example.com
http://example.com:80 "GET / HTTP/1.1" 200 606

I'm usually not interested in these messages, and would like to disable them. What would be the best way to silence those messages or decrease Requests' verbosity?

import logging
urllib3_logger = logging.getLogger('urllib3')
urllib3_logger.setLevel(logging.CRITICAL)

In this way all the messages of level=INFO from urllib3 won't be present in the logfile.

So you can continue to use the level=INFO for your log messages...just modify this for the library you are using.

I'm not sure if the previous approaches have stopped working, but in any case, here's another way of removing the warnings:

PYTHONWARNINGS="ignore:Unverified HTTPS request" ./do-insecure-request.py

Basically, adding an environment variable in the context of the script execution.

From the documentation: https://urllib3.readthedocs.org/en/latest/security.html#disabling-warnings

If You have configuration file, You can configure it.

Add urllib3 in loggers section:

[loggers]
keys = root, urllib3

Add logger_urllib3 section:

[logger_urllib3]
level = WARNING
handlers =
qualname = requests.packages.urllib3.connectionpool

This answer is here: Python: how to suppress logging statements from third party libraries?

You can leave the default logging level for basicConfig, and then you set the DEBUG level when you get the logger for your module.

logging.basicConfig(format='%(asctime)s %(module)s %(filename)s:%(lineno)s - %(message)s')
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

logger.debug("my debug message")

Setting the logger name as requests or requests.urllib3 did not work for me. I had to specify the exact logger name to change the logging level.

First See which loggers you have defined, to see which ones you want to remove

print(logging.Logger.manager.loggerDict)

And you will see something like this:

{...'urllib3.poolmanager': <logging.Logger object at 0x1070a6e10>, 'django.request': <logging.Logger object at 0x106d61290>, 'django.template': <logging.Logger object at 0x10630dcd0>, 'django.server': <logging.Logger object at 0x106dd6a50>, 'urllib3.connection': <logging.Logger object at 0x10710a350>,'urllib3.connectionpool': <logging.Logger object at 0x106e09690> ...}

Then configure the level for the exact logger:

   'loggers': {
    '': {
        'handlers': ['default'],
        'level': 'DEBUG',
        'propagate': True
    },
    'urllib3.connectionpool': {
        'handlers': ['default'],
        'level': 'WARNING',
        'propagate' : False
    },

simple: just add requests.packages.urllib3.disable_warnings() after import requests

Let me copy/paste the documentation section which it I wrote about week or two ago, after having a problem similar to yours:

import requests
import logging

# these two lines enable debugging at httplib level (requests->urllib3->httplib)
# you will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.
# the only thing missing will be the response.body which is not logged.
import httplib
httplib.HTTPConnection.debuglevel = 1

logging.basicConfig() # you need to initialize logging, otherwise you will not see anything from requests
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

requests.get('http://httpbin.org/headers')

In case you came here looking for a way to modify logging of any (possibly deeply nested) module, use logging.Logger.manager.loggerDict to get a dictionary of all of the logger objects. The returned names can then be used as the argument to logging.getLogger:

import requests
import logging
for key in logging.Logger.manager.loggerDict:
    print(key)
# requests.packages.urllib3.connectionpool
# requests.packages.urllib3.util
# requests.packages
# requests.packages.urllib3
# requests.packages.urllib3.util.retry
# PYREADLINE
# requests
# requests.packages.urllib3.poolmanager

logging.getLogger('requests').setLevel(logging.CRITICAL)
# Could also use the dictionary directly:
# logging.Logger.manager.loggerDict['requests'].setLevel(logging.CRITICAL)

Per user136036 in a comment, be aware that this method only shows you the loggers that exist at the time you run the above snippet. If, for example, a module creates a new logger when you instantiate a class, then you must put this snippet after creating the class in order to print its name.

Kbrose's guidance on finding which logger was generating log messages was immensely useful. For my Django project, I had to sort through 120 different loggers until I found that it was the elasticsearch Python library that was causing issues for me. As per the guidance in most of the questions, I disabled it by adding this to my loggers:

      ...
      'elasticsearch': {
          'handlers': ['console'],
          'level': logging.WARNING,
      },     
      ...

Posting here in case someone else is seeing the unhelpful log messages come through whenever they run an Elasticsearch query.

For anybody using logging.config.dictConfig you can alter the requests library log level in the dictionary like this:

'loggers': {
    '': {
        'handlers': ['file'],
        'level': level,
        'propagate': False
    },
    'requests.packages.urllib3': {
        'handlers': ['file'],
        'level': logging.WARNING
    }
}

import logging

# Only show warnings
logging.getLogger("urllib3").setLevel(logging.WARNING)

# Disable all child loggers of urllib3, e.g. urllib3.connectionpool
logging.getLogger("urllib3").propagate = False