How can I see the entire HTTP request that s being sent by my Python application

Question

In my case  I m using the requests library to call PayPal s API over HTTPS  Unfortunately  I m getting an error from PayPal  and PayPal support cannot figure out what the error is or what s causing it  They want me to  Please provide the entire request  headers included    How can I do that

User · Answer

The verbose configuration option might allow you to see what you want  There is an example in the documentation   NOTE  Read the comments below  The verbose config options doesn t seem to be available anymore

User · Answer

A simple method  enable logging in recent versions of Requests  1 x and higher      Requests uses the http client and logging module configuration to control logging verbosity  as described here     Demonstration  Code excerpted from the linked documentation   import requests import logging    These two lines enable debugging at httplib level  requests- gt urllib3- gt http client    You will see the REQUEST  including HEADERS and DATA  and RESPONSE with HEADERS but without DATA    The only thing missing will be the response body which is not logged  try      import http client as http client except ImportError        Python 2     import httplib as http client http client HTTPConnection debuglevel   1    You must initialize logging  otherwise you ll not see debug output  logging basicConfig   logging getLogger   setLevel logging DEBUG  requests log   logging getLogger  requests packages urllib3   requests log setLevel logging DEBUG  requests log propagate   True  requests get  https   httpbin org headers     Example Output    python requests-logging py  INFO requests packages urllib3 connectionpool Starting new HTTPS connection  1   httpbin org send   GET  headers HTTP 1 1 r nHost  httpbin org r nAccept-Encoding  gzip  deflate  compress r nAccept      r nUser-Agent  python-requests 1 2 0 CPython 2 7 3 Linux 3 2 0-48-generic r n r n  reply   HTTP 1 1 200 OK r n  header  Content-Type  application json header  Date  Sat  29 Jun 2013 11 19 34 GMT header  Server  gunicorn 0 17 4 header  Content-Length  226 header  Connection  keep-alive DEBUG requests packages urllib3 connectionpool  GET  headers HTTP 1 1  200 226

User · Answer

You can use HTTP Toolkit to do exactly this   It s especially useful if you need to do this quickly  with no code changes  you can open a terminal from HTTP Toolkit  run any Python code from there as normal  and you ll be able to see the full content of every HTTP HTTPS request immediately    There s a free version that can do everything you need  and it s 100  open source   I m the creator of HTTP Toolkit  I actually built it myself to solve the exact same problem for me a while back  I too was trying to debug a payment integration  but their SDK didn t work  I couldn t tell why  and I needed to know what was actually going on to properly fix it  It s very frustrating  but being able to see the raw traffic really helps

User · Answer

r   requests get  https   api github com   auth   user    pass      r is a response   It has a request attribute which has the information you need   r request allow redirects  r request headers          r request register hook r request auth             r request hooks            r request response r request cert             r request method           r request send r request config           r request params           r request sent r request cookies          r request path url         r request session r request data             r request prefetch         r request timeout r request deregister hook  r request proxies          r request url r request files            r request redirect         r request verify   r request headers gives the headers     Accept            Accept-Encoding    identity  deflate  compress  gzip     Authorization   u Basic dXNlcjpwYXNz     User-Agent    python-requests 0 12 1     Then r request data has the body as a mapping  You can convert this with urllib urlencode if they prefer   import urllib b   r request data encoded body   urllib urlencode b    depending on the type of the response the  data-attribute may be missing and a  body-attribute be there instead

User · Answer

If you re using Python 2 x  try installing a urllib2 opener  That should print out your headers  although you may have to combine that with other openers you re using to hit the HTTPS   import urllib2 urllib2 install opener urllib2 build opener urllib2 HTTPHandler debuglevel 1    urllib2 urlopen url

[python] How can I see the entire HTTP request that's being sent by my Python application?

Examples related to python

Examples related to debugging

Examples related to https

Examples related to python-requests