Python HTTPS GET with basic authentication

Question

Im trying to do a HTTPS GET with basic authentication using python  Im very new to python and the guides seem to use diffrent librarys to do things   http client  httplib and urllib   Can anyone show me how its done  How can you tell the standard library to use

User · Answer

using only standard modules and no manual header encoding

...which seems to be the intended and most portable way

the concept of python urllib is to group the numerous attributes of the request into various managers/directors/contexts... which then process their parts:

import urllib.request, ssl

# to avoid verifying ssl certificates
httpsHa = urllib.request.HTTPSHandler(context= ssl._create_unverified_context())

# setting up realm+urls+user-password auth
# (top_level_url may be sequence, also the complete url, realm None is default)
top_level_url = 'https://ip:port_or_domain'
# of the std managers, this can send user+passwd in one go,
# not after HTTP req->401 sequence
password_mgr = urllib.request.HTTPPasswordMgrWithPriorAuth()
password_mgr.add_password(None, top_level_url, "user", "password", is_authenticated=True)

handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
# create OpenerDirector
opener = urllib.request.build_opener(handler, httpsHa)

url = top_level_url + '/some_url?some_query...'
response = opener.open(url)

print(response.read())

User · Answer

A correct way to do basic auth in Python3 urllib request with certificate validation follows   Note that certifi is not mandatory  You can use your OS bundle  likely  nix only  or distribute Mozilla s CA Bundle yourself  Or if the hosts you communicate with are just a few  concatenate CA file yourself from the hosts  CAs  which can reduce the risk of MitM attack caused by another corrupt CA      usr bin env python3   import urllib request import ssl  import certifi   context   ssl SSLContext ssl PROTOCOL TLSv1  context verify mode   ssl CERT REQUIRED context load verify locations certifi where    httpsHandler   urllib request HTTPSHandler context   context   manager   urllib request HTTPPasswordMgrWithDefaultRealm   manager add password None   https   domain com     username    password   authHandler   urllib request HTTPBasicAuthHandler manager   opener   urllib request build opener httpsHandler  authHandler     Used globally for all urllib request requests    If it doesn t fit your design  use opener directly  urllib request install opener opener   response   urllib request urlopen  https   domain com some path   print response read

User · Answer

Use the power of Python and lean on one of the best libraries around  requests  import requests  r   requests get  https   my website com rest path   auth   myusername    mybasicpass    print r text    Variable r  requests response  has a lot more parameters that you can use   Best thing is to pop into the interactive interpreter and play around with it  and or read requests docs   ubuntu hostname  home ubuntu  python3 Python 3 4 3  default  Oct 14 2015  20 28 29   GCC 4 8 4  on linux Type  help    copyright    credits  or  license  for more information   gt  gt  gt  import requests  gt  gt  gt  r   requests get  https   my website com rest path   auth   myusername    mybasicpass     gt  gt  gt  dir r      attrs        bool        class        delattr        dict        dir        doc        eq        format        ge        getattribute        getstate        gt        hash        init        iter        le        lt        module        ne        new        nonzero        reduce        reduce ex        repr        setattr        setstate        sizeof        str        subclasshook        weakref       content     content consumed    apparent encoding    close    connection    content    cookies    elapsed    encoding    headers    history    iter content    iter lines    json    links    ok    raise for status    raw    reason    request    status code    text    url    gt  gt  gt  r content b   battery status  0  margin status  0  timestamp status  null  req status  0    gt  gt  gt  r text    battery status  0  margin status  0  timestamp status  null  req status  0    gt  gt  gt  r status code 200  gt  gt  gt  r headers CaseInsensitiveDict   x-powered-by    Express    content-length    77    date    Fri  20 May 2016 02 06 18 GMT    server    nginx 1 6 3    connection    keep-alive    content-type    application json  charset utf-8

User · Answer

Based on the  AndrewCox  s answer with some minor improvements   from http client import HTTPSConnection from base64 import b64encode   client   HTTPSConnection  www google com   user    user name  password    password  headers          Authorization    Basic     format          b64encode bytes f  user   password     utf-8    decode  ascii           client request  GET        headers headers  res   client getresponse   data   res read     Note  you should set encoding if you use bytes function instead of b

User · Answer

Update  OP uses Python 3  So adding an example using httplib2  import httplib2  h   httplib2 Http   cache    h add credentials  name    password     Basic authentication  resp  content   h request  https   host path to resource    POST   body  foobar     The below works for python 2 6   I use pycurl a lot in production for a process which does upwards of 10 million requests per day   You ll need to import the following first   import pycurl import cStringIO import base64   Part of the basic authentication header consists of the username and password encoded as Base64    headers      Authorization     Basic  s    base64 b64encode  username password       In the HTTP header you will see this line Authorization  Basic dXNlcm5hbWU6cGFzc3dvcmQ   The encoded string changes depending on your username and password   We now need a place to write our HTTP response to and a curl connection handle   response   cStringIO StringIO   conn   pycurl Curl     We can set various curl options  For a complete list of options  see this  The linked documentation is for the libcurl API  but the options does not change for other language bindings   conn setopt pycurl VERBOSE  1  conn setopt pycurlHTTPHEADER     s   s    t for t in headers items      conn setopt pycurl URL   https   host path to resource   conn setopt pycurl POST  1    If you do not need to verify certificate  Warning  This is insecure  Similar to running curl -k or curl --insecure   conn setopt pycurl SSL VERIFYPEER  False  conn setopt pycurl SSL VERIFYHOST  False    Call cStringIO write for storing the HTTP response   conn setopt pycurl WRITEFUNCTION  response write    When you re making a POST request   post body    foobar  conn setopt pycurl POSTFIELDS  post body    Make the actual request now   conn perform     Do something based on the HTTP response code   http code   conn getinfo pycurl HTTP CODE  if http code is 200     print response getvalue

User · Answer

In Python 3 the following will work   I am using the lower level http client from the standard library   Also check out section 2 of rfc2617 for details of basic authorization   This code won t check the certificate is valid  but will set up a https connection   See the http client docs on how to do that   from http client import HTTPSConnection from base64 import b64encode  This sets up the https connection c   HTTPSConnection  www google com    we need to base 64 encode it   and then decode it to acsii as python 3 stores it as a byte string userAndPass   b64encode b username password   decode  ascii   headers      Authorization     Basic  s     userAndPass    then connect c request  GET        headers headers   get the response back res   c getresponse     at this point you could check the status etc   this gets the page text data   res read

User · Answer

requests get url  auth requests auth HTTPBasicAuth username token  password       If with token  password should be     It works for me

[python] Python, HTTPS GET with basic authentication

using only standard modules and no manual header encoding

Examples related to python

Examples related to python-3.x