Python urllib2 Basic Auth Problem

Question

Update  based on Lee s comment I decided to condense my code to a really simple script and run it from the command line   import urllib2 import sys  username   sys argv 1  password   sys argv 2  url   sys argv 3  print  calling  s with  s  s n     url  username  password    passman   urllib2 HTTPPasswordMgrWithDefaultRealm   passman add password None  url  username  password  urllib2 install opener urllib2 build opener urllib2 HTTPBasicAuthHandler passman     req   urllib2 Request url  f   urllib2 urlopen req  data   f read   print data    Unfortunately it still won t generate the Authorization header  per Wireshark      I m having a problem sending basic AUTH over urllib2  I took a look at this article  and followed the example  My code   passman   urllib2 HTTPPasswordMgrWithDefaultRealm   passman add password None   api foursquare com   username  password  urllib2 install opener urllib2 build opener urllib2 HTTPBasicAuthHandler passman     req   urllib2 Request  http   api foursquare com v1 user       f   urllib2 urlopen req  data   f read     I m seeing the following on the Wire via wireshark   GET  v1 user HTTP 1 1 Host  api foursquare com Connection  close Accept-Encoding  gzip User-Agent  Python-urllib 2 5    You can see the Authorization is not sent  vs  when I send a request via curl  curl -u user password http   api foursquare com v1 user  GET  v1 user HTTP 1 1 Authorization  Basic  SNIP  User-Agent  curl 7 19 4  universal-apple-darwin10 0  libcurl 7 19 4 OpenSSL 0 9 8k zlib 1 2 3 Host  api foursquare com Accept        For some reason my code seems to not send the authentication - anyone see what I m missing   thanks  -simon

User · Answer

The second parameter must be a URI  not a domain name  i e   passman   urllib2 HTTPPasswordMgrWithDefaultRealm   passman add password None   http   api foursquare com    username  password

User · Answer

Here s what I m using to deal with a similar problem I encountered while trying to access MailChimp s API  This does the same thing  just formatted nicer   import urllib2 import base64  chimpConfig          headers           Content-Type    application json        Authorization    Basic     base64 encodestring  hayden MYSECRETAPIKEY   replace   n                   url    https   us12 api mailchimp com 3 0      perform authentication datas   None request   urllib2 Request chimpConfig  url    datas  chimpConfig  headers    result   urllib2 urlopen request

User · Answer

The problem could be that the Python libraries  per HTTP-Standard  first send an unauthenticated request  and then only if it s answered with a 401 retry  are the correct credentials sent  If the Foursquare servers don t do  totally standard authentication  then the libraries won t work   Try using headers to do authentication   import urllib2  base64  request   urllib2 Request  http   api foursquare com v1 user   base64string   base64 b64encode   s  s     username  password   request add header  Authorization    Basic  s    base64string     result   urllib2 urlopen request    Had the same problem as you and found the solution from this thread  http   forums shopify com categories 9 posts 27662

User · Answer

copy-paste adapted from https   stackoverflow com a 24048772 1733117    First you can subclass urllib2 BaseHandler or urllib2 HTTPBasicAuthHandler  and implement http request so that each request has the appropriate Authorization header   import urllib2 import base64  class PreemptiveBasicAuthHandler urllib2 HTTPBasicAuthHandler          Preemptive basic auth       Instead of waiting for a 403 to then retry with the credentials      send the credentials if the url is handled by the password manager      Note  please use realm None when calling add password         def http request self  req           url   req get full url           realm   None           this is very similar to the code from retry http basic auth             but returns a request object          user  pw   self passwd find user password realm  url          if pw              raw     s  s     user  pw              auth    Basic  s    base64 b64encode raw  strip               req add unredirected header self auth header  auth          return req      https request   http request   Then if you are lazy like me  install the handler globally  api url    http   api foursquare com   api username    johndoe  api password    some-cryptic-value   auth handler   PreemptiveBasicAuthHandler   auth handler add password      realm None    default realm      uri api url      user api username      passwd api password  opener   urllib2 build opener auth handler  urllib2 install opener opener

User · Answer

I would suggest that the current solution is to use my package urllib2 prior auth which solves this pretty nicely  I work on inclusion to the standard lib

[python] Python urllib2 Basic Auth Problem

Examples related to python

Examples related to authentication

Examples related to urllib2