Python urllib2 basic HTTP authentication and tr im

Question

I m playing around  trying to write some code to use the tr im APIs to shorten a URL   After reading http   docs python org library urllib2 html  I tried      TRIM API URL    http   api tr im api     auth handler   urllib2 HTTPBasicAuthHandler      auth handler add password realm  tr im                                uri TRIM API URL                               user USERNAME                               passwd PASSWORD     opener   urllib2 build opener auth handler     urllib2 install opener opener     response   urllib2 urlopen   s trim simple url  s                                   TRIM API URL  url to trim      url   response read   strip     response code is 200  I think it should be 202    url is valid  but the basic HTTP authentication doesn t seem to have worked  because the shortened URL isn t in my list of URLs  at http   tr im  page 1    After reading http   www voidspace org uk python articles authentication shtml doing-it-properly I also tried      TRIM API URL    api tr im api     password mgr   urllib2 HTTPPasswordMgrWithDefaultRealm      password mgr add password None  TRIM API URL  USERNAME  PASSWORD     auth handler   urllib2 HTTPBasicAuthHandler password mgr     opener   urllib2 build opener auth handler     urllib2 install opener opener     response   urllib2 urlopen  http    s trim simple url  s                                   TRIM API URL  url to trim      url   response read   strip     But I get the same results    response code is 200 and url is valid  but not recorded in my account at http   tr im     If I use query string parameters instead of basic HTTP authentication  like this      TRIM API URL    http   api tr im api     response   urllib2 urlopen   s trim simple url  s amp username  s amp password  s                                   TRIM API URL                                   url to trim                                   USERNAME                                   PASSWORD      url   response read   strip        then not only is url valid but it s recorded in my tr im account   Though response code is still 200    There must be something wrong with my code though  and not tr im s API   because    curl -u yacitus xxxx http   api tr im api trim url json url http   www google co uk      returns     trimpath   hfhb   reference   nH45bftZDWOX0QpVojeDbOvPDnaRaJ   trimmed   11  03  2009   destination   http     www google co uk     trim path   hfhb   domain   google co uk   url   http     tr im  hfhb   visits  0  status    result   OK   code   200   message   tr im URL Added     date time   2009-03-11T10 15 35-04 00        and the URL does appear in my list of URLs on http   tr im  page 1   And if I run     curl -u yacitus xxxx http   api tr im api trim url json url http   www google co uk      again  I get     trimpath   hfhb   reference   nH45bftZDWOX0QpVojeDbOvPDnaRaJ   trimmed   11  03  2009   destination   http     www google co uk     trim path   hfhb   domain   google co uk   url   http     tr im  hfhb   visits  0  status    result   OK   code   201   message   tr im URL Already Created  yacitus      date time   2009-03-11T10 15 35-04 00     Note code is 201  and message is  tr im URL Already Created  yacitus     I must not be doing the basic HTTP authentication correctly  in either attempt    Can you spot my problem   Perhaps I should look and see what s being sent over the wire   I ve never done that before   Are there Python APIs I can use  perhaps in pdb    Or is there another tool  preferably for Mac OS X  I can use

User · Answer

I would suggest that the current solution is to use my package urllib2 prior auth which solves this pretty nicely  I work on inclusion to the standard lib

User · Answer

Take a look at this SO post answer and also look at this basic authentication tutorial from the urllib2 missing manual   In order for urllib2 basic authentication to work  the http response must contain HTTP code 401 Unauthorized and a key  WWW-Authenticate  with the value  Basic  otherwise  Python won t send your login info  and you will need to either use Requests  or urllib urlopen url  with your login in the url  or add a the header like in  Flowpoke s answer   You can view your error by putting your urlopen in a try block   try      urllib2 urlopen urllib2 Request url   except urllib2 HTTPError  e      print e headers     print e headers has key  WWW-Authenticate

User · Answer

Try python-request or python-grab

User · Answer

This seems to work really well  taken from another thread   import urllib2  base64  request   urllib2 Request  http   api foursquare com v1 user   base64string   base64 encodestring   s  s     username  password   replace   n       request add header  Authorization    Basic  s    base64string     result   urllib2 urlopen request

User · Answer

The recommended way is to use requests module       usr bin env python import requests     python -m pip install requests     from pip  vendor import requests   bundled with python  url    https   httpbin org hidden-basic-auth user passwd  user  password    user    passwd   r   requests get url  auth  user  password     send auth unconditionally r raise for status     raise an exception if the authentication fails   Here s a single source Python 2 3 compatible urllib2-based variant      usr bin env python import base64 try      from urllib request import Request  urlopen except ImportError    Python 2     from urllib2 import Request  urlopen  credentials     user   password   format   vars    encode   urlopen Request url  headers   Authorization     send auth unconditionally     b Basic     base64 b64encode credentials     close     Python 3 5  introduces HTTPPasswordMgrWithPriorAuth   that allows        to eliminate unnecessary 401 response handling  or to unconditionally send credentials on the first request in order to communicate with servers that return a 404 response instead of a 401 if the Authorization header is not sent        usr bin env python3 import urllib request as urllib2  password manager   urllib2 HTTPPasswordMgrWithPriorAuth   password manager add password None  url  user  password                                is authenticated True    to handle 404 variant auth manager   urllib2 HTTPBasicAuthHandler password manager  opener   urllib2 build opener auth manager   opener open url  close     It is easy to replace HTTPBasicAuthHandler   with ProxyBasicAuthHandler   if necessary in this case

User · Answer

Really cheap solution   urllib urlopen  http   user xxxx api tr im api      which you may decide is not suitable for a number of reasons  like security of the url    Github API example    gt  gt  gt  import urllib  json  gt  gt  gt  result   urllib urlopen  https   personal-access-token x-oauth-basic api github com repos  owner  repo    gt  gt  gt  r   json load result fp   gt  gt  gt  result close

User · Answer

Same solutions as Python urllib2 Basic Auth Problem apply   see https   stackoverflow com a 24048852 1733117  you can subclass urllib2 HTTPBasicAuthHandler to add the Authorization header to each request that matches the known url   class PreemptiveBasicAuthHandler urllib2 HTTPBasicAuthHandler          Preemptive basic auth       Instead of waiting for a 403 to then retry with the credentials      send the credentials if the url is handled by the password manager      Note  please use realm None when calling add password         def http request self  req           url   req get full url           realm   None           this is very similar to the code from retry http basic auth             but returns a request object          user  pw   self passwd find user password realm  url          if pw              raw     s  s     user  pw              auth    Basic  s    base64 b64encode raw  strip               req add unredirected header self auth header  auth          return req      https request   http request

[python] Python urllib2, basic HTTP authentication, and tr.im

Examples related to python

Examples related to http

Examples related to authentication