Get webpage contents with Python

Question

I m using Python 3 1  if that helps   Anyways  I m trying to get the contents of this webpage  I Googled for a little bit and tried different things  but they didn t work  I m guessing that this should be an easy task  but   I can t get it       Results of urllib  urllib2    gt  gt  gt  import urllib2 Traceback  most recent call last     File   lt pyshell 0 gt    line 1  in  lt module gt      import urllib2 ImportError  No module named urllib2  gt  gt  gt  import urllib  gt  gt  gt  urllib urlopen  http   www python org   Traceback  most recent call last     File   lt pyshell 2 gt    line 1  in  lt module gt      urllib urlopen  http   www python org   AttributeError   module  object has no attribute  urlopen   gt  gt  gt     Python 3 solution  Thank you  Jason   D   import urllib request page   urllib request urlopen  http   services runescape com m hiscore ranking table 0 amp category type 0 amp time filter 0 amp date 1519066080774 amp user zezima   print page read

User · Accepted Answer

Because you re using Python 3 1  you need to use the new Python 3 1 APIs   Try   urllib request urlopen  http   www python org      Alternately  it looks like you re working from Python 2 examples  Write it in Python 2  then use the 2to3 tool to convert it  On Windows  2to3 py is in  python31 tools scripts  Can someone else point out where to find 2to3 py on other platforms   Edit  These days  I write Python 2 and 3 compatible code by using six   from six moves import urllib urllib request urlopen  http   www python org     Assuming you have six installed  that runs on both Python 2 and Python 3

User · Answer

You can use urlib2 and parse the HTML yourself   Or try Beautiful Soup to do some of the parsing for you

User · Answer

Also you can use faster than requests package  That s very fast and simple   import faster than requests as r content   r get2str  http   test com      Look at this comparison

User · Answer

Mechanize is a great package for  acting like a browser   if you want to handle cookie state  etc   http   wwwsearch sourceforge net mechanize

User · Answer

A solution with works with Python 2 X and Python 3 X   try        For Python 3 0 and later     from urllib request import urlopen except ImportError        Fall back to Python 2 s urllib2     from urllib2 import urlopen  url    http   hiscore runescape com index lite ws player zezima  response   urlopen url  data   str response read

User · Answer

Suppose you want to GET a webpage s content  The following code does it      - - coding  utf-8 - -   python    example of getting a web page  from urllib import urlopen print urlopen  http   xahlee info python python index html   read

User · Answer

The best way to do this these day is to use the  requests  library   import requests response   requests get  http   hiscore runescape com index lite ws player zezima   print  response status code  print  response content

User · Answer

If you ask me  try this one  import urllib2 resp   urllib2 urlopen  http   hiscore runescape com index lite ws player zezima     and read the normal way ie  page   resp read     Good luck though

[python] Get webpage contents with Python?

Examples related to python

Examples related to python-3.x