I looked at previous similar questions and got only more confused.
In python 3.4, I want to read an html page as a string, given the url.
In perl I do this with LWP::Simple, using get().
A matplotlib 1.3.1 example says:
import urllib; u1=urllib.urlretrieve(url).
python3 can't find
u1 = urllib.request.urlopen(url), which appears to get an
HTTPResponse object, but I can't print it or get a length on it or index it.
u1.body doesn't exist. I can't find a description of the
HTTPResponse in python3.
Is there an attribute in the
HTTPResponse object which will give me the raw bytes of the html page?
(Irrelevant stuff from other questions include
urllib2, which doesn't exist in my python, csv parsers, etc.)
I found something in a prior question which partially (mostly) does the job:
u2 = urllib.request.urlopen('http://finance.yahoo.com/q?s=aapl&ql=1') for lines in u2.readlines(): print (lines)
I say 'partially' because I don't want to read separate lines, but just one big string.
I could just concatenate the lines, but every line printed has a character 'b' prepended to it.
Where does that come from?
Again, I suppose I could delete the first character before concatenating, but that does get to be a kloodge.
This question is tagged with
~ Asked on 2014-06-11 01:56:17
Note that Python3 does not read the html code as a string but as a
bytearray, so you need to convert it to one with
import urllib.request fp = urllib.request.urlopen("http://www.python.org") mybytes = fp.read() mystr = mybytes.decode("utf8") fp.close() print(mystr)
~ Answered on 2015-06-17 11:18:20
Try the 'requests' module, it's much simpler.
#pip install requests for installation import requests url = 'https://www.google.com/' r = requests.get(url) r.text
more info here > http://docs.python-requests.org/en/master/
~ Answered on 2017-01-25 22:21:29