How to get JSON from webpage into Python script

Question

Got the following code in one of my scripts       url is defined above    jsonurl   urlopen url       While trying to debug  I put this in    print jsonurl      Was hoping text would contain the actual json crap from the URL  but seems not      text   json loads jsonurl  print text   What I want to do is get the        etc        stuff that I see on the URL when I load it in Firefox into my script so I can parse a value out of it  I ve Googled a ton but I haven t found a good answer as to how to actually get the         stuff from a URL ending in  json into an object in a Python script

User · Answer

you need import requests and use from json   method   source   requests get  quot url quot   json   print source   Of course  this method also works  import json urllib request data   urllib request urlopen  quot url quot   read   output   json loads data  print  output   json loads will decode it into a Python object using this table  for example a JSON object will become a Python dict

User · Answer

All that the call to urlopen   does  according to the docs  is return a file-like object  Once you have that  you need to call its read   method to actually pull the JSON data across the network   Something like   jsonurl   urlopen url   text   json loads jsonurl read    print text

User · Answer

There s no need to use an extra library to parse the json     json loads   returns a dictionary   So in your case  just do text  someValueKey

User · Answer

I ll take a guess that you actually want to get data from the URL   jsonurl   urlopen url  text   json loads jsonurl read       lt -- read from it   Or  check out JSON decoder in the requests library   import requests r   requests get  someurl   print r json     if response type was set to JSON  then you ll automatically have a JSON response here

User · Answer

I have found this to be the easiest and most efficient way to get JSON from a webpage when using Python 3    import json urllib request data   urllib request urlopen  https   api github com users since 100   read   output   json loads data  print  output

User · Answer

This gets a dictionary in JSON format from a webpage with Python 2 X and Python 3 X      usr bin env python  try        For Python 3 0 and later     from urllib request import urlopen except ImportError        Fall back to Python 2 s urllib2     from urllib2 import urlopen  import json   def get jsonparsed data url               Receive the content of   url    parse it as JSON and return the object       Parameters     ----------     url   str      Returns     -------     dict             response   urlopen url      data   response read   decode  utf-8       return json loads data    url     http   maps googleapis com maps api geocode json           address googleplex amp sensor false   print get jsonparsed data url     See also  Read and write example for JSON

User · Answer

you can use json dumps    import json    Hier comes you received data  data   json dumps response   print data    for loading json and write it on file the following code is useful   data   json loads json dumps Response  sort keys False  indent 4   with open  data json    w   as outfile  json dump data  outfile  sort keys False  indent 4

User · Answer

In Python 2  json load   will work instead of json loads    import json import urllib  url    https   api github com users since 100  output   json load urllib urlopen url   print output    Unfortunately  that doesn t work in Python 3  json load is just a wrapper around json loads that calls read   for a file-like object  json loads requires a string object and the output of urllib urlopen url  read   is a bytes object   So one has to get the file encoding in order to make it work in Python 3   In this example we query the headers for the encoding and fall back to utf-8 if we don t get one  The headers object is different between Python 2 and 3 so it has to be done different ways  Using requests would avoid all this  but sometimes you need to stick to the standard library   import json from six moves urllib request import urlopen  DEFAULT ENCODING    utf-8  url    https   api github com users since 100  urlResponse   urlopen url   if hasattr urlResponse headers   get content charset        encoding   urlResponse headers get content charset DEFAULT ENCODING  else      encoding   urlResponse headers getparam  charset   or DEFAULT ENCODING  output   json loads urlResponse read   decode encoding   print output

User · Answer

Get data from the URL and then call json loads e g   Python3 example   import urllib request  json  with urllib request urlopen  http   maps googleapis com maps api geocode json address google   as url      data   json loads url read   decode        print data    Python2 example   import urllib  json url    http   maps googleapis com maps api geocode json address google  response   urllib urlopen url  data   json loads response read    print data   The output would result in something like this      results                 address components                             long name     Charleston and Huff                short name     Charleston and Huff                types       establishment    point of interest                                      long name     Mountain View                short name     Mountain View                types       locality    political

User · Answer

Late answer  but for python gt  3 6 you can use   import dload j   dload json url      Install dload with   pip3 install dload

[python] How to get JSON from webpage into Python script

Examples related to python

Examples related to json