Loading and parsing a JSON file with multiple JSON objects

Question

I am trying to load and parse a JSON file in Python  But I m stuck trying to load the file   import json json data   open  file   data   json load json data    Yields   ValueError  Extra data  line 2 column 1 - line 225116 column 1  char 232 - 160128774    I looked at 18 2  json     JSON encoder and decoder in the Python documentation  but it s pretty discouraging to read through this horrible-looking documentation   First few lines  anonymized with randomized entries      votes     funny   2   useful   5   cool   1    user id    harveydennis    name    Jasmine Graham    url    http   example org user details userid harveydennis    average stars   3 5   review count   12   type    user     votes     funny   1   useful   2   cool   4    user id    njohnson    name    Zachary Ballard    url    https   www example com user details userid njohnson    average stars   3 5   review count   12   type    user     votes     funny   1   useful   0   cool   4    user id    david06    name    Jonathan George    url    https   example com user details userid david06    average stars   3 5   review count   12   type    user     votes     funny   6   useful   5   cool   0    user id    santiagoerika    name    Amanda Taylor    url    https   www example com user details userid santiagoerika    average stars   3 5   review count   12   type    user     votes     funny   1   useful   8   cool   2    user id    rodriguezdennis    name    Jennifer Roach    url    http   www example com user details userid rodriguezdennis    average stars   3 5   review count   12   type    user

User · Answer

for those stumbling upon this question  the python jsonlines library  much younger than this question  elegantly handles files with one json document per line  see https   jsonlines readthedocs io

User · Answer

You have a JSON Lines format text file  You need to parse your file line by line   import json  data      with open  file   as f      for line in f          data append json loads line     Each line contains valid JSON  but as a whole  it is not a valid JSON value as there is no top-level list or object definition   Note that because the file contains JSON per line  you are saved the headaches of trying to parse it all in one go or to figure out a streaming JSON parser  You can now opt to process each line separately before moving on to the next  saving memory in the process  You probably don t want to append each result to one list and then process everything if your file is really big   If you have a file containing individual JSON objects with delimiters in-between  use How do I use the   39 json  39  module to read in one JSON object at a time  to parse out individual objects using a buffered method

User · Answer

That is ill-formatted  You have one JSON object per line  but they are not contained in a larger data structure  ie an array   You ll either need to reformat it so that it begins with   and ends with   with a comma at the end of each line  or parse it line by line as separate dictionaries

[python] Loading and parsing a JSON file with multiple JSON objects

Examples related to python

Examples related to json

Examples related to file

Examples related to jsonlines