I have JSON data stored in the variable data
.
I want to write this to a text file for testing so I don't have to grab the data from the server each time.
Currently, I am trying this:
obj = open('data.txt', 'wb')
obj.write(data)
obj.close
And I am receiving this error:
TypeError: must be string or buffer, not dict
How to fix this?
if you are trying to write a pandas dataframe into a file using a json format i'd recommend this
destination='filepath'
saveFile = open(destination, 'w')
saveFile.write(df.to_json())
saveFile.close()
All previous answers are correct here is a very simple example:
#! /usr/bin/env python
import json
def write_json():
# create a dictionary
student_data = {"students":[]}
#create a list
data_holder = student_data["students"]
# just a counter
counter = 0
#loop through if you have multiple items..
while counter < 3:
data_holder.append({'id':counter})
data_holder.append({'room':counter})
counter += 1
#write the file
file_path='/tmp/student_data.json'
with open(file_path, 'w') as outfile:
print("writing file to: ",file_path)
# HERE IS WHERE THE MAGIC HAPPENS
json.dump(student_data, outfile)
outfile.close()
print("done")
write_json()
To write the JSON with indentation, "pretty print":
import json
outfile = open('data.json')
json.dump(data, outfile, indent=4)
Also, if you need to debug improperly formatted JSON, and want a helpful error message, use import simplejson
library, instead of import json
(functions should be the same)
I would answer with slight modification with aforementioned answers and that is to write a prettified JSON file which human eyes can read better. For this, pass sort_keys
as True
and indent
with 4 space characters and you are good to go. Also take care of ensuring that the ascii codes will not be written in your JSON file:
with open('data.txt', 'w') as outfile:
json.dump(jsonData, outfile, sort_keys = True, indent = 4,
ensure_ascii = False)
The JSON data can be written to a file as follows
hist1 = [{'val_loss': [0.5139984398465246],
'val_acc': [0.8002029867684085],
'loss': [0.593220705309384],
'acc': [0.7687131817929321]},
{'val_loss': [0.46456472964199463],
'val_acc': [0.8173602046780344],
'loss': [0.4932038113037539],
'acc': [0.8063946213802453]}]
Write to a file:
with open('text1.json', 'w') as f:
json.dump(hist1, f)
I don't have enough reputation to add in comments, so I just write some of my findings of this annoying TypeError here:
Basically, I think it's a bug in the json.dump()
function in Python 2 only - It can't dump a Python (dictionary / list) data containing non-ASCII characters, even you open the file with the encoding = 'utf-8'
parameter. (i.e. No matter what you do). But, json.dumps()
works on both Python 2 and 3.
To illustrate this, following up phihag's answer: the code in his answer breaks in Python 2 with exception TypeError: must be unicode, not str
, if data
contains non-ASCII characters. (Python 2.7.6, Debian):
import json
data = {u'\u0430\u0431\u0432\u0433\u0434': 1} #{u'?????': 1}
with open('data.txt', 'w') as outfile:
json.dump(data, outfile)
It however works fine in Python 3.
json.dump(data, open('data.txt', 'wb'))
To get utf8-encoded file as opposed to ascii-encoded in the accepted answer for Python 2 use:
import io, json
with io.open('data.txt', 'w', encoding='utf-8') as f:
f.write(json.dumps(data, ensure_ascii=False))
The code is simpler in Python 3:
import json
with open('data.txt', 'w') as f:
json.dump(data, f, ensure_ascii=False)
On Windows, the encoding='utf-8'
argument to open
is still necessary.
To avoid storing an encoded copy of the data in memory (result of dumps
) and to output utf8-encoded bytestrings in both Python 2 and 3, use:
import json, codecs
with open('data.txt', 'wb') as f:
json.dump(data, codecs.getwriter('utf-8')(f), ensure_ascii=False)
The codecs.getwriter
call is redundant in Python 3 but required for Python 2
Readability and size:
The use of ensure_ascii=False
gives better readability and smaller size:
>>> json.dumps({'price': '€10'})
'{"price": "\\u20ac10"}'
>>> json.dumps({'price': '€10'}, ensure_ascii=False)
'{"price": "€10"}'
>>> len(json.dumps({'?????': 1}))
37
>>> len(json.dumps({'?????': 1}, ensure_ascii=False).encode('utf8'))
17
Further improve readability by adding flags indent=4, sort_keys=True
(as suggested by dinos66) to arguments of dump
or dumps
. This way you'll get a nicely indented sorted structure in the json file at the cost of a slightly larger file size.
Write a data in file using JSON use json.dump() or json.dumps() used. write like this to store data in file.
import json
data = [1,2,3,4,5]
with open('no.txt', 'w') as txtfile:
json.dump(data, txtfile)
this example in list is store to a file.
Writing JSON to a File
import json
data = {}
data['people'] = []
data['people'].append({
'name': 'Scott',
'website': 'stackabuse.com',
'from': 'Nebraska'
})
data['people'].append({
'name': 'Larry',
'website': 'google.com',
'from': 'Michigan'
})
data['people'].append({
'name': 'Tim',
'website': 'apple.com',
'from': 'Alabama'
})
with open('data.txt', 'w') as outfile:
json.dump(data, outfile)
Reading JSON from a File
import json
with open('data.txt') as json_file:
data = json.load(json_file)
for p in data['people']:
print('Name: ' + p['name'])
print('Website: ' + p['website'])
print('From: ' + p['from'])
print('')
The accepted answer is fine. However, I ran into "is not json serializable" error using that.
Here's how I fixed it
with open("file-name.json", 'w')
as output:
output.write(str(response))
Although it is not a good fix as the json file it creates will not have double quotes, however it is great if you are looking for quick and dirty.
For those of you who are trying to dump greek or other "exotic" languages such as me but are also having problems (unicode errors) with weird characters such as the peace symbol (\u262E) or others which are often contained in json formated data such as Twitter's, the solution could be as follows (sort_keys is obviously optional):
import codecs, json
with codecs.open('data.json', 'w', 'utf8') as f:
f.write(json.dumps(data, sort_keys = True, ensure_ascii=False))
# -*- coding: utf-8 -*-
import json
# Make it work for Python 2+3 and with Unicode
import io
try:
to_unicode = unicode
except NameError:
to_unicode = str
# Define data
data = {'a list': [1, 42, 3.141, 1337, 'help', u'€'],
'a string': 'bla',
'another dict': {'foo': 'bar',
'key': 'value',
'the answer': 42}}
# Write JSON file
with io.open('data.json', 'w', encoding='utf8') as outfile:
str_ = json.dumps(data,
indent=4, sort_keys=True,
separators=(',', ': '), ensure_ascii=False)
outfile.write(to_unicode(str_))
# Read JSON file
with open('data.json') as data_file:
data_loaded = json.load(data_file)
print(data == data_loaded)
Explanation of the parameters of json.dump
:
indent
: Use 4 spaces to indent each entry, e.g. when a new dict is started (otherwise all will be in one line),sort_keys
: sort the keys of dictionaries. This is useful if you want to compare json files with a diff tool / put them under version control.separators
: To prevent Python from adding trailing whitespacesHave a look at my utility package mpu
for a super simple and easy to remember one:
import mpu.io
data = mpu.io.read('example.json')
mpu.io.write('example.json', data)
{
"a list":[
1,
42,
3.141,
1337,
"help",
"€"
],
"a string":"bla",
"another dict":{
"foo":"bar",
"key":"value",
"the answer":42
}
}
.json
For your application, the following might be important:
See also: Comparison of data serialization formats
In case you are rather looking for a way to make configuration files, you might want to read my short article Configuration files in Python
Source: Stackoverflow.com