I am having trouble using json.loads to convert to a dict object and I can't figure out what I'm doing wrong.The exact error I get running this is
ValueError: Expecting property name: line 1 column 2 (char 1)
Here is my code:
from kafka.client import KafkaClient
from kafka.consumer import SimpleConsumer
from kafka.producer import SimpleProducer, KeyedProducer
import pymongo
from pymongo import MongoClient
import json
c = MongoClient("54.210.157.57")
db = c.test_database3
collection = db.tweet_col
kafka = KafkaClient("54.210.157.57:9092")
consumer = SimpleConsumer(kafka,"myconsumer","test")
for tweet in consumer:
print tweet.message.value
jsonTweet=json.loads(({u'favorited': False, u'contributors': None})
collection.insert(jsonTweet)
I'm pretty sure that the error is occuring at the 2nd to last line
jsonTweet=json.loads({u'favorited': False, u'contributors': None})
but I do not know what to do to fix it. Any advice would be appreciated.
A different case in which I encountered this was when I was using echo
to pipe the JSON into my python script and carelessly wrapped the JSON string in double quotes:
echo "{"thumbnailWidth": 640}" | myscript.py
Note that the JSON string itself has quotes and I should have done:
echo '{"thumbnailWidth": 640}' | myscript.py
As it was, this is what the python script received: {thumbnailWidth: 640}
; the double quotes were effectively stripped.
used ast, example
In [15]: a = "[{'start_city': '1', 'end_city': 'aaa', 'number': 1},\
...: {'start_city': '2', 'end_city': 'bbb', 'number': 1},\
...: {'start_city': '3', 'end_city': 'ccc', 'number': 1}]"
In [16]: import ast
In [17]: ast.literal_eval(a)
Out[17]:
[{'end_city': 'aaa', 'number': 1, 'start_city': '1'},
{'end_city': 'bbb', 'number': 1, 'start_city': '2'},
{'end_city': 'ccc', 'number': 1, 'start_city': '3'}]
>> strs = "{u'key':u'val'}"
>> strs = strs.replace("'",'"')
>> json.loads(strs.replace('u"','"'))
All other answers may answer your query, but I faced same issue which was due to stray ,
which I added at the end of my json string like this:
{
"key":"123sdf",
"bus_number":"asd234sdf",
}
I finally got it working when I removed extra ,
like this:
{
"key":"123sdf",
"bus_number":"asd234sdf"
}
Hope this help! cheers.
I encountered another problem that returns the same error.
I used a json string with single quotes :
{
'property': 1
}
But json.loads
accepts only double quotes for json properties :
{
"property": 1
}
json.loads
doesn't accept a final comma:
{
"property": "text",
"property2": "text2",
}
ast
to solve single quote and final comma issuesYou can use ast
(part of standard library for both Python 2 and 3) for this processing. Here is an example :
import ast
# ast.literal_eval() return a dict object, we must use json.dumps to get JSON string
import json
# Single quote to double with ast.literal_eval()
json_data = "{'property': 'text'}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}
# ast.literal_eval() with double quotes
json_data = '{"property": "text"}'
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property": "text"}
# ast.literal_eval() with final coma
json_data = "{'property': 'text', 'property2': 'text2',}"
json_data = ast.literal_eval(json_data)
print(json.dumps(json_data))
# Displays : {"property2": "text2", "property": "text"}
Using ast
will prevent you from single quote and final comma issues by interpet the JSON like Python dictionnary (so you must follow the Python dictionnary syntax). It's a pretty good and safely alternative of eval()
function for literal structures.
Python documentation warned us of using large/complex string :
Warning It is possible to crash the Python interpreter with a sufficiently large/complex string due to stack depth limitations in Python’s AST compiler.
To use json.dumps
with single quotes easily you can use this code:
import ast
import json
data = json.dumps(ast.literal_eval(json_data_single_quote))
ast
documentationIf you frequently edit JSON, you may use CodeBeautify. It helps you to fix syntax error and minify/beautify JSON.
I hope it helps.
Source: Stackoverflow.com