I am trying to create a JSON string representation of a class instance and having difficulty. Let's say the class is built like this:
class testclass:
value1 = "a"
value2 = "b"
A call to the json.dumps is made like this:
t = testclass()
json.dumps(t)
It is failing and telling me that the testclass is not JSON serializable.
TypeError: <__main__.testclass object at 0x000000000227A400> is not JSON serializable
I have also tried using the pickle module :
t = testclass()
print(pickle.dumps(t, pickle.HIGHEST_PROTOCOL))
And it gives class instance information but not a serialized content of the class instance.
b'\x80\x03c__main__\ntestclass\nq\x00)\x81q\x01}q\x02b.'
What am I doing wrong?
This question is related to
python
json
serialization
pickle
Using jsonpickle
import jsonpickle
object = YourClass()
json_object = jsonpickle.encode(object)
Python3.x
The best aproach I could reach with my knowledge was this.
Note that this code treat set() too.
This approach is generic just needing the extension of class (in the second example).
Note that I'm just doing it to files, but it's easy to modify the behavior to your taste.
However this is a CoDec.
With a little more work you can construct your class in other ways. I assume a default constructor to instance it, then I update the class dict.
import json
import collections
class JsonClassSerializable(json.JSONEncoder):
REGISTERED_CLASS = {}
def register(ctype):
JsonClassSerializable.REGISTERED_CLASS[ctype.__name__] = ctype
def default(self, obj):
if isinstance(obj, collections.Set):
return dict(_set_object=list(obj))
if isinstance(obj, JsonClassSerializable):
jclass = {}
jclass["name"] = type(obj).__name__
jclass["dict"] = obj.__dict__
return dict(_class_object=jclass)
else:
return json.JSONEncoder.default(self, obj)
def json_to_class(self, dct):
if '_set_object' in dct:
return set(dct['_set_object'])
elif '_class_object' in dct:
cclass = dct['_class_object']
cclass_name = cclass["name"]
if cclass_name not in self.REGISTERED_CLASS:
raise RuntimeError(
"Class {} not registered in JSON Parser"
.format(cclass["name"])
)
instance = self.REGISTERED_CLASS[cclass_name]()
instance.__dict__ = cclass["dict"]
return instance
return dct
def encode_(self, file):
with open(file, 'w') as outfile:
json.dump(
self.__dict__, outfile,
cls=JsonClassSerializable,
indent=4,
sort_keys=True
)
def decode_(self, file):
try:
with open(file, 'r') as infile:
self.__dict__ = json.load(
infile,
object_hook=self.json_to_class
)
except FileNotFoundError:
print("Persistence load failed "
"'{}' do not exists".format(file)
)
class C(JsonClassSerializable):
def __init__(self):
self.mill = "s"
JsonClassSerializable.register(C)
class B(JsonClassSerializable):
def __init__(self):
self.a = 1230
self.c = C()
JsonClassSerializable.register(B)
class A(JsonClassSerializable):
def __init__(self):
self.a = 1
self.b = {1, 2}
self.c = B()
JsonClassSerializable.register(A)
A().encode_("test")
b = A()
b.decode_("test")
print(b.a)
print(b.b)
print(b.c.a)
Edit
With some more of research I found a way to generalize without the need of the SUPERCLASS register method call, using a metaclass
import json
import collections
REGISTERED_CLASS = {}
class MetaSerializable(type):
def __call__(cls, *args, **kwargs):
if cls.__name__ not in REGISTERED_CLASS:
REGISTERED_CLASS[cls.__name__] = cls
return super(MetaSerializable, cls).__call__(*args, **kwargs)
class JsonClassSerializable(json.JSONEncoder, metaclass=MetaSerializable):
def default(self, obj):
if isinstance(obj, collections.Set):
return dict(_set_object=list(obj))
if isinstance(obj, JsonClassSerializable):
jclass = {}
jclass["name"] = type(obj).__name__
jclass["dict"] = obj.__dict__
return dict(_class_object=jclass)
else:
return json.JSONEncoder.default(self, obj)
def json_to_class(self, dct):
if '_set_object' in dct:
return set(dct['_set_object'])
elif '_class_object' in dct:
cclass = dct['_class_object']
cclass_name = cclass["name"]
if cclass_name not in REGISTERED_CLASS:
raise RuntimeError(
"Class {} not registered in JSON Parser"
.format(cclass["name"])
)
instance = REGISTERED_CLASS[cclass_name]()
instance.__dict__ = cclass["dict"]
return instance
return dct
def encode_(self, file):
with open(file, 'w') as outfile:
json.dump(
self.__dict__, outfile,
cls=JsonClassSerializable,
indent=4,
sort_keys=True
)
def decode_(self, file):
try:
with open(file, 'r') as infile:
self.__dict__ = json.load(
infile,
object_hook=self.json_to_class
)
except FileNotFoundError:
print("Persistence load failed "
"'{}' do not exists".format(file)
)
class C(JsonClassSerializable):
def __init__(self):
self.mill = "s"
class B(JsonClassSerializable):
def __init__(self):
self.a = 1230
self.c = C()
class A(JsonClassSerializable):
def __init__(self):
self.a = 1
self.b = {1, 2}
self.c = B()
A().encode_("test")
b = A()
b.decode_("test")
print(b.a)
# 1
print(b.b)
# {1, 2}
print(b.c.a)
# 1230
print(b.c.c.mill)
# s
There are some good answers on how to get started on doing this. But there are some things to keep in mind:
__slots__ instead of __dict__?json-tricks is a library (that I made and others contributed to) which has been able to do this for quite a while. For example:
class MyTestCls:
def __init__(self, **kwargs):
for k, v in kwargs.items():
setattr(self, k, v)
cls_instance = MyTestCls(s='ub', dct={'7': 7})
json = dumps(cls_instance, indent=4)
instance = loads(json)
You'll get your instance back. Here the json looks like this:
{
"__instance_type__": [
"json_tricks.test_class",
"MyTestCls"
],
"attributes": {
"s": "ub",
"dct": {
"7": 7
}
}
}
If you like to make your own solution, you might look at the source of json-tricks so as not to forget some special cases (like __slots__).
It also does other types like numpy arrays, datetimes, complex numbers; it also allows for comments.
I believe instead of inheritance as suggested in accepted answer, it's better to use polymorphism. Otherwise you have to have a big if else statement to customize encoding of every object. That means create a generic default encoder for JSON as:
def jsonDefEncoder(obj):
if hasattr(obj, 'jsonEnc'):
return obj.jsonEnc()
else: #some default behavior
return obj.__dict__
and then have a jsonEnc() function in each class you want to serialize. e.g.
class A(object):
def __init__(self,lengthInFeet):
self.lengthInFeet=lengthInFeet
def jsonEnc(self):
return {'lengthInMeters': lengthInFeet * 0.3 } # each foot is 0.3 meter
Then you call json.dumps(classInstance,default=jsonDefEncoder)
You could use jsonic for to serialize pretty much anything to JSON:
https://github.com/OrrBin/Jsonic
Example:
class TestClass:
def __init__(self):
self.x = 1
self.y = 2
instance = TestClass()
s = serialize(instance): # instance s set to: {"x":1, "y":2}
d = deserialize(s) # d is a new class instance of TestClass
pythonic has some nice features like declaring class attributes transient and type safe deserialization.
(A few years late with the answer, but i think it might help others)
Use arbitrary, extensible object, and then serialize it to JSON:
import json
class Object(object):
pass
response = Object()
response.debug = []
response.result = Object()
# Any manipulations with the object:
response.debug.append("Debug string here")
response.result.body = "404 Not Found"
response.result.code = 404
# Proper JSON output, with nice formatting:
print(json.dumps(response, indent=4, default=lambda x: x.__dict__))
An approach which I have been using in my Flask app to serialize Class instance to JSON response.
from json import JSONEncoder
import json
from typing import List
class ResponseEncoder(JSONEncoder):
def default(self, o):
return o.__dict__
class ListResponse:
def __init__(self, data: List):
self.data = data
self.count = len(data)
class A:
def __init__(self, message: str):
self.message = message
class B:
def __init__(self, record: A):
self.record = record
class C:
def __init__(self, data: B):
self.data = data
Now create an instance of A, B, C then encode.
data_a = A('Test Data')
data_b = B(data_a)
data_c = C(data_b)
response = ResponseEncoder().encode(data_c)
json_response = json.loads(response)
Output
{
"data": {
"record": {
"message": "Test Data"
}
}
}
For list type response
records = ['One', 'Two', 'Three']
list_response = ListResponse(records)
response = ResponseEncoder().encode(list_response)
json_response = json.loads(response)
Output
{
"data": [
"One",
"Two",
"Three"
],
"count": 3
}
There's one way that works great for me that you can try out:
json.dumps() can take an optional parameter default where you can specify a custom serializer function for unknown types, which in my case looks like
def serialize(obj):
"""JSON serializer for objects not serializable by default json code"""
if isinstance(obj, date):
serial = obj.isoformat()
return serial
if isinstance(obj, time):
serial = obj.isoformat()
return serial
return obj.__dict__
First two ifs are for date and time serialization
and then there is a obj.__dict__ returned for any other object.
the final call looks like:
json.dumps(myObj, default=serialize)
It's especially good when you are serializing a collection and you don't want to call __dict__ explicitly for every object. Here it's done for you automatically.
So far worked so good for me, looking forward for your thoughts.
Here are two simple functions for serialization of any non-sophisticated classes, nothing fancy as explained before.
I use this for configuration type stuff because I can add new members to the classes with no code adjustments.
import json
class SimpleClass:
def __init__(self, a=None, b=None, c=None):
self.a = a
self.b = b
self.c = c
def serialize_json(instance=None, path=None):
dt = {}
dt.update(vars(instance))
with open(path, "w") as file:
json.dump(dt, file)
def deserialize_json(cls=None, path=None):
def read_json(_path):
with open(_path, "r") as file:
return json.load(file)
data = read_json(path)
instance = object.__new__(cls)
for key, value in data.items():
setattr(instance, key, value)
return instance
# Usage: Create class and serialize under Windows file system.
write_settings = SimpleClass(a=1, b=2, c=3)
serialize_json(write_settings, r"c:\temp\test.json")
# Read back and rehydrate.
read_settings = deserialize_json(SimpleClass, r"c:\temp\test.json")
# results are the same.
print(vars(write_settings))
print(vars(read_settings))
# output:
# {'c': 3, 'b': 2, 'a': 1}
# {'c': 3, 'b': 2, 'a': 1}
There's another really simple and elegant approach that can be applied here which is to just subclass 'dict' since it is serializable by default.
from json import dumps
class Response(dict):
def __init__(self, status_code, body):
super().__init__(
status_code = status_code,
body = body
)
r = Response()
dumps(r)
JSON is not really meant for serializing arbitrary Python objects. It's great for serializing dict objects, but the pickle module is really what you should be using in general. Output from pickle is not really human-readable, but it should unpickle just fine. If you insist on using JSON, you could check out the jsonpickle module, which is an interesting hybrid approach.
This can be easily handled with pydantic, as it already has this functionality built-in.
Option 1: normal way
from pydantic import BaseModel
class testclass(BaseModel):
value1: str = "a"
value2: str = "b"
test = testclass()
>>> print(test.json(indent=4))
{
"value1": "a",
"value2": "b"
}
Option 2: using pydantic's dataclass
import json
from pydantic.dataclasses import dataclass
from pydantic.json import pydantic_encoder
@dataclass
class testclass:
value1: str = "a"
value2: str = "b"
test = testclass()
>>> print(json.dumps(test, indent=4, default=pydantic_encoder))
{
"value1": "a",
"value2": "b"
}
I just do:
data=json.dumps(myobject.__dict__)
This is not the full answer, and if you have some sort of complicated object class you certainly will not get everything. However I use this for some of my simple objects.
One that it works really well on is the "options" class that you get from the OptionParser module. Here it is along with the JSON request itself.
def executeJson(self, url, options):
data=json.dumps(options.__dict__)
if options.verbose:
print data
headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
return requests.post(url, data, headers=headers)
You can specify the default named parameter in the json.dumps() function:
json.dumps(obj, default=lambda x: x.__dict__)
Explanation:
``default(obj)`` is a function that should return a serializable version
of obj or raise TypeError. The default simply raises TypeError.
(Works on Python 2.7 and Python 3.x)
Note: In this case you need instance variables and not class variables, as the example in the question tries to do. (I am assuming the asker meant class instance to be an object of a class)
I learned this first from @phihag's answer here. Found it to be the simplest and cleanest way to do the job.
Source: Stackoverflow.com