How to convert JSON data into a Python object

Question

I want to convert JSON data into a Python object  I receive JSON data objects from the Facebook API  which I want to store in my database  My current View in Django  Python   request POST contains the JSON   response   request POST user   FbApiUser user id   response  id    user name   response  name   user username   response  username   user save     This works fine  but how do I handle complex JSON data objects  Wouldn t it be much better if I could somehow convert this JSON object into a Python object for easy use

User · Answer

While searching for a solution  I ve stumbled upon this blog post   https   blog mosthege net 2016 11 12 json-deserialization-of-nested-objects   It uses the same technique as stated in previous answers but with a usage of decorators   Another thing I found useful is the fact that it returns a typed object at the end of deserialisation   class JsonConvert object       class mappings            classmethod     def class mapper cls  d           for keys  cls in clsself mappings items                if keys issuperset d keys         are all required arguments present                  return cls   d          else                Raise exception instead of silently returning None             raise ValueError  Unable to find a matching class for object    s   format d         classmethod     def complex handler cls  Obj           if hasattr Obj     dict                  return Obj   dict           else              raise TypeError  Object of type  s with value of  s is not JSON serializable     type Obj   repr Obj          classmethod     def register cls  claz           clsself mappings frozenset tuple  attr for attr val in cls     dict   items         cls         return cls       classmethod     def to json cls  obj           return json dumps obj   dict    default cls complex handler  indent 4        classmethod     def from json cls  json str           return json loads json str  object hook cls class mapper    Usage    JsonConvert register class Employee object       def   init   self  Name int None  Age int None           self Name   Name         self Age   Age         return   JsonConvert register class Company object       def   init   self  Name str     Employees  Employee  None           self Name   Name         self Employees      if Employees is None else Employees         return  company   Company  Contonso   company Employees append Employee  Werner   38   company Employees append Employee  Mary     as json   JsonConvert to json company  from json   JsonConvert from json as json  as json from json   JsonConvert to json from json   assert as json from json    as json   print as json from json

User · Answer

JSON to python object The follwing code creates dynamic attributes with the objects keys recursively  JSON object - fb data json         quot name quot    quot John Smith quot        quot hometown quot              quot name quot    quot New York quot            quot id quot   123             quot list quot              quot a quot            quot b quot            quot c quot           1                         quot key quot   1                       quot object quot              quot key quot                  quot key quot   1                    On the conversion we have 3 cases   lists dicts  new object  bool  int  float and str  import json   class AppConfiguration object       def   init   self  data None           if data is None              with open  quot fb data json quot   as fh                  data   json loads fh read            else              data   dict data           for key  val in data items                setattr self  key  self compute attr value val        def compute attr value self  value           if isinstance value  list               return  self compute attr value x  for x in value          elif isinstance value  dict               return AppConfiguration value          else              return value   if   name       quot   main   quot       instance   AppConfiguration        print instance name      print instance hometown name      print instance hometown id      print instance list 4  key      print instance object key key   Now the key  value pairs are attributes - objects  output  John Smith New York 123 1 1   Paste JSON as Code Supports TypeScript  Python  Go  Ruby  C   Java  Swift  Rust  Kotlin  C    Flow  Objective-C  JavaScript  Elm  and JSON Schema   Interactively generate types and  de- serialization code from JSON  JSON Schema  and TypeScript Paste JSON JSON Schema TypeScript as code   quicktype infers types from sample JSON data  then outputs strongly typed models and serializers for working with that data in your desired programming language  output    Generated by https   quicktype io     To change quicktype s target language  run command         quot Set quicktype target language quot   from typing import List  Union   class Hometown      name  str     id  int      def   init   self  name  str  id  int  - gt  None          self name   name         self id   id   class Key      key  int      def   init   self  key  int  - gt  None          self key   key   class Object      key  Key      def   init   self  key  Key  - gt  None          self key   key   class FbData      name  str     hometown  Hometown     list  List Union Key  int  str       object  Object      def   init   self  name  str  hometown  Hometown  list  List Union Key  int  str    object  Object  - gt  None          self name   name         self hometown   hometown         self list   list         self object   object  This extension is available for free in the Visual Studio Code Marketplace

User · Answer

This is not code golf  but here is my shortest trick  using types SimpleNamespace as the container for JSON objects   Compared to the leading namedtuple solution  it is    probably faster smaller as it does not create a class for each object shorter no rename option  and probably the same limitation on keys that are not valid identifiers  uses setattr under the covers    Example   from   future   import print function import json  try      from types import SimpleNamespace as Namespace except ImportError        Python 2 x fallback     from argparse import Namespace  data      name    John Smith    hometown     name    New York    id   123     x   json loads data  object hook lambda d  Namespace   d    print  x name  x hometown name  x hometown id

User · Answer

The answers given here does not return the correct object type  hence I created these methods below  They also fail if you try to add more fields to the class that does not exist in the given JSON   def dict to class class name  Any  dictionary  dict  - gt  Any      instance   class name       for key in dictionary keys            setattr instance  key  dictionary key       return instance   def json to class class name  Any  json string  str  - gt  Any      dict object   json loads json string      return dict to class class name  dict object

User · Answer

For complex objects  you can use JSON Pickle     Python library for serializing any arbitrary object graph into JSON    It can take almost any Python object and turn the object into JSON    Additionally  it can reconstitute the object back into Python

User · Answer

Modifying  DS response a bit  to load from a file   def  json object hook d   return namedtuple  X   d keys     d values    def load data file name     with open file name   r   as file data      return file data read   replace   n       def json2obj file name   return json loads load data file name   object hook  json object hook    One thing  this cannot load items with numbers ahead  Like this        1 first item          A    1        B    2          Because  1 first item  is not a valid python field name

User · Answer

If you re using Python 3 5   you can use jsons to serialize and deserialize to plain old Python objects   import jsons  response   request POST    You ll need your class attributes to match your dict keys  so in your case do  response  id     response pop  user id      Then you can load that dict into your class  user   jsons load response  FbApiUser   user save     You could also make FbApiUser inherit from jsons JsonSerializable for more elegance   user   FbApiUser from json response    These examples will work if your class consists of Python default types  like strings  integers  lists  datetimes  etc  The jsons lib will require type hints for custom types though

User · Answer

I have written a small  de serialization framework called any2any that helps doing complex transformations between two Python types   In your case  I guess you want to transform from a dictionary  obtained with json loads  to an complex object response education   response name  with a nested structure response education id  etc     So that s exactly what this framework is made for  The documentation is not great yet  but by using any2any simple MappingToObject  you should be able to do that very easily  Please ask if you need help

User · Answer

There are multiple viable answers already  but there are some minor libraries made by individuals that can do the trick for most users  An example would be json2object  Given a defined class  it deserialises json data to your custom model  including custom attributes and child objects  Its use is very simple  An example from the library wiki   x000D   x000D  from json2object import jsontoobject as jo  class Student      def   init   self           self firstName   None         self lastName   None         self courses    Course       class Course      def   init   self  name           self name   name  data         firstName    James    lastName    Bond    courses           name    Fighting               name    Shooting                model   Student   result   jo deserialize data  model  print result courses 0  name  x000D   x000D   x000D

User · Answer

If you are using python 3 6   you can use marshmallow-dataclass  Contrarily to all the solutions listed above  it is both simple  and type safe  from marshmallow dataclass import dataclass   dataclass class User      name  str  user   User Schema   load   quot name quot    quot Ramirez quot

User · Answer

If you re using Python 3 6 or newer  you could have a look at squema - a lightweight module for statically typed data structures  It makes your code easy to read while at the same time providing simple data validation  conversion and serialization without extra work  You can think of it as a more sophisticated and opinionated alternative to namedtuples and dataclasses  Here s how you could use it   from uuid import UUID from squema import Squema   class FbApiUser Squema       id  UUID     age  int     name  str      def save self           pass   user   FbApiUser   json loads response   user save

User · Answer

You can use  x   Map json loads response   x   class     MyClass   where  class Map dict       def   init   self   args    kwargs           super Map  self    init    args    kwargs          for arg in args              if isinstance arg  dict                   for k  v in arg iteritems                        self k    v                     if isinstance v  dict                           self k    Map v           if kwargs                for python 3 use kwargs items               for k  v in kwargs iteritems                    self k    v                 if isinstance v  dict                       self k    Map v       def   getattr   self  attr           return self get attr       def   setattr   self  key  value           self   setitem   key  value       def   setitem   self  key  value           super Map  self    setitem   key  value          self   dict   update  key  value        def   delattr   self  item           self   delitem   item       def   delitem   self  key           super Map  self    delitem   key          del self   dict   key    For a generic  future-proof solution

User · Answer

You could try this   class User object       def   init   self  name  username   args    kwargs           self name   name         self username   username  import json j   json loads your json  u   User   j    Just create a new Object  and pass the parameters as a map

User · Answer

UPDATE With Python3  you can do it in one line  using SimpleNamespace and object hook  import json from types import SimpleNamespace  data      quot name quot    quot John Smith quot    quot hometown quot     quot name quot    quot New York quot    quot id quot   123       Parse JSON into an object with attributes corresponding to dict keys  x   json loads data  object hook lambda d  SimpleNamespace   d   print x name  x hometown name  x hometown id   OLD ANSWER  Python2  In Python2  you can do it in one line  using namedtuple and object hook  but it s very slow with many nested objects   import json from collections import namedtuple  data      quot name quot    quot John Smith quot    quot hometown quot     quot name quot    quot New York quot    quot id quot   123       Parse JSON into an object with attributes corresponding to dict keys  x   json loads data  object hook lambda d  namedtuple  X   d keys     d values     print x name  x hometown name  x hometown id  or  to reuse this easily  def  json object hook d   return namedtuple  X   d keys     d values    def json2obj data   return json loads data  object hook  json object hook   x   json2obj data   If you want it to handle keys that aren t good attribute names  check out namedtuple s rename parameter

User · Answer

Since noone provided an answer quite like mine  I am going to post it here   It is a robust class that can easily convert back and forth between json str and dict that I have copied from my answer to another question   import json  class PyJSON object       def   init   self  d           if type d  is str              d   json loads d           self from dict d       def from dict self  d           self   dict                for key  value in d items                if type value  is dict                  value   PyJSON value              self   dict   key    value      def to dict self           d              for key  value in self   dict   items                if type value  is PyJSON                  value   value to dict               d key    value         return d      def   repr   self           return str self to dict         def   setitem   self  key  value           self   dict   key    value      def   getitem   self  key           return self   dict   key   json str          json string         py json   PyJSON json str

User · Answer

The lightest solution I think is import json from typing import NamedTuple   j      quot name quot   quot      quot   quot age quot  37  quot mother quot    quot name quot   quot       quot   quot age quot  58   quot children quot    quot      quot   quot       quot   quot      quot    quot married quot   true            quot dog quot  null      class PersonNameAge NamedTuple       name  str     age  int   class UserInfo NamedTuple       name  str     age  int     mother  PersonNameAge     children  list     married  bool     dog  str   j   json loads  j  u   UserInfo   j   print u name  u age  u mother  u children  u married  u dog    gt  gt  gt  Ivan 37   name    Olga    age   58    Mary    Igor    Jane   True None

User · Answer

Improving the lovasoa s very good answer     If you are using python 3 6   you can use  pip install marshmallow-enum and pip install marshmallow-dataclass    Its simple and type safe     You can transform your class in a string-json and vice-versa     From Object to String Json         from marshmallow dataclass import dataclass     user   User  Danilo   50   RedBull  15 OrderStatus CREATED      user json   User Schema   dumps user      user json str   user json data   From String Json to Object         json str      name   Danilo    orderId   50    productName   RedBull    quantity  15   status   Created        user  err   User Schema   loads json str      print user flush True    Class definitions     class OrderStatus Enum       CREATED    Created      PENDING    Pending      CONFIRMED    Confirmed      FAILED    Failed    dataclass class User      def   init   self  name  orderId  productName  quantity  status           self name   name         self orderId   orderId         self productName   productName         self quantity   quantity         self status   status      name  str     orderId  str     productName  str     quantity  int     status  OrderStatus

User · Answer

dacite may also be a solution for you  it supports following features   nested structures  basic  types checking optional fields  i e  typing Optional  unions forward references collections custom type hooks  https   pypi org project dacite  from dataclasses import dataclass from dacite import from dict    dataclass class User      name  str     age  int     is active  bool   data          name    John        age   30       is active   True     user   from dict data class User  data data   assert user    User name  John   age 30  is active True

User · Answer

Use the json module  new in Python 2 6  or the simplejson module which is almost always installed

User · Answer

Check out the section titled Specializing JSON object decoding in the json module documentation  You can use that to decode a JSON object into a specific Python type   Here s an example   class User object       def   init   self  name  username           self name   name         self username   username  import json def object decoder obj       if    type    in obj and obj    type         User           return User obj  name    obj  username        return obj  json loads      type      User    name    John Smith    username    jsmith                object hook object decoder   print type User     - gt   lt type  type  gt    Update  If you want to access data in a dictionary via the json module do this   user   json loads      type      User    name    John Smith    username    jsmith     print user  name   print user  username     Just like a regular dictionary

User · Answer

I was searching for a solution that worked with recordclass RecordClass  supports nested objects and works for both json serialization and json deserialization    Expanding on DS s answer  and expanding on solution from BeneStr  I came up with the following that seems to work   Code   import json import recordclass  class NestedRec recordclass RecordClass       a   int   0     b   int   0  class ExampleRec recordclass RecordClass       x   int         None     y   int         None     nested   NestedRec   NestedRec    class JsonSerializer       staticmethod     def dumps obj  ensure ascii True  indent None  sort keys False           return json dumps obj  default JsonSerializer   obj to dict  ensure ascii ensure ascii  indent indent  sort keys sort keys        staticmethod     def loads s  klass           return JsonSerializer   dict to obj klass  json loads s         staticmethod     def   obj to dict obj           if hasattr obj    asdict                return obj  asdict           else              return json JSONEncoder   default obj        staticmethod     def   dict to obj klass  s dict           kwargs                 key   JsonSerializer   dict to obj cls  s dict key   if hasattr cls   asdict   else s dict key                    for key cls in klass   annotations   items                         if s dict is not None and key in s dict                   return klass   kwargs    Usage   example 0   ExampleRec x   10  y   20  nested   NestedRec  a   30  b   40       Serialize to JSON  json str   JsonSerializer dumps example 0  print json str         x   10      y   20      nested           a   30        b   40            Deserialize from JSON example 1   JsonSerializer loads json str  ExampleRec  example 1 x    1 example 1 y    1 example 1 nested a    1 example 1 nested b    1  json str   JsonSerializer dumps example 1  print json str         x   11      y   21      nested           a   31        b   41

User · Answer

Here s a quick and dirty json pickle alternative  import json  class User      def   init   self  name  username           self name   name         self username   username      def to json self           return json dumps self   dict          classmethod     def from json cls  json str           json dict   json loads json str          return cls   json dict     example usage User  tbrown    Tom Brown   to json   User from json User  tbrown    Tom Brown   to json    to json

User · Answer

Expanding on DS s answer a bit  if you need the object to be mutable  which namedtuple is not   you can use the recordclass library instead of namedtuple   import json from recordclass import recordclass  data      name    John Smith    hometown     name    New York    id   123       Parse into a mutable object x   json loads data  object hook lambda d  recordclass  X   d keys     d values       The modified object can then be converted back to json very easily using simplejson   x name    John Doe  new json   simplejson dumps x

User · Answer

Python3 x  The best aproach I could reach with my knowledge was this  Note that this code treat set   too  This approach is generic just needing the extension of class  in the second example   Note that I m just doing it to files  but it s easy to modify the behavior to your taste   However this is a CoDec   With a little more work you can construct your class in other ways  I assume a default constructor to instance it  then I update the class dict   import json import collections   class JsonClassSerializable json JSONEncoder        REGISTERED CLASS           def register ctype           JsonClassSerializable REGISTERED CLASS ctype   name      ctype      def default self  obj           if isinstance obj  collections Set               return dict  set object list obj           if isinstance obj  JsonClassSerializable               jclass                  jclass  name     type obj    name               jclass  dict     obj   dict               return dict  class object jclass          else              return json JSONEncoder default self  obj       def json to class self  dct           if   set object  in dct              return set dct   set object            elif   class object  in dct              cclass   dct   class object               cclass name   cclass  name               if cclass name not in self REGISTERED CLASS                  raise RuntimeError                       Class    not registered in JSON Parser                       format cclass  name                                  instance   self REGISTERED CLASS cclass name                instance   dict     cclass  dict               return instance         return dct      def encode  self  file           with open file   w   as outfile              json dump                  self   dict    outfile                  cls JsonClassSerializable                  indent 4                  sort keys True                    def decode  self  file           try              with open file   r   as infile                  self   dict     json load                      infile                      object hook self json to class                           except FileNotFoundError              print  Persistence load failed                           do not exists  format file                        class C JsonClassSerializable        def   init   self           self mill    s    JsonClassSerializable register C    class B JsonClassSerializable        def   init   self           self a   1230         self c   C     JsonClassSerializable register B    class A JsonClassSerializable        def   init   self           self a   1         self b    1  2          self c   B    JsonClassSerializable register A   A   encode   test   b   A   b decode   test   print b a  print b b  print b c a      Edit  With some more of research I found a way to generalize without the need of the SUPERCLASS register method call  using a metaclass  import json import collections  REGISTERED CLASS       class MetaSerializable type        def   call   cls   args    kwargs           if cls   name   not in REGISTERED CLASS              REGISTERED CLASS cls   name      cls         return super MetaSerializable  cls    call    args    kwargs    class JsonClassSerializable json JSONEncoder  metaclass MetaSerializable        def default self  obj           if isinstance obj  collections Set               return dict  set object list obj           if isinstance obj  JsonClassSerializable               jclass                  jclass  name     type obj    name               jclass  dict     obj   dict               return dict  class object jclass          else              return json JSONEncoder default self  obj       def json to class self  dct           if   set object  in dct              return set dct   set object            elif   class object  in dct              cclass   dct   class object               cclass name   cclass  name               if cclass name not in REGISTERED CLASS                  raise RuntimeError                       Class    not registered in JSON Parser                       format cclass  name                                  instance   REGISTERED CLASS cclass name                instance   dict     cclass  dict               return instance         return dct      def encode  self  file           with open file   w   as outfile              json dump                  self   dict    outfile                  cls JsonClassSerializable                  indent 4                  sort keys True                    def decode  self  file           try              with open file   r   as infile                  self   dict     json load                      infile                      object hook self json to class                           except FileNotFoundError              print  Persistence load failed                           do not exists  format file                        class C JsonClassSerializable        def   init   self           self mill    s    class B JsonClassSerializable        def   init   self           self a   1230         self c   C     class A JsonClassSerializable        def   init   self           self a   1         self b    1  2          self c   B     A   encode   test   b   A   b decode   test   print b a    1 print b b     1  2  print b c a    1230 print b c c mill    s

[python] How to convert JSON data into a Python object

Examples related to python

Examples related to json

Examples related to django