I want to send a datetime.datetime object in serialized form from Python using JSON and de-serialize in JavaScript using JSON. What is the best way to do this?
This question is related to
javascript
python
json
I've worked it out.
Let's say you have a Python datetime object, d, created with datetime.now(). Its value is:
datetime.datetime(2011, 5, 25, 13, 34, 5, 787000)
You can serialize it to JSON as an ISO 8601 datetime string:
import json
json.dumps(d.isoformat())
The example datetime object would be serialized as:
'"2011-05-25T13:34:05.787000"'
This value, once received in the Javascript layer, can construct a Date object:
var d = new Date("2011-05-25T13:34:05.787000");
As of Javascript 1.8.5, Date objects have a toJSON method, which returns a string in a standard format. To serialize the above Javascript object back to JSON, therefore, the command would be:
d.toJSON()
Which would give you:
'2011-05-25T20:34:05.787Z'
This string, once received in Python, could be deserialized back to a datetime object:
datetime.strptime('2011-05-25T20:34:05.787Z', '%Y-%m-%dT%H:%M:%S.%fZ')
This results in the following datetime object, which is the same one you started with and therefore correct:
datetime.datetime(2011, 5, 25, 20, 34, 5, 787000)
My advice is to use a library. There are several available at pypi.org.
I use this one, it it works good: https://pypi.python.org/pypi/asjson
On python side:
import time, json
from datetime import datetime as dt
your_date = dt.now()
data = json.dumps(time.mktime(your_date.timetuple())*1000)
return data # data send to javascript
On javascript side:
var your_date = new Date(data)
where data is result from python
Late in the game... :)
A very simple solution is to patch the json module default. For example:
import json
import datetime
json.JSONEncoder.default = lambda self,obj: (obj.isoformat() if isinstance(obj, datetime.datetime) else None)
Now, you can use json.dumps() as if it had always supported datetime...
json.dumps({'created':datetime.datetime.now()})
This makes sense if you require this extension to the json module to always kick in and wish to not change the way you or others use json serialization (either in existing code or not).
Note that some may consider patching libraries in that way as bad practice. Special care need to be taken in case you may wish to extend your application in more than one way - is such a case, I suggest to use the solution by ramen or JT and choose the proper json extension in each case.
Using json, you can subclass JSONEncoder and override the default() method to provide your own custom serializers:
import json
import datetime
class DateTimeJSONEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, datetime.datetime):
return obj.isoformat()
else:
return super(DateTimeJSONEncoder, self).default(obj)
Then, you can call it like this:
>>> DateTimeJSONEncoder().encode([datetime.datetime.now()])
'["2010-06-15T14:42:28"]'
On python side:
import time, json
from datetime import datetime as dt
your_date = dt.now()
data = json.dumps(time.mktime(your_date.timetuple())*1000)
return data # data send to javascript
On javascript side:
var your_date = new Date(data)
where data is result from python
Here's a fairly complete solution for recursively encoding and decoding datetime.datetime and datetime.date objects using the standard library json module. This needs Python >= 2.6 since the %f format code in the datetime.datetime.strptime() format string is only supported in since then. For Python 2.5 support, drop the %f and strip the microseconds from the ISO date string before trying to convert it, but you'll loose microseconds precision, of course. For interoperability with ISO date strings from other sources, which may include a time zone name or UTC offset, you may also need to strip some parts of the date string before the conversion. For a complete parser for ISO date strings (and many other date formats) see the third-party dateutil module.
Decoding only works when the ISO date strings are values in a JavaScript literal object notation or in nested structures within an object. ISO date strings, which are items of a top-level array will not be decoded.
I.e. this works:
date = datetime.datetime.now()
>>> json = dumps(dict(foo='bar', innerdict=dict(date=date)))
>>> json
'{"innerdict": {"date": "2010-07-15T13:16:38.365579"}, "foo": "bar"}'
>>> loads(json)
{u'innerdict': {u'date': datetime.datetime(2010, 7, 15, 13, 16, 38, 365579)},
u'foo': u'bar'}
And this too:
>>> json = dumps(['foo', 'bar', dict(date=date)])
>>> json
'["foo", "bar", {"date": "2010-07-15T13:16:38.365579"}]'
>>> loads(json)
[u'foo', u'bar', {u'date': datetime.datetime(2010, 7, 15, 13, 16, 38, 365579)}]
But this doesn't work as expected:
>>> json = dumps(['foo', 'bar', date])
>>> json
'["foo", "bar", "2010-07-15T13:16:38.365579"]'
>>> loads(json)
[u'foo', u'bar', u'2010-07-15T13:16:38.365579']
Here's the code:
__all__ = ['dumps', 'loads']
import datetime
try:
import json
except ImportError:
import simplejson as json
class JSONDateTimeEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, (datetime.date, datetime.datetime)):
return obj.isoformat()
else:
return json.JSONEncoder.default(self, obj)
def datetime_decoder(d):
if isinstance(d, list):
pairs = enumerate(d)
elif isinstance(d, dict):
pairs = d.items()
result = []
for k,v in pairs:
if isinstance(v, basestring):
try:
# The %f format code is only supported in Python >= 2.6.
# For Python <= 2.5 strip off microseconds
# v = datetime.datetime.strptime(v.rsplit('.', 1)[0],
# '%Y-%m-%dT%H:%M:%S')
v = datetime.datetime.strptime(v, '%Y-%m-%dT%H:%M:%S.%f')
except ValueError:
try:
v = datetime.datetime.strptime(v, '%Y-%m-%d').date()
except ValueError:
pass
elif isinstance(v, (dict, list)):
v = datetime_decoder(v)
result.append((k, v))
if isinstance(d, list):
return [x[1] for x in result]
elif isinstance(d, dict):
return dict(result)
def dumps(obj):
return json.dumps(obj, cls=JSONDateTimeEncoder)
def loads(obj):
return json.loads(obj, object_hook=datetime_decoder)
if __name__ == '__main__':
mytimestamp = datetime.datetime.utcnow()
mydate = datetime.date.today()
data = dict(
foo = 42,
bar = [mytimestamp, mydate],
date = mydate,
timestamp = mytimestamp,
struct = dict(
date2 = mydate,
timestamp2 = mytimestamp
)
)
print repr(data)
jsonstring = dumps(data)
print jsonstring
print repr(loads(jsonstring))
If you're certain that only Javascript will be consuming the JSON, I prefer to pass Javascript Date objects directly.
The ctime() method on datetime objects will return a string that the Javascript Date object can understand.
import datetime
date = datetime.datetime.today()
json = '{"mydate":new Date("%s")}' % date.ctime()
Javascript will happily use that as an object literal, and you've got your Date object built right in.
Apparently The “right” JSON (well JavaScript) date format is 2012-04-23T18:25:43.511Z - UTC and "Z". Without this JavaScript will use the web browser's local timezone when creating a Date() object from the string.
For a "naive" time (what Python calls a time with no timezone and this assumes is local) the below will force local timezone so that it can then be correctly converted to UTC:
def default(obj):
if hasattr(obj, "json") and callable(getattr(obj, "json")):
return obj.json()
if hasattr(obj, "isoformat") and callable(getattr(obj, "isoformat")):
# date/time objects
if not obj.utcoffset():
# add local timezone to "naive" local time
# https://stackoverflow.com/questions/2720319/python-figure-out-local-timezone
tzinfo = datetime.now(timezone.utc).astimezone().tzinfo
obj = obj.replace(tzinfo=tzinfo)
# convert to UTC
obj = obj.astimezone(timezone.utc)
# strip the UTC offset
obj = obj.replace(tzinfo=None)
return obj.isoformat() + "Z"
elif hasattr(obj, "__str__") and callable(getattr(obj, "__str__")):
return str(obj)
else:
print("obj:", obj)
raise TypeError(obj)
def dump(j, io):
json.dump(j, io, indent=2, default=default)
why is this so hard.
For cross-language projects, I found out that strings containing RfC 3339 dates are the best way to go. An RfC 3339 date looks like this:
1985-04-12T23:20:50.52Z
I think most of the format is obvious. The only somewhat unusual thing may be the "Z" at the end. It stands for GMT/UTC. You could also add a timezone offset like +02:00 for CEST (Germany in summer). I personally prefer to keep everything in UTC until it is displayed.
For displaying, comparisons and storage you can leave it in string format across all languages. If you need the date for calculations easy to convert it back to a native date object in most language.
So generate the JSON like this:
json.dump(datetime.now().strftime('%Y-%m-%dT%H:%M:%SZ'))
Unfortunately, Javascript's Date constructor doesn't accept RfC 3339 strings but there are many parsers available on the Internet.
huTools.hujson tries to handle the most common encoding issues you might come across in Python code including date/datetime objects while handling timezones correctly.
Not much to add to the community wiki answer, except for timestamp!
Javascript uses the following format:
new Date().toJSON() // "2016-01-08T19:00:00.123Z"
Python side (for the json.dumps handler, see the other answers):
>>> from datetime import datetime
>>> d = datetime.strptime('2016-01-08T19:00:00.123Z', '%Y-%m-%dT%H:%M:%S.%fZ')
>>> d
datetime.datetime(2016, 1, 8, 19, 0, 0, 123000)
>>> d.isoformat() + 'Z'
'2016-01-08T19:00:00.123000Z'
If you leave that Z out, frontend frameworks like angular can not display the date in browser-local timezone:
> $filter('date')('2016-01-08T19:00:00.123000Z', 'yyyy-MM-dd HH:mm:ss')
"2016-01-08 20:00:00"
> $filter('date')('2016-01-08T19:00:00.123000', 'yyyy-MM-dd HH:mm:ss')
"2016-01-08 19:00:00"
Here's a fairly complete solution for recursively encoding and decoding datetime.datetime and datetime.date objects using the standard library json module. This needs Python >= 2.6 since the %f format code in the datetime.datetime.strptime() format string is only supported in since then. For Python 2.5 support, drop the %f and strip the microseconds from the ISO date string before trying to convert it, but you'll loose microseconds precision, of course. For interoperability with ISO date strings from other sources, which may include a time zone name or UTC offset, you may also need to strip some parts of the date string before the conversion. For a complete parser for ISO date strings (and many other date formats) see the third-party dateutil module.
Decoding only works when the ISO date strings are values in a JavaScript literal object notation or in nested structures within an object. ISO date strings, which are items of a top-level array will not be decoded.
I.e. this works:
date = datetime.datetime.now()
>>> json = dumps(dict(foo='bar', innerdict=dict(date=date)))
>>> json
'{"innerdict": {"date": "2010-07-15T13:16:38.365579"}, "foo": "bar"}'
>>> loads(json)
{u'innerdict': {u'date': datetime.datetime(2010, 7, 15, 13, 16, 38, 365579)},
u'foo': u'bar'}
And this too:
>>> json = dumps(['foo', 'bar', dict(date=date)])
>>> json
'["foo", "bar", {"date": "2010-07-15T13:16:38.365579"}]'
>>> loads(json)
[u'foo', u'bar', {u'date': datetime.datetime(2010, 7, 15, 13, 16, 38, 365579)}]
But this doesn't work as expected:
>>> json = dumps(['foo', 'bar', date])
>>> json
'["foo", "bar", "2010-07-15T13:16:38.365579"]'
>>> loads(json)
[u'foo', u'bar', u'2010-07-15T13:16:38.365579']
Here's the code:
__all__ = ['dumps', 'loads']
import datetime
try:
import json
except ImportError:
import simplejson as json
class JSONDateTimeEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, (datetime.date, datetime.datetime)):
return obj.isoformat()
else:
return json.JSONEncoder.default(self, obj)
def datetime_decoder(d):
if isinstance(d, list):
pairs = enumerate(d)
elif isinstance(d, dict):
pairs = d.items()
result = []
for k,v in pairs:
if isinstance(v, basestring):
try:
# The %f format code is only supported in Python >= 2.6.
# For Python <= 2.5 strip off microseconds
# v = datetime.datetime.strptime(v.rsplit('.', 1)[0],
# '%Y-%m-%dT%H:%M:%S')
v = datetime.datetime.strptime(v, '%Y-%m-%dT%H:%M:%S.%f')
except ValueError:
try:
v = datetime.datetime.strptime(v, '%Y-%m-%d').date()
except ValueError:
pass
elif isinstance(v, (dict, list)):
v = datetime_decoder(v)
result.append((k, v))
if isinstance(d, list):
return [x[1] for x in result]
elif isinstance(d, dict):
return dict(result)
def dumps(obj):
return json.dumps(obj, cls=JSONDateTimeEncoder)
def loads(obj):
return json.loads(obj, object_hook=datetime_decoder)
if __name__ == '__main__':
mytimestamp = datetime.datetime.utcnow()
mydate = datetime.date.today()
data = dict(
foo = 42,
bar = [mytimestamp, mydate],
date = mydate,
timestamp = mytimestamp,
struct = dict(
date2 = mydate,
timestamp2 = mytimestamp
)
)
print repr(data)
jsonstring = dumps(data)
print jsonstring
print repr(loads(jsonstring))
You can add the 'default' parameter to json.dumps to handle this:
date_handler = lambda obj: (
obj.isoformat()
if isinstance(obj, (datetime.datetime, datetime.date))
else None
)
json.dumps(datetime.datetime.now(), default=date_handler)
'"2010-04-20T20:08:21.634121"'
Which is ISO 8601 format.
A more comprehensive default handler function:
def handler(obj):
if hasattr(obj, 'isoformat'):
return obj.isoformat()
elif isinstance(obj, ...):
return ...
else:
raise TypeError, 'Object of type %s with value of %s is not JSON serializable' % (type(obj), repr(obj))
Update: Added output of type as well as value.
Update: Also handle date
Late in the game... :)
A very simple solution is to patch the json module default. For example:
import json
import datetime
json.JSONEncoder.default = lambda self,obj: (obj.isoformat() if isinstance(obj, datetime.datetime) else None)
Now, you can use json.dumps() as if it had always supported datetime...
json.dumps({'created':datetime.datetime.now()})
This makes sense if you require this extension to the json module to always kick in and wish to not change the way you or others use json serialization (either in existing code or not).
Note that some may consider patching libraries in that way as bad practice. Special care need to be taken in case you may wish to extend your application in more than one way - is such a case, I suggest to use the solution by ramen or JT and choose the proper json extension in each case.
If you're certain that only Javascript will be consuming the JSON, I prefer to pass Javascript Date objects directly.
The ctime() method on datetime objects will return a string that the Javascript Date object can understand.
import datetime
date = datetime.datetime.today()
json = '{"mydate":new Date("%s")}' % date.ctime()
Javascript will happily use that as an object literal, and you've got your Date object built right in.
Apparently The “right” JSON (well JavaScript) date format is 2012-04-23T18:25:43.511Z - UTC and "Z". Without this JavaScript will use the web browser's local timezone when creating a Date() object from the string.
For a "naive" time (what Python calls a time with no timezone and this assumes is local) the below will force local timezone so that it can then be correctly converted to UTC:
def default(obj):
if hasattr(obj, "json") and callable(getattr(obj, "json")):
return obj.json()
if hasattr(obj, "isoformat") and callable(getattr(obj, "isoformat")):
# date/time objects
if not obj.utcoffset():
# add local timezone to "naive" local time
# https://stackoverflow.com/questions/2720319/python-figure-out-local-timezone
tzinfo = datetime.now(timezone.utc).astimezone().tzinfo
obj = obj.replace(tzinfo=tzinfo)
# convert to UTC
obj = obj.astimezone(timezone.utc)
# strip the UTC offset
obj = obj.replace(tzinfo=None)
return obj.isoformat() + "Z"
elif hasattr(obj, "__str__") and callable(getattr(obj, "__str__")):
return str(obj)
else:
print("obj:", obj)
raise TypeError(obj)
def dump(j, io):
json.dump(j, io, indent=2, default=default)
why is this so hard.
You can add the 'default' parameter to json.dumps to handle this:
date_handler = lambda obj: (
obj.isoformat()
if isinstance(obj, (datetime.datetime, datetime.date))
else None
)
json.dumps(datetime.datetime.now(), default=date_handler)
'"2010-04-20T20:08:21.634121"'
Which is ISO 8601 format.
A more comprehensive default handler function:
def handler(obj):
if hasattr(obj, 'isoformat'):
return obj.isoformat()
elif isinstance(obj, ...):
return ...
else:
raise TypeError, 'Object of type %s with value of %s is not JSON serializable' % (type(obj), repr(obj))
Update: Added output of type as well as value.
Update: Also handle date
My advice is to use a library. There are several available at pypi.org.
I use this one, it it works good: https://pypi.python.org/pypi/asjson
Using json, you can subclass JSONEncoder and override the default() method to provide your own custom serializers:
import json
import datetime
class DateTimeJSONEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, datetime.datetime):
return obj.isoformat()
else:
return super(DateTimeJSONEncoder, self).default(obj)
Then, you can call it like this:
>>> DateTimeJSONEncoder().encode([datetime.datetime.now()])
'["2010-06-15T14:42:28"]'
If you're certain that only Javascript will be consuming the JSON, I prefer to pass Javascript Date objects directly.
The ctime() method on datetime objects will return a string that the Javascript Date object can understand.
import datetime
date = datetime.datetime.today()
json = '{"mydate":new Date("%s")}' % date.ctime()
Javascript will happily use that as an object literal, and you've got your Date object built right in.
I've worked it out.
Let's say you have a Python datetime object, d, created with datetime.now(). Its value is:
datetime.datetime(2011, 5, 25, 13, 34, 5, 787000)
You can serialize it to JSON as an ISO 8601 datetime string:
import json
json.dumps(d.isoformat())
The example datetime object would be serialized as:
'"2011-05-25T13:34:05.787000"'
This value, once received in the Javascript layer, can construct a Date object:
var d = new Date("2011-05-25T13:34:05.787000");
As of Javascript 1.8.5, Date objects have a toJSON method, which returns a string in a standard format. To serialize the above Javascript object back to JSON, therefore, the command would be:
d.toJSON()
Which would give you:
'2011-05-25T20:34:05.787Z'
This string, once received in Python, could be deserialized back to a datetime object:
datetime.strptime('2011-05-25T20:34:05.787Z', '%Y-%m-%dT%H:%M:%S.%fZ')
This results in the following datetime object, which is the same one you started with and therefore correct:
datetime.datetime(2011, 5, 25, 20, 34, 5, 787000)
For cross-language projects, I found out that strings containing RfC 3339 dates are the best way to go. An RfC 3339 date looks like this:
1985-04-12T23:20:50.52Z
I think most of the format is obvious. The only somewhat unusual thing may be the "Z" at the end. It stands for GMT/UTC. You could also add a timezone offset like +02:00 for CEST (Germany in summer). I personally prefer to keep everything in UTC until it is displayed.
For displaying, comparisons and storage you can leave it in string format across all languages. If you need the date for calculations easy to convert it back to a native date object in most language.
So generate the JSON like this:
json.dump(datetime.now().strftime('%Y-%m-%dT%H:%M:%SZ'))
Unfortunately, Javascript's Date constructor doesn't accept RfC 3339 strings but there are many parsers available on the Internet.
huTools.hujson tries to handle the most common encoding issues you might come across in Python code including date/datetime objects while handling timezones correctly.
Not much to add to the community wiki answer, except for timestamp!
Javascript uses the following format:
new Date().toJSON() // "2016-01-08T19:00:00.123Z"
Python side (for the json.dumps handler, see the other answers):
>>> from datetime import datetime
>>> d = datetime.strptime('2016-01-08T19:00:00.123Z', '%Y-%m-%dT%H:%M:%S.%fZ')
>>> d
datetime.datetime(2016, 1, 8, 19, 0, 0, 123000)
>>> d.isoformat() + 'Z'
'2016-01-08T19:00:00.123000Z'
If you leave that Z out, frontend frameworks like angular can not display the date in browser-local timezone:
> $filter('date')('2016-01-08T19:00:00.123000Z', 'yyyy-MM-dd HH:mm:ss')
"2016-01-08 20:00:00"
> $filter('date')('2016-01-08T19:00:00.123000', 'yyyy-MM-dd HH:mm:ss')
"2016-01-08 19:00:00"
If you're certain that only Javascript will be consuming the JSON, I prefer to pass Javascript Date objects directly.
The ctime() method on datetime objects will return a string that the Javascript Date object can understand.
import datetime
date = datetime.datetime.today()
json = '{"mydate":new Date("%s")}' % date.ctime()
Javascript will happily use that as an object literal, and you've got your Date object built right in.
For cross-language projects, I found out that strings containing RfC 3339 dates are the best way to go. An RfC 3339 date looks like this:
1985-04-12T23:20:50.52Z
I think most of the format is obvious. The only somewhat unusual thing may be the "Z" at the end. It stands for GMT/UTC. You could also add a timezone offset like +02:00 for CEST (Germany in summer). I personally prefer to keep everything in UTC until it is displayed.
For displaying, comparisons and storage you can leave it in string format across all languages. If you need the date for calculations easy to convert it back to a native date object in most language.
So generate the JSON like this:
json.dump(datetime.now().strftime('%Y-%m-%dT%H:%M:%SZ'))
Unfortunately, Javascript's Date constructor doesn't accept RfC 3339 strings but there are many parsers available on the Internet.
huTools.hujson tries to handle the most common encoding issues you might come across in Python code including date/datetime objects while handling timezones correctly.
For the Python to JavaScript date conversion, the date object needs to be in specific ISO format, i.e. ISO format or UNIX number. If the ISO format lacks some info, then you can convert to the Unix number with Date.parse first. Moreover, Date.parse works with React as well while new Date might trigger an exception.
In case you have a DateTime object without milliseconds, the following needs to be considered. :
var unixDate = Date.parse('2016-01-08T19:00:00')
var desiredDate = new Date(unixDate).toLocaleDateString();
The example date could equally be a variable in the result.data object after an API call.
For options to display the date in the desired format (e.g. to display long weekdays) check out the MDN doc.
Source: Stackoverflow.com