JSON character encoding - is UTF-8 well-supported by browsers or should I use numeric escape sequences

Question

I am writing a webservice that uses json to represent its resources  and I am a bit stuck thinking about the best way to encode the json  Reading the json rfc  http   www ietf org rfc rfc4627 txt  it is clear that the preferred encoding is utf-8  But the rfc also describes a string escaping mechanism for specifying characters  I assume this would generally be used to escape non-ascii characters  thereby making the resulting utf-8 valid ascii   So let s say I have a json string that contains unicode characters  code-points  that are non-ascii  Should my webservice just utf-8 encoding that and return it  or should it escape all those non-ascii characters and return pure ascii   I d like browsers to be able to execute the results using jsonp or eval  Does that effect the decision  My knowledge of various browser s javascript support for utf-8 is lacking   EDIT  I wanted to clarify that my main concern about how to encode the results is really about browser handling of the results  What I ve read indicates that browsers may be sensitive to the encoding when using JSONP in particular  I haven t found any really good info on the subject  so I ll have to start doing some testing to see what happens  Ideally I d like to only escape those few characters that are required and just utf-8 encode the results

User · Accepted Answer

The JSON spec requires UTF-8 support by decoders  As a result  all JSON decoders can handle UTF-8 just as well as they can handle the numeric escape sequences   This is also the case for Javascript interpreters  which means JSONP will handle the UTF-8 encoded JSON as well   The ability for JSON encoders to use the numeric escape sequences instead just offers you more choice   One reason you may choose the numeric escape sequences would be if a transport mechanism in between your encoder and the intended decoder is not binary-safe   Another reason you may want to use numeric escape sequences is to prevent certain characters appearing in the stream  such as  lt    amp  and    which may be interpreted as HTML sequences if the JSON code is placed without escaping into HTML or a browser wrongly interprets it as HTML  This can be a defence against HTML injection or cross-site scripting  note  some characters MUST be escaped in JSON  including   and      Some frameworks  including PHP s implementation of JSON  always do the numeric escape sequences on the encoder side for any character outside of ASCII   This is intended for maximum compatibility with limited transport mechanisms and the like   However  this should not be interpreted as an indication that JSON decoders have a problem with UTF-8   So  I guess you just could decide which to use like this    Just use UTF-8  unless your method of storage or transport between encoder and decoder is not binary-safe  Otherwise  use the numeric escape sequences

User · Answer

I had a similar problem with    char    I think the comment  it s possible that the text you re feeding it isn t UTF-8  is probably close to the mark here  I have a feeling the default collation in my instance was something else until I realized and changed to utf8    problem is the data was already there  so not sure if it converted the data or not when i changed it  displays fine in mysql workbench  End result is that php will not json encode the data  just returns false  Doesn t matter what browser you use as its the server causing my issue  php will not parse the data to utf8 if this char is present  Like i say not sure if it is due to converting the schema to utf8 after data was present or just a php bug  In this case use json encode utf8 encode  string

User · Answer

I had a problem there  When I JSON encode a string with a character like       every browsers will return the same       except IE which will return   u00e9    Then with PHP json decode    it will fail if it find       so for Firefox  Opera  Safari and Chrome  I ve to call utf8 encode   before json decode     Note   with my tests  IE and Firefox are using their native JSON object  others browsers are using json2 js

User · Answer

I was facing the same problem  It works for me  Please check this   json encode  array JSON UNESCAPED UNICODE

User · Answer

Reading the json rfc  http   www ietf org rfc rfc4627 txt  it is clear that the preferred encoding is utf-8     FYI  RFC 4627 is no longer the official JSON spec  It was obsoleted in 2014 by RFC 7159  which was then obsoleted in 2017 by RFC 8259  which is the current spec   RFC 8259 states      8 1   Character Encoding      JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8  RFC3629        Previous specifications of JSON have not required the use of UTF-8 when transmitting JSON text   However  the vast majority of JSON-based software implementations have chosen to use the UTF-8 encoding  to the extent that it is the only encoding that achieves interoperability       Implementations MUST NOT add a byte order mark  U FEFF  to the beginning of a networked-transmitted JSON text   In the interests of interoperability  implementations that parse JSON texts MAY ignore the presence of a byte order mark rather than treating it as an error

User · Answer

ASCII isn t in it any more   Using UTF-8 encoding means that you aren t using ASCII encoding   What you should use the escaping mechanism for is what the RFC says      All Unicode characters may be placed   within the    quotation marks except   for the characters that must be   escaped     quotation mark  reverse   solidus  and the control characters    U 0000    through U 001F

[web-services] JSON character encoding - is UTF-8 well-supported by browsers or should I use numeric escape sequences?

Examples related to web-services

Examples related to json

Examples related to unicode

Examples related to utf-8