JSON Invalid UTF-8 middle byte

Question

This error happens when the  Jackson  this case  JSON engine tries to parse some JSON that is not encoded in UTF-8   How to tell the engine that it should expect something different from UTF-8  such as UTF-16   HttpHeaders requestHeaders   createSomeHeader    RestTemplate restTemplate   new RestTemplate    HttpEntity lt   gt  requestEntity   new HttpEntity lt Object gt  requestHeaders   String url    someurl  ResponseEntity lt MyObject   gt  arrayResponseEntity   restTemplate exchange url  HttpMethod GET  requestEntity  MyObject   class     error log   Caused by  org springframework http converter HttpMessageNotReadableException  Could not read JSON  Invalid UTF-8 middle byte 0x20 at  Source  org apache http conn EofSensorInputStream 44d397b0  line  92  column  42   nested exception is org codehaus jackson JsonParseException  Invalid UTF-8 middle byte 0x20 at  Source  org apache http conn EofSensorInputStream 44d397b0  line  92  column  42  at org springframework http converter json MappingJacksonHttpMessageConverter readInternal MappingJacksonHttpMessageConverter java 138  at org springframework http converter AbstractHttpMessageConverter read AbstractHttpMessageConverter java 154  at org springframework web client HttpMessageConverterExtractor extractData HttpMessageConverterExtractor java 74  at org springframework web client RestTemplate ResponseEntityResponseExtractor extractData RestTemplate java 622  at org springframework web client RestTemplate ResponseEntityResponseExtractor extractData RestTemplate java 608  at org springframework web client RestTemplate doExecute RestTemplate java 449  at org springframework web client RestTemplate execute RestTemplate java 404  at org springframework web client RestTemplate exchange RestTemplate java 380      4 more Caused by  org codehaus jackson JsonParseException  Invalid UTF-8 middle byte 0x20 at  Source  org apache http conn EofSensorInputStream 44d397b0  line  92  column  42  at org codehaus jackson JsonParser  constructError JsonParser java 1213  at org codehaus jackson impl JsonParserMinimalBase  reportError JsonParserMinimalBase java 375  at org codehaus jackson impl Utf8StreamParser  reportInvalidOther Utf8StreamParser java 2132  at org codehaus jackson impl Utf8StreamParser  reportInvalidOther Utf8StreamParser java 2139  at org codehaus jackson impl Utf8StreamParser  decodeUtf8 3fast Utf8StreamParser java 1962

User · Answer

client text protocol  POST http   127 0 0 1 bom create HTTP 1 1 Content-Type  application json User-Agent  PostmanRuntime 7 25 0 Accept      Postman-Token  50ecfbfe-741f-4a2b-a3d3-cdf162ada27f Host  127 0 0 1 Accept-Encoding  gzip  deflate  br Connection  keep-alive Content-Length  405       fwoid   1     list                  bomIndex    10001          desc     GH 1 25 13pin                  pn    084 0001 0036          preUse   1         type          -  PCB                         bomIndex    10002          desc        -               pn    Z 08 013 0051          preUse   1         type    E060A0302301              HTTP 1 1 200 OK Connection  keep-alive Vary  Origin Vary  Access-Control-Request-Method Vary  Access-Control-Request-Headers Content-Type  application json Date  Mon  01 Jun 2020 11 23 42 GMT Content-Length  40    code   0   message   BOM         a springboot Controller code as below    PostMapping   bom create    ApiOperation value      BOM    BusinessOperation module    BOM  methods      BOM   public JsonResult save  RequestBody BOMSaveQuery query        return bomService saveBomList query       when i debug on loopback interface it works ok  while deploy on internet server via bat command  i got an error  ServletInvocableHandlerMethod - Could not resolve parameter  0  in public XXXController save com h2 mes query BOMSaveQuery   JSON parse error  Invalid UTF-8 middle byte 0x3f  nested exception is com fasterxml jackson databind JsonMappingException  Invalid UTF-8 middle byte 0x3f  at  Source   PushbackInputStream   line  9  column  32   through reference chain  com h2 mes query BOMSaveQuery  list  - gt java util ArrayList 0 - gt com h2 mes vo BOMVO  type    2020-06-01 15 37 50 251 MES  XNIO-1 task-13  WARN  o s w s m s DefaultHandlerExceptionResolver - Resolved  org springframework http converter HttpMessageNotReadableException  JSON parse error  Invalid UTF-8 middle byte 0x3f  nested exception is com fasterxml jackson databind JsonMappingException  Invalid UTF-8 middle byte 0x3f  at  Source   PushbackInputStream   line  9  column  32   through reference chain  com h2 mes query BOMSaveQuery  list  - gt java util ArrayList 0 - gt com h2 mes vo BOMVO  type     2020-06-01 15 37 50 251 MES  XNIO-1 task-13  DEBUG o s web servlet DispatcherServlet - Completed 400 BAD REQUEST 2020-06-01 15 37 50 251 MES  XNIO-1 task-13  DEBUG o s web servlet DispatcherServlet -  ERROR  dispatch for POST   error   parameters      add a jvm arguement works for me  java -Dfile encoding UTF-8

User · Answer

I had this problem inconsistently between different platforms  as I got JSON as String from Mapper and did the writing myself  Sometimes it went into file as ansi and other times correctly as UTF8  I switched to   mapper writeValue file  data     letting Mapper do the file operations  and it started working fine

User · Answer

This awnser solved my problem  Below is a copy of it      Make sure to start you JVM with -Dfile encoding UTF-8  You JVM   defaults to the operating system charset   This is a JVM argument which could be added  for example  either to JBoss standalone or JBoss running from Eclipse   In my case  this problem happened isolatelly on only one of my team people s computer  All the others was working without this problem

User · Answer

JSON data must be encoded as UTF-8  UTF-16 or UTF-32  The JSON decoder can determine the encoding by examining the first four octets of the byte stream          00 00 00 xx  UTF-32BE        00 xx 00 xx  UTF-16BE        xx 00 00 00  UTF-32LE        xx 00 xx 00  UTF-16LE        xx xx xx xx  UTF-8   It sounds like the server is encoding data in some illegal encoding  ISO-8859-1  windows-1252  etc

User · Answer

On the off chance it may help others I ll share a related anecdote   I encountered this exact error  Invalid UTF-8 middle byte 0x3f  running a PowerShell script via the PowerShell Integrated Script Environment  ISE   The identical script  executed outside the ISE  works fine  The code uses the Confluence v3 and v5 x REST APIs and this error is logged on the Confluence v5 x server - presumably because the ISE somehow mucks with the request

User · Answer

I got this after saving the JSON file using Notepad2  so I had to open it with Notepad   and then say  Convert to UTF-8    Then it worked

User · Answer

I got this exception when in the Java Client Application I was serializing a JSON like this  String json   mapper writeValueAsString contentBean     and on the Server Side I was using Spring Boot as REST Endpoint  Exception was    nested exception is com fasterxml jackson databind JsonMappingException  Invalid UTF-8 start byte 0xaa  My problem was  that I was not setting the correct encoding on the HTTP Client   This solved my problem   updateRequest setHeader  Content-Type    application json charset UTF-8    StringEntity entity  new StringEntity json   UTF-8    updateRequest setEntity entity     Android set content type HttpPost

[json] JSON Invalid UTF-8 middle byte

Examples related to json

Examples related to jackson