How to repair a serialized string which has been corrupted by an incorrect byte count length

Question

I am using Hotaru CMS with the Image Upload plugin  I get this error if I try to attach an image to a post  otherwise there is no error      unserialize    function unserialize   Error at offset   The offending code  error points to line with                  Retrieve submission step data                param  key - empty when setting         return bool             public function loadSubmitData  h   key                        delete everything in this table older than 30 minutes           this- gt deleteTempData  h- gt db            if    key    return false              cleanKey   preg replace     a-z0-9         key           if  strcmp  key  cleanKey     0                return false            else                sql    SELECT tempdata value FROM     TABLE TEMPDATA     WHERE tempdata key    s ORDER BY tempdata updatedts DESC LIMIT 1                submitted data    h- gt db- gt get var  h- gt db- gt prepare  sql   key                  if   submitted data    return unserialize  submitted data     else   return false                         Data from the table  notice the end bit has the image info  I am not an expert in PHP so I was wondering what you guys gals might think   tempdata value   a 10  s 16  submit editorial  b 0 s 15  submit orig url  s 13  www bbc co uk  s 12  submit title  s 14  No title found  s 14  submit content  s 12  dnfsdkfjdfdf  s 15  submit category  i 2 s 11  submit tags  s 3  bbc  s 9  submit id  b 0 s 16  submit subscribe  i 0 s 15  submit comments  s 4  open  s 5  image  s 19  C fakepath100 jpg      Edit  I think I ve found the serialize bit                Save submission step data                return bool             public function saveSubmitData  h                   delete everything in this table older than 30 minutes           this- gt deleteTempData  h- gt db             sid   preg replace     a-z0-9   i       session id              key   md5 microtime      sid   rand              sql    INSERT INTO     TABLE TEMPDATA      tempdata key  tempdata value  tempdata updateby  VALUES   s  s   d             h- gt db- gt query  h- gt db- gt prepare  sql   key  serialize  h- gt vars  submitted data      h- gt currentUser- gt id            return  key

User · Answer

This error is caused because your charset is wrong.

Set charset after open tag:

header('Content-Type: text/html; charset=utf-8');

And set charset utf8 in your database :

mysql_query("SET NAMES 'utf8'");

User · Answer

You can fix broken serialize string using following function  with multibyte character handling   function repairSerializeString  value          regex     s   0-9                    return preg replace callback           regex  function  match                return  s   mb strlen  match 2          match 2                             value

User · Answer

badData    a 2  i 0 s 16  as 45  d   Is  n  i 1 s 19  as 45  d   Is  r n        You can not fix a broken serialize string using the proposed regexes    data   preg replace   s   d            e     s   strlen   2        2        badData   var dump  unserialize  data       Output  bool false      or   data   preg replace callback        s   d                   function  m           return  s     strlen  m 2             m 2                      badData    var dump  unserialize  data       Output  bool false    You can fix broken serialize string using following regex    data   preg replace callback           lt         s   d                 asbdiO    d N        s       function  m           return  s     strlen  m 2             m 2                      badData     var dump  unserialize  data      Output  array 2       0    gt    string 17   as 45  d   Is  n     1    gt    string 19   as 45  d   Is  r n      or  array 2       0    gt    string 16   as 45  d   Is  n     1    gt    string 18   as 45  d   Is  r n

User · Answer

You will have to alter the collation type to utf8 unicode ci and the problem will be fixed

User · Answer

the official docs says it should return false and set E NOTICE  but since you got error then the error reporting is set to be triggered by E NOTICE  here is a fix to allow you detect false returned by unserialize   old err error reporting     error reporting  old err  amp   E NOTICE    object   unserialize  serialized data   error reporting  old err     you might want to consider use base64 encode decode   string base64 encode serialize  obj    unserialize base64 decode  string

User · Answer

In my case I was storing serialized data in BLOB field of MySQL DB which apparently wasn t big enough to contain the whole value and truncated it  Such a string obviously could not be unserialized  Once converted that field to MEDIUMBLOB the problem dissipated  Also it may be needed to switch in table options ROW FORMAT to DYNAMIC or COMPRESSED

User · Answer

public function unserializeKeySkills  string         output   array         string   trim preg replace    s s          string         string   preg replace callback   s   d    quot       quot      function  m    return  s   strlen  m 2      quot    m 2    quot        utf8 encode  trim preg replace    s s          string           try            output    unserialize  string         catch   Exception  e             Log  error  quot unserialize Data    quot   print r  string true              return  output

User · Answer

After having tried some things on this page without success I had a look in the page-source and remarked that all quotes in the serialized string have been replaced by html-entities  Decoding these entities helps avoiding much headache    myVar   html entity decode  myVar

User · Answer

unserialize    function unserialize   Error at offset was dues to invalid serialization data due to invalid length  Quick Fix   What you can do is is recalculating the length of the elements in serialized array  You current serialized data    data    a 10  s 16  submit editorial  b 0 s 15  submit orig url  s 13  www bbc co uk  s 12  submit title  s 14  No title found  s 14  submit content  s 12  dnfsdkfjdfdf  s 15  submit category  i 2 s 11  submit tags  s 3  bbc  s 9  submit id  b 0 s 16  submit subscribe  i 0 s 15  submit comments  s 4  open  s 5  image  s 19  C fakepath100 jpg        Example without recalculation   var dump unserialize  data      Output   Notice  unserialize    function unserialize   Error at offset 337 of 338 bytes   Recalculating    data   preg replace   s   d            e     s   strlen   2        2        data   var dump unserialize  data      Output  array    submit editorial    gt  boolean false    submit orig url    gt  string  www bbc co uk   length 13     submit title    gt  string  No title found   length 14     submit content    gt  string  dnfsdkfjdfdf   length 12     submit category    gt  int 2    submit tags    gt  string  bbc   length 3     submit id    gt  boolean false    submit subscribe    gt  int 0    submit comments    gt  string  open   length 4     image    gt  string  C fakepath100 jpg   length 17    Recommendation    I  Instead of using this kind of quick fix     i ll advice you update the question with    How you are serializing your data How you are Saving it                                         EDIT 1                                    The Error   The Error was generated because of use of double quote   instead single quote   that is why C  fakepath 100 png was converted to C fakepath100 jpg   To fix the error   You need to change  h- gt vars  submitted data   From   Note the singe quite      Replace    h- gt vars  submitted data    image      C  fakepath 100 png      With    h- gt vars  submitted data    image      C  fakepath 100 png      Additional Filter    You can also add this simple filter before you call serialize   function satitize  amp  value   key         value   addslashes  value      array walk  h- gt vars  submitted data     satitize      If you have UTF Characters you can also run     h- gt vars  submitted data     array map  utf8 encode   h- gt vars  submitted data       How to detect the problem in future serialized data     findSerializeError    data1       Output  Diffrence 9    7     - gt  ORD number 57    55     - gt  Line Number   315     - gt  Section Data1    pen  s 5  image  s 19  C fakepath100 jpg     - gt  Section Data2    pen  s 5  image  s 17  C fakepath100 jpg                                              ------- The Error  Element Length    findSerializeError Function   function findSerializeError  data1        echo   lt pre gt         data2   preg replace     s   d            e     s   strlen   2        2       data1         max    strlen    data1    gt  strlen    data2      strlen    data1     strlen    data2         echo  data1   PHP EOL      echo  data2   PHP EOL       for  i   0   i  lt   max   i                if    data1   i        data2   i                  echo  Diffrence      data1   i             data2   i   PHP EOL              echo   t- gt  ORD number    ord     data1   i             ord     data2   i     PHP EOL              echo   t- gt  Line Number    i    PHP EOL                start     i - 20                start     start  lt  0    0    start               length   40                point    max -  i              if   point  lt  20                     rlength   1                   rpoint   -  point                else                    rpoint    length - 20                   rlength   1                             echo   t- gt  Section Data1       substr replace   substr    data1   start   length      lt b style   color green   gt   data1   i   lt  b gt     rpoint   rlength    PHP EOL              echo   t- gt  Section Data2       substr replace   substr    data2   start   length      lt b style   color red   gt   data2   i   lt  b gt     rpoint   rlength    PHP EOL                        A better way to save to Database    toDatabse   base64 encode serialize  data        Save to database  fromDatabase   unserialize base64 decode  data      Getting Save Format

User · Answer

Another reason of this problem can be column type of  payload  sessions table  If you have huge data on session  a text column wouldn t be enough  You will need MEDIUMTEXT or even LONGTEXT

User · Answer

The corruption in this question is isolated to a single substring at the end of the serialized string with was probably manually replaced by someone who lazily wanted to update the image filename   This fact will be apparent in my demonstration link below using the OP s posted data -- in short  C fakepath100 jpg does not have a length of 19  it should be 17   Since the serialized string corruption is limited to an incorrect byte character count number  the following will do a fine job of updating the corrupted string with the correct byte count value   The following regex based replacement will only be effective in remedying byte counts  nothing more   It looks like many of the earlier posts are just copy-pasting a regex pattern from someone else   There is no reason to capture the potentially corrupted byte count number if it isn t going to be used in the replacement   Also  adding the s pattern modifier is a reasonable inclusion in case a string value contains newlines line returns    For those that are not aware of the treatment of multibyte characters with serializing  you must not use mb strlen   in the custom callback because it is the byte count that is stored not the character count  see my output     Code   Demo with OP s data   Demo with arbitrary sample data   Demo with condition replacing    corrupted    lt  lt  lt STRING a 4  i 0 s 3  three  i 1 s 5  five  i 2 s 2  newline1 newline2  i 3 s 6  gar  on    STRING    repaired   preg replace callback            s  d           s                  - matched consumed but not captured because not used in replacement         function   m                return  s     strlen  m 1           m 1                            corrupted         echo  corrupted     n     repaired  echo   n--- n   var export unserialize  repaired      Output   a 4  i 0 s 3  three  i 1 s 5  five  i 2 s 2  newline1 Newline2  i 3 s 6  gar  on    a 4  i 0 s 5  three  i 1 s 4  five  i 2 s 17  newline1 Newline2  i 3 s 7  gar  on    --- array     0   gt   three     1   gt   five     2   gt   newline1 Newline2     3   gt   gar  on         One leg down the rabbit hole    The above works fine even if double quotes occur in a string value  but if a string value contains    or some other monkeywrenching sbustring  you ll need to go a little further and implement  lookarounds    My new pattern  checks that the leading s is    the start of the entire input string or preceded by     and checks that the    is    at the end of the entire input string or followed by   or followed by a string or integer declaration s  or i    I haven t test each and every possibility  in fact  I am relatively unfamiliar with all of the possibilities in a serialized string because I never elect to work with serialized data -- always json in modern applications  If there are additional possible leading or trailing characters  leave a comment and I ll extend the lookarounds   Extended snippet    Demo    corrupted byte counts    lt  lt  lt STRING a 12  i 0 s 3  three  i 1 s 5  five  i 2 s 2  newline1 newline2  i 3 s 6  gar  on  i 4 s 111  double   quote   escaped  i 5 s 1  a comma  i 6 s 9  a colon  i 7 s 0  single  quote  i 8 s 999  semi colon  s 5  assoc  s 3  yes  i 9 s 1  monkey  wrenching doublequote-semicolon  s 3  s   s 9  val s  val    STRING    repaired   preg replace callback               lt      s  d                  si    s                     --------------             -- some additional validation         function   m                return  s     strlen  m 1           m 1                            corrupted byte counts         echo  corrupted serialized array  n corrupted byte counts   echo   n--- n   echo  repaired serialized array  n repaired   echo   n--- n   print r unserialize  repaired      Output   corrupted serialized array  a 12  i 0 s 3  three  i 1 s 5  five  i 2 s 2  newline1 newline2  i 3 s 6  gar  on  i 4 s 111  double   quote   escaped  i 5 s 1  a comma  i 6 s 9  a colon  i 7 s 0  single  quote  i 8 s 999  semi colon  s 5  assoc  s 3  yes  i 9 s 1  monkey  wrenching doublequote-semicolon  s 3  s   s 9  val s  val    --- repaired serialized array  a 12  i 0 s 5  three  i 1 s 4  five  i 2 s 17  newline1 newline2  i 3 s 7  gar  on  i 4 s 24  double   quote   escaped  i 5 s 7  a comma  i 6 s 7  a colon  i 7 s 13  single  quote  i 8 s 10  semi colon  s 5  assoc  s 3  yes  i 9 s 39  monkey  wrenching doublequote-semicolon  s 2  s   s 10  val s  val    --- Array        0    gt  three      1    gt  five      2    gt  newline1 newline2      3    gt  gar  on      4    gt  double   quote   escaped      5    gt  a comma      6    gt  a colon      7    gt  single  quote      8    gt  semi colon      assoc    gt  yes      9    gt  monkey  wrenching doublequote-semicolon      s     gt  val s  val

User · Answer

There s another reason unserialize   failed because you improperly put serialized data into the database see Official Explanation here  Since serialize   returns binary data and php variables don t care encoding methods  so that putting it into TEXT  VARCHAR   will cause this error   Solution  store serialized data into BLOB in your table

User · Answer

Quick Fix  Recalculating the length of the elements in serialized array - but don t use  preg replace  it s deprecated - better use preg replace callback   Edit  New Version now not just wrong length but it also fix line-breaks and count correct characters with aczent  thanks to mickmackusa      New Version  data   preg replace callback   s  d           s   function  m    return  s     strlen  m 1           m 1            data

User · Answer

I don t have enough reputation to comment  so I hope this is seen by people using the above  correct  answer   Since php 5 5 the  e modifier in preg replace   has been deprecated completely and the preg match above will error out  The php documentation recommends using preg match callback in its place   Please find the following solution as an alternative to the above proposed preg match    fixed data   preg replace callback     s   d               function  match              return   match 1     strlen  match 2       match 0     s     strlen  match 2             match 2             bad data

User · Answer

Here is an Online Tool for fixing a corrupted serialized string   I d like to add that this mostly happens due to a search and replace done on the DB and the serialization data specially the key length  doesn t get updated as per the replace and that causes the  corruption    Nonetheless  The above tool uses the following logic to fix the serialization data  Copied From Here    function error correction serialise  string          at first  check if  fixing  is really needed at all  After that  security checkup      if   unserialize  string      true  amp  amp   preg match     aOs       string                string   preg replace callback    s    d               s      function  matches  return  s   strlen  matches 2         matches 2              string              return  string

[php] How to repair a serialized string which has been corrupted by an incorrect byte count length?

Examples related to php

Examples related to mysql

Examples related to serialization

Examples related to content-management-system