string sanitizer for filename

Question

I m looking for a php function that will sanitize a string and make it ready to use for a filename  Anyone know of a handy one      I could write one  but I m worried that I ll overlook a character     Edit  for saving files on a Windows NTFS filesystem

User · Answer

The best I know today is static method Strings::webalize from Nette framework.

BTW, this translates all diacritic signs to their basic.. š=>s ü=>u ß=>ss etc.

For filenames you have to add dot "." to allowed characters parameter.

/**
 * Converts to ASCII.
 * @param  string  UTF-8 encoding
 * @return string  ASCII
 */
public static function toAscii($s)
{
    static $transliterator = NULL;
    if ($transliterator === NULL && class_exists('Transliterator', FALSE)) {
        $transliterator = \Transliterator::create('Any-Latin; Latin-ASCII');
    }

    $s = preg_replace('#[^\x09\x0A\x0D\x20-\x7E\xA0-\x{2FF}\x{370}-\x{10FFFF}]#u', '', $s);
    $s = strtr($s, '`\'"^~?', "\x01\x02\x03\x04\x05\x06");
    $s = str_replace(
        array("\xE2\x80\x9E", "\xE2\x80\x9C", "\xE2\x80\x9D", "\xE2\x80\x9A", "\xE2\x80\x98", "\xE2\x80\x99", "\xC2\xB0"),
        array("\x03", "\x03", "\x03", "\x02", "\x02", "\x02", "\x04"), $s
    );
    if ($transliterator !== NULL) {
        $s = $transliterator->transliterate($s);
    }
    if (ICONV_IMPL === 'glibc') {
        $s = str_replace(
            array("\xC2\xBB", "\xC2\xAB", "\xE2\x80\xA6", "\xE2\x84\xA2", "\xC2\xA9", "\xC2\xAE"),
            array('>>', '<<', '...', 'TM', '(c)', '(R)'), $s
        );
        $s = @iconv('UTF-8', 'WINDOWS-1250//TRANSLIT//IGNORE', $s); // intentionally @
        $s = strtr($s, "\xa5\xa3\xbc\x8c\xa7\x8a\xaa\x8d\x8f\x8e\xaf\xb9\xb3\xbe\x9c\x9a\xba\x9d\x9f\x9e"
            . "\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3"
            . "\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8"
            . "\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf8\xf9\xfa\xfb\xfc\xfd\xfe"
            . "\x96\xa0\x8b\x97\x9b\xa6\xad\xb7",
            'ALLSSSSTZZZallssstzzzRAAAALCCCEEEEIIDDNNOOOOxRUUUUYTsraaaalccceeeeiiddnnooooruuuuyt- <->|-.');
        $s = preg_replace('#[^\x00-\x7F]++#', '', $s);
    } else {
        $s = @iconv('UTF-8', 'ASCII//TRANSLIT//IGNORE', $s); // intentionally @
    }
    $s = str_replace(array('`', "'", '"', '^', '~', '?'), '', $s);
    return strtr($s, "\x01\x02\x03\x04\x05\x06", '`\'"^~?');
}


/**
 * Converts to web safe characters [a-z0-9-] text.
 * @param  string  UTF-8 encoding
 * @param  string  allowed characters
 * @param  bool
 * @return string
 */
public static function webalize($s, $charlist = NULL, $lower = TRUE)
{
    $s = self::toAscii($s);
    if ($lower) {
        $s = strtolower($s);
    }
    $s = preg_replace('#[^a-z0-9' . preg_quote($charlist, '#') . ']+#i', '-', $s);
    $s = trim($s, '-');
    return $s;
}

User · Answer

It seems this all hinges on the question  is it possible to create a filename that can be used to hack into a server  or do some-such other damage    If not  then it seems the simple answer to is try creating the file wherever it will  ultimately  be used  since that will be the operating system of choice  no doubt    Let the operating system sort it out   If it complains  port that complaint back to the User as a Validation Error   This has the added benefit of being reliably portable  since all  I m pretty sure  operating systems will complain if the filename is not properly formed for that OS   If it is possible to do nefarious things with a filename  perhaps there are measures that can be applied before testing the filename on the resident operating system -- measures less complicated than a full  sanitation  of the filename

User · Answer

fname   str replace         fname     Since users might use the slash to separate two words it would be better to replace with a dash instead of NULL

User · Answer

PHP provides a function to sanitize a text to different format  filter filters sanitize  How to     echo filter var      Lorem Ipsum has been the industry s  FILTER SANITIZE URL          Blockquote LoremIpsumhasbeentheindustry s

User · Answer

Well  tempnam   will do it for you   http   us2 php net manual en function tempnam php  but that creates an entirely new name   To sanitize an existing string just restrict what your users can enter and make it letters  numbers  period  hyphen and underscore then sanitize with a simple regex  Check what characters need to be escaped or you could get false positives    sanitized   preg replace     a-zA-Z0-9 -            filename

User · Answer

safe  replace every sequence of NOT  quot a-zA-Z0-9 - quot  to a dash  add an extension yourself   name   preg replace     a-zA-Z0-9 -       -   strtolower  name        extension

User · Answer

SOLUTION 1 - simple and effective   file name   preg replace      a-z0-9       -   strtolower   url        strtolower   guarantees the filename is lowercase  since case does not matter inside the URL  but in the NTFS filename    a-z0-9   will ensure  the filename only keeps letters and numbers Substitute invalid characters with  -  keeps the filename readable   Example    URL   http   stackoverflow com questions 2021624 string-sanitizer-for-filename File  http-stackoverflow-com-questions-2021624-string-sanitizer-for-filename     SOLUTION 2 - for very long URLs  You want to cache the URL contents and just need to have unique filenames  I would use this function    file name   md5  strtolower   url      this will create a filename with fixed length  The MD5 hash is in most cases unique enough for this kind of usage   Example   URL   https   www amazon com Interstellar-Matthew-McConaughey dp B00TU9UFTS ref s9 nwrsa gw g318 i10 r  encoding UTF8 amp fpl fresh amp pf rd m ATVPDKIKX0DER amp pf rd s desktop-1 amp pf rd r BS5M1H560SMAR2JDKYX3 amp pf rd r BS5M1H560SMAR2JDKYX3 amp pf rd t 36701 amp pf rd p 6822bacc-d4f0-466d-83a8-2c5e1d703f8e amp pf rd p 6822bacc-d4f0-466d-83a8-2c5e1d703f8e amp pf rd i desktop File  51301f3edb513f6543779c3a5433b01c

User · Answer

Making a small adjustment to Tor Valamo s solution to fix the problem noticed by Dominic Rodger  you could use      Remove anything which isn t a word  whitespace  number    or any of the following caracters -             If you don t need to handle multi-byte characters    you can use preg replace rather than mb ereg replace    Thanks  Lukasz Rysiak   file   mb ereg replace      w s d -                       file      Remove any runs of periods  thanks falstro    file   mb ereg replace        2           file

User · Answer

What about using rawurlencode     http   www php net manual en function rawurlencode php  Here is a function that sanitize even Chinese Chars   public static function normalizeString   str              str   strip tags  str         str   preg replace     r n t             str        str   preg replace              lt   gt                   str        str   strtolower  str        str   html entity decode   str  ENT QUOTES   utf-8          str   htmlentities  str  ENT QUOTES   utf-8         str   preg replace     amp    a-z    a-z     i     2    str        str   str replace       -    str        str   rawurlencode  str        str   str replace       -    str       return  str      Here is the explaination   Strip HTML Tags Remove Break Tabs Return Carriage Remove Illegal Chars for folder and filename Put the string in lower case Remove foreign accents such as        by convert it into html entities and then remove the code and keep the letter  Replace Spaces with dashes Encode special chars that could pass the previous steps and enter in conflict filename on server  ex          Replace     with dashes to make sure the link of the file will not be rewritten by the browser when querying th file    OK  some filename will not be releavant but in most case it will work   ex   Original Name          -  -             jpg   Output Name   -E1-83-A1-E1-83-90-E1-83-91-E1-83-94-E1-83-AD-E1-83-93-E1-83-98--E1-83-93-E1-83-90--E1-83-A2-E1-83-98-E1-83-9E-E1-83-9D-E1-83-92-E1-83-A0-E1-83-90-E1-83-A4-E1-83-98-E1-83-A3-E1-83-9A-E1-83-98 jpg   It s better like that than an 404 error   Hope that was helpful   Carl

User · Answer

preg replace     w s d   -                      file    Add remove more valid characters depending on what is allowed for your system   Alternatively you can try to create the file and then return an error if it s bad

User · Answer

These may be a bit heavy  but they re flexible enough to sanitize whatever string into a  safe  en style filename or folder name  or heck  even scrubbed slugs and things if you bend it    1  Building a full filename  with fallback name in case input is totally truncated    str file  raw string   word separator   file extension   fallback name   length     2  Or using just the filter util without building a full filename  strict mode true will not allow    or    in filename    str file filter  string   separator   strict   length     3  And here are those functions      Returns filesystem-safe string after cleaning  filtering  and trimming input function str file filter       str       sep             strict   false       trim   248          str   strip tags htmlspecialchars decode strtolower  str        lowercase - gt  decode - gt  strip tags      str   str replace   20         str      convert rogue  20s into spaces      str   preg replace     a-z0-9  1 2  i        str      remove hexy things      str   str replace   amp nbsp          str      convert all nbsp into space      str   preg replace    amp    a-z0-9  2 8   i        str      remove the other non-tag things      str   preg replace    s      sep   str      filter multiple spaces      str   preg replace                str      filter multiple periods      str   preg replace                str      trim leading period      if   strict             str   preg replace       w d       sep                str      only allow words and digits       else            str   preg replace       w d       sep                        str      allow words  digits      and                str   preg replace          sep          sep   str      filter multiple separators      str   substr  str  0   trim      trim filename to desired length  note 255 char limit on windows      return  str         Returns full file name including fallback and extension function str file       str       sep             ext            default            trim   248            Run  str and or  ext through filters to clean up strings      str   str file filter  str   sep        ext         str file filter  ext      true           Default file name in case all chars are trimmed from  str  then ensure there is an id at tail     if  empty  str   amp  amp  empty  default              str    no name      date  Y-m-d H-m A            uniqid          elseif  empty  str              str    default                Return completed string     if   empty  ext             return  str    ext        else           return  str            So let s say some user input is        amp lt div amp gt  amp lt  div amp gt  lt script gt  lt  script gt  amp amp  Wei   G  bel      File name   20    20  21  2C D  cor             z      y         x        This name    is  amp  462   not  amp nbsp  amp nbsp  amp nbsp  amp nbsp  amp nbsp    that grrrreat -  09   1234747         -  -              And we wanna convert it to something friendlier to make a tar gz with a file name length of 255 chars  Here is an example use  Note  this example includes a malformed tar gz extension as a proof of concept  you should still filter the ext after string is built against your whitelist s     raw str          amp lt div amp gt  amp lt  div amp gt  lt script gt  lt  script gt  amp amp  Wei   G  bel      File name   20    20  21  2C D  cor             z      y         x        This name    is  amp  462   not  amp nbsp  amp nbsp  amp nbsp  amp nbsp  amp nbsp    that grrrreat -  09   1234747         -  -                fallback str    generated     date  Y-m-d H-m A     bad extension        t amp    a  r gz      echo str file  raw str        bad extension   fallback str     The output would be   wei gbel file name dcor       z   y   x   this name is 462 not that grrrreat   09   1234747   tar gz  You can play with it here  https   3v4l org iSgi8  Or a Gist  https   gist github com dhaupin b109d3a8464239b7754a  EDIT   updated script filter for  amp nbsp  instead of space  updated 3v4l link

User · Answer

and    in the user provided file name can be harmful  So you should get rid of these by something like    fname   str replace            fname    fname   str replace            fname

User · Answer

Making a small adjustment to Sean Vieira s solution to allow for single dots  you could use   preg replace      w s d   -                    2           file

User · Answer

The following expression creates a nice  clean  and usable string      a-z0-9   -   gi   Turning today s financial  billing into today-s-financial-billing

User · Answer

Instead of worrying about overlooking characters - how about using a whitelist of characters you are happy to be used  For example  you could allow just good ol  a-z  0-9     and a single instance of a period      That s obviously more limiting than most filesystems  but should keep you safe

User · Answer

one way   bad           lt  gt        string    fi le     function sanitize  str  pat        return preg replace  pat     str      echo sanitize  string  bad

User · Answer

This is how you can sanitize for a file system as asked  function filter filename  name           remove illegal file system characters https   en wikipedia org wiki Filename Reserved characters and words      name   str replace array merge          array map  chr   range 0  31            array   lt      gt                                                    name          maximise filename length to 255 bytes http   serverfault com a 9548 44086      ext   pathinfo  name  PATHINFO EXTENSION        name  mb strcut pathinfo  name  PATHINFO FILENAME   0  255 -   ext   strlen  ext    1   0   mb detect encoding  name       ext          ext            return  name      Everything else is allowed in a filesystem  so the question is perfectly answered         but it could be dangerous to allow for example single quotes   in a filename if you use it later in an unsafe HTML context because this absolutely legal filename      onerror   alert document cookie  jpg   becomes an XSS hole    lt img src   lt   echo  image   gt     gt     output   lt img src     onerror   alert document cookie     gt    Because of that  the popular CMS software Wordpress removes them  but they covered all relevant chars only after some updates    special chars   array                                  lt      gt                                amp                                                                     chr 0           a few rows later are whitespaces removed as well     preg replace      r n t -       -    filename     Finally their list includes now most of the characters that are part of the URI rerserved-characters and URL unsafe characters list    Of course you could simply encode all these chars on HTML output  but most developers and me too  follow the idiom  Better safe than sorry  and delete them in advance   So finally I would suggest to use this   function filter filename  filename   beautify true           sanitize filename      filename   preg replace                       lt  gt                         file system reserved https   en wikipedia org wiki Filename Reserved characters and words           x00- x1F                 control characters http   msdn microsoft com en-us library windows desktop aa365247 28v vs 85 29 aspx           x7F xA0 xAD              non-printing characters DEL  NO-BREAK SPACE  SOFT HYPHEN                   amp                  URI reserved https   tools ietf org html rfc3986 section-2 2                                    URL unsafe characters https   www ietf org rfc rfc1738 txt          x            -    filename          avoids           or   hiddenFiles       filename   ltrim  filename    -           optional beautification     if   beautify   filename   beautify filename  filename          maximize filename length to 255 bytes http   serverfault com a 9548 44086      ext   pathinfo  filename  PATHINFO EXTENSION        filename   mb strcut pathinfo  filename  PATHINFO FILENAME   0  255 -   ext   strlen  ext    1   0   mb detect encoding  filename       ext          ext            return  filename      Everything else that does not cause problems with the file system should be part of an additional function   function beautify filename  filename           reduce consecutive characters      filename   preg replace array              file   name zip  becomes  file-name zip                              file   name zip  becomes  file-name zip                              file---name zip  becomes  file-name zip            -            -    filename        filename   preg replace array              file-- -- - --name zip  becomes  file name zip            -   -                 file   name  zip  becomes  file name zip               2                  filename          lowercase for windows unix interoperability http   support microsoft com kb 100625      filename   mb strtolower  filename  mb detect encoding  filename             file-name -  becomes  file-name       filename   trim  filename    -        return  filename      And at this point you need to generate a filename if the result is empty and you can decide if you want to encode UTF-8 characters  But you do not need that as UTF-8 is allowed in all file systems that are used in web hosting contexts   The only thing you have to do is to use urlencode    as you hopefully do it with all your URLs  so the filename                 jpg becomes this URL as your  lt img src gt  or  lt a href gt   http   www maxrev de html img  E1 83 A1 E1 83 90 E1 83 91 E1 83 94 E1 83 AD E1 83 93 E1 83 98  E1 83 9B E1 83 90 E1 83 9C E1 83 A5 E1 83 90 E1 83 9C E1 83 90 jpg  Stackoverflow does that  so I can post this link as a user would do it  http   www maxrev de html img                 jpg  So this is a complete legal filename and not a problem as  SequenceDigitale com mentioned in his answer

[php] string sanitizer for filename

Examples related to php

Examples related to string

Examples related to sanitization