[php] Sanitizing strings to make them URL and filename safe?

I am trying to come up with a function that does a good job of sanitizing certain strings so that they are safe to use in the URL (like a post slug) and also safe to use as file names. For example, when someone uploads a file I want to make sure that I remove all dangerous characters from the name.

So far I have come up with the following function which I hope solves this problem and allows foreign UTF-8 data also.

/**
 * Convert a string to the file/URL safe "slug" form
 *
 * @param string $string the string to clean
 * @param bool $is_filename TRUE will allow additional filename characters
 * @return string
 */
function sanitize($string = '', $is_filename = FALSE)
{
 // Replace all weird characters with dashes
 $string = preg_replace('/[^\w\-'. ($is_filename ? '~_\.' : ''). ']+/u', '-', $string);

 // Only allow one dash separator at a time (and make string lowercase)
 return mb_strtolower(preg_replace('/--+/u', '-', $string), 'UTF-8');
}

Does anyone have any tricky sample data I can run against this - or know of a better way to safeguard our apps from bad names?

$is-filename allows some additional characters like temp vim files

update: removed the star character since I could not think of a valid use

I have adapted from another source and added a couple extra, maybe a little overkill

/**
 * Convert a string into a url safe address.
 *
 * @param string $unformatted
 * @return string
 */
public function formatURL($unformatted) {

    $url = strtolower(trim($unformatted));

    //replace accent characters, forien languages
    $search = array('À', 'Á', 'Â', 'Ã', 'Ä', 'Å', 'Æ', 'Ç', 'È', 'É', 'Ê', 'Ë', 'Ì', 'Í', 'Î', 'Ï', 'Ð', 'Ñ', 'Ò', 'Ó', 'Ô', 'Õ', 'Ö', 'Ø', 'Ù', 'Ú', 'Û', 'Ü', 'Ý', 'ß', 'à', 'á', 'â', 'ã', 'ä', 'å', 'æ', 'ç', 'è', 'é', 'ê', 'ë', 'ì', 'í', 'î', 'ï', 'ñ', 'ò', 'ó', 'ô', 'õ', 'ö', 'ø', 'ù', 'ú', 'û', 'ü', 'ý', 'ÿ', 'A', 'a', 'A', 'a', 'A', 'a', 'C', 'c', 'C', 'c', 'C', 'c', 'C', 'c', 'D', 'd', 'Ð', 'd', 'E', 'e', 'E', 'e', 'E', 'e', 'E', 'e', 'E', 'e', 'G', 'g', 'G', 'g', 'G', 'g', 'G', 'g', 'H', 'h', 'H', 'h', 'I', 'i', 'I', 'i', 'I', 'i', 'I', 'i', 'I', 'i', '?', '?', 'J', 'j', 'K', 'k', 'L', 'l', 'L', 'l', 'L', 'l', '?', '?', 'L', 'l', 'N', 'n', 'N', 'n', 'N', 'n', '?', 'O', 'o', 'O', 'o', 'O', 'o', 'Œ', 'œ', 'R', 'r', 'R', 'r', 'R', 'r', 'S', 's', 'S', 's', 'S', 's', 'Š', 'š', 'T', 't', 'T', 't', 'T', 't', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'W', 'w', 'Y', 'y', 'Ÿ', 'Z', 'z', 'Z', 'z', 'Ž', 'ž', '?', 'ƒ', 'O', 'o', 'U', 'u', 'A', 'a', 'I', 'i', 'O', 'o', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', '?', '?', '?', '?', '?', '?'); 
    $replace = array('A', 'A', 'A', 'A', 'A', 'A', 'AE', 'C', 'E', 'E', 'E', 'E', 'I', 'I', 'I', 'I', 'D', 'N', 'O', 'O', 'O', 'O', 'O', 'O', 'U', 'U', 'U', 'U', 'Y', 's', 'a', 'a', 'a', 'a', 'a', 'a', 'ae', 'c', 'e', 'e', 'e', 'e', 'i', 'i', 'i', 'i', 'n', 'o', 'o', 'o', 'o', 'o', 'o', 'u', 'u', 'u', 'u', 'y', 'y', 'A', 'a', 'A', 'a', 'A', 'a', 'C', 'c', 'C', 'c', 'C', 'c', 'C', 'c', 'D', 'd', 'D', 'd', 'E', 'e', 'E', 'e', 'E', 'e', 'E', 'e', 'E', 'e', 'G', 'g', 'G', 'g', 'G', 'g', 'G', 'g', 'H', 'h', 'H', 'h', 'I', 'i', 'I', 'i', 'I', 'i', 'I', 'i', 'I', 'i', 'IJ', 'ij', 'J', 'j', 'K', 'k', 'L', 'l', 'L', 'l', 'L', 'l', 'L', 'l', 'l', 'l', 'N', 'n', 'N', 'n', 'N', 'n', 'n', 'O', 'o', 'O', 'o', 'O', 'o', 'OE', 'oe', 'R', 'r', 'R', 'r', 'R', 'r', 'S', 's', 'S', 's', 'S', 's', 'S', 's', 'T', 't', 'T', 't', 'T', 't', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'W', 'w', 'Y', 'y', 'Y', 'Z', 'z', 'Z', 'z', 'Z', 'z', 's', 'f', 'O', 'o', 'U', 'u', 'A', 'a', 'I', 'i', 'O', 'o', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'U', 'u', 'A', 'a', 'AE', 'ae', 'O', 'o'); 
    $url = str_replace($search, $replace, $url);

    //replace common characters
    $search = array('&', '£', '$'); 
    $replace = array('and', 'pounds', 'dollars'); 
    $url= str_replace($search, $replace, $url);

    // remove - for spaces and union characters
    $find = array(' ', '&', '\r\n', '\n', '+', ',', '//');
    $url = str_replace($find, '-', $url);

    //delete and replace rest of special chars
    $find = array('/[^a-z0-9\-<>]/', '/[\-]+/', '/<[^>]*>/');
    $replace = array('', '-', '');
    $uri = preg_replace($find, $replace, $url);

    return $uri;
}

I've always thought Kohana did a pretty good job of it.

public static function title($title, $separator = '-', $ascii_only = FALSE)
{
if ($ascii_only === TRUE)
{
// Transliterate non-ASCII characters
$title = UTF8::transliterate_to_ascii($title);

// Remove all characters that are not the separator, a-z, 0-9, or whitespace
$title = preg_replace('![^'.preg_quote($separator).'a-z0-9\s]+!', '', strtolower($title));
}
else
{
// Remove all characters that are not the separator, letters, numbers, or whitespace
$title = preg_replace('![^'.preg_quote($separator).'\pL\pN\s]+!u', '', UTF8::strtolower($title));
}

// Replace all separator characters and whitespace by a single separator
$title = preg_replace('!['.preg_quote($separator).'\s]+!u', $separator, $title);

// Trim separators from the beginning and end
return trim($title, $separator);
}

The handy UTF8::transliterate_to_ascii() will turn stuff like ñ => n.

Of course, you could replace the other UTF8::* stuff with mb_* functions.

Here's CodeIgniter's implementation.

/**
 * Sanitize Filename
 *
 * @param   string  $str        Input file name
 * @param   bool    $relative_path  Whether to preserve paths
 * @return  string
 */
public function sanitize_filename($str, $relative_path = FALSE)
{
    $bad = array(
        '../', '<!--', '-->', '<', '>',
        "'", '"', '&', '$', '#',
        '{', '}', '[', ']', '=',
        ';', '?', '%20', '%22',
        '%3c',      // <
        '%253c',    // <
        '%3e',      // >
        '%0e',      // >
        '%28',      // (
        '%29',      // )
        '%2528',    // (
        '%26',      // &
        '%24',      // $
        '%3f',      // ?
        '%3b',      // ;
        '%3d'       // =
    );

    if ( ! $relative_path)
    {
        $bad[] = './';
        $bad[] = '/';
    }

    $str = remove_invisible_characters($str, FALSE);
    return stripslashes(str_replace($bad, '', $str));
}

And the remove_invisible_characters dependency.

function remove_invisible_characters($str, $url_encoded = TRUE)
{
    $non_displayables = array();

    // every control character except newline (dec 10),
    // carriage return (dec 13) and horizontal tab (dec 09)
    if ($url_encoded)
    {
        $non_displayables[] = '/%0[0-8bcef]/';  // url encoded 00-08, 11, 12, 14, 15
        $non_displayables[] = '/%1[0-9a-f]/';   // url encoded 16-31
    }

    $non_displayables[] = '/[\x00-\x08\x0B\x0C\x0E-\x1F\x7F]+/S';   // 00-08, 11, 12, 14-31, 127

    do
    {
        $str = preg_replace($non_displayables, '', $str, -1, $count);
    }
    while ($count);

    return $str;
}

This is a nice way to secure an upload filename:

$file_name = trim(basename(stripslashes($name)), ".\x00..\x20");

I have entry titles with all kinds of weird latin characters as well as some HTML tags that I needed to translate into a useful dash-delimited filename format. I combined @SoLoGHoST's answer with a couple of items from @Xeoncross's answer and customized a bit.

    function sanitize($string,$force_lowercase=true) {
    //Clean up titles for filenames
    $clean = strip_tags($string);
    $clean = strtr($clean, array('Š' => 'S','Ž' => 'Z','š' => 's','ž' => 'z','Ÿ' => 'Y','À' => 'A','Á' => 'A','Â' => 'A','Ã' => 'A','Ä' => 'A','Å' => 'A','Ç' => 'C','È' => 'E','É' => 'E','Ê' => 'E','Ë' => 'E','Ì' => 'I','Í' => 'I','Î' => 'I','Ï' => 'I','Ñ' => 'N','Ò' => 'O','Ó' => 'O','Ô' => 'O','Õ' => 'O','Ö' => 'O','Ø' => 'O','Ù' => 'U','Ú' => 'U','Û' => 'U','Ü' => 'U','Ý' => 'Y','à' => 'a','á' => 'a','â' => 'a','ã' => 'a','ä' => 'a','å' => 'a','ç' => 'c','è' => 'e','é' => 'e','ê' => 'e','ë' => 'e','ì' => 'i','í' => 'i','î' => 'i','ï' => 'i','ñ' => 'n','ò' => 'o','ó' => 'o','ô' => 'o','õ' => 'o','ö' => 'o','ø' => 'o','ù' => 'u','ú' => 'u','û' => 'u','ü' => 'u','ý' => 'y','ÿ' => 'y'));
    $clean = strtr($clean, array('Þ' => 'TH', 'þ' => 'th', 'Ð' => 'DH', 'ð' => 'dh', 'ß' => 'ss', 'Œ' => 'OE', 'œ' => 'oe', 'Æ' => 'AE', 'æ' => 'ae', 'µ' => 'u','—' => '-'));
    $clean = str_replace("--", "-", preg_replace("/[^a-z0-9-]/i", "", preg_replace(array('/\s/', '/[^\w-\.\-]/'), array('-', ''), $clean)));

    return ($force_lowercase) ?
        (function_exists('mb_strtolower')) ?
            mb_strtolower($clean, 'UTF-8') :
            strtolower($clean) :
        $clean;
}

I needed to manually add the em dash character (—) to the translation array. There may be others but so far my file names are looking good.

So:

Part 1: My dad’s “Žurburts”?—they’re (not) the best!

becomes:

part-1-my-dads-zurburts-theyre-not-the-best

I just add ".html" to the returned string.

why not simply use php's urlencode? it replaces "dangerous" characters with their hex representation for urls (i.e. %20 for a space)

This isn't exactly an answer as it doesn't provide any solutions (yet!), but it's too big to fit on a comment...

I did some testing (regarding file names) on Windows 7 and Ubuntu 12.04 and what I found out was that:

1. PHP Can't Handle non-ASCII Filenames

Although both Windows and Ubuntu can handle Unicode filenames (even RTL ones as it seems) PHP 5.3 requires hacks to deal even with the plain old ISO-8859-1, so it's better to keep it ASCII only for safety.

2. The Lenght of the Filename Matters (Specially on Windows)

On Ubuntu, the maximum length a filename can have (incluinding extension) is 255 (excluding path):

/var/www/uploads/123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345/

However, on Windows 7 (NTFS) the maximum lenght a filename can have depends on it's absolute path:

(0 + 0 + 244 + 11 chars) C:\1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234\1234567.txt
(0 + 3 + 240 + 11 chars) C:\123\123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890\1234567.txt
(3 + 3 + 236 + 11 chars) C:\123\456\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456\1234567.txt

Wikipedia says that:

NTFS allows each path component (directory or filename) to be 255 characters long.

To the best of my knowledge (and testing), this is wrong.

In total (counting slashes) all these examples have 259 chars, if you strip the C:\ that gives 256 characters (not 255?!). The directories where created using the Explorer and you'll notice that it restrains itself from using all the available space for the directory name. The reason for this is to allow the creation of files using the 8.3 file naming convention. The same thing happens for other partitions.

Files don't need to reserve the 8.3 lenght requirements of course:

(255 chars) E:\12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901.txt

You can't create any more sub-directories if the absolute path of the parent directory has more than 242 characters, because 256 = 242 + 1 + \ + 8 + . + 3. Using Windows Explorer, you can't create another directory if the parent directory has more than 233 characters (depending on the system locale), because 256 = 233 + 10 + \ + 8 + . + 3; the 10 here is the length of the string New folder.

Windows file system poses a nasty problem if you want to assure inter-operability between file systems.

3. Beware of Reserved Characters and Keywords

Aside from removing non-ASCII, non-printable and control characters, you also need to re(place/move):

"*/:<>?\|

Just removing these characters might not be the best idea because the filename might lose some of it's meaning. I think that, at the very least, multiple occurences of these characters should be replaced by a single underscore (_), or perhaps something more representative (this is just an idea):

• "*? -> _
• /\| -> -
• : -> [ ]-[ ]
• < -> (
• > -> )

There are also special keywords that should be avoided (like NUL), although I'm not sure how to overcome that. Perhaps a black list with a random name fallback would be a good approach to solve it.

4. Case Sensitiveness

This should go without saying, but if you want so ensure file uniqueness across different operating systems you should transform file names to a normalized case, that way my_file.txt and My_File.txt on Linux won't both become the same my_file.txt file on Windows.

5. Make Sure It's Unique

If the file name already exists, a unique identifier should be appended to it's base file name.

Common unique identifiers include the UNIX timestamp, a digest of the file contents or a random string.

6. Hidden Files

Just because it can be named doesn't mean it should...

Dots are usually white-listed in file names but in Linux a hidden file is represented by a leading dot.

7. Other Considerations

If you have to strip some chars of the file name, the extension is usually more important than the base name of the file. Allowing a considerable maximum number of characters for the file extension (8-16) one should strip the characters from the base name. It's also important to note that in the unlikely event of having a more than one long extension - such as _.graphmlz.tag.gz - _.graphmlz.tag only _ should be considered as the file base name in this case.

8. Resources

Calibre handles file name mangling pretty decently:

• /src/calibre/utils/filenames.py
• /src/calibre/library/save_to_disk.py

Wikipedia page on file name mangling and linked chapter from Using Samba.

If for instance, you try to create a file that violates any of the rules 1/2/3, you'll get a very useful error:

Warning: touch(): Unable to create file ... because No error in ... on line ...

I don't think having a list of chars to remove is safe. I would rather use the following:

For filenames: Use an internal ID or a hash of the filecontent. Save the document name in a database. This way you can keep the original filename and still find the file.

For url parameters: Use urlencode() to encode any special characters.

In terms of file uploads, you would be safest to prevent the user from controlling the file name. As has already been hinted at, store the canonicalised filename in a database along with a randomly chosen and unique name which you'll use as the actual filename.

Using OWASP ESAPI, these names could be generated thus:

$userFilename   = ESAPI::getEncoder()->canonicalize($input_string);
$safeFilename   = ESAPI::getRandomizer()->getRandomFilename();

You could append a timestamp to the $safeFilename to help ensure that the randomly generated filename is unique without even checking for an existing file.

In terms of encoding for URL, and again using ESAPI:

$safeForURL     = ESAPI::getEncoder()->encodeForURL($input_string);

This method performs canonicalisation before encoding the string and will handle all character encodings.

Solution #1: You have ability to install PHP extensions on server (hosting)

For transliteration of "almost every single language on the planet Earth" to ASCII characters.

1. Install PHP Intl extension first. This is command for Debian (Ubuntu): sudo aptitude install php5-intl

2. This is my fileName function (create test.php and paste there following code):

<!doctype html>_x000D_
<html lang="en">_x000D_
<head>_x000D_
<meta charset="utf-8">_x000D_
<title>Test</title>_x000D_
</head>_x000D_
<body>_x000D_
<?php_x000D_
_x000D_
function pr($string) {_x000D_
  print '<hr>';_x000D_
  print '"' . fileName($string) . '"';_x000D_
  print '<br>';_x000D_
  print '"' . $string . '"';_x000D_
}_x000D_
_x000D_
function fileName($string) {_x000D_
  // remove html tags_x000D_
  $clean = strip_tags($string);_x000D_
  // transliterate_x000D_
  $clean = transliterator_transliterate('Any-Latin;Latin-ASCII;', $clean);_x000D_
  // remove non-number and non-letter characters_x000D_
  $clean = str_replace('--', '-', preg_replace('/[^a-z0-9-\_]/i', '', preg_replace(array(_x000D_
    '/\s/', _x000D_
    '/[^\w-\.\-]/'_x000D_
  ), array(_x000D_
    '_', _x000D_
    ''_x000D_
  ), $clean)));_x000D_
  // replace '-' for '_'_x000D_
  $clean = strtr($clean, array(_x000D_
    '-' => '_'_x000D_
  ));_x000D_
  // remove double '__'_x000D_
  $positionInString = stripos($clean, '__');_x000D_
  while ($positionInString !== false) {_x000D_
    $clean = str_replace('__', '_', $clean);_x000D_
    $positionInString = stripos($clean, '__');_x000D_
  }_x000D_
  // remove '_' from the end and beginning of the string_x000D_
  $clean = rtrim(ltrim($clean, '_'), '_');_x000D_
  // lowercase the string_x000D_
  return strtolower($clean);_x000D_
}_x000D_
pr('_replace(\'~&([a-z]{1,2})(ac134/56f4315981743 8765475[]lt7nl2ú5änú138yé73tž7ýlute|');_x000D_
pr(htmlspecialchars('<script>alert(\'hacked\')</script>'));_x000D_
pr('Álix----_Ãxel!?!?');_x000D_
pr('áéíóúÁÉÍÓÚ');_x000D_
pr('üÿÄËÏÖÜ.ŸåÅ');_x000D_
pr('nie4c a a§ônäääaš');_x000D_
pr('??? ??????');_x000D_
pr('???');_x000D_
pr('??? ??? ????');_x000D_
pr('???? ????????');_x000D_
pr('??? ???-????');_x000D_
pr('??? ??????');_x000D_
pr('Mao Tr?ch Ðông');_x000D_
pr('???');_x000D_
pr('???? ??????');_x000D_
?>_x000D_
</body>_x000D_
</html>

This line is core:

  // transliterate
  $clean = transliterator_transliterate('Any-Latin;Latin-ASCII;', $clean);

Answer based on this post.

Solution #2: You don't have ability to install PHP extensions on server (hosting)

Pretty good job is done in transliteration module for CMS Drupal. It supports almost every single language on the planet Earth. I suggest to check plugin repository if you want to have really complete solution sanitizing strings.

I found this larger function in the Chyrp code:

/**
 * Function: sanitize
 * Returns a sanitized string, typically for URLs.
 *
 * Parameters:
 *     $string - The string to sanitize.
 *     $force_lowercase - Force the string to lowercase?
 *     $anal - If set to *true*, will remove all non-alphanumeric characters.
 */
function sanitize($string, $force_lowercase = true, $anal = false) {
    $strip = array("~", "`", "!", "@", "#", "$", "%", "^", "&", "*", "(", ")", "_", "=", "+", "[", "{", "]",
                   "}", "\\", "|", ";", ":", "\"", "'", "‘", "’", "“", "”", "–", "—",
                   "—", "–", ",", "<", ".", ">", "/", "?");
    $clean = trim(str_replace($strip, "", strip_tags($string)));
    $clean = preg_replace('/\s+/', "-", $clean);
    $clean = ($anal) ? preg_replace("/[^a-zA-Z0-9]/", "", $clean) : $clean ;
    return ($force_lowercase) ?
        (function_exists('mb_strtolower')) ?
            mb_strtolower($clean, 'UTF-8') :
            strtolower($clean) :
        $clean;
}

and this one in the wordpress code

/**
 * Sanitizes a filename replacing whitespace with dashes
 *
 * Removes special characters that are illegal in filenames on certain
 * operating systems and special characters requiring special escaping
 * to manipulate at the command line. Replaces spaces and consecutive
 * dashes with a single dash. Trim period, dash and underscore from beginning
 * and end of filename.
 *
 * @since 2.1.0
 *
 * @param string $filename The filename to be sanitized
 * @return string The sanitized filename
 */
function sanitize_file_name( $filename ) {
    $filename_raw = $filename;
    $special_chars = array("?", "[", "]", "/", "\\", "=", "<", ">", ":", ";", ",", "'", "\"", "&", "$", "#", "*", "(", ")", "|", "~", "`", "!", "{", "}");
    $special_chars = apply_filters('sanitize_file_name_chars', $special_chars, $filename_raw);
    $filename = str_replace($special_chars, '', $filename);
    $filename = preg_replace('/[\s-]+/', '-', $filename);
    $filename = trim($filename, '.-_');
    return apply_filters('sanitize_file_name', $filename, $filename_raw);
}

Update Sept 2012

Alix Axel has done some incredible work in this area. His phunction framework includes several great text filters and transformations.

• Unaccent
• Slug
• Filter

This should make your filenames safe...

$string = preg_replace(array('/\s/', '/\.[\.]+/', '/[^\w_\.\-]/'), array('_', '.', ''), $string);

and a deeper solution to this is:

// Remove special accented characters - ie. sí.
$clean_name = strtr($string, array('Š' => 'S','Ž' => 'Z','š' => 's','ž' => 'z','Ÿ' => 'Y','À' => 'A','Á' => 'A','Â' => 'A','Ã' => 'A','Ä' => 'A','Å' => 'A','Ç' => 'C','È' => 'E','É' => 'E','Ê' => 'E','Ë' => 'E','Ì' => 'I','Í' => 'I','Î' => 'I','Ï' => 'I','Ñ' => 'N','Ò' => 'O','Ó' => 'O','Ô' => 'O','Õ' => 'O','Ö' => 'O','Ø' => 'O','Ù' => 'U','Ú' => 'U','Û' => 'U','Ü' => 'U','Ý' => 'Y','à' => 'a','á' => 'a','â' => 'a','ã' => 'a','ä' => 'a','å' => 'a','ç' => 'c','è' => 'e','é' => 'e','ê' => 'e','ë' => 'e','ì' => 'i','í' => 'i','î' => 'i','ï' => 'i','ñ' => 'n','ò' => 'o','ó' => 'o','ô' => 'o','õ' => 'o','ö' => 'o','ø' => 'o','ù' => 'u','ú' => 'u','û' => 'u','ü' => 'u','ý' => 'y','ÿ' => 'y'));
$clean_name = strtr($clean_name, array('Þ' => 'TH', 'þ' => 'th', 'Ð' => 'DH', 'ð' => 'dh', 'ß' => 'ss', 'Œ' => 'OE', 'œ' => 'oe', 'Æ' => 'AE', 'æ' => 'ae', 'µ' => 'u'));

$clean_name = preg_replace(array('/\s/', '/\.[\.]+/', '/[^\w_\.\-]/'), array('_', '.', ''), $clean_name);

This assumes that you want a dot in the filename. if you want it transferred to lowercase, just use

$clean_name = strtolower($clean_name);

for the last line.

and this is Joomla 3.3.2 version from JFile::makeSafe($file)

public static function makeSafe($file)
{
    // Remove any trailing dots, as those aren't ever valid file names.
    $file = rtrim($file, '.');

    $regex = array('#(\.){2,}#', '#[^A-Za-z0-9\.\_\- ]#', '#^\.#');

    return trim(preg_replace($regex, '', $file));
}

Depending on how you will use it, you might want to add a length limit to protect against buffer overflows.

There are already several solutions provided for this question but I have read and tested most of the code here and I ended up with this solution which is a mix of what I learned here:

The function

The function is bundled here in a Symfony2 bundle but it can be extracted to be used as plain PHP, it only has a dependency with the iconv function that must be enabled:

Filesystem.php:

<?php

namespace COil\Bundle\COilCoreBundle\Component\HttpKernel\Util;

use Symfony\Component\HttpKernel\Util\Filesystem as BaseFilesystem;

/**
 * Extends the Symfony filesystem object.
 */
class Filesystem extends BaseFilesystem
{
    /**
     * Make a filename safe to use in any function. (Accents, spaces, special chars...)
     * The iconv function must be activated.
     *
     * @param string  $fileName       The filename to sanitize (with or without extension)
     * @param string  $defaultIfEmpty The default string returned for a non valid filename (only special chars or separators)
     * @param string  $separator      The default separator
     * @param boolean $lowerCase      Tells if the string must converted to lower case
     *
     * @author COil <https://github.com/COil>
     * @see    http://stackoverflow.com/questions/2668854/sanitizing-strings-to-make-them-url-and-filename-safe
     *
     * @return string
     */
    public function sanitizeFilename($fileName, $defaultIfEmpty = 'default', $separator = '_', $lowerCase = true)
    {
    // Gather file informations and store its extension
    $fileInfos = pathinfo($fileName);
    $fileExt   = array_key_exists('extension', $fileInfos) ? '.'. strtolower($fileInfos['extension']) : '';

    // Removes accents
    $fileName = @iconv('UTF-8', 'us-ascii//TRANSLIT', $fileInfos['filename']);

    // Removes all characters that are not separators, letters, numbers, dots or whitespaces
    $fileName = preg_replace("/[^ a-zA-Z". preg_quote($separator). "\d\.\s]/", '', $lowerCase ? strtolower($fileName) : $fileName);

    // Replaces all successive separators into a single one
    $fileName = preg_replace('!['. preg_quote($separator).'\s]+!u', $separator, $fileName);

    // Trim beginning and ending seperators
    $fileName = trim($fileName, $separator);

    // If empty use the default string
    if (empty($fileName)) {
        $fileName = $defaultIfEmpty;
    }

    return $fileName. $fileExt;
    }
}

The unit tests

What is interesting is that I have created PHPUnit tests, first to test edge cases and so you can check if it fits your needs: (If you find a bug, feel free to add a test case)

FilesystemTest.php:

<?php

namespace COil\Bundle\COilCoreBundle\Tests\Unit\Helper;

use COil\Bundle\COilCoreBundle\Component\HttpKernel\Util\Filesystem;

/**
 * Test the Filesystem custom class.
 */
class FilesystemTest extends \PHPUnit_Framework_TestCase
{
    /**
     * test sanitizeFilename()
     */
    public function testFilesystem()
    {
    $fs = new Filesystem();

    $this->assertEquals('logo_orange.gif', $fs->sanitizeFilename('--logö  _  __   ___   ora@@ñ--~gé--.gif'), '::sanitizeFilename() handles complex filename with specials chars');
    $this->assertEquals('coilstack', $fs->sanitizeFilename('cOiLsTaCk'), '::sanitizeFilename() converts all characters to lower case');
    $this->assertEquals('cOiLsTaCk', $fs->sanitizeFilename('cOiLsTaCk', 'default', '_', false), '::sanitizeFilename() lower case can be desactivated, passing false as the 4th argument');
    $this->assertEquals('coil_stack', $fs->sanitizeFilename('coil stack'), '::sanitizeFilename() convert a white space to a separator');
    $this->assertEquals('coil-stack', $fs->sanitizeFilename('coil stack', 'default', '-'), '::sanitizeFilename() can use a different separator as the 3rd argument');
    $this->assertEquals('coil_stack', $fs->sanitizeFilename('coil          stack'), '::sanitizeFilename() removes successive white spaces to a single separator');
    $this->assertEquals('coil_stack', $fs->sanitizeFilename('       coil stack'), '::sanitizeFilename() removes spaces at the beginning of the string');
    $this->assertEquals('coil_stack', $fs->sanitizeFilename('coil   stack         '), '::sanitizeFilename() removes spaces at the end of the string');
    $this->assertEquals('coilstack', $fs->sanitizeFilename('coil,,,,,,stack'), '::sanitizeFilename() removes non-ASCII characters');
    $this->assertEquals('coil_stack', $fs->sanitizeFilename('coil_stack  '), '::sanitizeFilename() keeps separators');
    $this->assertEquals('coil_stack', $fs->sanitizeFilename(' coil________stack'), '::sanitizeFilename() converts successive separators into a single one');
    $this->assertEquals('coil_stack.gif', $fs->sanitizeFilename('cOil Stack.GiF'), '::sanitizeFilename() lower case filename and extension');
    $this->assertEquals('copy_of_coil.stack.exe', $fs->sanitizeFilename('Copy of coil.stack.exe'), '::sanitizeFilename() keeps dots before the extension');
    $this->assertEquals('default.doc', $fs->sanitizeFilename('____________.doc'), '::sanitizeFilename() returns a default file name if filename only contains special chars');
    $this->assertEquals('default.docx', $fs->sanitizeFilename('     ___ -  --_     __%%%%__¨¨¨***____      .docx'), '::sanitizeFilename() returns a default file name if filename only contains special chars');
    $this->assertEquals('logo_edition_1314352521.jpg', $fs->sanitizeFilename('logo_edition_1314352521.jpg'), '::sanitizeFilename() returns the filename untouched if it does not need to be modified');
    $userId = rand(1, 10);
    $this->assertEquals('user_doc_'. $userId. '.doc', $fs->sanitizeFilename('?????.doc', 'user_doc_'. $userId), '::sanitizeFilename() returns the default string (the 2nd argument) if it can\'t be sanitized');
    }
}

The test results: (checked on Ubuntu with PHP 5.3.2 and MacOsX with PHP 5.3.17:

All tests pass:

phpunit -c app/ src/COil/Bundle/COilCoreBundle/Tests/Unit/Helper/FilesystemTest.php
PHPUnit 3.6.10 by Sebastian Bergmann.

Configuration read from /var/www/strangebuzz.com/app/phpunit.xml.dist

.

Time: 0 seconds, Memory: 5.75Mb

OK (1 test, 17 assertions)

This is a good function:

public function getFriendlyURL($string) {
    setlocale(LC_CTYPE, 'en_US.UTF8');
    $string = iconv('UTF-8', 'ASCII//TRANSLIT//IGNORE', $string);
    $string = preg_replace('~[^\-\pL\pN\s]+~u', '-', $string);
    $string = str_replace(' ', '-', $string);
    $string = trim($string, "-");
    $string = strtolower($string);
    return $string;
} 

I recommend* URLify for PHP (480+ stars on Github) - "the PHP port of URLify.js from the Django project. Transliterates non-ascii characters for use in URLs".

Basic usage:

To generate slugs for URLs:

<?php

echo URLify::filter (' J\'étudie le français ');
// "jetudie-le-francais"

echo URLify::filter ('Lo siento, no hablo español.');
// "lo-siento-no-hablo-espanol"

?>

To generate slugs for file names:

<?php

echo URLify::filter ('????.jpg', 60, "", true);
// "foto.jpg"

?>

*None of the other suggestions matched my criteria:

• Should be installable via composer
• Should not depend on iconv since it behaves differently on different systems
• Should be extendable to allow overrides and custom character replacements
• Popular (for instance many stars on Github)
• Has tests

As a bonus, URLify also removes certain words and strips away all characters not transliterated.

Here is a test case with tons of foreign characters being transliterated properly using URLify: https://gist.github.com/motin/a65e6c1cc303e46900d10894bf2da87f

Try this:

function normal_chars($string)
{
    $string = htmlentities($string, ENT_QUOTES, 'UTF-8');
    $string = preg_replace('~&([a-z]{1,2})(acute|cedil|circ|grave|lig|orn|ring|slash|th|tilde|uml);~i', '$1', $string);
    $string = html_entity_decode($string, ENT_QUOTES, 'UTF-8');
    $string = preg_replace(array('~[^0-9a-z]~i', '~[ -]+~'), ' ', $string);

    return trim($string, ' -');
}

Examples:

echo normal_chars('Álix----_Ãxel!?!?'); // Alix Axel
echo normal_chars('áéíóúÁÉÍÓÚ'); // aeiouAEIOU
echo normal_chars('üÿÄËÏÖÜŸåÅ'); // uyAEIOUYaA

Based on the selected answer in this thread: URL Friendly Username in PHP?

There is 2 good answers to slugfy your data, use it https://stackoverflow.com/a/3987966/971619 or it https://stackoverflow.com/a/7610586/971619

This post seems to work the best among all that I have tied. http://gsynuh.com/php-string-filename-url-safe/205

This is the code used by Prestashop to sanitize urls :

replaceAccentedChars

is used by

str2url

to remove diacritics

function replaceAccentedChars($str)
{
    $patterns = array(
        /* Lowercase */
        '/[\x{0105}\x{00E0}\x{00E1}\x{00E2}\x{00E3}\x{00E4}\x{00E5}]/u',
        '/[\x{00E7}\x{010D}\x{0107}]/u',
        '/[\x{010F}]/u',
        '/[\x{00E8}\x{00E9}\x{00EA}\x{00EB}\x{011B}\x{0119}]/u',
        '/[\x{00EC}\x{00ED}\x{00EE}\x{00EF}]/u',
        '/[\x{0142}\x{013E}\x{013A}]/u',
        '/[\x{00F1}\x{0148}]/u',
        '/[\x{00F2}\x{00F3}\x{00F4}\x{00F5}\x{00F6}\x{00F8}]/u',
        '/[\x{0159}\x{0155}]/u',
        '/[\x{015B}\x{0161}]/u',
        '/[\x{00DF}]/u',
        '/[\x{0165}]/u',
        '/[\x{00F9}\x{00FA}\x{00FB}\x{00FC}\x{016F}]/u',
        '/[\x{00FD}\x{00FF}]/u',
        '/[\x{017C}\x{017A}\x{017E}]/u',
        '/[\x{00E6}]/u',
        '/[\x{0153}]/u',

        /* Uppercase */
        '/[\x{0104}\x{00C0}\x{00C1}\x{00C2}\x{00C3}\x{00C4}\x{00C5}]/u',
        '/[\x{00C7}\x{010C}\x{0106}]/u',
        '/[\x{010E}]/u',
        '/[\x{00C8}\x{00C9}\x{00CA}\x{00CB}\x{011A}\x{0118}]/u',
        '/[\x{0141}\x{013D}\x{0139}]/u',
        '/[\x{00D1}\x{0147}]/u',
        '/[\x{00D3}]/u',
        '/[\x{0158}\x{0154}]/u',
        '/[\x{015A}\x{0160}]/u',
        '/[\x{0164}]/u',
        '/[\x{00D9}\x{00DA}\x{00DB}\x{00DC}\x{016E}]/u',
        '/[\x{017B}\x{0179}\x{017D}]/u',
        '/[\x{00C6}]/u',
        '/[\x{0152}]/u');

    $replacements = array(
            'a', 'c', 'd', 'e', 'i', 'l', 'n', 'o', 'r', 's', 'ss', 't', 'u', 'y', 'z', 'ae', 'oe',
            'A', 'C', 'D', 'E', 'L', 'N', 'O', 'R', 'S', 'T', 'U', 'Z', 'AE', 'OE'
        );

    return preg_replace($patterns, $replacements, $str);
}

function str2url($str)
{
    if (function_exists('mb_strtolower'))
        $str = mb_strtolower($str, 'utf-8');

    $str = trim($str);
    if (!function_exists('mb_strtolower'))
        $str = replaceAccentedChars($str);

    // Remove all non-whitelist chars.
    $str = preg_replace('/[^a-zA-Z0-9\s\'\:\/\[\]-\pL]/u', '', $str);
    $str = preg_replace('/[\s\'\:\/\[\]-]+/', ' ', $str);
    $str = str_replace(array(' ', '/'), '-', $str);

    // If it was not possible to lowercase the string with mb_strtolower, we do it after the transformations.
    // This way we lose fewer special chars.
    if (!function_exists('mb_strtolower'))
        $str = strtolower($str);

    return $str;
}

// CLEAN ILLEGAL CHARACTERS
function clean_filename($source_file)
{
    $search[] = " ";
    $search[] = "&";
    $search[] = "$";
    $search[] = ",";
    $search[] = "!";
    $search[] = "@";
    $search[] = "#";
    $search[] = "^";
    $search[] = "(";
    $search[] = ")";
    $search[] = "+";
    $search[] = "=";
    $search[] = "[";
    $search[] = "]";

    $replace[] = "_";
    $replace[] = "and";
    $replace[] = "S";
    $replace[] = "_";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";
    $replace[] = "";

    return str_replace($search,$replace,$source_file);

}