Question mark characters displaying within text why is this

Question

I have a backup server that automatically backs up my live site  both files and database   On the live site  the text looks fine  but when you view the mirrored version of it  it displays     within some of the text  This text is stored within the news database table   Here is a screen shot of it being on the live server and of it on the mirrored server   What could happen within the process of backing it up to the mirrored server

User · Accepted Answer

The following articles will be useful

http://dev.mysql.com/doc/refman/5.0/en/charset-syntax.html

http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html

After you connect to the database issue the following command:

SET NAMES 'utf8';

Ensure that your web page also uses the UTF-8 encoding:

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

PHP also offers several function that will be useful for conversions:

http://us3.php.net/manual/en/function.iconv.php

http://us.php.net/mb_convert_encoding

User · Answer

Your browser hasn t interpretted the encoding of the page correctly  either because you ve forced it to a particular setting  or the page is set incorrectly   and thus cannot display some of the characters

User · Answer

I usually curse MS word and then run the following Wscript      replace with path to a file that needs cleaning PATH    test html   var go WScript CreateObject  Scripting FileSystemObject    var content go GetFile PATH  OpenAsTextStream   ReadAll    var out go CreateTextFile  clean-  PATH  true       symbols content content replace      g       content content replace      g       content content replace      g       content content replace      g  -    content content replace     g   amp copy     content content replace     g   amp reg     content content replace     g   amp deg     content content replace     g   lt p gt     content content replace     g   amp iquest     content content replace     g   amp iexcl     content content replace     g   amp cent     content content replace     g   amp pound     content content replace     g   amp yen      out Write content

User · Answer

Your browser hasn t interpretted the encoding of the page correctly  either because you ve forced it to a particular setting  or the page is set incorrectly   and thus cannot display some of the characters

User · Answer

Check the character set being emitted by your mirrored server  There appears to be a difference from that to the main server -- the live site appears to be outputting Unicode  where the mirror is not  Also  it s usually a good idea to scrub Unicode characters in your incoming content and replace them with their appropriate HTML entities   Your specific issue regards  smart quotes    em dashes  and  en dashes   I know you can replace em dashes with  amp mdash  and n-dashes with  amp ndash   which should be done on the input side of your database   I don t know what the correct replacement for the smart quotes would be   I usually just replace all curly single quotes with   and all curly double quotes with       Typography geeks may feel free to shoot me on sight    I should note that some browsers are more forgiving than others with this issue -- Internet Explorer on Windows tends to auto-magically detect and  fix  this  Firefox and most other browsers display the question marks

User · Answer

Check the character set being emitted by your mirrored server  There appears to be a difference from that to the main server -- the live site appears to be outputting Unicode  where the mirror is not  Also  it s usually a good idea to scrub Unicode characters in your incoming content and replace them with their appropriate HTML entities   Your specific issue regards  smart quotes    em dashes  and  en dashes   I know you can replace em dashes with  amp mdash  and n-dashes with  amp ndash   which should be done on the input side of your database   I don t know what the correct replacement for the smart quotes would be   I usually just replace all curly single quotes with   and all curly double quotes with       Typography geeks may feel free to shoot me on sight    I should note that some browsers are more forgiving than others with this issue -- Internet Explorer on Windows tends to auto-magically detect and  fix  this  Firefox and most other browsers display the question marks

User · Answer

Unicode or other character set characters falling through   I have seen similar  strange  characters show up on sites I have worked on often when the text is copied from an email or some other document format  e g  word  into a text editor   The editor can display the non ASCII characters but the browser can t   For the website  I would suggest looking up the HTML entity code for the character and inserting that instead     or switch to more standard ones

User · Answer

Unicode or other character set characters falling through   I have seen similar  strange  characters show up on sites I have worked on often when the text is copied from an email or some other document format  e g  word  into a text editor   The editor can display the non ASCII characters but the browser can t   For the website  I would suggest looking up the HTML entity code for the character and inserting that instead     or switch to more standard ones

User · Answer

I usually curse MS word and then run the following Wscript      replace with path to a file that needs cleaning PATH    test html   var go WScript CreateObject  Scripting FileSystemObject    var content go GetFile PATH  OpenAsTextStream   ReadAll    var out go CreateTextFile  clean-  PATH  true       symbols content content replace      g       content content replace      g       content content replace      g       content content replace      g  -    content content replace     g   amp copy     content content replace     g   amp reg     content content replace     g   amp deg     content content replace     g   lt p gt     content content replace     g   amp iquest     content content replace     g   amp iexcl     content content replace     g   amp cent     content content replace     g   amp pound     content content replace     g   amp yen      out Write content

User · Answer

This is going to be something to do with character encodings   Are you sure the mirrored site has the same properties with regards to character encodings as your main server   Depending on what sort of server you have  this may be a property of the server process itself  or it could be an environment variable   For example  if this is a UNIX environment  perhaps try comparing LANG or LC ALL   See also here

User · Answer

Check the character set being emitted by your mirrored server  There appears to be a difference from that to the main server -- the live site appears to be outputting Unicode  where the mirror is not  Also  it s usually a good idea to scrub Unicode characters in your incoming content and replace them with their appropriate HTML entities   Your specific issue regards  smart quotes    em dashes  and  en dashes   I know you can replace em dashes with  amp mdash  and n-dashes with  amp ndash   which should be done on the input side of your database   I don t know what the correct replacement for the smart quotes would be   I usually just replace all curly single quotes with   and all curly double quotes with       Typography geeks may feel free to shoot me on sight    I should note that some browsers are more forgiving than others with this issue -- Internet Explorer on Windows tends to auto-magically detect and  fix  this  Firefox and most other browsers display the question marks

User · Answer

This is going to be something to do with character encodings   Are you sure the mirrored site has the same properties with regards to character encodings as your main server   Depending on what sort of server you have  this may be a property of the server process itself  or it could be an environment variable   For example  if this is a UNIX environment  perhaps try comparing LANG or LC ALL   See also here

User · Answer

I usually curse MS word and then run the following Wscript      replace with path to a file that needs cleaning PATH    test html   var go WScript CreateObject  Scripting FileSystemObject    var content go GetFile PATH  OpenAsTextStream   ReadAll    var out go CreateTextFile  clean-  PATH  true       symbols content content replace      g       content content replace      g       content content replace      g       content content replace      g  -    content content replace     g   amp copy     content content replace     g   amp reg     content content replace     g   amp deg     content content replace     g   lt p gt     content content replace     g   amp iquest     content content replace     g   amp iexcl     content content replace     g   amp cent     content content replace     g   amp pound     content content replace     g   amp yen      out Write content

User · Answer

Unicode or other character set characters falling through   I have seen similar  strange  characters show up on sites I have worked on often when the text is copied from an email or some other document format  e g  word  into a text editor   The editor can display the non ASCII characters but the browser can t   For the website  I would suggest looking up the HTML entity code for the character and inserting that instead     or switch to more standard ones

User · Answer

I got here looking for a solution for JavaScript displayed in the browser and although not directly related with a database     In my case I copied and pasted some text I found on the internet into a JavaScript file and saved it with Windows Notepad   When the page that uses that JavaScript file output the strings there were question marks  like the ones shown in the question  instead of the special characters like accented letters  etc   I opened the file using Notepad    Right after opening the file I saw that the character encoding was set as ANSI as you can see  mouse cursor on footer  in the following screenshot     To solve the issue  click the Encoding menu in Notepad   and select Encode in UTF-8  You should be good to go

User · Answer

Unicode or other character set characters falling through   I have seen similar  strange  characters show up on sites I have worked on often when the text is copied from an email or some other document format  e g  word  into a text editor   The editor can display the non ASCII characters but the browser can t   For the website  I would suggest looking up the HTML entity code for the character and inserting that instead     or switch to more standard ones

User · Answer

I had this issue so I just took all my content  copy pasted it into notepad  made a new php file  pasted back in  re-saved and overwrote  and   that worked   It really was some relic of Microsoft Word editing

User · Answer

I usually curse MS word and then run the following Wscript      replace with path to a file that needs cleaning PATH    test html   var go WScript CreateObject  Scripting FileSystemObject    var content go GetFile PATH  OpenAsTextStream   ReadAll    var out go CreateTextFile  clean-  PATH  true       symbols content content replace      g       content content replace      g       content content replace      g       content content replace      g  -    content content replace     g   amp copy     content content replace     g   amp reg     content content replace     g   amp deg     content content replace     g   lt p gt     content content replace     g   amp iquest     content content replace     g   amp iexcl     content content replace     g   amp cent     content content replace     g   amp pound     content content replace     g   amp yen      out Write content

User · Answer

This is going to be something to do with character encodings   Are you sure the mirrored site has the same properties with regards to character encodings as your main server   Depending on what sort of server you have  this may be a property of the server process itself  or it could be an environment variable   For example  if this is a UNIX environment  perhaps try comparing LANG or LC ALL   See also here

User · Answer

Edit your Apache configuration file on the  mirror  server  the server with the problem   and comment-out the following line   AddDefaultCharset UTF-8   Then restart Apache   service httpd restart   The problem is that the  AddDefaultCharset UTF-8  line overrides the Content-Type specified in the  html files  e g     lt meta http-equiv Content-Type content  text html  charset windows-1252  gt    The most common symptom is that character codes above 127 display as black diamonds with question marks on them  in Chrome  Safari or Firefox   or as little boxes  in IE and Opera    HTML files generated by Microsoft Word usually have many such characters  the most common one being character code 160   0xA0  which is equivalent to   amp nbsp   in the Windows-1252 encoding  and is often found between span tags  like this    lt span style  mso-spacerun  yes  gt         lt  span gt

User · Answer

I got here looking for a solution for JavaScript displayed in the browser and although not directly related with a database     In my case I copied and pasted some text I found on the internet into a JavaScript file and saved it with Windows Notepad   When the page that uses that JavaScript file output the strings there were question marks  like the ones shown in the question  instead of the special characters like accented letters  etc   I opened the file using Notepad    Right after opening the file I saw that the character encoding was set as ANSI as you can see  mouse cursor on footer  in the following screenshot     To solve the issue  click the Encoding menu in Notepad   and select Encode in UTF-8  You should be good to go

User · Answer

Your browser hasn t interpretted the encoding of the page correctly  either because you ve forced it to a particular setting  or the page is set incorrectly   and thus cannot display some of the characters

User · Answer

This is going to be something to do with character encodings   Are you sure the mirrored site has the same properties with regards to character encodings as your main server   Depending on what sort of server you have  this may be a property of the server process itself  or it could be an environment variable   For example  if this is a UNIX environment  perhaps try comparing LANG or LC ALL   See also here

User · Answer

Your browser hasn t interpretted the encoding of the page correctly  either because you ve forced it to a particular setting  or the page is set incorrectly   and thus cannot display some of the characters

User · Answer

Check the character set being emitted by your mirrored server  There appears to be a difference from that to the main server -- the live site appears to be outputting Unicode  where the mirror is not  Also  it s usually a good idea to scrub Unicode characters in your incoming content and replace them with their appropriate HTML entities   Your specific issue regards  smart quotes    em dashes  and  en dashes   I know you can replace em dashes with  amp mdash  and n-dashes with  amp ndash   which should be done on the input side of your database   I don t know what the correct replacement for the smart quotes would be   I usually just replace all curly single quotes with   and all curly double quotes with       Typography geeks may feel free to shoot me on sight    I should note that some browsers are more forgiving than others with this issue -- Internet Explorer on Windows tends to auto-magically detect and  fix  this  Firefox and most other browsers display the question marks

User · Answer

I had this issue so I just took all my content  copy pasted it into notepad  made a new php file  pasted back in  re-saved and overwrote  and   that worked   It really was some relic of Microsoft Word editing

User · Answer

Edit your Apache configuration file on the  mirror  server  the server with the problem   and comment-out the following line   AddDefaultCharset UTF-8   Then restart Apache   service httpd restart   The problem is that the  AddDefaultCharset UTF-8  line overrides the Content-Type specified in the  html files  e g     lt meta http-equiv Content-Type content  text html  charset windows-1252  gt    The most common symptom is that character codes above 127 display as black diamonds with question marks on them  in Chrome  Safari or Firefox   or as little boxes  in IE and Opera    HTML files generated by Microsoft Word usually have many such characters  the most common one being character code 160   0xA0  which is equivalent to   amp nbsp   in the Windows-1252 encoding  and is often found between span tags  like this    lt span style  mso-spacerun  yes  gt         lt  span gt

[html] Question mark characters displaying within text, why is this?

Examples related to html

Examples related to backup

Examples related to character-encoding

Examples related to mirror