Best way to encode text data for XML

Question

I was looking for a generic method in  Net to encode a string for use in an Xml element or attribute  and was surprised when I didn t immediately find one   So  before I go too much further  could I just be missing the built-in function     Assuming for a moment that it really doesn t exist  I m putting together my own generic EncodeForXml string data  method  and I m thinking about the best way to do this     The data I m using that prompted this whole thing could contain bad characters like  amp    lt    quot   etc   It could also contains on occasion the properly escaped entities   amp amp    amp lt   and  amp quot   which means just using a CDATA section may not be the best idea   That seems kinda klunky anyay  I d much rather end up with a nice string value that can be used directly in the xml   I ve used a regular expression in the past to just catch bad ampersands  and I m thinking of using it to catch them in this case as well as the first step  and then doing a simple replace for other characters     So  could this be optimized further without making it too complex  and is there anything I m missing     Function EncodeForXml ByVal data As String  As String     Static badAmpersand As new Regex   amp     a-zA-Z  2 6     0-9  2 4           data   badAmpersand Replace data    amp amp         return data Replace   lt      amp lt    Replace         amp quot    Replace   gt     gt    End Function   Sorry for all you C  -only folks-- I don t really care which language I use  but I wanted to make the Regex static and you can t do that in C  without declaring it outside the method  so this will be VB Net  Finally  we re still on  Net 2 0 where I work  but if someone could take the final product and turn it into an extension method for the string class  that d be pretty cool too   Update The first few responses indicate that  Net does indeed have built-in ways of doing this   But now that I ve started  I kind of want to finish my EncodeForXml   method just for the fun of it  so I m still looking for ideas for improvement   Notably  a more complete list of characters that should be encoded as entities  perhaps stored in a list map   and something that gets better performance than doing a  Replace   on immutable strings in serial

User · Answer

You can use the built-in class XAttribute  which handles the encoding automatically   using System Xml Linq   XDocument doc   new XDocument     List lt XAttribute gt  attributes   new List lt XAttribute gt     attributes Add new XAttribute  key1    val1 amp val11     attributes Add new XAttribute  key2    val2      XElement elem   new XElement  test   attributes ToArray      doc Add elem    string xmlStr   doc ToString

User · Answer

Depending on how much you know about the input  you may have to take into account that not all Unicode characters are valid XML characters   Both Server HtmlEncode and System Security SecurityElement Escape seem to ignore illegal XML characters  while System XML XmlWriter WriteString throws an ArgumentException when it encounters illegal characters  unless you disable that check in which case it ignores them   An overview of library functions is available here   Edit 2011 8 14  seeing that at least a few people have consulted this answer in the last couple years  I decided to completely rewrite the original code  which had numerous issues  including horribly mishandling UTF-16   using System  using System Collections Generic  using System IO  using System Linq        lt summary gt      Encodes data so that it can be safely embedded as text in XML documents       lt  summary gt  public class XmlTextEncoder   TextReader       public static string Encode string s            using  var stream   new StringReader s           using  var encoder   new XmlTextEncoder stream                 return encoder ReadToEnd                              lt param name  source  gt The data to be encoded in UTF-16 format  lt  param gt           lt param name  filterIllegalChars  gt It is illegal to encode certain         characters in XML  If true  silently omit these characters from the         output  if false  throw an error when encountered  lt  param gt      public XmlTextEncoder TextReader source  bool filterIllegalChars true             source   source           filterIllegalChars   filterIllegalChars             readonly Queue lt char gt   buf   new Queue lt char gt         readonly bool  filterIllegalChars      readonly TextReader  source       public override int Peek             PopulateBuffer            if   buf Count    0  return -1          return  buf Peek               public override int Read             PopulateBuffer            if   buf Count    0  return -1          return  buf Dequeue               void PopulateBuffer             const int endSentinel   -1          while   buf Count    0  amp  amp   source Peek      endSentinel                   Strings in  NET are assumed to be UTF-16 encoded  1               var c    char   source Read                if  Entities ContainsKey c                        Encode all entities defined in the XML spec  2                   foreach  var i in Entities c    buf Enqueue i                 else if    0x0  lt   c  amp  amp  c  lt   0x8   amp  amp                          new     0xB  0xC   Contains c   amp  amp                           0xE  lt   c  amp  amp  c  lt   0x1F   amp  amp                           0x7F  lt   c  amp  amp  c  lt   0x84   amp  amp                           0x86  lt   c  amp  amp  c  lt   0x9F   amp  amp                           0xD800  lt   c  amp  amp  c  lt   0xDFFF   amp  amp                          new     0xFFFE  0xFFFF   Contains c                        Allow if the Unicode codepoint is legal in XML  3                    buf Enqueue c                 else if  char IsHighSurrogate c   amp  amp                          source Peek      endSentinel  amp  amp                         char IsLowSurrogate  char   source Peek                          Allow well-formed surrogate pairs  1                    buf Enqueue c                    buf Enqueue  char   source Read                   else if    filterIllegalChars                       Note that we cannot encode illegal characters as entity                    references due to the  Legal Character  constraint of                    XML  4   Nor are they allowed in CDATA sections  5                   throw new ArgumentException                      String Format  Illegal character    0 X      int  c                                       static readonly Dictionary lt char string gt  Entities           new Dictionary lt char string gt                         amp quot          amp      amp amp               amp apos                      lt      amp lt          gt      amp gt                         References          1  http   en wikipedia org wiki UTF-16 UCS-2         2  http   www w3 org TR xml11  sec-predefined-ent         3  http   www w3 org TR xml11  charsets         4  http   www w3 org TR xml11  sec-references         5  http   www w3 org TR xml11  sec-cdata-sect     Unit tests and full code can be found here

User · Answer

XmlTextWriter WriteString   does the escaping

User · Answer

In the past I have used HttpUtility HtmlEncode to encode text for xml  It performs the same task  really  I havent ran into any issues with it yet  but that s not to say I won t in the future  As the name implies  it was made for HTML  not XML   You ve probably already read it  but here is an article on xml encoding and decoding   EDIT  Of course  if you use an xmlwriter or one of the new XElement classes  this encoding is done for you  In fact  you could just take the text  place it in a new XElement instance  then return the string   tostring  version of the element  I ve heard that SecurityElement Escape will perform the same task as your utility method as well  but havent read much about it or used it   EDIT2  Disregard my comment about XElement  since you re still on 2 0

User · Answer

If this is an ASP NET app why not use Server HtmlEncode

User · Answer

SecurityElement Escape  documented here

User · Answer

This might be the case where you could benefit from using the WriteCData method   public override void WriteCData string text      Member of System Xml XmlTextWriter  Summary  Writes out a  lt   CDATA       gt  block containing the specified text   Parameters  text  Text to place inside the CDATA block    A simple example would look like the following   writer WriteStartElement  name    writer WriteCData   lt unsafe characters gt     writer WriteFullEndElement      The result looks like    lt name gt  lt   CDATA  lt unsafe characters gt    gt  lt  name gt    When reading the node values the XMLReader automatically strips out the CData part of the innertext so you don t have to worry about it   The only catch is that you have to store the data as an innerText value to an XML node   In other words  you can t insert CData content into an attribute value

User · Answer

In  net 3 5   new XText  I  lt want gt  to  amp  encode this for XML   ToString      Gives you   I  amp lt want amp gt  to  amp amp  encode this for XML   Turns out that this method doesn t encode some things that it should  like quotes    SecurityElement Escape  workmad3 s answer  seems to do a better job with this and it s included in earlier versions of  net   If you don t mind 3rd party code and want to ensure no illegal characters make it into your XML  I would recommend Michael Kropat s answer

User · Answer

Brilliant  That s all I can say   Here is a VB variant of the updated code  not in a class  just a function  that will clean up and also sanitize the xml  Function cXML ByVal  buf As String  As String     Dim textOut As New StringBuilder     Dim c As Char     If  buf Trim Is Nothing OrElse  buf   String Empty Then Return String Empty     For i As Integer   0 To  buf Length - 1         c    buf i          If Entities ContainsKey c  Then             textOut Append Entities Item c           ElseIf  AscW c     amp H9 OrElse AscW c     amp HA OrElse AscW c     amp HD  OrElse   AscW c   gt    amp H20  AndAlso  AscW c   lt    amp HD7FF                 OrElse   AscW c   gt    amp HE000  AndAlso  AscW c   lt    amp HFFFD   OrElse   AscW c   gt    amp H10000  AndAlso  AscW c   lt    amp H10FFFF   Then             textOut Append c          End If     Next     Return textOut ToString  End Function  Shared ReadOnly Entities As New Dictionary Of Char  String    From       c    amp quot        amp  c    amp amp         c    amp apos        lt  c    amp lt        gt  c    amp gt

User · Answer

This might be the case where you could benefit from using the WriteCData method   public override void WriteCData string text      Member of System Xml XmlTextWriter  Summary  Writes out a  lt   CDATA       gt  block containing the specified text   Parameters  text  Text to place inside the CDATA block    A simple example would look like the following   writer WriteStartElement  name    writer WriteCData   lt unsafe characters gt     writer WriteFullEndElement      The result looks like    lt name gt  lt   CDATA  lt unsafe characters gt    gt  lt  name gt    When reading the node values the XMLReader automatically strips out the CData part of the innertext so you don t have to worry about it   The only catch is that you have to store the data as an innerText value to an XML node   In other words  you can t insert CData content into an attribute value

User · Answer

Here is a single line solution using the XElements  I use it in a very small tool  I don t need it a second time so I keep it this way   Its dirdy doug   StrVal     lt x a  lt    StrVal   gt  gt END lt  x gt   ToString   Replace   lt x a          Replace   gt END lt  x gt          Oh and it only works in VB not in C

User · Answer

System XML handles the encoding for you  so you don t need a method like this

User · Answer

Microsoft s AntiXss library AntiXssEncoder Class in System Web dll has methods for this   AntiXss XmlEncode string s  AntiXss XmlAttributeEncode string s    it has HTML as well   AntiXss HtmlEncode string s  AntiXss HtmlAttributeEncode string s

User · Answer

If you re serious about handling all of the invalid characters  not just the few  html  ones   and you have access to System Xml  here s the simplest way to do proper Xml encoding of value data   string theTextToEscape    Something  x1d else  x1D  lt script gt alert  123    lt  script gt    var x   new XmlDocument    x LoadXml   lt r  gt        simple  empty root element x DocumentElement InnerText   theTextToEscape     put in raw string string escapedText   x DocumentElement InnerXml     Returns   Something  amp  x1D  else  amp  x1D   amp lt script amp gt alert  123    amp lt  script amp gt      Repeat the last 2 lines to escape additional strings    It s important to know that XmlConvert EncodeName   is not appropriate  because that s for entity tag names  not values  Using that would be like Url-encoding when you needed to Html-encode

[.net] Best way to encode text data for XML

Examples related to .net

Examples related to xml

Examples related to encoding

Examples related to .net-2.0