Efficient way to remove ALL whitespace from String

Question

I m calling a REST API and am receiving an XML response back  It returns a list of a workspace names  and I m writing a quick IsExistingWorkspace   method  Since all workspaces consist of contiguous characters with no whitespace  I m assuming the easiest way to find out if a particular workspace is in the list is to remove all whitespace  including newlines  and doing this  XML is the string received from the web request    XML Contains   lt name gt     workspaceName     lt  name gt       I know it s case-sensitive  and I m relying on that  I just need a way to remove all whitespace in a string efficiently  I know RegEx and LINQ can do it  but I m open to other ideas  I am mostly just concerned about speed

User · Answer

I found a nice write-up on this on CodeProject by Felipe Machado (with help by Richard Robertson)

He tested ten different methods. This one is the fastest unsafe version...

public static unsafe string TrimAllWithStringInplace(string str) {
    fixed (char* pfixed = str) {
        char* dst = pfixed;
        for (char* p = pfixed; *p != 0; p++)

            switch (*p) {

                case '\u0020': case '\u00A0': case '\u1680': case '\u2000': case '\u2001':

                case '\u2002': case '\u2003': case '\u2004': case '\u2005': case '\u2006':

                case '\u2007': case '\u2008': case '\u2009': case '\u200A': case '\u202F':

                case '\u205F': case '\u3000': case '\u2028': case '\u2029': case '\u0009':

                case '\u000A': case '\u000B': case '\u000C': case '\u000D': case '\u0085':
                    continue;

                default:
                    *dst++ = *p;
                    break;
            }

        return new string(pfixed, 0, (int)(dst - pfixed));
    }
}

And the fastest safe version...

public static string TrimAllWithInplaceCharArray(string str) {

    var len = str.Length;
    var src = str.ToCharArray();
    int dstIdx = 0;

    for (int i = 0; i < len; i++) {
        var ch = src[i];

        switch (ch) {

            case '\u0020': case '\u00A0': case '\u1680': case '\u2000': case '\u2001':

            case '\u2002': case '\u2003': case '\u2004': case '\u2005': case '\u2006':

            case '\u2007': case '\u2008': case '\u2009': case '\u200A': case '\u202F':

            case '\u205F': case '\u3000': case '\u2028': case '\u2029': case '\u0009':

            case '\u000A': case '\u000B': case '\u000C': case '\u000D': case '\u0085':
                continue;

            default:
                src[dstIdx++] = ch;
                break;
        }
    }
    return new string(src, 0, dstIdx);
}

There are also some nice independent benchmarks on Stack Overflow by Stian Standahl that also show how Felipe's function is about 300% faster than the next fastest function.

User · Answer

I needed to replace white space in a string with spaces  but not duplicate spaces  e g   I needed to convert something like the following    a b   c r n d t t t e    to   a b c d e    I used the following method  private static string RemoveWhiteSpace string value        if  value    null    return null        var sb   new StringBuilder         var lastCharWs   false      foreach  var c in value                if  char IsWhiteSpace c                         if  lastCharWs    continue                sb Append                   lastCharWs   true                    else                       sb Append c               lastCharWs   false                      return sb ToString

User · Answer

Building on Henks answer I have created some test methods with his answer and some added  more optimized  methods  I found the results differ based on the size of the input string  Therefore  I have tested with two result sets  In the fastest method  the linked source has a even faster way  But  since it is characterized as unsafe I have left this out   Long input string results    InPlaceCharArray  2021 ms  Sunsetquest s answer  -  Original source  String split then join  4277ms  Kernowcode s answer  String reader  6082 ms LINQ using native char IsWhitespace  7357 ms LINQ  7746 ms  Henk s answer  ForLoop  32320 ms RegexCompiled  37157 ms Regex  42940 ms   Short input string results    InPlaceCharArray  108 ms   Sunsetquest s answer  -  Original source  String split then join  294 ms  Kernowcode s answer  String reader  327 ms ForLoop  343 ms LINQ using native char IsWhitespace  624 ms LINQ  645ms  Henk s answer  RegexCompiled  1671 ms Regex  2599 ms   Code   public class RemoveWhitespace       public static string RemoveStringReader string input                var s   new StringBuilder input Length       input Length           using  var reader   new StringReader input                         int i   0              char c              for    i  lt  input Length  i                                  c    char reader Read                    if   char IsWhiteSpace c                                         s Append c                                                      return s ToString               public static string RemoveLinqNativeCharIsWhitespace string input                return new string input ToCharArray                Where c   gt   char IsWhiteSpace c                ToArray                public static string RemoveLinq string input                return new string input ToCharArray                Where c   gt   Char IsWhiteSpace c                ToArray                public static string RemoveRegex string input                return Regex Replace input     s                    private static Regex compiled   new Regex    s    RegexOptions Compiled       public static string RemoveRegexCompiled string input                return compiled Replace input                  public static string RemoveForLoop string input                for  int i   input Length - 1  i  gt   0  i--                        if  char IsWhiteSpace input i                                  input   input Remove i  1                                   return input             public static string StringSplitThenJoin this string str                return string Join     str Split default string     StringSplitOptions RemoveEmptyEntries               public static string RemoveInPlaceCharArray string input                var len   input Length          var src   input ToCharArray            int dstIdx   0          for  int i   0  i  lt  len  i                          var ch   src i               switch  ch                                case   u0020                   case   u00A0                   case   u1680                   case   u2000                   case   u2001                   case   u2002                   case   u2003                   case   u2004                   case   u2005                   case   u2006                   case   u2007                   case   u2008                   case   u2009                   case   u200A                   case   u202F                   case   u205F                   case   u3000                   case   u2028                   case   u2029                   case   u0009                   case   u000A                   case   u000B                   case   u000C                   case   u000D                   case   u0085                       continue                  default                      src dstIdx      ch                      break                                  return new string src  0  dstIdx             Tests    TestFixture  public class Test          Short input       private const string input    123 123  t 1adc  n 222         private const string expected    1231231adc222           Long input     private const string input    123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222123 123  t 1adc  n 222       private const string expected    1231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc2221231231adc222        private const int iterations   1000000        Test      public void RemoveInPlaceCharArray                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace RemoveInPlaceCharArray input                      stopwatch Stop            Console WriteLine  InPlaceCharArray      stopwatch ElapsedMilliseconds     ms            Assert AreEqual expected  s               Test      public void RemoveStringReader                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace RemoveStringReader input                      stopwatch Stop            Console WriteLine  String reader      stopwatch ElapsedMilliseconds     ms            Assert AreEqual expected  s               Test      public void RemoveLinqNativeCharIsWhitespace                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace RemoveLinqNativeCharIsWhitespace input                      stopwatch Stop            Console WriteLine  LINQ using native char IsWhitespace      stopwatch ElapsedMilliseconds     ms            Assert AreEqual expected  s               Test      public void RemoveLinq                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace RemoveLinq input                      stopwatch Stop            Console WriteLine  LINQ      stopwatch ElapsedMilliseconds     ms            Assert AreEqual expected  s               Test      public void RemoveRegex                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace RemoveRegex input                      stopwatch Stop            Console WriteLine  Regex      stopwatch ElapsedMilliseconds     ms             Assert AreEqual expected  s               Test      public void RemoveRegexCompiled                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace RemoveRegexCompiled input                      stopwatch Stop            Console WriteLine  RegexCompiled      stopwatch ElapsedMilliseconds     ms             Assert AreEqual expected  s               Test      public void RemoveForLoop                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace RemoveForLoop input                      stopwatch Stop            Console WriteLine  ForLoop      stopwatch ElapsedMilliseconds     ms             Assert AreEqual expected  s               TestMethod      public void StringSplitThenJoin                 string s   null          var stopwatch   Stopwatch StartNew            for  int i   0  i  lt  iterations  i                          s   RemoveWhitespace StringSplitThenJoin input                      stopwatch Stop            Console WriteLine  StringSplitThenJoin      stopwatch ElapsedMilliseconds     ms             Assert AreEqual expected  s             Edit  Tested a nice one liner from Kernowcode

User · Answer

Just an alternative because it looks quite nice    - NOTE  Henks answer is the quickest of these  input ToCharArray     Where c   gt   Char IsWhiteSpace c     Select c   gt  c ToString      Aggregate  a  b    gt  a   b    Testing 1 000 000 loops on  quot This is a simple Test quot  This method   1 74 seconds Regex   2 58 seconds new String  Henks    0 82 seconds

User · Answer

Using Linq  you can write a readable method this way        public static string RemoveAllWhitespaces this string source                return string IsNullOrEmpty source    source   new string source Where x   gt   char IsWhiteSpace x   ToArray

User · Answer

We can use       public static string RemoveWhitespace this string input                if  input    null              return null          return new string input ToCharArray                Where c   gt   Char IsWhiteSpace c                ToArray

User · Answer

I have found different results to be true  I am trying to replace all whitespace with a single space and the regex was extremely slow   return  Regex  Replace  text  L  s    L           What worked the most optimally for me  in C   cli  was   String  ReduceWhitespace  String  text       String  newText    bool    inWhitespace   false    Int32   posStart   0    Int32   pos        0    for  pos   0  pos  lt  text- gt Length    pos           wchar t cc   text pos       if  Char  IsWhiteSpace  cc                 if   inWhitespace                   if  pos  gt  posStart   newText    text- gt Substring  posStart  pos - posStart            inWhitespace   true          newText    L                   posStart   pos   1            else             if  inWhitespace                   inWhitespace   false          posStart   pos                       if  pos  gt  posStart   newText    text- gt Substring  posStart  pos - posStart       return  newText        I tried the above routine first by replacing each character separately  but had to switch to doing substrings for the non-space sections  When applying to a 1 200 000 character string    the above routine gets it done in 25 seconds the above routine   separate character replacement in 95 seconds the regex aborted after 15 minutes

User · Answer

Regex is overkill  just use extension on string  thanks Henk   This is trivial and should have been part of the framework  Anyhow  here s my implementation   public static partial class Extension       public static string RemoveWhiteSpace this string self                return new string self Where c   gt   Char IsWhiteSpace c   ToArray

User · Answer

My solution is to use Split and Join and it is surprisingly fast  in fact the fastest of the top answers here   str   string Join     str Split default string     StringSplitOptions RemoveEmptyEntries      Timings for 10 000 loop on simple string with whitespace inc new lines and tabs   split join   60 milliseconds linq chararray   94 milliseconds regex   437 milliseconds   Improve this by wrapping it up in method to give it meaning  and also make it an extension method while we are at it      public static string RemoveWhitespace this string str        return string Join     str Split default string     StringSplitOptions RemoveEmptyEntries

User · Answer

Try the replace method of the string in C    XML Replace      string Empty

User · Answer

This is fastest way I know of  even though you said you didn t want to use regular expressions   Regex Replace XML     s

User · Answer

I have an alternative way without regexp  and it seems to perform pretty good  It is a continuation on Brandon Moretz answer    public static string RemoveWhitespace this string input         return new string input ToCharArray            Where c   gt   Char IsWhiteSpace c            ToArray          I tested it in a simple unit test    Test   TestCase  123 123 1adc  n 222    1231231adc222    public void RemoveWhiteSpace1 string input  string expected        string s   null      for  int i   0  i  lt  1000000  i                  s   input RemoveWhitespace              Assert AreEqual expected  s       Test   TestCase  123 123 1adc  n 222    1231231adc222    public void RemoveWhiteSpace2 string input  string expected        string s   null      for  int i   0  i  lt  1000000  i                  s   Regex Replace input     s                   Assert AreEqual expected  s       For 1 000 000 attempts the first option  without regexp  runs in less then a second  700 nbsp ms on my machine   and the second takes 3 5 seconds

User · Answer

If you need superb performance  you should avoid LINQ and regular expressions in this case  I did some performance benchmarking  and it seems that if you want to strip white space from beginning and end of the string  string Trim   is your ultimate function   If you need to strip all white spaces from a string  the following method works fastest of all that has been posted here       public static string RemoveWhitespace this string input                int j   0  inputlen   input Length          char   newarr   new char inputlen            for  int i   0  i  lt  inputlen    i                        char tmp   input i                if   char IsWhiteSpace tmp                                 newarr j    tmp                    j                                  return new String newarr  0  j

User · Answer

Here is a simple linear alternative to the RegEx solution  I am not sure which is faster  you d have to benchmark it   static string RemoveWhitespace string input        StringBuilder output   new StringBuilder input Length        for  int index   0  index  lt  input Length  index                  if   Char IsWhiteSpace input  index                         output Append input index                        return output ToString

User · Answer

I assume your XML response looks like this   var xml      lt names gt                   lt name gt                      foo                  lt  name gt                   lt name gt                      bar                  lt  name gt               lt  names gt      The best way to process XML is to use an XML parser  such as LINQ to XML   var doc   XDocument Parse xml    var containsFoo   doc Root                       Elements  name                         Any e   gt    string e  Trim       foo

User · Answer

Here is yet another variant   public static string RemoveAllWhitespace string aString      return String Join String Empty  aString Where aChar   gt  aChar  Char IsWhiteSpace aChar         As with most of the other solutions  I haven t performed exhaustive benchmark tests  but this works well enough for my purposes

[c#] Efficient way to remove ALL whitespace from String?

Examples related to c#

Examples related to removing-whitespace