Does C have a String Tokenizer like Java s

Question

I m doing simple string input parsing and I am in need of a string tokenizer  I am new to C  but have programmed Java  and it seems natural that C  should have a string tokenizer  Does it  Where is it  How do I use it

User · Answer

The split method of a string is what you need  In fact the tokenizer class in Java is deprecated in favor of Java s string split method

User · Answer

use Regex Split string

User · Answer

The split method of a string is what you need  In fact the tokenizer class in Java is deprecated in favor of Java s string split method

User · Answer

read this  split function has an overload takes an array consist of seperators http   msdn microsoft com en-us library system stringsplitoptions aspx

User · Answer

The similar to Java s method is   Regex Split string  pattern     where   string - the text you need to split pattern - string type pattern  what is splitting the text

User · Answer

I just want to highlight the power of C  s Split method and give a more detailed comparison  particularly from someone who comes from a Java background   Whereas StringTokenizer in Java only allows a single delimiter  we can actually split on multiple delimiters making regular expressions less necessary  although if one needs regex  use regex by all means    Take for example this   str Split new char                        This splits on three different delimiters returning an array of tokens   We can also remove empty arrays with what would be a second parameter for the above example   str Split new char                      StringSplitOptions RemoveEmptyEntries    One thing Java s String tokenizer does have that I believe C  is lacking  at least Java 7 has this feature  is the ability to keep the delimiter s  as tokens   C  s Split will discard the tokens   This could be important in say some NLP applications  but for more general purpose applications this might not be a problem

User · Answer

If you re trying to do something like splitting command line arguments in a  NET Console app  you re going to have issues because  NET is either broken or is trying to be clever  which means it s as good as broken   I needed to be able to split arguments by the space character  preserving any literals that were quoted so they didn t get split in the middle  This is the code I wrote to do the job   private static List lt String gt  Tokenise string value  char seperator        List lt string gt  result   new List lt string gt         value   value Replace            Replace            Trim        StringBuilder sb   new StringBuilder        bool insideQuote   false      foreach char c in value ToCharArray                  if c                               insideQuote    insideQuote                    if  c    seperator   amp  amp   insideQuote                        if  sb ToString   Trim   Length  gt  0                                result Add sb ToString   Trim                     sb Clear                                    else                       sb Append c                       if  sb ToString   Trim   Length  gt  0                result Add sb ToString   Trim                return result

User · Answer

If you are using C  3 5 you could write an extension method to System String that does the splitting you need  You then can then use syntax   string SplitByMyTokens      More info and a useful example from MS here http   msdn microsoft com en-us library bb383977 aspx

User · Answer

words   new List lt string gt  YourText ToLower   Trim   n     r   Split                   Select x   gt  new string x Where Char IsLetter  ToArray          Or    words   new List lt string gt  YourText Trim   n     r   Split                   Select x   gt  new string x Where Char IsLetterOrDigit  ToArray

User · Answer

For complex splitting you could use a regex creating a match collection

User · Answer

You could use String Split method   class ExampleClass       public ExampleClass                 string exampleString    there is a cat              Split string on spaces  This will separate all the words in a string         string   words   exampleString Split               foreach  string word in words                        Console WriteLine word                  there                is                a                cat                     For more information see Sam Allen s article about splitting strings in c   Performance  Regex

User · Answer

I think the nearest in the  NET Framework is    string Split

User · Answer

I think the nearest in the  NET Framework is    string Split

User · Answer

words   new List lt string gt  YourText ToLower   Trim   n     r   Split                   Select x   gt  new string x Where Char IsLetter  ToArray          Or    words   new List lt string gt  YourText Trim   n     r   Split                   Select x   gt  new string x Where Char IsLetterOrDigit  ToArray

User · Answer

For complex splitting you could use a regex creating a match collection

User · Answer

I think the nearest in the  NET Framework is    string Split

User · Answer

If you are using C  3 5 you could write an extension method to System String that does the splitting you need  You then can then use syntax   string SplitByMyTokens      More info and a useful example from MS here http   msdn microsoft com en-us library bb383977 aspx

User · Answer

You could use String Split method   class ExampleClass       public ExampleClass                 string exampleString    there is a cat              Split string on spaces  This will separate all the words in a string         string   words   exampleString Split               foreach  string word in words                        Console WriteLine word                  there                is                a                cat                     For more information see Sam Allen s article about splitting strings in c   Performance  Regex

User · Answer

If you re trying to do something like splitting command line arguments in a  NET Console app  you re going to have issues because  NET is either broken or is trying to be clever  which means it s as good as broken   I needed to be able to split arguments by the space character  preserving any literals that were quoted so they didn t get split in the middle  This is the code I wrote to do the job   private static List lt String gt  Tokenise string value  char seperator        List lt string gt  result   new List lt string gt         value   value Replace            Replace            Trim        StringBuilder sb   new StringBuilder        bool insideQuote   false      foreach char c in value ToCharArray                  if c                               insideQuote    insideQuote                    if  c    seperator   amp  amp   insideQuote                        if  sb ToString   Trim   Length  gt  0                                result Add sb ToString   Trim                     sb Clear                                    else                       sb Append c                       if  sb ToString   Trim   Length  gt  0                result Add sb ToString   Trim                return result

User · Answer

The split method of a string is what you need  In fact the tokenizer class in Java is deprecated in favor of Java s string split method

User · Answer

The split method of a string is what you need  In fact the tokenizer class in Java is deprecated in favor of Java s string split method

User · Answer

use Regex Split string

User · Answer

If you are using C  3 5 you could write an extension method to System String that does the splitting you need  You then can then use syntax   string SplitByMyTokens      More info and a useful example from MS here http   msdn microsoft com en-us library bb383977 aspx

User · Answer

I just want to highlight the power of C  s Split method and give a more detailed comparison  particularly from someone who comes from a Java background   Whereas StringTokenizer in Java only allows a single delimiter  we can actually split on multiple delimiters making regular expressions less necessary  although if one needs regex  use regex by all means    Take for example this   str Split new char                        This splits on three different delimiters returning an array of tokens   We can also remove empty arrays with what would be a second parameter for the above example   str Split new char                      StringSplitOptions RemoveEmptyEntries    One thing Java s String tokenizer does have that I believe C  is lacking  at least Java 7 has this feature  is the ability to keep the delimiter s  as tokens   C  s Split will discard the tokens   This could be important in say some NLP applications  but for more general purpose applications this might not be a problem

User · Answer

I think the nearest in the  NET Framework is    string Split

User · Answer

You could use String Split method   class ExampleClass       public ExampleClass                 string exampleString    there is a cat              Split string on spaces  This will separate all the words in a string         string   words   exampleString Split               foreach  string word in words                        Console WriteLine word                  there                is                a                cat                     For more information see Sam Allen s article about splitting strings in c   Performance  Regex

User · Answer

You could use String Split method   class ExampleClass       public ExampleClass                 string exampleString    there is a cat              Split string on spaces  This will separate all the words in a string         string   words   exampleString Split               foreach  string word in words                        Console WriteLine word                  there                is                a                cat                     For more information see Sam Allen s article about splitting strings in c   Performance  Regex

User · Answer

The similar to Java s method is   Regex Split string  pattern     where   string - the text you need to split pattern - string type pattern  what is splitting the text

User · Answer

If you are using C  3 5 you could write an extension method to System String that does the splitting you need  You then can then use syntax   string SplitByMyTokens      More info and a useful example from MS here http   msdn microsoft com en-us library bb383977 aspx

User · Answer

For complex splitting you could use a regex creating a match collection

User · Answer

For complex splitting you could use a regex creating a match collection

User · Answer

read this  split function has an overload takes an array consist of seperators http   msdn microsoft com en-us library system stringsplitoptions aspx

[c#] Does C# have a String Tokenizer like Java's?

Examples related to c#

Examples related to string

Examples related to parsing