Scanner vs StringTokenizer vs String Split

Question

I just learned about Java s Scanner class and now I m wondering how it compares competes with the StringTokenizer and String Split  I know that the StringTokenizer and String Split only work on Strings  so why would I want to use the Scanner for a String  Is Scanner just intended to be one-stop-shopping for spliting

User · Answer

One important difference is that both String.split() and Scanner can produce empty strings but StringTokenizer never does it.

For example:

String str = "ab cd  ef";

StringTokenizer st = new StringTokenizer(str, " ");
for (int i = 0; st.hasMoreTokens(); i++) System.out.println("#" + i + ": " + st.nextToken());

String[] split = str.split(" ");
for (int i = 0; i < split.length; i++) System.out.println("#" + i + ": " + split[i]);

Scanner sc = new Scanner(str).useDelimiter(" ");
for (int i = 0; sc.hasNext(); i++) System.out.println("#" + i + ": " + sc.next());

Output:

//StringTokenizer
#0: ab
#1: cd
#2: ef
//String.split()
#0: ab
#1: cd
#2: 
#3: ef
//Scanner
#0: ab
#1: cd
#2: 
#3: ef

This is because the delimiter for String.split() and Scanner.useDelimiter() is not just a string, but a regular expression. We can replace the delimiter " " with " +" in the example above to make them behave like StringTokenizer.

User · Answer

String split   works very good but has its own boundaries  like if you wanted to split a string as shown below based on single or double pipe     symbol  it doesn t work  In this situation you can use StringTokenizer   ABC IJK

User · Answer

String split seems to be much slower than StringTokenizer  The only advantage with split is that you get an array of the tokens  Also you can use any regular expressions in split       org apache commons lang StringUtils has a split method which works much more faster than any of two viz  StringTokenizer or String split       But the CPU utilization for all the three is nearly the same  So we also need a method which is less CPU intensive  which I am still not able to find

User · Answer

Split is slow  but not as slow as Scanner   StringTokenizer is faster than split  However  I found that I could obtain double the speed  by trading some flexibility  to get a speed-boost  which I did at JFastParser https   github com hughperkins jfastparser  Testing on a string containing one million doubles   Scanner  10642 ms Split  715 ms StringTokenizer  544ms JFastParser  290ms

User · Answer

I recently did some experiments about the bad performance of String split   in highly performance sensitive situations  You may find this useful     http   eblog chrononsystems com hidden-evils-of-javas-stringsplit-and-stringr  The gist is that String split   compiles a Regular Expression pattern each time and can thus slow down your program  compared to if you use a precompiled Pattern object and use it directly to operate on a String

User · Answer

StringTokenizer was always there  It is the fastest of all  but the enumeration-like idiom might not look as elegant as the others   split came to existence on JDK 1 4  Slower than tokenizer but easier to use  since it is callable from the String class   Scanner came to be on JDK 1 5  It is the most flexible and fills a long standing gap on the Java API to support an equivalent of the famous Cs scanf function family

User · Answer

For the default scenarios I would suggest Pattern split   as well but if you need maximum performance  especially on Android all solutions I tested are quite slow  and you only need to split by a single char  I now use my own method   public static ArrayList lt String gt  splitBySingleChar final char   s          final char splitChar        final ArrayList lt String gt  result   new ArrayList lt String gt         final int length   s length      int offset   0      int count   0      for  int i   0  i  lt  length  i              if  s i     splitChar                if  count  gt  0                    result add new String s  offset  count                              offset   i   1              count   0            else               count                        if  count  gt  0            result add new String s  offset  count              return result      Use  abc  toCharArray   to get the char array for a String  For example   String s         a bb   ccc  dddd eeeee  ffffff    ggggggg    ArrayList lt String gt  result   splitBySingleChar s toCharArray

User · Answer

If you have a String object you want to tokenize  favor using String s split method over a StringTokenizer   If you re parsing text data from a source outside your program  like from a file  or from the user  that s where a Scanner comes in handy

User · Answer

Let s start by eliminating StringTokenizer  It is getting old and doesn t even support regular expressions  Its documentation states      StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code  It is recommended that anyone seeking this functionality use the split method of String or the java util regex package instead    So let s throw it out right away  That leaves split   and Scanner  What s the difference between them   For one thing  split   simply returns an array  which makes it easy to use a foreach loop   for  String token   input split    s              Scanner is built more like a stream   while  myScanner hasNext          String token   myScanner next                or  while  myScanner hasNextDouble          double token   myScanner nextDouble                 It has a rather large API  so don t think that it s always restricted to such simple things    This stream-style interface can be useful for parsing simple text files or console input  when you don t have  or can t get  all the input before starting to parse   Personally  the only time I can remember using Scanner is for school projects  when I had to get user input from the command line  It makes that sort of operation easy  But if I have a String that I want to split up  it s almost a no-brainer to go with split

User · Answer

They re essentially horses for courses    Scanner is designed for cases where you need to parse a string  pulling out data of different types  It s very flexible  but arguably doesn t give you the simplest API for simply getting an array of strings delimited by a particular expression  String split   and Pattern split   give you an easy syntax for doing the latter  but that s essentially all that they do  If you want to parse the resulting strings  or change the delimiter halfway through depending on a particular token  they won t help you with that  StringTokenizer is even more restrictive than String split    and also a bit fiddlier to use  It is essentially designed for pulling out tokens delimited by fixed substrings  Because of this restriction  it s about twice as fast as String split     See my comparison of String split   and StringTokenizer   It also predates the regular expressions API  of which String split   is a part    You ll note from my timings that String split   can still tokenize thousands of strings in a few milliseconds on a typical machine  In addition  it has the advantage over StringTokenizer that it gives you the output as a string array  which is usually what you want  Using an Enumeration  as provided by StringTokenizer  is too  syntactically fussy  most of the time  From this point of view  StringTokenizer is a bit of a waste of space nowadays  and you may as well just use String split

[java] Scanner vs. StringTokenizer vs. String.Split

Examples related to java

Examples related to java.util.scanner

Examples related to tokenize

Examples related to split