StringTokenizer class in Java

By Anubhav Srivastava

Hi coders! In this tutorial, we are going to learn the StringTokenizer class in Java. StringTokenizer class is used to break a string into parts. Individual part of the string is known as a token hence, the process of converting a string into a token is called tokenization. The process of tokenization is done with the help of delimiters. There are various delimiters (“,” or “/” or white space). The default delimiter used for breaking the string into tokens is space or tab or a new-line.

Constructors of StringTokenizer class in Java

• StringTokenizer(String str): This constructor is used to break the string into tokens with the default delimiter space.
• StringTokenizer(String str, String delimit): This constructor tokenizes the string with the given delimiter.
• StringTokenizer(String str, String delimit, boolean delimiterAsToken): This constructor tokenizes the string with the given delimiter including the given delimiter also as a token.

Methods of StringTokenizer class

• boolean hasMoreToken: This function returns true till there is token in the string.
• int countTokens: This function returns the number of tokens in the string.
• String nextToken: This function returns the token.

Java Code for StringTokenizer class

import java.util.StringTokenizer;

public class StringTokenize {

  // driver function

  public static void main(String[] args) {
    String str = "This is, an example, of StringTokenizer, class";
    /* object creation of StringTokenizer */
    StringTokenizer st = new StringTokenizer( str);
    /* countTokens() is used to count number of tokens*/
    System.out.println("number of tokens in the given string: "+ st.countTokens());
    System.out.println("Tokens in the string: ");
    /* hasMoreTokens() return true while there is token */
    while(st.hasMoreTokens()) {
      /* returns the token as string */
      System.out.println(st.nextToken());
    }
    /* "," as delimiter */
    StringTokenizer st1 = new StringTokenizer( str,",");
    System.out.println("number of tokens in the given string: "+ st1.countTokens());
    System.out.println("Tokens in the string: ");
    while(st1.hasMoreTokens()) {
      System.out.println(st1.nextToken());
    
  } /* "," as delimiter and also as token " */
    StringTokenizer st2 = new StringTokenizer( str,",",true);
    System.out.println("number of tokens in the given string: "+ st2.countTokens());
    System.out.println("Tokens in the string: ");
    while(st2.hasMoreTokens()) {
      System.out.println(st2.nextToken());
    
  }

}
}
//Code is provided by Anubhav Srivastava

Output:

number of tokens in the given string: 7
Tokens in the string: 
This
is,
an
example,
of
StringTokenizer,
class
number of tokens in the given string: 4
Tokens in the string: 
This is
an example
of StringTokenizer
class
number of tokens in the given string: 7
Tokens in the string: 
This is
,
an example
,
of StringTokenizer
,
class

Also, read:

How does a carriage return (\r) works in Java

How to convert Uppercase String to Lowercase in Java