The StringTokenizer is a class in the java.util package which is used to "token"ize strings. A token is a maximal sequence of consecutive characters that are not delimiters (default), or either one delimiter character, or a maximal sequence of consecutive characters that are not delimiters (the flag must be set to true).
It has 3 different constructors.
You have already learned about the Scanner class and how it parses the input for tokens. Remember that the input was delimited with a whitespace character unless you change it with the useDelimiter(aString) method.
Thus, unless you specify the delimiter to be used as the second argument value, the StringTokenizer will look for a white space character to tokenize the String in the first argument.
The third argument may be supplied to acccept the delimiter also as a token (true).
See the following program and its output:
import java.util.*;
public class tokenizer1 {
public static void main (String[] args) {
StringTokenizer st1 = new StringTokenizer("this is a test");
while (st1.hasMoreTokens()) {
System.out.println(st1.nextToken());
}
System.out.println ("--------------");
StringTokenizer st2 = new StringTokenizer("this is a test","s");
while (st2.hasMoreTokens()) {
System.out.println(st2.nextToken());
}
System.out.println ("--------------");
StringTokenizer st3 = new StringTokenizer("this is a test","s",true);
while (st3.hasMoreTokens()) {
System.out.println(st3.nextToken());
}
System.out.println ("--------------");
Scanner st4 = new Scanner ("this is a test");
while (st4.hasNext()){
System.out.println(st4.next());
}
System.out.println ("--------------");
Scanner st5 = new Scanner ("this is a test");
st5.useDelimiter ("s");
while (st5.hasNext()){
System.out.println(st5.next());
}
System.out.println ("--------------");
}
}
this
is
a
test
--------------
thi
i
a te
t
--------------
thi
s
i
s
a te
s
t
--------------
this
is
a
test
--------------
thi
i
a te
t
--------------
Implement a program that does the four basic operations (add, subtract, multiply, and divide) on two integers, and prints the result. Also count and display the number of valid basic operations performed. Read the data from a text file. Each line of file is organized as
first operand;operation;second operand
Note that first and second operands are integers and operation is one of "add", "subtract", "multiply", or "divide".
Sample file for input (mydata.txt) contains:
34;multiply;5 23;subtract;22 5;add;7 32;divide;5 -20;add;70 13;mod;5
/* * tokenizerSample version 1 * Author: Özlem ÖZGÜ * Date: 21/11/07 * Desc: Demonstrate the StringTokenizer class */
import java.util.StringTokenizer; import java.util.Scanner; import java.io.*;
public class tokenizerSample {
public int readAndCompute( String filename) throws IOException {
Scanner scan; StringTokenizer tokens; String line, one, two, three; int num1, num3, result, count; char c;
count = 0; result = 0; /* just to make the compiler happy */
scan = new Scanner( new File( filename));
while ( scan.hasNext()) {
line = scan.nextLine(); System.out.println( line);
tokens = new StringTokenizer( line, ";");
one = tokens.nextToken(); two = tokens.nextToken(); three = tokens.nextToken();
num1 = Integer.parseInt( one ); num3 = Integer.parseInt( three );
if (two.equalsIgnoreCase("add")) {
result = num1 + num3 ;
c = '+';
}
else if (two.equalsIgnoreCase("subtract")) {
result = num1 - num3 ;
c = '-';
}
else if (two.equalsIgnoreCase("multiply")) {
result = num1 * num3 ;
c = '*';
}
else if (two.equalsIgnoreCase("divide")) {
result = num1 / num3 ;
c = '/';
}
else {
c = '#';
}
if (c !='#') {
System.out.println (""+num1+c+num3+"="+result);
count ++;
}
else
System.out.println ("The operation, "+two+" is undefined!") ;
} return count; }
public static void main(String[] args) throws IOException {
tokenizerSample thiss = new tokenizerSample () ;
int correctOperators = thiss.readAndCompute( "mydata.txt" ) ;
System.out.println ("mydata.txt contains "+correctOperators+" correct operators.");
}
}
Count and display the number of words in an input text (single line). Each word is separated with a single space. There is no space at the end of the input string. Consider the following program execution as an example (the input is in italics for clarity):
This is Didem. She is my sister. She teaches English in Buyukhanli
Ilkogretim Okulu.
The input you have supplied has 14 words.
import java.util.*;
public class tokenizer2 {
public static void main (String[] args) {
Scanner oku;
oku = new Scanner (System.in);
String input; input = oku.nextLine(); StringTokenizer st1 = new StringTokenizer(input);
System.out.print ("The input you have supplied has ");
System.out.print ( st1.countTokens() );
System.out.println (" words.");
} }
import java.util.*;
public class scanner2 {
public static void main (String[] args) {
Scanner oku;
oku = new Scanner (System.in);
String input; input = oku.nextLine(); oku = new Scanner(input);
int count=0;
String S;
while (oku.hasNext()){
S = oku.next();
count++;
}
System.out.print ("The input you have supplied has ");
System.out.print ( count );
System.out.println (" words.");
} }
import java.util.*;
public class string2 {
public static void main (String[] args) {
Scanner oku;
oku = new Scanner (System.in);
String input; input = oku.nextLine();
int count = 1;/*because there is no space at the end of the last word*/
for (int index=0;index<input.length();index++) {
if (input.charAt(index) == ' ' ) { // search for a space
count++;
}
}
System.out.print ("The input you have supplied has ");
System.out.print ( count );
System.out.println (" words.");
} }
For more details, visit this web page: http://java.sun.com/javase/6/docs/api/java/util/StringTokenizer.html