---

Nested loop example: parsing

---

• Problem description
  • Problem description:

    You are given a line of text Example: " Hello Mr. Sunshine, How are you ?" Break the line of text into individual words (tokens) The words (tokens) are separated by at least 1 space Example output: input: " Hello Mr. Sunshine, How are you ?" output: "Hello" "Mr." "Sunshine," "How" "are" "you" "?"

    Note:

    The process of finding words (or tokens) in an input text is one of the (many) tasks in the technique called parsing There may be leading space in the input line !!!

  ---

  ---

• Reading in a line (with space)
  • The following method in the java.util.Scanner class will return every character typed until the RETURN key is pressed:

    nextLine()

  • Sample usage:

    Scanner input = new Scanner(System.in); String s; s = input.nextLine(); // Read a line and store in s

  ---

  ---

  ---

  ---

  ---

  ---

• Developing the algorithm: how would you handle the problem ?
  • A very rudimentary algorithm:

    scan_pos = 0; while ( not done scanning input line ) { scan for the next word starting at "scan_pos" advance "scan_pos" to the end of the next word }

    Example:

    0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | scan_pos=0 1a. scan for the next word starting at "scan_pos=0" 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | scan_pos=0 1b. advance "scan_pos" to the end of the next word 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | scan_pos=6 2a. scan for the next word starting at "scan_pos=6" 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | scan_pos=6 2b. advance "scan_pos" to the end of the next word 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | scan_pos=13 and so on...

    ---

    ---

  • Refining the algorithm using a concrete example:

    Input line: 0123456789012345678901234 <--- ruler s = " abc Hello 123 " We start scanning the input line from the first character: 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | left=0 Repeat the following tasks: Find the start of the next word. How to do it: Skip over leading spaces: 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | left=3 Result: we found the start of a new word Find the end of the current word How to do it: find the first space (while holding on to the start of the current word !!!) 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ ^ | | left=3 right=6 We must use another variable (called: right) to find the end of the current word ! Yippiee ! We found the word. We can extract the located word with the substring() method: String t; t = s.substring( left, right ); // extra word found We are done finding one word Advance the scan position to the end of the word found This is achieved with the following statement: left = right; Explanation: Situation before the statement "left = right;": 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ ^ | | left=3 right=6 Situation after the statement "left = right;": 0123456789012345678901234 <--- ruler s = " abc Hello 123 " ^ | left=6 ready to scan for the next word !!! (right=6, but this variable is not relevant) Important note: At the end of the loop, we must set up the variables so that the loop can be re-started !!!

    ---

  • Pseudo code:

    s = input line; left = 0; while ( left < s.length() ) { make the "left" index variable to skip over leading space; // Result: left points to the start of the next word use "right" index variable to find the end of the current word; t = s.substring( left, right ); // extract word System.out.println( t ); // Print it left = right; // advance scan position to the end of the next word }

    ---

    ---

  • Java program:

    import java.util.Scanner; public class Parse01 { public static void main(String[] args) { Scanner input = new Scanner(System.in); String s, t; System.out.print("Enter line:"); s = input.nextLine(); System.out.println("Input line = `" + s + "'"); int left, right; left = 0; while ( left < s.length() ) { /* ---------------------------------------------------------- Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ | left=0 Next task: Find start of word by: Skipping leading spaces ---------------------------------------------------------- */ while ( s.charAt(left) == ' ' ) left++; /* ------------------------------------------------------ Achieved: left points to the first non-space character Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ | left=3 I.e.: we found the start of the next word Next task: find the end of the word ------------------------------------------------------ */ /* ------------------------------ Look for the end of the word ------------------------------ */ right = left+1; while ( s.charAt(right) != ' ' ) right++; /* ------------------------------------------------------ Achieved: right points to the first space character I.e.: we found the end of the next word Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ ^ | | left=3 right=9 ------------------------------------------------------ */ t = s.substring(left, right); // Extract word found System.out.println("Next word found: `" + t + "'"); /* ------------------------------------------------------ PREPARE to LOOP !!! First task of loop is: Skip leading spaces We need to set up left !!! Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ ^ | | left=3 right=9 Situation you need to be to restart loop: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ | left=9 (so the loop can find next word !) ------------------------------------------------------ */ left = right; } } }

    ---

  • Example Program: (Demo above code)                                                
    • Prog file: click here

    How to run the program:

    Right click on link and save in a scratch directory To compile:   javac Parse01.java To run:          java Parse01

    ---

  • There is some errors in the above program (because we are not careful)

    Example:

    >> java Parse01 Enter line: abc def hij klm Input line = ` abc def hij klm ' Next word found: `abc' Next word found: `def' Next word found: `hij' Next word found: `klm' Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 23 at java.lang.String.charAt(String.java:558) at Parse01.main(Parse01.java:35) Line 35: // Next task: Skip leading spaces while ( s.charAt(left) == ' ' ) left++; Problem: Input line = ` abc def hij klm ' ^ | left You will scan PASS the end of the line !!!

  ---

  ---

• Corrected version of the algorithm: make sure you don't pass the end of the line when you scan !
  • Corrected algorithm: (I have highlighted the places where I made changes to the above algorithm in red)

    import java.util.Scanner; public class Parse02 { public static void main(String[] args) { Scanner input = new Scanner(System.in); String s, t; System.out.print("Enter line:"); s = input.nextLine(); System.out.println("Input line = `" + s + "'"); int left, right; left = 0; while ( left < s.length() ) { /* -------------------------------------------------------- Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ | left=0 Next task: Skip leading spaces CAREFULLY -------------------------------------------------------- */ while ( left < s.length() && s.charAt(left) == ' ' ) left++; /* ------------------------------------------------------ Achieved: left points to the first non-space character Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ | left=3 I.e.: we found the start of the next word OR: we reached the end ! Next task: find the end of the word ------------------------------------------------------ */ /* --------------------------------------- Check if we have reached the end... --------------------------------------- */ if ( left == s.length() ) break; // break out of the loop.... /* ------------------------------------------------ No, so look for the end of the word CAREFULLY ------------------------------------------------ */ right = left+1; while ( right < s.length() && s.charAt(right) != ' ' ) right++; /* ------------------------------------------------------ Achieved: right points to the first space character I.e.: we found the end of the next word Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ ^ | | left=3 right=9 ------------------------------------------------------ */ t = s.substring(left, right); // Extract word found System.out.println("Next word found: `" + t + "'"); /* ------------------------------------------------------ PREPARE to LOOP !!! First task of loop is: Skip leading spaces We need to set up left !!! Situation right now: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ ^ | | left=3 right=9 Situation you need to be to restart loop: 0123456789012345 sssxxxxxxsssxxxs (s = space, x = non-space) ^ | left=9 (so the loop can find next word !) ------------------------------------------------------ */ left = right; } } }

    ---

  • Example Program: (Demo above code)                                                
    • Prog file: click here

    How to run the program:

    Right click on link and save in a scratch directory To compile:   javac Parse02.java To run:          java Parse02