4
\$\begingroup\$

This is a small program written for a small assignment to check spelling. The assignment asks for the following output:

  1. A list of words that are misspelled in the file
  2. A count of the whitespace delimited words contained in the file
  3. For all words that appear in the file, a count of the number of times they appear. This should be in alphabetical order
  4. A list of the top five words in terms of frequencey of appearance from the file.

My program runs successfully. I just want to get some suggestions to improve it.

import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.Scanner;


public class SpellChecker {

private final HashDict dict;
private final HashDict wordFile;
final static String dictionary = "dict.txt";
final static String file = ("big_flat_file.txt");

/**
 * Constructor of spellChecker
 */
public SpellChecker() {

    dict = new HashDict<>();
    wordFile = new HashDict<>();
    read(dictionary);

}

/**
 * @param args the command line arguments
 */
public static void main(String[] args) {
    SpellChecker checker = new SpellChecker();
    int wordCount = checker.count(file);

    System.out.println("The file contains following misspelled words: ");
    checker.spellCheck();
    System.out.println("The file contains " + wordCount + " words in total");
    System.out.println();
    System.out.println("The frequency of all words are listed below: ");
    checker.wordFreq();

}

/**
 * read dictionary into a hashed dictionary
 *
 * @param fileName the file of dictionary
 */
public void read(String fileName) {
    File theFile = new File(fileName);

    try {
        Scanner reader = new Scanner(theFile);

        while (reader.hasNext()) {
            String input = reader.next();
            dict.add(input, 0);
        }

    } catch (FileNotFoundException e) {
        System.out.print("file not found");
    }

}

/**
 * add every word into a hashed dictionary as the key, its frequency as
 * value, and count the total words
 *
 * @param fileName a given .txt file
 * @return an integer of total words in the file
 */
public int count(String fileName) {
    File theFile = new File(fileName);
    int totalCount = 0;

    try {
        Scanner sc = new Scanner(theFile);

        while (sc.hasNext()) {
            int freq = 0;
            String word = sc.next().replaceAll("[^A-Za-z]+", "").toLowerCase();

            totalCount++;
            if (wordFile.contains(word)) {
                freq = (int) wordFile.getValue(word.toLowerCase());
            }
            freq++;
            wordFile.add(word, freq);
        }
    } catch (FileNotFoundException ex) {

    }
    return totalCount;

}

/**
 * Check every word in the file to see if it is misspelled by comparing it
 * with the dictionary. Ignore all the single letters. Print out the word
 * that is not contained in the dictionary
 */
public void spellCheck() {
    Iterator traverse = wordFile.getKeyIterator();

    while (traverse.hasNext()) {
        String e = (String) traverse.next();
        if (!e.matches("[A-Za-z]{1}")) {
            if (!dict.contains(e)) {
                System.out.println(e);
            }
        }
    }


}

/**
 * Count the time of all words appear in the file, and list them in 
 * alphabetical order.
 * List top five words that appear most. 
 */
public void wordFreq() {
    Iterator traverse = wordFile.getKeyIterator();
    ArrayList<String> list = new ArrayList<>();
    int top1 = 0, top2 = 0, top3 = 0, top4 = 0, top5 = 0;
    int top1Index = 0, top2Index = 0, top3Index = 0, top4Index = 0, 
            top5Index = 0;
    while (traverse.hasNext()) {
        String e = (String) traverse.next();
        list.add(e);
        Collections.sort(list, String.CASE_INSENSITIVE_ORDER);

    }
    // find the top 5 words that appear most frequent
    for (int i = 0; i < list.size(); i++) {
        System.out.println(list.get(i) + " " + wordFile.getValue(list.get(i)));
        int freq = (int) wordFile.getValue(list.get(i));
        if (freq > top1) {
            top1 = freq;
            top1Index = i;
        } else if (freq > top2) {
            top2 = freq;
            top2Index = i;
        } else if (freq > top3) {
            top3 = freq;
            top3Index = i;
        } else if (freq > top4) {
            top4 = freq;
            top4Index = i;
        } else if (freq > top5) {
            top5 = freq;
            top5Index = i;
        }

    }
    System.out.println();
    System.out.println("The top 5 frequent used words in the file are");
    System.out.println(list.get(top1Index) + " " + top1);
    System.out.println(list.get(top2Index) + " " + top2);
    System.out.println(list.get(top3Index) + " " + top3);
    System.out.println(list.get(top4Index) + " " + top4);
    System.out.println(list.get(top5Index) + " " + top5);
}

}

The HashDict is my own written chained dictionary and is checked by my instructor. I am not going to put it here, but I will if somebody asks for it.

----------------------------------update code-------------------------------------- Following are my updated code based on suggestions I've got.

import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Iterator;
import java.util.Scanner;


public class SpellChecker {

private final HashDict dict;
private final HashDict wordFile;
final static String dictionary = "dict.txt";


/**
 * Constructor of spellChecker
 */
public SpellChecker() {

    dict = new HashDict<>();
    wordFile = new HashDict<>();
    readDict(dictionary);

}

/**
 * @param args the command line arguments
 */
public static void main(String[] args) {
    Scanner in = new Scanner(System.in);
    System.out.println("What is the filename?");
    String inputFileName = in.nextLine();

    SpellChecker checker = new SpellChecker();

    int wordCount = checker.loadFileAndCount(inputFileName);
    checker.printMisspelledWords();
    System.out.println();
    System.out.println("The file contains " + wordCount + " words in total");
    System.out.println();

    checker.wordFreq();

}

/**
 * readDict dictionary into a hashed dictionary
 *
 * @param fileName the file of dictionary
 */
public void readDict(String fileName) {
    File theFile = new File(fileName);

    try {
        Scanner reader = new Scanner(theFile);

        while (reader.hasNext()) {
            String input = reader.next();
            dict.add(input, 0);
        }

    } catch (FileNotFoundException e) {
        System.out.print("file not found");
    }

}

/**
 * add every word into a hashed dictionary as the key, its frequency as
 * value, and loadFileAndCount the total words
 *
 * @param fileName a given .txt file
 * @return an integer of total words in the file
 */
public int loadFileAndCount(String fileName) {
    File theFile = new File(fileName);
    int totalCount = 0;

    try {
        Scanner sc = new Scanner(theFile);

        while (sc.hasNext()) {
            int freq = 0;
            String word = sc.next().replaceAll("[^A-Za-z]+", "").toLowerCase();

            totalCount++;
            if (wordFile.contains(word)) {
                freq = (int) wordFile.getValue(word.toLowerCase());
            }
            freq++;
            wordFile.add(word, freq);
        }
    } catch (FileNotFoundException ex) {
        System.out.println("File not found.");
    }
    return totalCount;

}

/**
 * Check every word in the file to see if it is misspelled by comparing it
 * with the dictionary. Ignore all the single letters. Print out the word
 * that is not contained in the dictionary
 */
public void printMisspelledWords() {
    Iterator traverse = wordFile.getKeyIterator();
    System.out.println("The file contains following misspelled words: ");
    while (traverse.hasNext()) {
        String e = (String) traverse.next();
        if (!(e.length() == 1 && Character.isLetter(e.toCharArray()[0]))) {
            if (!dict.contains(e)) {
                System.out.println(e);
            }
        }
    }

}

/**
 * Count the time of all words appear in the file, and list them in
 * alphabetical order. List top five words that appear most.
 */
public void wordFreq() {
    Iterator traverse = wordFile.getKeyIterator();
    ArrayList<String> list = new ArrayList<>();
    int[] topFreq = new int[5];
    int[] topFreqIndex = new int[5];

    while (traverse.hasNext()) {
        String e = (String) traverse.next();
        list.add(e);
    }
    Collections.sort(list, String.CASE_INSENSITIVE_ORDER);
    System.out.println("The frequency of all words are listed below: ");
    // print out all words and their frequencies, and
    // find the top 5 words that appear most frequent
    for (int i = 0; i < list.size(); i++) {
        int freq = (int) wordFile.getValue(list.get(i));
        System.out.println(list.get(i) + " " + freq);
        for (int m = topFreq.length - 1; m >= 0; m--) {
            if (freq > topFreq[m]) {
                if (m == topFreq.length - 1) {
                    topFreq[m] = freq;
                    topFreqIndex[m] = i;
                } else {
                    int tempFreq = topFreq[m];
                    int tempFreqIndex = topFreqIndex[m];
                    topFreq[m] = freq;
                    topFreqIndex[m] = i;
                    topFreq[m + 1] = tempFreq;
                    topFreqIndex[m + 1] = tempFreqIndex;
                }

            }

        }

    }
    System.out.println();
    System.out.println("The top 5 frequent used words in the file are");

    for (int m = 0; m < topFreq.length; m++) {
        System.out.println(list.get(topFreqIndex[m]) + " " + topFreq[m]);

    }
}

}

Most of the changes are in the wordFreq method.

The output is like this:

run:
What is the filename?
big_flat_file.txt
The file contains following misspelled words: 
// all misspelled words
The file contains 6629 words in total
The frequency of all words are listed below: 
// all words and their frequency. TL
The top 5 frequent used words in the file are
of 262
and 248
to 246
the 183
 167

One thing concerned me is that the last top frequent word is a blank space. The blank space also appears in the word count, so that it is not the topFreq method's problem. I am not sure why it is counted as a word.

\$\endgroup\$
2
\$\begingroup\$

I don't know Java well, so I'll just be writing about more general ideas.

Zero One Infinity Rule: This is a guideline which says: "Allow none of foo, one of foo, or any number of foo." This applies in your wordFreq() method. When you have a lot of variables like top1, top2, etc. you're giving yourself a lot more opportunity to make a typo, and there's more code you have to change if you want the top 10 instead of the top 5. Instead, store these in an array (or ArrayList, or whatever the most appropriate container is)

Bug: I'm not convinced wordFreq is correct. I'm unsure of the ethics or the CodeReview policy on describing bugs for homework assignments, so I will be cautious and say nothing more at this time.

Performance: Look at

while (traverse.hasNext()) {
        String e = (String) traverse.next();
        list.add(e);
        Collections.sort(list, String.CASE_INSENSITIVE_ORDER);

    }

First, why do you need this to be sorted? Second, I suspect resorting the list every time you add a new element is wasteful. Of course, it will have no noticeable impact in a program this small, but it is good to be mindful of algorithmic inefficiencies.

I/O: To improve flexibility, avoid hard-coding pathnames. It would be better to accept them as command-line arguments. You could use the hard-coded names as default values if the user doesn't supply an arguments. This is helpful if, for example, you want to use a script to run your program against a lot of input-files as a test of some sort.

Error handling: If you are going to hardcode file names, it would be good to make that clear in your exception text. Right now if it can't find the dictionary, I will just see the message "file not found", and I have no way to know what you want without opening the code. Including the name of the file not found in the error message will make the program easier to use.

public int count(String fileName) {
    try {
            ...
        } catch (FileNotFoundException ex) {

        }
}

If the intent is to keep going with an empty word list if there is no file, this needs to be more explicit. A comment would help clarify. Or better, an explicit check to see if the file exists. (I am unsure how using exceptions as control-flow is considered in the Java world. It is usually frowned upon). If this is not the intent, the error needs to be handled.

Naming: I found your variable names to be mostly clear. It confused me that spellCheck wrote output though. printMisspelledWords would be much more clear. In general, it is recommended for methods to be verbs and classes and objects to be nouns.

Some of your names are vague. I had to keep checking which file read() was supposed to be read. It is surprising that count() not only returns the word count, but loads the dictionary too. You could be more clear my having functions like loadDictionary() loadWordList(), and getWordCount().

\$\endgroup\$
  • \$\begingroup\$ Thank you very much for all the suggestions. They are very useful. I just want to explain some of your confusions. 1. I want the list of words to be sorted because the assignment asks for the word and their frequency to be listed in alphabetical order. But you are correct that the sorting should not be in the while loop. 2. I edited the error handling part by adding a line of "file not found". But I am not sure if that is what you mean by being user friendly. 3. I do all the counting and loading file in one method because I thought it is efficient to count the frequency when reading it in. \$\endgroup\$ – Yuhe Zhu May 10 '18 at 15:52
1
\$\begingroup\$

since I know Java :) I have the following comments to add to the previous answer:

resources handling

Resources, in this context, are the input files you are reading. Their lifecycle is not handled properly. In other words, you do not close the files, leaving OS resources open after the file was read and parsed into memory. This is not a big deal in your program since you only open two files, but it is still a glaring omission.

Since proper handling of lifecycle of resources can be tricky, starting with version 7, the Java compiler gives you the feature of try-with-resources, offering automatic closure (and better exception handling)

try (Scanner reader = new Scanner(theFile)) {
    ...

} catch (FileNotFoundException e) {
    ...
}

regular expression usage

regarding this line of code

if (!e.matches("[A-Za-z]{1}")) {

while it is convinient to use String.matches(), under the hood, the method will perform all the steps that are required to match an input to a regular expression, including compiling the pattern. since this line is performed inside a loop that reads a (possibly big) input file, you may have a performance issue here. A better approach would be to create a Pattern instance via the compile(regex) factory method and then use a Matcher over the input.

an even better approach (in terms of performance) would be to ask if a regular expression is required. yuo are testing whether the string is a single letter. This can be replaced by quering the String's length and then quering the contents of the (only) character. Something like

if (!(e.length() == 1 && Character.isLetter(e.toCharArray()[0])) {

this way, a string that is longer than 1 letter will pass the condition without any examination of contents.

\$\endgroup\$
  • \$\begingroup\$ I understand the second part and I made the change. But I am not sure I understand how to "close the files". I did the try-and-catch part. Do you mean to exit the program when file is not found, or exit the program when finished? \$\endgroup\$ – Yuhe Zhu May 10 '18 at 15:59
  • \$\begingroup\$ with try-with-resources the closure is handled for you by the compiler. no need to close the file explicitly \$\endgroup\$ – Sharon Ben Asher May 10 '18 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.