I'm working on a small program to perform various tasks on text content, primarily at the word level. I wrote these methods as ways to help prepare the raw text file into something more malleable, like a List<String>
, where at a later point I can perform various routines such as counting and sorting words, and so on.
Concerns:
In
splitTextStringIntoWordList
I found myself having to splittextString
param intoString[]
array, and immediately afterwards adding the elements from the array one at a time, parsing with regex, into aList<String>
. Is there a better way to do this that might not need as much manipulation?Are some of my methods just doing too many things between parameters and return?
Is the JavaDoc clear, concise and descriptive?
What are some beginner mistakes I might be making?
import java.util.Arrays;
import java.util.ArrayList;
import java.util.List;
import java.util.Collections;
import java.util.regex.*;
import java.net.URL;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.IOException;
public class TextFileWordSplitter {
/**
* Remove punctuation marks from a String using regular expressions.
* <p>
* This will account for contractions such as "it's" and "can't", as well as
* hyphenated words such as "first-class" and "low-budget", which in both cases
* will be considered as whole words.
* </p>
* @param input The String from which to remove punctuation
* @return The String with the punctuation removed, or empty String
*/
static String removePunctuationFromString(String input) {
Pattern regex = Pattern.compile("([A-Za-z]?[\\-']?[A-Za-z])+");
Matcher matcher = regex.matcher(input);
if (matcher.find()) {
return matcher.group();
} else {
return "";
}
}
/**
* Create a String by fetching a text file at the provided URL.
* @param url The URL where the text file is located.
* @return The content of the text file, or null
* @throws IOException
*/
static String readUrlTextContent(String url) throws IOException {
URL source = new URL(url);
try (BufferedReader reader = new BufferedReader(new InputStreamReader(source.openStream()))) {
StringBuilder builder = new StringBuilder();
String line = reader.readLine();
while (line != null) {
builder.append(line);
builder.append("\n");
line = reader.readLine();
}
return builder.toString();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
/**
* Split a String into an ArrayList of individual words as separate elements.
* <p>
* The words are all converted to uppercase, such that "Hello", "hello" and "HELLO"
* will all become the same word string, "HELLO".
* </p>
* @param textString The String which is intended to be split into a list of words
* @return An ArrayList containing one word per element, or null
*/
static List<String> splitTextStringIntoWordList (String textString) {
try {
String allWhiteSpace = "\\s+";
String[] splitText = textString.toUpperCase().split(allWhiteSpace);
List<String> wordList = new ArrayList();
for (String word : splitText) {
Collections.addAll(wordList, removePunctuationFromString(word));
}
wordList.removeAll(Arrays.asList("", null));
return wordList;
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
public static void main(String[] args) {
String textString = null;
try {
textString = readUrlTextContent("http://textfiles.com/stories/antcrick.txt");
} catch (IOException e) {
e.printStackTrace();
}
List<String> wordList = splitTextStringIntoWordList(textString);
/* Print each word along with its index */
int wordIndex = 0;
for (String word : wordList) {
System.out.println("[" + wordIndex++ + "] " + word);
}
}
}
The output of the program, using The Ant and the Cricket as a source, is as follows:
[0] THE [1] ANT [2] AND [3] THE [4] CRICKET [5] ONCE [6] UPON [7] A [8] TIME ... [368] WELL [369] TRY [370] DANCING [371] NOW