The following program is a lexical analyser for a simple and small grammar.
Please read my code and answer me these questions:
- Is my code easy to understand?
- Is my code well organized?
- Are there bugs in my code?
- Is the lack of comments a problem?
- Can you make my code more performatic without modifying all the structure?
A general criticism is also welcome.
A note about performance: The program is really bad in performance. I already coded lexical analysers much better in this point, but this one is easier to extend and understand, so I do not want to lose the current structure.
Token.java
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public enum Token {
LET_KEYWORD ("let"),
IF_KEYWORD ("if"),
WHILE_KEYWORD ("while"),
ELSE_KEYWORD ("else"),
SEMICOLON (";"),
COMMA (","),
OPEN_BRACKET ("\\{"),
CLOSE_BRACKET ("\\}"),
OPEN_PARENTHESIS ("\\("),
CLOSE_PARENTHESIS ("\\)"),
EQUAL ("=="),
DIFFERENT ("<>"),
GREATER_EQUAL (">="),
LESSER_EQUAL ("<="),
ATTRIBUTION ("="),
GREATER (">"),
LESSER ("<"),
ADDITION ("\\+"),
SUBTRACTION ("-"),
MULTIPLICATION ("\\*"),
DIVISION ("/"),
MODULUS ("%"),
STRING ("\"[^\"]+\""),
NUMBER ("\\d+(\\.\\d+)?"),
IDENTIFIER ("\\w+");
private final Pattern pattern;
Token(String regex) {
pattern = Pattern.compile("^" + regex);
}
int endOfMatch(String s) {
Matcher m = pattern.matcher(s);
if (m.find()) {
return m.end();
}
return -1;
}
}
Lexer.java
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.HashSet;
import java.util.Set;
import java.util.stream.Stream;
public class Lexer {
private StringBuilder input = new StringBuilder();
private Token token;
private String lexema;
private boolean exausthed = false;
private String errorMessage = "";
private Set<Character> blankChars = new HashSet<Character>();
public Lexer(String filePath) {
try (Stream<String> st = Files.lines(Paths.get(filePath))) {
st.forEach(input::append);
} catch (IOException ex) {
exausthed = true;
errorMessage = "Could not read file: " + filePath;
return;
}
blankChars.add('\r');
blankChars.add('\n');
blankChars.add((char) 8);
blankChars.add((char) 9);
blankChars.add((char) 11);
blankChars.add((char) 12);
blankChars.add((char) 32);
moveAhead();
}
public void moveAhead() {
if (exausthed) {
return;
}
if (input.length() == 0) {
exausthed = true;
return;
}
ignoreWhiteSpaces();
if (findNextToken()) {
return;
}
exausthed = true;
if (input.length() > 0) {
errorMessage = "Unexpected symbol: '" + input.charAt(0) + "'";
}
}
private void ignoreWhiteSpaces() {
int charsToDelete = 0;
while (blankChars.contains(input.charAt(charsToDelete))) {
charsToDelete++;
}
if (charsToDelete > 0) {
input.delete(0, charsToDelete);
}
}
private boolean findNextToken() {
for (Token t : Token.values()) {
int end = t.endOfMatch(input.toString());
if (end != -1) {
token = t;
lexema = input.substring(0, end);
input.delete(0, end);
return true;
}
}
return false;
}
public Token currentToken() {
return token;
}
public String currentLexema() {
return lexema;
}
public boolean isSuccessful() {
return errorMessage.isEmpty();
}
public String errorMessage() {
return errorMessage;
}
public boolean isExausthed() {
return exausthed;
}
}
You probably want to run my code, so use this:
Start.java
public class Start {
public static void main(String s[]) {
Lexer lexer = new Lexer("C:/teste/font.txt");
while (!lexer.isExausthed()) {
System.out.printf("%-18s %s\n", lexer.currentToken(), lexer.currentLexema());
lexer.moveAhead();
}
if (lexer.isSuccessful()) {
System.out.println("Ok! :D");
} else {
System.out.println(lexer.errorMessage());
}
}
}
As you can see, the lexer reads C:/teste/font.txt
. The content is:
let i = 0;
let do_task (name, secs) = {
if secs == 0 {
write("Completed" + name + "instantaneously");
} else {
sleep(secs * 1000);
write("After" + secs + "completed" + name);
}
}
while i <= 10 {
do_task(i, "some task");
i = i + 1;
}