I'm building an IntelliJ Plugin which will allow a person to select an XML file (and even a snippet of XML in the file, if they like) and, using a really brief GUI, generate classes / code in one of a selection of languages that will read the XML via native functions and generate code to do so on a regular basis.
The idea here is that a user should be able to generate a deserialization function that would not rely on any third-party libraries, but only on the native built-in features of the language and framework. Additionally, this would mean no reflection / introspection would be necessary: just read the XML into a fully strong-typed thing.
I'm going to post this as several "parts", because it's a large project. I'll do so incrementally: starting from the first component and moving towards the last.
The first step in the adventure is to create and use a Parser
which can read our XML file into some nodes / objects. The idea here is that it only cares about the structure of the file: node names, keys, and value types, not the actual values. (So <node attr="test">
would say "this is a node named 'node', with an attribute named 'attr', that is a string.")
To start with, we should begin at the, well, beginning: reading the file.
For simplicity, I've built a Reading
class that can take a document as a string and attempt to parse the XML out of it:
ArrayList<Node> nodes = Reading.readDocument(document);
This is a very simple interface, but underneath it starts doing a lot of work:
package Xml;
import javax.xml.stream.XMLEventReader;
import javax.xml.stream.XMLInputFactory;
import javax.xml.stream.XMLStreamException;
import javax.xml.stream.events.Attribute;
import javax.xml.stream.events.Characters;
import javax.xml.stream.events.StartElement;
import javax.xml.stream.events.XMLEvent;
import java.io.StringReader;
import java.util.ArrayList;
import java.util.Iterator;
public class Reading {
public static ArrayList<Node> readDocument(String document) throws XMLStreamException {
XMLInputFactory factory = XMLInputFactory.newInstance();
XMLEventReader reader = factory.createXMLEventReader(new StringReader(document));
ArrayList<Node> nodes = new ArrayList<>();
int indentation = 0;
while (reader.hasNext()) {
XMLEvent event = reader.nextEvent();
if (event.isStartElement()) {
StartElement startElement = event.asStartElement();
Node node = new Node(startElement.getName().getLocalPart());
Iterator attributes = startElement.getAttributes();
while (attributes.hasNext()) {
Attribute attribute = (Attribute)attributes.next();
if (attribute.isSpecified()) {
node.add_attribute(attribute.getName().getLocalPart(), attribute.getValue());
}
}
ArrayList<Node> nodeAddition = getActiveNode(nodes, indentation);
nodeAddition.add(node);
indentation++;
} else if (event.isCharacters()) {
Characters characters = event.asCharacters();
String data = characters.getData();
ArrayList<Node> nodeAddition = getActiveNode(nodes, indentation - 1);
nodeAddition.get(nodeAddition.size() - 1).set_value(data);
} else if (event.isEndElement()) {
indentation--;
}
}
return nodes;
}
private static ArrayList<Node> getActiveNode(ArrayList<Node> nodes, int indentation) {
ArrayList<Node> nodeAddition = nodes;
while (indentation > 0) {
indentation--;
nodeAddition = nodeAddition.get(nodeAddition.size() - 1).get_nodes();
}
return nodeAddition;
}
}
Here, we read the XML in and start processing it into our list of array nodes. We keep the nested node structure, and we have the following definition of a Node
within our program:
package Xml;
import java.util.ArrayList;
import java.util.HashMap;
public class Node {
private String _name;
private String _value;
private ArrayList<Node> _nodes;
private HashMap<String, String> _attributes;
public Node(String name) {
_name = name;
_nodes = new ArrayList<>();
_attributes = new HashMap<>();
}
public String get_name() {
return _name;
}
public String get_value() {
return _value;
}
public void set_value(String _value) {
this._value = _value;
}
public HashMap<String, String> get_attributes() {
return _attributes;
}
public void add_attribute(String key, String value) {
_attributes.put(key, value);
}
public ArrayList<Node> get_nodes() {
return _nodes;
}
}
Obviously we're reading the value in here, but we'll eliminate that later. That was the easy part.
Next, we need to parse the nodes into some type of "object", here we'll use our Parser
to begin doing some real work:
Parser parser = new Parser();
parser.set_integerRule(getIntegerRule(wrapper.getSelectedNumberRule()));
parser.set_nullDefault(wrapper.getUseStringForNull() ? Type.STRING : Type.NULL);
parser.set_dateFormat(wrapper.getDateFormat());
parser.set_allNodesAreArrays(wrapper.getAllNodesAreArrays());
for (Tuple2<String, String> rule : wrapper.getBooleanRules()) {
parser.add_booleanRule(new ParserBooleanRule(rule.getFirst(), rule.getSecond()));
}
ArrayList<CodeObject> codeObjects = parser.parse(nodes);
The Parser
is much more complex, and has a lot more going on. To start: it has a selection of "rules" that determine how and why it parses things, particularly how to handle different "features" of our parser. We have a few major features:
- Number Rules: when the parser encounters a number type, how should it treat the type? We can use the minimal data-size, or we can assume the minimum is
Int
,Long
, orDouble
. - Use String for Null: when the parser encounters a null value, how should it proceed? If selected, the parser should intrinsically treat the lack of a value as a
String
. - Date Format: how should the parser detect dates? If specified, the parser should use the specified format to detect date types.
- Boolean Rules: how should the parser detect true/false values? Three built-in rules exist for true/false respectively:
Y
/N
,Yes
/No
,True
/False
. - Result Language: this is less for the parser, and more for the generator. What is the destination output language?
So, with that said, we'll look at the Parser
:
package Generation;
import Xml.Node;
import org.jetbrains.annotations.NotNull;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.HashMap;
public class Parser {
private String _dateFormat;
private ParserIntegerRule _integerRule;
private Type _nullDefault;
private ArrayList<ParserBooleanRule> _booleanRules;
private boolean _allNodesAreArrays;
public Parser() {
_integerRule = ParserIntegerRule.Minimal;
_nullDefault = Type.NULL;
_booleanRules = new ArrayList<>();
}
public String get_dateFormat() { return _dateFormat; }
public void set_dateFormat(String _dateFormat) { this._dateFormat = _dateFormat; }
public ParserIntegerRule get_integerRule() { return _integerRule; }
public void set_integerRule(ParserIntegerRule _integerRule) { this._integerRule = _integerRule; }
public Type get_nullDefault() { return _nullDefault; }
public void set_nullDefault(Type _nullDefault) { this._nullDefault = _nullDefault; }
public ArrayList<ParserBooleanRule> get_booleanRules() { return _booleanRules; }
public void add_booleanRule(ParserBooleanRule rule) { _booleanRules.add(rule); }
public boolean is_allNodesAreArrays() { return _allNodesAreArrays; }
public void set_allNodesAreArrays(boolean _allNodesAreArrays) { this._allNodesAreArrays = _allNodesAreArrays; }
private Type getType(String value) {
SimpleDateFormat format = new SimpleDateFormat(_dateFormat);
format.setLenient(false);
if (value == null || value.equals("")) {
return _nullDefault;
}
if (_dateFormat != null) {
try {
format.parse(value);
return Type.DATE;
} catch (ParseException ignored) { }
}
for (ParserBooleanRule rule : _booleanRules) {
String tempValue = value.toLowerCase();
if (tempValue.equals(rule.get_true()) || tempValue.equals(rule.get_false())) {
return Type.BOOLEAN;
}
}
if (_integerRule == ParserIntegerRule.Minimal) {
try {
//noinspection ResultOfMethodCallIgnored
Byte.parseByte(value);
return Type.INT8;
} catch (NumberFormatException ignored) { }
}
if (_integerRule == ParserIntegerRule.Minimal) {
try {
//noinspection ResultOfMethodCallIgnored
Short.parseShort(value);
return Type.INT16;
} catch (NumberFormatException ignored) { }
}
if (_integerRule == ParserIntegerRule.Minimal || _integerRule == ParserIntegerRule.AssumeInt) {
try {
Integer.parseInt(value);
return Type.INT32;
} catch (NumberFormatException ignored) { }
}
if (_integerRule == ParserIntegerRule.Minimal || _integerRule == ParserIntegerRule.AssumeInt || _integerRule == ParserIntegerRule.AssumeLong) {
try {
Long.parseLong(value);
return Type.INT64;
} catch (NumberFormatException ignored) { }
}
try {
Double.parseDouble(value);
return Type.DOUBLE;
} catch (NumberFormatException ignored) { }
return Type.STRING;
}
private boolean isNumeric(Type type) {
switch (type) {
case INT8:
return true;
case INT16:
return true;
case INT32:
return true;
case INT64:
return true;
case DOUBLE:
return true;
default:
return false;
}
}
private Type lowestCommonNumericType(Type type1, Type type2) {
if (type1 == Type.INT8) {
return type2;
}
if (type1 == Type.INT16) {
if (type2 == Type.INT8) {
return Type.INT16;
}
return type2;
}
if (type1 == Type.INT32) {
if (type2 == Type.INT8 || type2 == Type.INT16) {
return Type.INT32;
}
return type2;
}
if (type1 == Type.INT64) {
if (type2 == Type.INT8 || type2 == Type.INT16 || type2 == Type.INT32) {
return Type.INT64;
}
return type2;
}
return Type.DOUBLE;
}
private Type compareTypes(Type type1, Type type2) {
if (type1 == Type.NULL) {
return type2;
}
if (type2 == Type.NULL) {
return type1;
}
if (type1 == Type.OBJECT || type2 == Type.OBJECT) {
return Type.OBJECT;
}
if (_dateFormat != null
&& ((type1 == Type.DATE && isNumeric(type2))
|| (isNumeric(type1) && type2 == Type.DATE))) {
SimpleDateFormat format = new SimpleDateFormat(_dateFormat);
format.setLenient(false);
if (isNumeric(getType(format.format(Date.from(Instant.now()))))) {
return type1 == Type.DATE ? type2 : type1;
}
}
if (type1 == Type.BOOLEAN && type2 == Type.BOOLEAN) {
return Type.BOOLEAN;
}
if (type1 == Type.DATE && type2 == Type.DATE) {
return Type.DATE;
}
if (isNumeric(type1) && isNumeric(type2)) {
return lowestCommonNumericType(type1, type2);
}
return Type.STRING;
}
private CodeObject objectWithName(Collection<CodeObject> codeObjects, String name) {
for (CodeObject codeObject : codeObjects) {
if (codeObject.get_name().equals(name)) {
return codeObject;
}
}
return null;
}
@NotNull
private ArrayList<CodeObject> getMainObjects(@NotNull ArrayList<Node> nodes) {
ArrayList<CodeObject> codeObjects = new ArrayList<>();
for (Node node : nodes) {
CodeObject object = new CodeObject(node.get_name());
codeObjects.add(object);
HashMap<String, String> attributes = node.get_attributes();
for (String key : attributes.keySet()) {
CodeObject attrCodeObject = new CodeObject(key, getType(attributes.get(key)));
object.add_object(attrCodeObject);
attrCodeObject.set_from(From.ATTRIBUTE);
}
ArrayList<Node> subNodes = node.get_nodes();
if (subNodes.size() == 0) {
object.set_type(getType(node.get_value()));
object.set_from(attributes.size() > 0 ? From.NODE : From.VALUE);
if (node.get_value() != null) {
Type myAttrType = getType(attributes.get("_Text"));
if (myAttrType == Type.NULL) {
myAttrType = Type.STRING;
}
CodeObject attrCodeObject = new CodeObject("_Text", myAttrType);
object.add_object(attrCodeObject);
attrCodeObject.set_from(From.RAW_VALUE);
}
} else {
object.add_objects(getMainObjects(subNodes));
object.set_from(From.NODE);
}
if (attributes.size() > 0 || subNodes.size() > 0) {
object.set_type(Type.OBJECT);
}
}
return codeObjects;
}
private void reduceObjects(@NotNull CodeObject existing, @NotNull CodeObject object) {
existing.set_type(compareTypes(existing.get_type(), object.get_type()));
existing.set_isArray(existing.is_isArray() || object.is_isArray());
ArrayList<CodeObject> reducedObjects = reduceObjects(object.get_codeObjects());
for (CodeObject reducedObject : reducedObjects) {
CodeObject existingSubObject = objectWithName(existing.get_codeObjects(), reducedObject.get_name());
if (existingSubObject == null) {
existing.add_object(reducedObject);
continue;
}
reduceObjects(existingSubObject, reducedObject);
}
}
@NotNull
private ArrayList<CodeObject> reduceObjects(@NotNull ArrayList<CodeObject> objects) {
ArrayList<CodeObject> result = new ArrayList<>();
for (CodeObject object : objects) {
CodeObject existingObject = objectWithName(result, object.get_name());
if (existingObject == null) {
existingObject = new CodeObject(object.get_name(), object.get_type());
existingObject.set_from(object.get_from());
existingObject.set_isArray((_allNodesAreArrays && existingObject.get_from() == From.NODE) || object.is_isArray());
existingObject.add_objects(reduceObjects(object.get_codeObjects()));
result.add(existingObject);
} else {
existingObject.set_isArray(true);
object.set_isArray(true);
reduceObjects(existingObject, object);
}
}
return result;
}
@NotNull
public ArrayList<CodeObject> parse(@NotNull ArrayList<Node> nodes) {
ArrayList<CodeObject> objects = getMainObjects(nodes);
return reduceObjects(objects);
}
}
This class is somewhat long, and has a few other definitions we need to include:
The CodeObject
:
package Generation;
import java.util.ArrayList;
import java.util.Collection;
public class CodeObject {
private String _name;
private Type _type;
private From _from;
private boolean _isArray;
private ArrayList<CodeObject> _codeObjects;
public CodeObject(String name) {
_name = name;
_codeObjects = new ArrayList<>();
}
public CodeObject(String name, Type type) {
this(name);
_type = type;
}
public String get_name() {
return _name;
}
public Type get_type() {
return _type;
}
public void set_type(Type _type) {
this._type = _type;
}
public ArrayList<CodeObject> get_codeObjects() {
return _codeObjects;
}
public void add_object(CodeObject codeObject) {
_codeObjects.add(codeObject);
}
public void add_objects(Collection<CodeObject> codeObject) {
_codeObjects.addAll(codeObject);
}
public From get_from() {
return _from;
}
public void set_from(From _from) {
this._from = _from;
}
public boolean is_isArray() {
return _isArray;
}
public void set_isArray(boolean _isArray) {
this._isArray = _isArray;
}
}
The Type
:
package Generation;
public enum Type {
NULL,
STRING,
INT8,
INT16,
INT32,
INT64,
DOUBLE,
DATE,
BOOLEAN,
OBJECT
}
The From
:
package Generation;
public enum From {
ATTRIBUTE,
NODE,
VALUE,
RAW_VALUE
}
The ParserBooleanRule
:
package Generation;
public class ParserBooleanRule {
private String _true;
private String _false;
public ParserBooleanRule(String _true, String _false) {
this._true = _true;
this._false = _false;
}
public String get_true() {
return _true;
}
public String get_false() {
return _false;
}
}
And the ParserIntegerRule
:
package Generation;
public enum ParserIntegerRule {
Minimal,
AssumeInt,
AssumeLong,
AssumeDouble
}
All these components, together, implement the logic of the parser. Read the Node
objects into a CodeObject
array.
Additionally, one should note all of the Type
functions in the Parser
: these help determine type-rules: given two types, which type should the result be? We use these to determine what we need to do with the data-types, and how an attribute of 5
and 14.2
and test
should be compared, to get the lowest-common type: the one that will accurately represent all values in our input.
Finally, the generation also has three more components relevant to this parser:
private LinkedHashMap<String, ParserIntegerRule> _parserIntegerRules;
public GenerateAction() {
super("Hello");
_parserIntegerRules = new LinkedHashMap<>();
_parserIntegerRules.put("Minimal", ParserIntegerRule.Minimal);
_parserIntegerRules.put("Assume Int", ParserIntegerRule.AssumeInt);
_parserIntegerRules.put("Assume Long", ParserIntegerRule.AssumeLong);
_parserIntegerRules.put("Assume Double", ParserIntegerRule.AssumeDouble);
}
@NotNull
private ParserIntegerRule getIntegerRule(String rule) {
return _parserIntegerRules.getOrDefault(rule, ParserIntegerRule.Minimal);
}