14
\$\begingroup\$

My home-made wannabe-a-parser was wet behind the ears, so I started seriously looking into more mature alternatives, and ended up adding a dependency to the ANTLR lexer/parser generator tool, and using that along with a slightly modified Visual Basic 6 grammar I was able to generate the real deal - a full-blown real parser that actually understands [almost] everything there is to understand about VBA code.

Here's how I'm exposing it to the rest of Rubberduck:

public interface IRubberduckParser
{
    /// <summary>
    /// Parses specified code into a code tree.
    /// </summary>
    /// <param name="projectName">The name of the VBA project the code belongs to.</param>
    /// <param name="componentName">The name of the VBA component (module) the code belongs to.</param>
    /// <param name="code">The VBA code fragment to be parsed.</param>
    /// <returns></returns>
    Node Parse(string projectName, string componentName, string code);
}

It's implemented like this:

namespace Rubberduck.VBA
{
    public class VBParser : IRubberduckParser
    {
        public Node Parse(string projectName, string componentName, string code)
        {
            var result = ParseInternal(code);
            var walker = new ParseTreeWalker();

            var listener = new VBTreeListener(projectName, componentName);
            walker.Walk(listener, result);

            return listener.Root;
        }

        private IParseTree ParseInternal(string code)
        {
            var input = new AntlrInputStream(code);
            var lexer = new VisualBasic6Lexer(input);
            var tokens = new CommonTokenStream(lexer);
            var parser = new VisualBasic6Parser(tokens);

            return parser.startRule();
        }
    }
}

The generated IParseTree can't quite be passed to the outside world as is - it's ugly generated code, with methods that essentially match grammar rules 1:1... Implementing VBA code inspections with this API would have been a nightmare. So I wrote this Node class:

namespace Rubberduck.VBA.Nodes
{
    /// <summary>
    /// The base class for all nodes in a Rubberduck code tree.
    /// </summary>
    public abstract class Node
    {
        private readonly ParserRuleContext _context;
        private readonly Selection _selection;
        private readonly string _parentScope;

        private readonly string _localScope;
        private readonly ICollection<Node> _childNodes;

        /// <summary>
        /// Represents a node in the code tree.
        /// </summary>
        /// <param name="context">The parser rule context, obtained from an ANTLR-generated parser method.</param>
        /// <param name="parentScope">The scope this node belongs to. <c>null</c> for the root node.</param>
        /// <param name="localScope">The scope this node defines, if any. <c>null</c> if omitted.</param>
        /// <param name="childNodes">The child nodes.</param>
        /// <remarks>
        /// Specifying a <c>localScope</c> ensures child nodes can be added, regardless of 
        /// </remarks>
        protected Node(ParserRuleContext context, string parentScope, string localScope = null, ICollection<Node> childNodes = null)
        {
            _context = context;
            _selection = context.GetSelection();
            _parentScope = parentScope;

            _localScope = localScope;

            _childNodes = (localScope != null && childNodes == null)
                            ? new List<Node>()
                            : childNodes;
        }

        /// <summary>
        /// Gets the parser rule context for the node.
        /// </summary>
        protected ParserRuleContext Context { get { return _context; } }

        /// <summary>
        /// Gets the name of the scope this node belongs to.
        /// </summary>
        public string ParentScope { get { return _parentScope; } }

        /// <summary>
        /// Gets the name of the scope defined by this node. <c>null</c> if node cannot be a parent.
        /// </summary>
        public string LocalScope { get { return _localScope; } }

        /// <summary>
        /// Gets a value representing the position of the node in the code module.
        /// </summary>
        public Selection Selection { get { return _selection; } }

        /// <summary>
        /// Gets a the child nodes. <c>null</c> if node cannot be a parent.
        /// </summary>
        public IEnumerable<Node> Children { get { return _childNodes; } }

        /// <summary>
        /// Adds a child node.
        /// </summary>
        /// <param name="node">The child node to be added.</param>
        /// <exception cref="InvalidOperationException">Thrown if node cannot have child nodes.</exception>
        public void AddChild(Node node)
        {
            if (_childNodes == null)
            {
                throw new InvalidOperationException("This node cannot have child nodes.");
            }

            _childNodes.Add(node);
        }
    }
}

The idea is to implement this class for everything that's useful in the ANTLR parse tree, into a structure that's easier to work with. Such as an EnumNode:

namespace Rubberduck.VBA.Nodes
{
    public class EnumNode : Node
    {
        private readonly VisualBasic6Parser.EnumerationStmtContext _context;
        private readonly IdentifierNode _identifier;

        public EnumNode(VisualBasic6Parser.EnumerationStmtContext context, string scope)
            :base(context, scope, null, new List<Node>())
        {
            _context = context;
            _identifier = new IdentifierNode(_context.ambiguousIdentifier(), scope);

            var children = context.enumerationStmt_Constant();
            foreach (var child in children)
            {
                AddChild(new EnumConstNode(child, scope));
            }
        }

        public IdentifierNode Identifier
        {
            get { return _identifier; }
        }

        public VBAccessibility Accessibility
        {
            get { return (VBAccessibility) Enum.Parse(typeof (VBAccessibility), _context.visibility().GetText()); }
        }
    }

All child nodes of an EnumNode are EnumConstNode instances:

    public class EnumConstNode : Node
    {
        private readonly VisualBasic6Parser.EnumerationStmt_ConstantContext _context;
        private readonly IdentifierNode _identifier;

        public EnumConstNode(VisualBasic6Parser.EnumerationStmt_ConstantContext context, string scope)
            :base(context, scope)
        {
            _context = context;
            _identifier = new IdentifierNode(_context.ambiguousIdentifier(), scope);
        }

        public string SpecifiedValue
        {
            get { return _context.valueStmt().GetText(); }
        }
    }
}

Given a VisualBasic6.g4 grammar file, ANTLR generates a VisualBasic6BaseListener class, that the tree walker makes virtual method calls onto as it walks the tree - I only needed to implement it, so I wrote this VBTreeListener:

public partial class VBTreeListener : VisualBasic6BaseListener
{
    private readonly string _project;
    private readonly string _module;
    private readonly IList<Node> _members = new List<Node>();

    private string _currentScope;
    private Node _currentNode;

    public VBTreeListener(string project, string module)
    {
        _project = project;
        _module = module;
        _currentScope = project + "." + module;
    }

    public Node Root
    {
        get { return new ModuleNode(null, _project, _module, _members); }
    }

    private void AddCurrentMember()
    {
        _members.Add(_currentNode);
        _currentNode = null;
    }

    public override void EnterSubStmt(VisualBasic6Parser.SubStmtContext context)
    {
        _currentNode = CreateProcedureNode(context);
        _currentScope = _project + "." + _module + "." + ((ProcedureNode) _currentNode).Name;
    }

    public override void EnterFunctionStmt(VisualBasic6Parser.FunctionStmtContext context)
    {
        _currentNode = CreateProcedureNode(context);
        _currentScope = _project + "." + _module + "." + ((ProcedureNode)_currentNode).Name;
    }

    public override void EnterPropertyGetStmt(VisualBasic6Parser.PropertyGetStmtContext context)
    {
        _currentNode = CreateProcedureNode(context);
        _currentScope = _project + "." + _module + "." + ((ProcedureNode)_currentNode).Name;
    }

    public override void EnterPropertyLetStmt(VisualBasic6Parser.PropertyLetStmtContext context)
    {
        _currentNode = CreateProcedureNode(context);
        _currentScope = _project + "." + _module + "." + ((ProcedureNode)_currentNode).Name;
    }

    public override void EnterPropertySetStmt(VisualBasic6Parser.PropertySetStmtContext context)
    {
        _currentNode = CreateProcedureNode(context);
        _currentScope = _project + "." + _module + "." + ((ProcedureNode)_currentNode).Name;
    }

    private ProcedureNode CreateProcedureNode(dynamic context)
    {
        var procedureName = context.ambiguousIdentifier().IDENTIFIER()[0].Symbol.Text;
        var node = new ProcedureNode(context, _currentScope, procedureName);

        var args = context.argList().arg() as IReadOnlyList<VisualBasic6Parser.ArgContext>;
        if (args != null)
        {
            foreach (var arg in args)
            {
                node.AddChild(new ParameterNode(arg, _currentScope));
            }
        }

        return node;
    }

    public override void ExitOptionExplicitStmt(VisualBasic6Parser.OptionExplicitStmtContext context)
    {
        _members.Add(new OptionNode(context, _currentScope));
    }

    public override void ExitOptionBaseStmt(VisualBasic6Parser.OptionBaseStmtContext context)
    {
        _members.Add(new OptionNode(context, _currentScope));
    }

    public override void ExitOptionCompareStmt(VisualBasic6Parser.OptionCompareStmtContext context)
    {
        _members.Add(new OptionNode(context, _currentScope));
    }

    public override void ExitEnumerationStmt(VisualBasic6Parser.EnumerationStmtContext context)
    {
        _members.Add(new EnumNode(context, _currentScope));
    }

    public override void ExitSubStmt(VisualBasic6Parser.SubStmtContext context)
    {
        AddCurrentMember();
    }

    public override void ExitFunctionStmt(VisualBasic6Parser.FunctionStmtContext context)
    {
        AddCurrentMember();
    }

    public override void ExitPropertyGetStmt(VisualBasic6Parser.PropertyGetStmtContext context)
    {
        AddCurrentMember();
    }

    public override void ExitPropertyLetStmt(VisualBasic6Parser.PropertyLetStmtContext context)
    {
        AddCurrentMember();
    }

    public override void ExitPropertySetStmt(VisualBasic6Parser.PropertySetStmtContext context)
    {
        AddCurrentMember();
    }

}

How's this? I intend to continue implementing overrides in the VBTreeListener class, and I made it partial so that when I feel like it I can move related overrides to a dedicated code file. Or could this be a job for #region (there will be a lot of them)?

\$\endgroup\$
  • 2
    \$\begingroup\$ a full-blown real parser that actually understands [almost] everything there is to understand about VBA code. -- well the syntax, at least ;) \$\endgroup\$ – Mathieu Guindon Jan 23 '15 at 7:01
3
\$\begingroup\$

This is a big improvement from that last post! It so great that makes my eyes sparkle and brings joy.

I think you are making the right use of the keyword partial here. This may be my personal taste but I don't really like to see #region keywords in the code.

The only thing I see you could improve in your code at this moment is to extract those two lines:

_currentNode = CreateProcedureNode(context);
_currentScope = _project + "." + _module + "." + ((ProcedureNode)_currentNode).Name;

into a new method, like you have done with AddCurrentMember:

private void CreateNodeFromContext(dynamic context){
    var node = CreateProcedureNode(context); // I prefered to store this in a variable to avoid the cast 
    _currentNode = node;
    _currentScope = _project + "." + _module + "." + node.Name;
}

EDIT: To address your comment. I dont really know how node could be a bother, you already had the care to make the Node class to ease your code inspections, like you stated. I also like the fact of having a class that represents each element needed in the tree.

One thing that you could do though is to remove the context from the specific class, you already have it in the Node class. In the subclass you could have a property wich implementation would be to do a cast of the node to the suitable type:

public abstract class Node
{
    protected readonly ParserRuleContext _context;
    //...
}

public class EnumNode : Node
{
    private VisualBasic6Parser.EnumerationStmtContext Context{
        get{ return (VisualBasic6Parser.EnumerationStmtContext)_context; }
    }
    //...
}

It doesn't bring much benefit, but welll is always worth saving 4 or 8 bytes of memory per object...

\$\endgroup\$
  • \$\begingroup\$ Thanks! Could Node be a leaky abstraction because of its dependency on ANTLR? Thoughts? \$\endgroup\$ – Mathieu Guindon Jan 25 '15 at 2:35
  • \$\begingroup\$ I'm actually torn between injecting the context like I'm doing, and using it in the derived types to expose getters for, say, the enum's Name and Accessibility (accessing the encapsulated context every time) - or injecting string name and string accessibility values to initialize private readonly fields (and possibly bloating up constructors when the node has many properties); the former has a dependency on ANTLR, the latter doesn't... I like the private getter that casts to the actual context type though.. decisions, decisions... ;) \$\endgroup\$ – Mathieu Guindon Jan 25 '15 at 16:00
  • \$\begingroup\$ @Mat'sMug If for some reason the ANTLR dependency doesn't guives you much, meaning that you use few of the properties of the ANTLR object, then you may create a class to contain the information that you want to import from ANTLR (let's say you start by having two or three properties here). If that object begins to increase then there is no problem to bring that depedency. You can also create an Interface that provides the properties that you need so you may change that easly in future I guess. \$\endgroup\$ – Bruno Costa Jan 25 '15 at 21:37
  • \$\begingroup\$ You might want to take a look at this post, where I show ProcedureNode and VariableDeclarationNode implementations. I'd love to hear your thoughts on whether that's headed in the right direction. \$\endgroup\$ – Mathieu Guindon Jan 26 '15 at 0:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.