Parsing library interface

Question

I have been working on a parser generator, in which I want to employ a sort of fluent configuration interface - avoiding the pre-compile step that usually comes with parser generators. It's supposed to be a tool not just for language construction, but also for generalized parsing tasks that you usually end up coding manually since it's not worth the effort to drag in a full-blown language construction kit.

What I'm concerned about is if I'm missing the mark, since the code quickly becomes a bit complex when configuring a parser. Here's my demo for a JSON parser:

using System.Collections.Generic;
using Piglet.Parser;

namespace Piglet.Demo.Parser
{
    public class JsonParser
    {
        public class JsonElement
        {
            public string Name { get; set; }
            public object Value { get; set; }
        };

        public class JsonObject
        {
            public List<JsonElement> Elements { get; set; }
        };

        public static void Run()
        {
            var parser = ParserFactory.Configure<object>(configurator =>
            {
                var quotedString = configurator.Terminal("\"(\\.|[^\"])*\"",    f => f.Substring(1, f.Length - 2));
                var doubleValue = configurator.Terminal(@"\d+\.\d+",            f => double.Parse(f));
                var integerValue = configurator.Terminal(@"\d+",                f => int.Parse(f));

                var jsonObject = configurator.NonTerminal();
                var optionalElementList = configurator.NonTerminal();
                var elementList = configurator.NonTerminal();
                var element = configurator.NonTerminal();
                var value = configurator.NonTerminal();
                var array = configurator.NonTerminal();
                var optionalValueList = configurator.NonTerminal();
                var valueList = configurator.NonTerminal();

                jsonObject.Productions(p => p.Production("{", optionalElementList, "}")
                                             .OnReduce( f => new JsonObject { Elements = (List<JsonElement>) f[1]} ));

                optionalElementList.Productions(p =>
                {
                    p.Production(elementList)
                     .OnReduce(f => f[0]);

                    p.Production()
                     .OnReduce(f => new List<JsonElement>());
                });

                elementList.Productions(p =>
                {
                    p.Production(elementList, ",", element)
                     .OnReduce(f => {
                                       var list = (List<JsonElement>)f[0];
                                       list.Add((JsonElement)f[2]);
                                       return list;
                                   });

                    p.Production(element)
                     .OnReduce(f => new List<JsonElement> { (JsonElement)f[0] });
                });

                element.Productions(p => p.Production(quotedString, ":", value)
                                          .OnReduce(f => new JsonElement { Name = (string)f[0], Value = f[2]}));

                value.Productions(p =>
                {
                    p.Production(quotedString)  .OnReduce(f => f[0]);
                    p.Production(integerValue)  .OnReduce(f => f[0]);
                    p.Production(doubleValue)   .OnReduce(f => f[0]);
                    p.Production(jsonObject)    .OnReduce(f => f[0]);
                    p.Production(array)         .OnReduce(f => f[0]);
                    p.Production("true")        .OnReduce(f => true);
                    p.Production("false")       .OnReduce(f => false);
                    p.Production("null")        .OnReduce(f => null);
                });

                array.Productions(p => p.Production("[", optionalValueList, "]")
                                        .OnReduce( f => ((List<object>)f[1]).ToArray() ));

                optionalValueList.Productions(p =>
                {
                    p.Production(valueList)
                     .OnReduce(f => f[0]);
                    p.Production()
                     .OnReduce(f => new List<object>());
                });

                valueList.Productions(p =>
                {
                    p.Production(valueList, ",", value)
                     .OnReduce(f => {
                                       var list = (List<object>)f[0];
                                       list.Add(f[2]);
                                       return list;
                                   });
                    p.Production(value)
                     .OnReduce(f => new List<object>{f[0]});
                });

                configurator.LexerSettings.EscapeLiterals = true;
                configurator.LexerSettings.Ignore = new [] { @"\s+" };
            });


            var jObject = (JsonObject)parser.Parse("{ \"Property1\":\"value\", \"IntegerProperty\" : 1234 }");
        }
    }
}

Questions:

1. Is it clear what this configuration achieves to you?
2. Is there any way you can see to improve the interface of the library?

Of course, this should be considerably more understandable if you have previous experience of parser generators, but even if you don't it would be valuable to me to know how much of this someone unfamiliar with parser generators can figure out from just seeing the code.

The full library is on GitHub, if you feel the need to delve deeper into the internals. But since this is the interface that users of the library would see, this is probably the most important part I want to get right.

To clarify exactly what I would like people to take a shot at is not the actual JSON parsing grammar in this example, or the Run method in general. Rather it is the structure of the ParserFactory and the way the Production, OnParse and such methods are called. For example:

1. Is my way of designing the library configuration interface good?
2. Does it use the lambdas in a good way?
3. Are the method names understandable?
4. Do you understand what it does given whatever background you have on context free grammar parsing?

Answer 1

I like the idea of a Fluent Parser, but I find the syntax you're using very confusing, which probably isn't what you wanted to hear.

Some feedback:

• Productions, Production, Onreduce, these terms have no meaning for me. So they don't feel fluent at all. They might be fluent in the world of a parser, but as you want to abstract the parser away, I think you need to think more in terms of the resulting structure or the expressions. Kind of Parser.Add("ElementList").AddOption().Start().FollowedByLiteral("{").FollowedByList<int>(",").FollowedByLiteral("}"); or something, that expresses the structure being parsed, but not the process underneath. (not the best example, but still more understandable than Productions.AddProduction.OnReduce(f => f[0][1].ToString());
• What's meant by f => f[1]? It's unclear for the reader, can't you write this as OnReduce<returnType>().SkipLiteral("}").Take<returnType>(); or something else that stays fluent and removed the need for these lambda's if possible?
• I'd try to refactor OnReduce (whatever it does) to be a generic method, so that OnReduce(f => ((List<object>)f[1]).ToArray()) would become OnReduce<object[]>(f => f[1]); And then you need to figure out a way to remove the f => f[1] part.

Answer 2

Refactor the Run()-method into seperate methods. It's barely readable now.

Answer 3

The Productions() method is clearly something with side effects, since you are calling it and not saving any return value; shouldn't its name include a verb, then?

Something like SetProductions() or AddProductions().