4
\$\begingroup\$

I'm sill experimenting with different design patterns for full integration tests for Console applications (and later also Windows Services) and I wasn't quite happy with the result of the refactoring of my last question. I've changed a few things and this is what I've come up this time.

The main application code that starts an application is now divided into two files Program.Entry.cs and Program.Instance.cs that are partial classes of the Program (otherwise I needed a new name for the other file and this isn't actually necessary). The Entry file contains only the Main method and redirects this call to the Start method that besides the args parameter also takes a few other ones that I'm using for testing. It also contains other static methods necessary for the initialization. The Instance file contains only instance code that runs the application. Having them both split allows me to better separate the two tasks. The new code:

Program.Entry.cs

internal partial class Program
{
    internal static int Main(string[] args)
    {
        return Start(
            args, 
            InitializeLogging, 
            InitializeConfiguration, 
            configuration => InitializeContainer(configuration, Enumerable.Empty<Autofac.Module>()));
    }

    public static int Start(
        string[] args, 
        Action initializeLogging, 
        Func<Configuration> initializeConfiguration, 
        Func<Configuration, IContainer> initializeContainer)
    {
        initializeLogging();

        var mainLogger = LoggerFactory.CreateLogger(nameof(Program)); LogEntry.New().Debug().Message("Logging initialized.").Log(mainLogger);
        var mainLogEntry = LogEntry.New().Stopwatch(sw => sw.Start());

        try
        {
            var configuration = initializeConfiguration(); LogEntry.New().Debug().Message("Configuration initialized.").Log(mainLogger);
            var container = initializeContainer(configuration); LogEntry.New().Debug().Message("IoC initialized.").Log(mainLogger);

            using (var scope = container.BeginLifetimeScope())
            {
                var program = scope.Resolve<Program>();
                LogEntry.New().Info().Message($"Created {Name} v{Version}").Log(mainLogger);
                program.Start(args);
            }

            mainLogEntry.Info().Message("Completed.");
            return 0;
        }
        catch (Exception ex)
        {
            mainLogEntry.Fatal().Message("Crashed.").Exception(ex);
            return 1;
        }
        finally
        {
            mainLogEntry.Log(mainLogger);
            LogEntry.New().Info().Message("Exited.").Log(mainLogger);
        }
    }

    #region Initialization

    internal static void InitializeLogging()
    {
        Reusable.Logging.NLog.Tools.LayoutRenderers.InvariantPropertiesLayoutRenderer.Register();

        Reusable.Logging.Logger.ComputedProperties.Add(new Reusable.Logging.ComputedProperties.AppSetting(name: "Environment", key: $"Gunter.Program.Config.Environment"));
        Reusable.Logging.Logger.ComputedProperties.Add(new Reusable.Logging.ComputedProperties.ElapsedSeconds());

        Reusable.Logging.LoggerFactory.Initialize<Reusable.Logging.Adapters.NLogFactory>();
    }

    internal static Configuration InitializeConfiguration()
    {
        try
        {
            return new Configuration(new AppSettings());
        }
        catch (Exception ex)
        {
            throw new InitializationException("Could not initialize configuration.", ex);
        }
    }

    internal static IContainer InitializeContainer(Configuration configuration, IEnumerable<Autofac.Module> moduleOverrides)
    {
        try
        {
            var builder = new ContainerBuilder();

            builder.RegisterInstance(configuration.Load<Program, Workspace>());

            builder.RegisterModule<SystemModule>();
            builder.RegisterModule<DataModule>();
            builder.RegisterModule<ReportingModule>();
            builder.RegisterModule<HtmlModule>();

            builder
                .RegisterType<TestRunner>()
                .WithParameter(new TypedParameter(typeof(ILogger), LoggerFactory.CreateLogger(nameof(TestRunner))));

            builder
                .RegisterType<Program>()
                .WithParameter(new TypedParameter(typeof(ILogger), LoggerFactory.CreateLogger(nameof(Program))))
                .PropertiesAutowired();

            foreach (var module in moduleOverrides)
            {
                builder.RegisterModule(module);
            }

            return builder.Build();
        }
        catch (Exception ex)
        {
            throw new InitializationException("Could not initialize container.", ex);
        }
    }

    #endregion
}

Program.Instance.cs

internal partial class Program
{
    public static readonly string Name = Assembly.GetAssembly(typeof(Program)).GetName().Name;
    public static readonly string Version = "2.0.0";
    private static readonly string GlobalFileName = "_Global.json";

    private readonly ILogger _logger;
    private readonly IPathResolver _pathResolver;
    private readonly IFileSystem _fileSystem;
    private readonly IVariableBuilder _variableBuilder;
    private readonly AutofacContractResolver _autofacContractResolver;
    private readonly TestRunner _testRunner;

    public Program(
        ILogger logger,
        IPathResolver pathResolver,
        IFileSystem fileSystem,
        IVariableBuilder variableBuilder,
        AutofacContractResolver autofacContractResolver,
        TestRunner testRunner)
    {
        _logger = logger;
        _pathResolver = pathResolver;
        _fileSystem = fileSystem;
        _variableBuilder = variableBuilder;
        _autofacContractResolver = autofacContractResolver;
        _testRunner = testRunner;
    }

    public Workspace Workspace { get; set; }

    public void Start(string[] args)
    {
        var globalFile = LoadGlobalFile();

        var globals = VariableResolver.Empty
            .MergeWith(globalFile.Globals)
            .MergeWith(_variableBuilder.BuildVariables(Workspace));

        var testFiles = LoadTestFiles().ToList();

        LogEntry.New().Debug().Message($"Test files ({testFiles.Count}) loaded.").Log(_logger);
        LogEntry.New().Info().Message($"*** {Name} v{Version} started. ***").Log(_logger);

        _testRunner.RunTestFiles(testFiles, args, globals);
    }

    private GlobalFile LoadGlobalFile()
    {
        var targetsDirectoryName = _pathResolver.ResolveDirectoryPath(Workspace.Targets);
        var fileName = Path.Combine(targetsDirectoryName, GlobalFileName);

        if (!File.Exists(fileName)) { return new GlobalFile(); }

        try
        {
            var globalFileJson = _fileSystem.ReadAllText(fileName);
            var globalFile = JsonConvert.DeserializeObject<GlobalFile>(globalFileJson);

            VariableValidator.ValidateNamesNotReserved(globalFile.Globals, _variableBuilder.Names);

            LogEntry.New().Debug().Message($"{Path.GetFileName(fileName)} loaded.").Log(_logger);

            return globalFile;
        }
        catch (Exception ex)
        {
            throw new InitializationException($"Could not load {Path.GetFileName(fileName)}.", ex);
        }
    }

    [NotNull, ItemNotNull]
    private IEnumerable<TestFile> LoadTestFiles()
    {
        LogEntry.New().Debug().Message("Initializing tests...").Log(_logger);

        return
            GetTestFileNames()
                .Select(LoadTest)
                .Where(Conditional.IsNotNull);
    }

    [NotNull, ItemNotNull]
    private IEnumerable<string> GetTestFileNames()
    {
        var targetsDirectoryName = _pathResolver.ResolveDirectoryPath(Workspace.Targets);

        return
            from fullName in _fileSystem.GetFiles(targetsDirectoryName, "*.json")
            where !Path.GetFileName(fullName).StartsWith("_", StringComparison.OrdinalIgnoreCase)
            select fullName;
    }

    [CanBeNull]
    private TestFile LoadTest(string fileName)
    {
        var logEntry = LogEntry.New().Info();
        try
        {
            var json = _fileSystem.ReadAllText(fileName);
            var testFile = JsonConvert.DeserializeObject<TestFile>(json, new JsonSerializerSettings
            {
                ContractResolver = _autofacContractResolver,
                DefaultValueHandling = DefaultValueHandling.Populate,
                TypeNameHandling = TypeNameHandling.Auto,
            });
            testFile.FullName = fileName;

            VariableValidator.ValidateNamesNotReserved(testFile.Locals, _variableBuilder.Names);

            logEntry.Message($"Test initialized: {fileName}");
            return testFile;
        }
        catch (Exception ex)
        {
            logEntry.Error().Message($"Could not initialize test: {fileName}").Exception(ex);
            return null;
        }
        finally
        {
            logEntry.Log(_logger);
        }
    }
}

Testing

Now I am able to override the default modules with my test versions to actually use other files that I store in a test project as Embeded Resources. This way I can simulate various scenarios with non-existing or invalid files, not working database connections etc and verfiy that the application behaves as I want it to. This is, it should exit in some situations and survive in others or log something etc.

Here's one of the first tests I wrote for it and a new TestFileSystem that I use to fake files. Additionaly I can now test it with different command line arguments.

[TestClass]
public class ProgramTest
{
    [TestMethod]
    public void Start_NoArguments_RunsAllTests()
    {
        var exitCode = Program.Start(
            new string[0],
            Program.InitializeLogging,
            Program.InitializeConfiguration,
            configuration => Program.InitializeContainer(configuration, new[]
            {
                new TestModule
                {
                    FileSystem = new TestFileSystem
                    {
                        Files = { @"t:\tests\assets\targets\single-test.json" }
                    }
                }
            }));

        Assert.AreEqual(0, exitCode);
        Assert.AreEqual(1, TestAlert.GetReports("single-test.json").Count);
    }
}

internal class TestModule : Autofac.Module
{
    public TestFileSystem FileSystem { get; set; }

    protected override void Load(ContainerBuilder builder)
    {
        builder
            .RegisterType<TestAlert>();

        builder
            .RegisterType<TestPathResolver>()
            .As<IPathResolver>();

        builder
            .RegisterInstance(FileSystem)
            .As<IFileSystem>();
    }
}

internal class TestFileSystem : IFileSystem
{
    public List<string> Files { get; } = new List<string>();

    public string ReadAllText(string fileName)
    {
        switch (fileName.ToLower())
        {
            case @"t:\tests\_Global.json": return ResourceReader.ReadEmbeddedResource<ProgramTest>("Resources.assets.targets._Global.json");
            case @"t:\tests\single-test.json": return ResourceReader.ReadEmbeddedResource<ProgramTest>("Resources.assets.targets.single-test.json");
            default: throw new FileNotFoundException($"File \"{fileName}\" does not exist.");
        }
    }

    public string[] GetFiles(string path, string searchPattern)
    {
        return Files.ToArray();
    }
}

What do you think of this design? Can it still be improved in any way?

\$\endgroup\$
5
\$\begingroup\$
var mainLogger = LoggerFactory.CreateLogger(nameof(Program)); LogEntry.New().Debug().Message("Logging initialized.").Log(mainLogger);

This should really be split into two lines. Not sure what happened, but I'm sure it was a mistake.

var mainLogger = LoggerFactory.CreateLogger(nameof(Program)); 
LogEntry.New().Debug().Message("Logging initialized.").Log(mainLogger);

Same with the very next line.

Maybe it wasn't a mistake. Maybe you did this intentionally because of all the noisy daisy chaining your logger needs. Try extracting a few small helpers to reduce the noise.

private void logDebug(string message, ILogger logger)
{
    LogEntry.New().Debug().Message(message).Log(logger);
}

Otherwise, I find your entry point well structured and readable. I like that you ensure the program returns a non-zero exit code on failure. We windows devs tend to forget these kinds of things, but it's important if you ever program ever gets used as part of some shell script. +1


Using a couple of variables here could clean this up.

internal static void InitializeLogging()
{
    Reusable.Logging.NLog.Tools.LayoutRenderers.InvariantPropertiesLayoutRenderer.Register();

    Reusable.Logging.Logger.ComputedProperties.Add(new Reusable.Logging.ComputedProperties.AppSetting(name: "Environment", key: $"Gunter.Program.Config.Environment"));
    Reusable.Logging.Logger.ComputedProperties.Add(new Reusable.Logging.ComputedProperties.ElapsedSeconds());

    Reusable.Logging.LoggerFactory.Initialize<Reusable.Logging.Adapters.NLogFactory>();
}
internal static void InitializeLogging()
{
    Reusable.Logging.NLog.Tools.LayoutRenderers.InvariantPropertiesLayoutRenderer.Register();

    var properties = Reusable.Logging.Logger.ComputedProperties;
    properties.Add(new Reusable.Logging.ComputedProperties.AppSetting(name: "Environment", key: $"Gunter.Program.Config.Environment"));
    properties.Add(new Reusable.Logging.ComputedProperties.ElapsedSeconds());

    Reusable.Logging.LoggerFactory.Initialize<Reusable.Logging.Adapters.NLogFactory>();
}

If I got that wrong and those are namespaces, not static properties, for goodness sake use some imports.


One last thought...

If your entry point has become so large and complex that you're worried about testing the code directly, or are tempted to use a partial class, then your domain is likely missing some concepts.

I think it would be useful to ask yourself why you want to test these conditions, and why is it hard to test them? Could you test them by just running the program under different contexts? Could you use a shell script to just run it with different inputs? Why not? Because you're testing embedded resources? Why not pass those files via args instead? Why don't your other existing tests inside of the actual domain logic test those embedded resources efficiently? Have you put a ton of time and effort into a problem that didn't need to be solved out of some dogmatic reasoning that everything must be tested extremely thoroughly? What benefit did it give you? Sometimes simpler is better and we need to know when enough is enough. I'm not saying that you were wrong to do any of this, I'm just asking you to reflect on whether or not you were.

\$\endgroup\$
  • \$\begingroup\$ You're right with almost everything. I'm working on such a big integration test because I need a reusable pattern for other console tools and services that can do a lot of really important things. I'd like to be able to validate that especially windows services won't crash while starting and not giving any usable feedback what happened. Some tools (like this one) can also be used and configured by mulitple uses and I'd like to make sure they're get a valueable feedback before asking me what's wrong and won't crash on silly mistakes. \$\endgroup\$ – t3chb0t Jun 25 '17 at 12:24
  • \$\begingroup\$ It's not possible to validate all that with simple unit tests (that I also have). The json files have many optional and requried attributes so I'd like to use fake-files to verify the app can use or reject them propertly instead of running the application by myself several times. It's always nice to be able to hit a button and see how the tests turn green :) \$\endgroup\$ – t3chb0t Jun 25 '17 at 12:25
  • \$\begingroup\$ My goal is to be able to write tests that cover most of the valid and even more important invalid usage of the app in real world. I can verify the logic of the test-runner but I cannot build a unit test to check whether the app won't crash if some json file contains invalid data. I think otherwise all the dependency-injection would be in vain if I wasn't replacing services with fakes etc. I think that's the whole purpose of doing it. \$\endgroup\$ – t3chb0t Jun 25 '17 at 12:29
  • 2
    \$\begingroup\$ Right. I'm not saying that you shouldn't automate those tests. I'm simply suggesting that you may be using the wrong approach to automate those tests. The whole deal might be simpler if you just started and ran your program. No reason you couldn't do that with Process.Start() (or however you run cli programs in C#; I'm a bit rusty). \$\endgroup\$ – RubberDuck Jun 25 '17 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.