Assigning test-data parameters by name

Question

When creating data-driven tests with xUnit we can use the MemberDataAttribute to get the data from a member of this or other class. This is very nice but it has one disadvantage, you have to specify parameters in the same order as the test method so you need to be very careful. I prefer to use names for that purpose because my data-items already have them so I'd like to reuse them with the test signature.

---

SmartMemberDataAttribute

In order to make it work this way I created my own SmartMemberDataAttribute. I think it's smart because it can not only match data-item and parameters by name but at the same time it's case-insensitive and also validates whether data-item values can be assigned to the parameters or are optional.

Warning: DynamicExceptions at work here! Please ignore this question if you don't like them ;-]

Yes, this code throws three DynamicExceptions to make debugging easier in case I made a mistake and gave properties or parameters non-matching names or used incompatible types.

[DataDiscoverer("Xunit.Sdk.MemberDataDiscoverer", "xunit.core")]
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true)]
public class SmartMemberDataAttribute : MemberDataAttributeBase
{
    public SmartMemberDataAttribute(string memberName, params object[] parameters) : base(memberName, parameters) { }

    protected override object[] ConvertDataItem(MethodInfo testMethod, object item)
    {
        try
        {
            return CreateDataItem(testMethod, item);
        }
        catch (Exception inner)
        {
            throw DynamicException.Create
            (
                $"DataItemConversion",
                $"Could not convert '{item.GetType().ToPrettyString()}' for '{GetTestMethodInfo()}'. See the inner exception for details.",
                inner
            );
        }

        // Creates text: MyTest.TestMethod
        string GetTestMethodInfo() => $"{testMethod.DeclaringType.ToPrettyString()}.{testMethod.Name}";
    }

    private static object[] CreateDataItem(MethodInfo testMethod, object item)
    {
        var itemProperties = item.GetType().GetProperties().ToDictionary(p => p.Name, p => p, SoftString.Comparer);
        var testMethodParameters = testMethod.GetParameters();
        var dataItem = new object[testMethodParameters.Length];

        // We need the index to set the correct item in the result array.
        foreach (var (testMethodParameter, i) in testMethodParameters.Select((x, i) => (x, i)))
        {
            if (itemProperties.TryGetValue(testMethodParameter.Name, out var itemProperty))
            {
                if (testMethodParameter.ParameterType.IsAssignableFrom(itemProperty.PropertyType))
                {
                    dataItem[i] = itemProperty.GetValue(item);
                }
                else
                {
                    throw DynamicException.Create
                    (
                        $"ParameterTypeMismatch",                            
                        $"Cannot assign value of type '{itemProperty.PropertyType.ToPrettyString()}' " +
                        $"to the parameter '{testMethodParameter.Name}' of type '{testMethodParameter.ParameterType.ToPrettyString()}'."
                    );
                }
            }
            else
            {
                if (testMethodParameter.IsOptional)
                {
                    dataItem[i] = testMethodParameter.DefaultValue;
                }
                else
                {
                    throw DynamicException.Create
                    (
                        $"ParameterNotOptional",
                        $"Data item does not specify the required parameter '{testMethodParameter.Name}'."
                    );
                }
            }
        }

        return dataItem;
    }
}

---

Example

I use it the same way as the original attribute:

[Theory]
[SmartMemberData(nameof(GetData))]
public void Can_evaluate_supported_expressions(string useCaseName, object expected, bool throws)
{
    var useCase = _helper.GetExpressions().Single(e => e.Name == useCaseName);
    ExpressionAssert.Equal(expected, useCase, ctx => ctx.WithReferences(_helper.GetReferences()), _output, throws);
}

It gets the data from anonymous-objects like this one that I create from tuples:

public static IEnumerable<object> GetData() => new (string UseCaseName, object Expected, bool Throws)[]
{
    ("Any", true, false),
    ("Sum", 3.0, false),
    ("ToDouble", 1.0, false),
    ("True.Not", false, false),
    ("Double.ToDouble", 1.0, true),
    // ... more use cases
}.Select(uc => new { uc.UseCaseName, uc.Expected, uc.Throws });

---

What do you say? Is this solution robust enough and debug-friendly? Can it be even smarter?

Answer 1

I would use a relaxation on type matching. Since an entire type conversion API is available in the .NET Framework, why not take advantage of it?

if (testMethodParameter.ParameterType.IsAssignableFrom(itemProperty.PropertyType))
{
    dataItem[i] = itemProperty.GetValue(item);
}
else
{
    throw DynamicException.Create
    (
        $"ParameterTypeMismatch",                            
        $"Cannot assign value of type '{itemProperty.PropertyType.ToPrettyString()}' " +
        $"to the parameter '{testMethodParameter.Name}' of type '{testMethodParameter.ParameterType.ToPrettyString()}'."
    );
}

You can still decide to wrap it with an exception handler and do some trickery with Nullable and Null-assignable type checks like here.

dataItem[i] = Convert.ChangeType(itemProperty.GetValue(item), testMethodParameter.ParameterType);