418 I'm a TeapotServer for RESTful integration-tests

Question

I created a test-server for RESTful integration-tests that I call TeapotServer because I like the status-code 418 that the server returns when the client makes a mistake and sends it an invalid request.

Example

Its API is similar to JustMock. I first tell it to what API I want to test, how the request should look like, how many of them I expect and what response(s) it should return.

There are two types of assertions: - as requests arrive to validate them as soon as possible - after the request to validate whether they have been even called

public class HttpProviderExtensionsTest : IDisposable, IClassFixture<TeapotServerFixture>
{
    private readonly ITeapotServerContext _serverContext;

    private readonly IResourceProvider _http;

    public HttpProviderExtensionsTest(TeapotServerFixture teapotServerFixture)
    {
        _serverContext = teapotServerFixture.GetServer("http://localhost:30002").BeginScope();
        _http = HttpProvider.FromBaseUri("http://localhost:30002/api");
    }

    [Fact]
    public async Task Can_send_email_and_receive_html()
    {
        _serverContext
            .MockPost("/api/mailr/messages/test", request =>
            {
                request
                    .AcceptsHtml()
                    .AsUserAgent("xunit", "1.0")
                    .WithContentTypeJson(content =>
                    {
                        content
                            .HasProperty("$.To")
                            .HasProperty("$.Subject")
                            //.HasProperty("$.From") // Boom! This property does not exist.
                            .HasProperty("$.Body.Greeting");
                    })
                    .Occurs(1); // <-- short-circuit on 2nd request
            })
            .ArrangeResponse(builder =>
            {
                builder
                    .Once(200, "OK!");
            });

        var email = new Email.Html(new[] { "[email protected]" }, "Test-mail")
        {
            Body = new { Greeting = "Hallo Mailr!" }
        };

        var response = await _http.SendEmailAsync("mailr/messages/test", new UserAgent("xunit", "1.0"), email);

        _serverContext.Assert();
        Assert.Equal("OK!", response);
    }

    public void Dispose()
    {
        _serverContext.Dispose();
        _http.Dispose();
    }
}

Server + Middleware

The TeapotServer is based on Kestrel and uses a single TeapotMiddleware. I use it to forward requests to assertions that can short-circuit the pipeline (unlike Flurl that I don't like because of its assertions that take place too late - they cannot protect you from infinite request loops).

[PublicAPI]
public class TeapotServer : IDisposable
{
    private readonly IWebHost _host;

    private readonly ConcurrentDictionary<Guid, ITeapotServerContext> _serverContexts = new ConcurrentDictionary<Guid, ITeapotServerContext>();

    public TeapotServer(string url)
    {
        var configuration =
            new ConfigurationBuilder()
                // Tests can use their own urls so let's not use hosting.json but in-memory-collection
                .AddInMemoryCollection(new Dictionary<string, string>
                {
                    ["urls"] = url // <-- this is the only way that works with Kestrel
                })
                .Build();

        _host =
            new WebHostBuilder()
                .UseKestrel()
                .UseConfiguration(configuration)
                .ConfigureServices(services =>
                {
                    // Allows to validate requests as they arrive.
                    services.AddSingleton((RequestAssertDelegate)Assert);
                    // Allows to provide custom responses for each request.
                    services.AddSingleton((ResponseMockDelegate)GetResponseFactory);
                })
                .Configure(app =>
                {
                    app.UseMiddleware<TeapotMiddleware>();
                })
                .Build();

        _host.StartAsync().GetAwaiter().GetResult(); // <-- asp.net-core TestServer is doing this too.
    }

    //public Task Task { get; set; } // <-- I think I don't need this anymore...

    // Creates a new server-context that separates api-mocks.
    public ITeapotServerContext BeginScope()
    {
        return _serverContexts.GetOrAdd(Guid.NewGuid(), id => new TeapotServerContext(Disposable.Create(() => _serverContexts.TryRemove(id, out _))));
    }

    private void Assert(RequestCopy requestCopy)
    {
        FindApiMock(requestCopy.Method, requestCopy.Uri)?.Assert(requestCopy);
    }

    private Func<HttpRequest, ResponseMock> GetResponseFactory(HttpMethod method, UriString uri)
    {
        return FindApiMock(method, uri)?.GetResponseFactory();
    }

    // Finds an api-mock that should handle the current request.
    [CanBeNull]
    private ApiMock FindApiMock(HttpMethod method, UriString uri)
    {
        if (_serverContexts.IsEmpty) throw new InvalidOperationException($"Cannot get response without a server-context. Call '{nameof(BeginScope)}' first.");

        var mocks =
            from tc in _serverContexts.Values
            from rm in tc
            where rm.Method == method && rm.Uri == uri
            select rm;

        return mocks.FirstOrDefault();
    }

    public void Dispose()
    {
        _host.StopAsync().GetAwaiter().GetResult();
        _host.Dispose();
    }
}

public delegate void RequestAssertDelegate(RequestCopy requestCopy);

[CanBeNull]
public delegate Func<HttpRequest, ResponseMock> ResponseMockDelegate(HttpMethod method, UriString path);

[UsedImplicitly]
internal class TeapotMiddleware
{
    private readonly RequestDelegate _next;
    private readonly RequestAssertDelegate _requestAssert;
    private readonly ResponseMockDelegate _nextResponseMock;

    public TeapotMiddleware
    (
        RequestDelegate next,
        RequestAssertDelegate requestAssert,
        ResponseMockDelegate nextResponseMock
    )
    {
        _next = next;
        _requestAssert = requestAssert;
        _nextResponseMock = nextResponseMock;
    }

    public async Task Invoke(HttpContext context)
    {
        try
        {
            var uri = context.Request.Path + context.Request.QueryString;

            // You need this later to restore the body so don't dispose it.
            var bodyBackup = new MemoryStream();

            // You copy it because the original stream does not support seeking.
            await context.Request.Body.CopyToAsync(bodyBackup);

            // You copy the request because otherwise it won't pass the "barrier" between the middleware and the assert.
            using (var bodyCopy = new MemoryStream())
            {
                await bodyBackup.Rewind().CopyToAsync(bodyCopy);

                var request = new RequestCopy
                {
                    Uri = uri,
                    Method = new HttpMethod(context.Request.Method),
                    // There is no copy-constructor.
                    Headers = new HeaderDictionary(context.Request.Headers.ToDictionary(x => x.Key, x => x.Value)),
                    ContentLength = context.Request.ContentLength,
                    Content = bodyCopy
                };

                _requestAssert(request);
            }

            // Restore body.
            context.Request.Body = bodyBackup;

            await _next(context);

            var responseMock = _nextResponseMock(new HttpMethod(context.Request.Method), uri);
            if (responseMock is null)
            {
                context.Response.StatusCode = StatusCodes.Status404NotFound;
            }
            else
            {
                using (var response = responseMock(context.Request))
                {
                    context.Response.StatusCode = response.StatusCode;
                    context.Response.ContentType = response.ContentType;

                    // Let's see what kind of content we got and handle it appropriately...

                    if (response.ContentType == MimeType.Plain)
                    {
                        await context.Response.WriteAsync((string)response.Content);
                    }

                    if (response.ContentType == MimeType.Binary)
                    {
                        await ((Stream)response.Content).Rewind().CopyToAsync(context.Response.Body);
                    }

                    if (response.ContentType == MimeType.Json)
                    {
                        await context.Response.WriteAsync(JsonConvert.SerializeObject(response.Content));
                    }
                }
            }
        }
        catch (Exception ex)
        {
            // It looks like the client sent an invalid request.
            if (ex is DynamicException dex && dex.NameMatches("AssertException"))
            {
                // "Response status code does not indicate success: 418 (I'm a teapot)."
                context.Response.StatusCode = StatusCodes.Status418ImATeapot; // <-- I find this one funny. 
            }
            // Nope, there is a problem with the server.
            else
            {
                context.Response.StatusCode = StatusCodes.Status500InternalServerError;
            }

            context.Response.ContentType = MimeType.Plain;
            await context.Response.WriteAsync(ex.ToString()); // <-- dump the exception to the response stream.
        }
    }
}

Helpers

There is a hole bunch of helpers that provide (hopefully) a user-friendly API.

---

They start with the TeapotServerContext that separates api-mocks between test methods. I created a couple of convenience shortcut extensions like MockGet to save typing. Each of those returns an ApiMock that stores a pair of request and response builders for a single API. They are used to configure request asserts and responses.

public interface ITeapotServerContext : IEnumerable<ApiMock>, IDisposable
{
    ApiMock MockApi(HttpMethod method, UriString uri);

    void Assert();
}

internal class TeapotServerContext : List<ApiMock>, ITeapotServerContext
{
    private readonly IDisposable _disposer;

    public TeapotServerContext(IDisposable disposer)
    {
        _disposer = disposer;
    }

    public ApiMock MockApi(HttpMethod method, UriString uri)
    {
        var mock = new ApiMock(method, uri);
        Add(mock);
        return mock;
    }

    public void Assert()
    {
        foreach (var apiMock in this)
        {
            apiMock.Assert();
        }
    }

    public Func<HttpRequest, ResponseMock> GetResponseFactory(HttpMethod method, UriString uri)
    {
        return this.FirstOrDefault(m => m.Method == method && m.Uri == uri)?.GetResponseFactory();
    }

    public void Dispose()
    {
        _disposer.Dispose();
    }
}

public static class TeapotServerContextExtensions
{
    public static ApiMock MockGet(this ITeapotServerContext context, string uri, Action<IRequestBuilder> configureRequest)
    {
        return
            context
                .MockApi(HttpMethod.Get, uri)
                .ArrangeRequest(configureRequest);
    }

    public static ApiMock MockPost(this ITeapotServerContext context, string uri, Action<IRequestBuilder> configureRequest)
    {
        return
            context
                .MockApi(HttpMethod.Post, uri)
                .ArrangeRequest(configureRequest);
    }

    public static ApiMock MockPut(this ITeapotServerContext context, string uri, Action<IRequestBuilder> configureRequest)
    {
        return
            context
                .MockApi(HttpMethod.Put, uri)
                .ArrangeRequest(configureRequest);
    }

    public static ApiMock MockDelete(this ITeapotServerContext context, string uri, Action<IRequestBuilder> configureRequest)
    {
        return
            context
                .MockApi(HttpMethod.Delete, uri)
                .ArrangeRequest(configureRequest);
    }
}

// Represents a single api-mock.
public class ApiMock
{
    private readonly IRequestBuilder _request;
    private readonly IResponseBuilder _response;

    public ApiMock(HttpMethod method, UriString uri)
    {
        Method = method;
        Uri = uri;

        _request = new RequestBuilder();
        _response = new ResponseBuilder().Always(200, new { Message = "OK" }, MimeType.Json);
    }

    public HttpMethod Method { get; }

    public UriString Uri { get; }

    // Allows to configure request asserts.
    public ApiMock ArrangeRequest(Action<IRequestBuilder> configure)
    {
        configure(_request);
        return this;
    }

    // Allows to configure responses.
    public ApiMock ArrangeResponse(Action<IResponseBuilder> configure)
    {
        configure(_response.Clear());
        return this;
    }

    // Validates the request either as it arrives (not-null) or afterwards (null).
    public void Assert(RequestCopy requestCopy = default)
    {
        try
        {
            _request.Assert(requestCopy);
        }
        catch (Exception inner)
        {
            throw DynamicException.Create("Assert", $"{Method} {Uri}", inner);
        }
    }

    // Tries to get the nest response.
    public Func<HttpRequest, ResponseMock> GetResponseFactory()
    {
        return request => _response?.Next(request);
    }
}

Request and response builders

• The RequestBuilder keeps a collection of assert Actions and calls each of them when a request arrives. Assertions are created with convenience extensions that can validate headers, content etc.
• The ResponseBuilder keeps a collection of response factories. It's a queue where each item can produce either a response or null in which case it's removed from the queue an the next factory is used. When there are no more response factories an exception is thrown.

public interface IRequestBuilder
{
    [NotNull]
    IRequestBuilder Add(Action<RequestCopy> assert, bool canShortCircuit);

    void Assert(RequestCopy requestCopy);
}

// Maintains a collection of request-asserts.
internal class RequestBuilder : IRequestBuilder
{
    private readonly IList<(Action<RequestCopy> Assert, bool CanShortCircuit)> _asserts = new List<(Action<RequestCopy>, bool CanShortCircuit)>();

    // Adds a request assert. If it can-short-circuit then it can validate requests as they arrive.
    public IRequestBuilder Add(Action<RequestCopy> assert, bool canShortCircuit)
    {
        _asserts.Add((assert, canShortCircuit));
        return this;
    }

    // Fires all asserts.
    public void Assert(RequestCopy requestCopy)
    {
        foreach (var (assert, canShortCircuit) in _asserts)
        {
            if (requestCopy is null && !canShortCircuit)
            {
                continue;
            }

            assert(requestCopy);
        }
    }
}

[PublicAPI]
public static class RequestBuilderExtensions
{
    public static IRequestBuilder Occurs(this IRequestBuilder builder, int exactly)
    {
        var counter = 0;
        return builder.Add(request =>
        {
            if (request.CanShortCircuit())
            {
                if (++counter > exactly)
                {
                    throw DynamicException.Create(nameof(Occurs), $"Api was called {counter} time(s) but expected {exactly}.");
                }
            }
            else
            {
                if (counter != exactly)
                {
                    throw DynamicException.Create(nameof(Occurs), $"Api was called {counter} time(s) but expected {exactly}.");
                }
            }
        }, true);
    }

    public static IRequestBuilder WithHeader(this IRequestBuilder builder, string header, params string[] expectedValues)
    {
        return builder.Add(request =>
        {
            if (request.CanShortCircuit())
            {
                if (request.Headers.TryGetValue(header, out var actualValues))
                {
                    if (actualValues.Intersect(expectedValues).Count() != expectedValues.Count())
                    {
                        throw DynamicException.Create
                        (
                            "DifferentHeader",
                            $"Expected: [{expectedValues.Join(", ")}]{Environment.NewLine}" +
                            $"Actual:   [{actualValues.Join(", ")}]"
                        );
                    }
                }
                else
                {
                    throw DynamicException.Create
                    (
                        "HeaderNotFound",
                        $"Header '{header}' is missing."
                    );
                }
            }
        }, false);
    }

    public static IRequestBuilder WithApiVersion(this IRequestBuilder builder, string version) => builder.WithHeader("Api-Version", version);

    public static IRequestBuilder WithContentType(this IRequestBuilder builder, string mediaType) => builder.WithHeader("Content-Type", mediaType);

    public static IRequestBuilder WithContentTypeJson(this IRequestBuilder builder, Action<ContentSection<JToken>> contentAssert)
    {
        return builder.Add(request =>
        {
            var content = request.DeserializeAsJToken();
            contentAssert(ContentSection.FromJToken(content));
        }, false);
    }

    public static IRequestBuilder AsUserAgent(this IRequestBuilder builder, string product, string version) => builder.WithHeader("User-Agent", $"{product}/{version}");

    public static IRequestBuilder Accepts(this IRequestBuilder builder, string mediaType) => builder.WithHeader("Accept", mediaType);

    public static IRequestBuilder AcceptsJson(this IRequestBuilder builder) => builder.Accepts("application/json");

    public static IRequestBuilder AcceptsHtml(this IRequestBuilder builder) => builder.Accepts("text/html");
}

public interface IResponseBuilder
{
    [NotNull]
    ResponseBuilder Enqueue(Func<HttpRequest, ResponseMock> next);

    ResponseBuilder Clear();

    [NotNull]
    ResponseMock Next(HttpRequest request);
}

public class ResponseBuilder : IResponseBuilder
{
    private readonly Queue<Func<HttpRequest, ResponseMock>> _responses = new Queue<Func<HttpRequest, ResponseMock>>();

    public ResponseBuilder Enqueue(Func<HttpRequest, ResponseMock> next)
    {
        _responses.Enqueue(next);
        return this;
    }

    public ResponseBuilder Clear()
    {
        _responses.Clear();
        return this;
    }

    // Gets the next response or throws when there are none.
    public ResponseMock Next(HttpRequest request)
    {
        while (_responses.Any())
        {
            var createResponse = _responses.Peek();
            var response = createResponse(request);

            // This response factory is empty.
            if (response is null)
            {
                // Remove it from the queue an try again.
                _responses.Dequeue();
                continue;
            }

            return response;
        }

        throw DynamicException.Create("OutOfResponses", "There are no more responses");
    }
}

[PublicAPI]
public static class ResponseBuilderExtensions
{
    public static IResponseBuilder Once(this IResponseBuilder response, int statusCode, object content, string contentType = default)
    {
        return response.Exactly(statusCode, content, content.DetermineContentType(contentType), 1);
    }

    public static IResponseBuilder Always(this IResponseBuilder response, int statusCode, object content, string contentType = default)
    {
        return response.Enqueue(request => new ResponseMock(statusCode, content, content.DetermineContentType(contentType)));
    }

    public static IResponseBuilder Exactly(this IResponseBuilder response, int statusCode, object content, string contentType, int count)
    {
        var counter = 0;
        return response.Enqueue(request => counter++ < count ? new ResponseMock(statusCode, content, contentType) : default);
    }

    // Forwards request to response.
    public static IResponseBuilder Echo(this IResponseBuilder response)
    {
        return response.Enqueue(request =>
        {
            var requestCopy = new MemoryStream();
            request.Body.Rewind().CopyTo(requestCopy);
            return new ResponseMock(200, requestCopy, MimeType.Binary);
        });
    }

    private static string DetermineContentType(this object content, string contentType)
    {
        return contentType ?? (content is string ? MimeType.Plain : MimeType.Json);
    }
}

Data

Other classes used here are mainly of DTO nature that just pass data from one place to the other.

public class ContentSection<TContent> where TContent : class
{
    public ContentSection([NotNull] TContent value, string path = ".")
    {
        Value = value ?? throw new ArgumentNullException(nameof(value));
        Path = path;
    }

    public TContent Value { get; }

    public string Path { get; }
}

public static class ContentSection
{
    // You use ?? to support the null-pattern in case there is no response.

    public static ContentSection<JToken> FromJToken(JToken content, string path = ".")
    {
        return new ContentSection<JToken>(content ?? JToken.Parse("{}"), path);
    }

    public static ContentSection<JValue> FromJValue(JValue content, string path = ".")
    {
        return new ContentSection<JValue>(content ?? JValue.CreateNull(), path);
    }
}

// This needs to be copy because it otherwise won't pass the middleware/assert "barrier"
// between the server and the "client".
public class RequestCopy
{
    public UriString Uri { get; set; }

    public HttpMethod Method { get; set; }

    public IHeaderDictionary Headers { get; set; }

    public long? ContentLength { get; set; }

    public Stream Content { get; set; }
}

public static class RequestCopyExtensions
{
    public static bool CanShortCircuit(this RequestCopy request) => !(request is null);

    // Deserializes the content of request-copy for further analysis.
    public static JToken DeserializeAsJToken(this RequestCopy request)
    {
        if (request.Headers.TryGetValue("Content-Type", out var contentType) && contentType != MimeType.Json)
        {
            throw new ArgumentOutOfRangeException($"This method can deserialize only {MimeType.Json} content.");
        }

        if (request.ContentLength == 0)
        {
            return default;
        }

        using (var memory = new MemoryStream())
        {
            // You copy it because otherwise it'll get disposed when the request-copy vanishes.
            request.Content.Rewind().CopyTo(memory);

            using (var reader = new StreamReader(memory.Rewind()))
            {
                var body = reader.ReadToEnd();
                return JToken.Parse(body);
            }
        }
    }
}

public class ResponseMock : IDisposable
{
    public ResponseMock(int statusCode, object content, string contentType)
    {
        StatusCode = statusCode;
        Content = content;
        ContentType = contentType;
    }

    public int StatusCode { get; }

    [CanBeNull]
    public object Content { get; }

    public string ContentType { get; }

    public void Dispose()
    {
        (Content as IDisposable)?.Dispose();
    }
}

Json assertions

There is also a set of extensions that simplify content validation. I currently work only with JSON so only this kind of extensions exist. However, the ContentSection<T> is built to support other content-types if necessary.

public static class ContentAssert
{
    // Use this the same pattern for each assert: condition ? throw : content

    #region JToken helpers 

    [NotNull]
    public static ContentSection<JValue> Value(this ContentSection<JToken> content, string jsonPath)
    {
        return
            !(content.Value.SelectToken(jsonPath) is JValue value)
                ? throw DynamicException.Create("ContentPropertyNotFound", $"There is no such property as '{jsonPath}'")
                : ContentSection.FromJValue(value, jsonPath);
    }

    [NotNull]
    public static ContentSection<JToken> HasProperty(this ContentSection<JToken> content, string jsonPath)
    {
        return
            content.Value.SelectToken(jsonPath) is null
                ? throw DynamicException.Create("ContentPropertyNotFound", $"There is no such property as '{jsonPath}'")
                : content;
    }

    #endregion

    #region JValue helpers

    [NotNull]
    public static ContentSection<JValue> IsNotNull(this ContentSection<JValue> content)
    {
        return
            content.Value.Equals(JValue.CreateNull())
                ? throw DynamicException.Create("ValueNull", $"Value at '{content.Path}' is null.")
                : content;
    }

    [NotNull]
    public static ContentSection<JValue> IsEqual(this ContentSection<JValue> content, object expected)
    {
        return
            !content.Value.Equals(new JValue(expected))
                ? throw DynamicException.Create("ValueNotEqual", $"Value at '{content.Path}' is '{content.Value}' but should be '{expected}'.")
                : content;
    }

    [NotNull]
    public static ContentSection<JValue> IsLike(this ContentSection<JValue> content, [RegexPattern] string pattern, RegexOptions options = RegexOptions.IgnoreCase)
    {
        return
            !Regex.IsMatch(content.Value.Value<string>() ?? string.Empty, pattern, options)
                ? throw DynamicException.Create("ValueNotLike", $"Value at '{content.Path}' is '{content.Value}' but should be like '{pattern}'.")
                : content;
    }

    #endregion
}

---

Extensibility

I build this framework as I use it and new assertion use-cases emerge with each project so I'd like it to be primarily extendible at this endpoint.

I currently can do the following:

• use different URL for each server.
• add other content-types because ContentSection<T> is extensible via T and consequently I can chain specific extension to it
• add other extensions to the RequestBuilder supporting more request assertions

Questions

Besides extensibilty I'd like this framework to be easy to use and intuitive. It should also provide useful feedback about what went wrong and where.

What is your rating?

---

You can also take a look at it on github.

Answer 1

You allow a server to reuse an existing context when calling BeginScope. Does this mean multiple integration tests could reuse a context? If so, how do you handle disposal of the context? Your example shows one test that handles disposal of a possibly shared context.

public ITeapotServerContext BeginScope()
{
     // CR: who should be allowed to handle the lifetime of a shared context?
    return _serverContexts.GetOrAdd(Guid.NewGuid(), 
        id => new TeapotServerContext(Disposable.Create(
             () => _serverContexts.TryRemove(id, out _))));
}

Maybe consumers of the API should not be able to dispose a context, but could register and unregister on a shared context. The server would then be responsible for lifetime management of each context. I would remove IDisposable from the interface definition of ITeapotServerContext.

public void Dispose()
{
    // CR: what if context is shared?
    _serverContext.Dispose();
    _http.Dispose();
}

---

The constructor of TeapotServer performs some configuration, followed by starting the host. I'm personally not a fan of doing anything more than storing state and configuring the instance in a constructor. Further initialisation - like IO operations - should be done in a method called Initialise().

_host.StartAsync().GetAwaiter().GetResult();

---

Talking about extensibility, I'm not sure whether ITeapotServerContext should be an interface. I don't see any other possible implementation besides the one you provided. Its lifetime and dispose pattern is also an internal occasion.