I have built an open-source project based on FluentAssertions in order to solve some recurring tasks I was doing while testing the .Net Core APIs I'm developing.
I find the capability of doing in memory end-to-end tests of the APIs I build a great addition in terms of productivity and validity.
With this project I want to solve two problems: to have a Assert API oriented for HttpResponseMessage, the type of response we get when we do an HTTP request with HttpClient and to get immediate feedback about what went wrong when the test fails, by providing the necessary details in the Test Explorer window from Visual Studio, so I can spend less time in debugging.
I choose FluentAssertions because it treats extensibility as a feature, so they have a great API for doing so. Also, they provide a mechanism to build your own failure messages and link them to a specific type, which is exactly what I needed.
The project is located here.
I have prepared a branch for codereview.stackexchange.com to comply with the rule of not changing the code while you are reviewing it. Hopefully, this is OK, otherwise please let me know how I should proceed.
The nuget package built from this version is
Install-Package FluentAssertions.Web -Version 1.0.113
Next is an example of how it can be used
using FluentAssertions;
using Microsoft.AspNetCore.Mvc.Testing;
using Sample.Api.Net30.Controllers;
using System.Collections.Generic;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using Xunit;
namespace Sample.Api.Net30.Tests
{
public class CommentsControllerTests : IClassFixture<WebApplicationFactory<Startup>>
{
private readonly WebApplicationFactory<Startup> _factory;
public CommentsControllerTests(WebApplicationFactory<Startup> factory)
{
_factory = factory;
}
[Fact]
public async Task Get_Returns_Ok_With_CommentsList()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync("/api/comments");
// Assert
response.Should().Be200Ok().And.BeAs(new[]
{
new { Author = "Adrian", Content = "Hey" },
new { Author = "Johnny", Content = "Hey!" }
});
}
[Fact]
public async Task Get_WithCommentId_Returns_Ok_With_The_Expected_Comment()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync("/api/comments/1");
// Assert
response.Should().Be200Ok().And.BeAs(new
{
Author = "Adrian",
Content = "Hey"
});
}
[Fact]
public async Task Get_Returns_Ok_With_CommentsList_With_TwoUniqueComments()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync("/api/comments");
// Assert
response.Should().Satisfy<IReadOnlyCollection<Comment>>(
model =>
{
model.Should().HaveCount(2);
model.Should().OnlyHaveUniqueItems(c => c.CommentId);
}
);
}
[Fact]
public async Task Get_WithCommentId_Returns_A_NonSpam_Comment()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync("/api/comments/1");
// Assert
response.Should().Satisfy(givenModelStructure: new
{
Author = default(string),
Content = default(string)
}, assertion: model =>
{
model.Author.Should().NotBe("I DO SPAM!");
model.Content.Should().NotContain("BUY MORE");
});
}
[Fact]
public async Task Get_WithCommentId_Returns_Response_That_Satisfies_Several_Assertions()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync("/api/comments/1");
// Assert
response.Should().Satisfy(
r =>
{
r.Content.Headers.ContentRange.Should().BeNull();
r.Content.Headers.Allow.Should().NotBeNull();
}
);
}
[Fact]
public async Task Post_ReturnsOk()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.PostAsync("/api/comments", new StringContent(@"{
""author"": ""John"",
""content"": ""Hey, you...""
}", Encoding.UTF8, "application/json"));
// Assert
response.Should().Be200Ok();
}
[Fact]
public async Task Post_ReturnsOkAndWithContent()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.PostAsync("/api/comments", new StringContent(@"{
""author"": ""John"",
""content"": ""Hey, you...""
}", Encoding.UTF8, "application/json"));
// Assert
response.Should().Be200Ok().And.BeAs(new
{
Author = "John",
Content = "Hey, you..."
});
}
[Fact]
public async Task Post_WithNoContent_ReturnsBadRequest()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.PostAsync("/api/comments", new StringContent("", Encoding.UTF8, "application/json"));
// Assert
response.Should().Be400BadRequest()
.And.HaveErrorMessage("*The input does not contain any JSON tokens*");
}
[Fact]
public async Task Post_WithNoAuthorAndNoContent_ReturnsBadRequest()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.PostAsync("/api/comments", new StringContent(@"{
""author"": """",
""content"": """"
}", Encoding.UTF8, "application/json"));
// Assert
response.Should().Be400BadRequest()
.And.HaveError("Author", "The Author field is required.")
.And.HaveError("Content", "The Content field is required.");
}
[Fact]
public async Task Post_WithNoAuthor_ReturnsBadRequestWithUsefulMessage()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.PostAsync("/api/comments", new StringContent(@"{
""content"": ""Hey, you...""
}", Encoding.UTF8, "application/json"));
// Assert
response.Should().Be400BadRequest()
.And.HaveError("Author", "The Author field is required.")
.And.NotHaveError("content");
}
[Fact]
public async Task Post_WithNoAuthorButWithContent_ReturnsBadRequestWithAnErrorMessageRelatedToAuthorOnly()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.PostAsync("/api/comments", new StringContent(@"{
""content"": ""Hey, you...""
}", Encoding.UTF8, "application/json"));
// Assert
response.Should().Be400BadRequest()
.And.OnlyHaveError("Author", "The Author field is required.");
}
}
}
And the controller that handles the HTTP requests and the corresponding model looks like:
using Microsoft.AspNetCore.Mvc;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
namespace Sample.Api.Net30.Controllers
{
[Route("api/[controller]")]
[ApiController]
public class CommentsController : ControllerBase
{
[HttpGet]
public IEnumerable<Comment> Get() => new[]
{
new Comment { Author = "Adrian", Content = "Hey", CommentId = 1},
new Comment { Author = "Johnny", Content = "Hey!", CommentId = 2 }
};
[HttpGet("{id}")]
public Comment Get(int id) => new Comment { Author = "Adrian", Content = "Hey", CommentId = id };
[HttpPost]
public Comment Post([FromBody] Comment value) => value;
}
public class Comment
{
[Required]
public string Author { get; set; }
[Required, StringLength(maximumLength: 500)]
public string Content { get; set; }
public int CommentId { get; set; }
}
}
The assertion API aims to move the burden of remembering/typing the HTTP status codes as you have the option of recalling either the name or the number. Another common task is to deserialize the response to a certain model type and then to assert on.
The other feature is to show as much information as possible from the request-response pair, when a test fails. So in this example, for the following test, I will get the subsequent messages.
[Fact]
public async Task Get_Returns_Ok_With_CommentsList()
{
// Arrange
var client = _factory.CreateClient();
// Act
var response = await client.GetAsync("/api/comments");
// Assert
response.Should().Be200Ok().And.BeAs(new[]
{
new { Author = "Adrian", Content = "Hey" },
new { Author = "Johnny", Content = "Hey!" },
new { Author = "John", Content = "Hey!" }
});
}
this test fails so the message is
Sample.Api.Net30.Tests.CommentsControllerTests.Get_Returns_Ok_With_CommentsList
Source: CommentsControllerTests.cs line 22
Duration: 128 ms
Message:
Expected response to have a content equivalent to a model, but is has differences:
- expected subjectModel to be a collection with 3 item(s), but {{ Author = Adrian, Content = Hey }, { Author = Johnny, Content = Hey! }}"
"contains 1 item(s) less than"
"{{ Author = Adrian, Content = Hey }, { Author = Johnny, Content = Hey! }, { Author = John, Content = Hey! }}.
.
The HTTP response was:
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Content-Length: 102
[
{
"author": "Adrian",
"content": "Hey",
"commentId": 1
},
{
"author": "Johnny",
"content": "Hey!",
"commentId": 2
}
]
The originated HTTP request was:
GET http://localhost/api/comments HTTP 1.1
Cookie:
Content-Length: 0
Stack Trace:
XUnit2TestFramework.Throw(String message)
TestFrameworkProvider.Throw(String message)
DefaultAssertionStrategy.HandleFailure(String message)
AssertionScope.FailWith(Func`1 failReasonFunc)
AssertionScope.FailWith(Func`1 failReasonFunc)
AssertionScope.FailWith(String message, Object[] args)
HttpResponseMessageAssertions.BeAs[TModel](TModel expectedModel, String because, Object[] becauseArgs) line 52
CommentsControllerTests.Get_Returns_Ok_With_CommentsList() line 31
--- End of stack trace from previous location where exception was thrown ---
This particular test uses Be200Ok and BeAs<> assertions. I will provide next the BeAs assertions to make a picture of how these are implemented
using FluentAssertions.Execution;
using FluentAssertions.Web.Internal;
using System;
using System.Net.Http;
namespace FluentAssertions.Web
{
/// <summary>
/// Contains a number of methods to assert that an <see cref="HttpResponseMessage"/> is in the expected state related to the HTTP content.
/// </summary>
public partial class HttpResponseMessageAssertions
{
/// <summary>
/// Asserts that HTTP response content can be an equivalent representation of the expected model.
/// </summary>
/// <param name="expectedModel">
/// The expected model.
/// </param>
/// <param name="because">
/// A formatted phrase as is supported by <see cref="string.Format(string,object[])" /> explaining why the assertion
/// is needed. If the phrase does not start with the word <i>because</i>, it is prepended automatically.
/// </param>
/// <param name="becauseArgs">
/// Zero or more objects to format using the placeholders in <see paramref="because" />.
/// </param>
public AndConstraint<HttpResponseMessageAssertions> BeAs<TModel>(TModel expectedModel, string because = "", params object[] becauseArgs)
{
ExecuteSubjectNotNull(because, becauseArgs);
if (expectedModel == null)
{
throw new ArgumentNullException(nameof(expectedModel), "Cannot verify having a content equivalent to a model against a <null> model.");
}
var success = TryGetSubjectModel<TModel>(out var subjectModel);
Execute.Assertion
.BecauseOf(because, becauseArgs)
.ForCondition(success)
.FailWith("Expected {context:response} to have a content equivalent to a model, but the JSON representation could not be parsed{reason}. {0}",
Subject);
string[] failures;
using (var scope = new AssertionScope())
{
subjectModel.Should().BeEquivalentTo(expectedModel);
failures = scope.Discard();
}
Execute.Assertion
.BecauseOf(because, becauseArgs)
.ForCondition(failures.Length == 0)
.FailWith("Expected {context:response} to have a content equivalent to a model, but is has differences:{0}{reason}. {1}",
new AssertionsFailures(failures),
Subject);
return new AndConstraint<HttpResponseMessageAssertions>(this);
}
/// <summary>
/// Asserts that HTTP response has content that matches a wildcard pattern.
/// </summary>
/// <param name="expectedWildcardText">
/// The wildcard pattern with which the subject is matched, where * and ? have special meanings.
/// </param>
/// <param name="because">
/// A formatted phrase as is supported by <see cref="string.Format(string,object[])" /> explaining why the assertion
/// is needed. If the phrase does not start with the word <i>because</i>, it is prepended automatically.
/// </param>
/// <param name="becauseArgs">
/// Zero or more objects to format using the placeholders in <see paramref="because" />.
/// </param>
public AndConstraint<HttpResponseMessageAssertions> MatchInContent(string expectedWildcardText, string because = "", params object[] becauseArgs)
{
Guard.ThrowIfArgumentIsNull(expectedWildcardText, nameof(expectedWildcardText), "Cannot verify a HTTP response content match a <null> wildcard pattern.");
ExecuteSubjectNotNull(because, becauseArgs);
var content = GetContent();
if (string.IsNullOrEmpty(content))
{
Execute.Assertion
.BecauseOf(because, becauseArgs)
.FailWith("Expected {context:response} to match the wildcard pattern {0} in its content, but content was <null>{reason}. {1}",
expectedWildcardText,
Subject);
}
string[] failures;
using (var scope = new AssertionScope())
{
content.Should().Match(expectedWildcardText);
failures = scope.Discard();
}
Execute.Assertion
.BecauseOf(because, becauseArgs)
.ForCondition(failures.Length == 0)
.FailWith("Expected {context:response} to match a wildcard pattern in its content, but does not since:{0}{reason}. {1}",
new AssertionsFailures(failures),
Subject);
return new AndConstraint<HttpResponseMessageAssertions>(this);
}
}
}
In order to tell FluentAssertions how to display failure messages, you can implement the IValueFormatter so I have one for HttpResponseMessage
using FluentAssertions.Formatting;
using FluentAssertions.Web.Internal;
using FluentAssertions.Web.Internal.ContentProcessors;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
namespace FluentAssertions.Web
{
internal class HttpResponseMessageFormatter : IValueFormatter
{
public bool CanHandle(object value) => value is HttpResponseMessage;
/// <inheritdoc />
public string Format(object value, FormattingContext context, FormatChild formatChild)
{
var response = (HttpResponseMessage)value;
var messageBuilder = new StringBuilder();
messageBuilder.AppendLine();
messageBuilder.AppendLine();
messageBuilder.AppendLine("The HTTP response was:");
Func<Task> contentResolver = async () => await AppendHttpResponseMessage(messageBuilder, response);
contentResolver.ExecuteInDefaultSynchronizationContext().GetAwaiter().GetResult();
return messageBuilder.ToString();
}
private static async Task AppendHttpResponseMessage(StringBuilder messageBuilder, HttpResponseMessage response)
{
await AppendResponse(messageBuilder, response);
await AppendRequest(messageBuilder, response);
}
private static async Task AppendResponse(StringBuilder messageBuilder, HttpResponseMessage response)
{
AppendProtocolAndStatusCode(messageBuilder, response);
Appender.AppendHeaders(messageBuilder, response.GetHeaders());
AppendContentLength(messageBuilder, response);
await AppendResponseContent(messageBuilder, response);
}
private static async Task AppendRequest(StringBuilder messageBuilder, HttpResponseMessage response)
{
var request = response.RequestMessage;
messageBuilder.AppendLine();
if (request == null)
{
messageBuilder.AppendLine("The originated HTTP request was <null>.");
return;
}
messageBuilder.AppendLine("The originated HTTP request was:");
messageBuilder.AppendLine();
messageBuilder.AppendLine($"{request.Method.ToString().ToUpper()} {request.RequestUri} HTTP {request.Version}");
Appender.AppendHeaders(messageBuilder, request.GetHeaders());
AppendContentLength(messageBuilder, request);
messageBuilder.AppendLine();
await AppendRequestContent(messageBuilder, request.Content);
}
private static async Task AppendResponseContent(StringBuilder messageBuilder, HttpResponseMessage response)
{
var content = response.Content;
if (content == null)
{
return;
}
var processors = new List<IContentProcessor>();
processors.Add(new InternalServerErrorProcessor(response, content));
processors.AddRange(ProcessorsRunner.CommonProcessors(content));
var contentBuilder = await ProcessorsRunner.RunProcessors(processors);
messageBuilder.AppendLine();
messageBuilder.Append(contentBuilder);
}
private static async Task AppendRequestContent(StringBuilder messageBuilder, HttpContent content)
{
await Appender.AppendContent(messageBuilder, content);
}
private static void AppendProtocolAndStatusCode(StringBuilder messageBuilder, HttpResponseMessage response)
{
messageBuilder.AppendLine();
messageBuilder.AppendLine($@"HTTP/{response.Version} {(int)response.StatusCode} {response.StatusCode}");
}
private static void AppendContentLength(StringBuilder messageBuilder, HttpResponseMessage response)
{
if (!response.GetHeaders().Any(c => string.Equals(c.Key, "Content-Length", StringComparison.OrdinalIgnoreCase)))
{
messageBuilder.AppendLine($"Content-Length: {response.Content?.Headers.ContentLength ?? 0}");
}
}
private static void AppendContentLength(StringBuilder messageBuilder, HttpRequestMessage request)
{
if (!request.GetHeaders()
.Any(c => string.Equals(c.Key, "Content-Length", StringComparison.OrdinalIgnoreCase))
)
{
request.Content.TryGetContentLength(out long contentLength);
messageBuilder.AppendLine($"Content-Length: {contentLength}");
}
}
}
}
This class and its collaborators have increased complexity and unfortunately, fewer unit tests. Thus this is also a source of pain points lately, so I will have at some moment to invest more in this regard. The collaborators are some classes that have the knowledge of building failure messages based on certain predicates, like the HttpContent type and/or other aspects of the subjected HttpResponse. So for example for a JSON it will print it and beautify it, for a binary type like data, just an informative error message and for the an internal server error it will try to parse the stack trace embedded in the developer page. Next is the JsonProcessor.
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
namespace FluentAssertions.Web.Internal.ContentProcessors
{
internal class JsonProcessor : ProcessorBase
{
private readonly HttpContent _httpContent;
public JsonProcessor(HttpContent httpContent)
{
_httpContent = httpContent;
}
protected override async Task Handle(StringBuilder contentBuilder)
{
var content = await _httpContent.SafeReadAsStringAsync();
var beautified = content?.BeautifyJson();
if (!string.IsNullOrEmpty(beautified))
{
contentBuilder.Append(beautified);
}
}
protected override bool CanHandle()
{
if (_httpContent == null)
{
return false;
}
_httpContent.TryGetContentLength(out long length);
var mediaType = _httpContent.Headers.ContentType?.MediaType;
return length <= ContentFormatterOptions.MaximumReadableBytes
&& (mediaType.EqualsCaseInsensitive("application/json") || mediaType.EqualsCaseInsensitive("application/problem+json"));
}
}
}
I feel at this moment that I'm close to a first production release, but I didn't get yet through a code review, to see what others think about this tool, so I kindly ask you to help me out.
So there would be some of my questions:
- How complete do you find the assertion API, does the current one fit the day-to-day work? Do you have more suggestions?
- I focused on writing quality code and went also with some of the C# 8.0 features. I most likely missed some aspects, please point them
- Do the samples fulfill their purpose?
- Is the documentation relevant? Are the unit tests complete, beyond code coverage?
- Any other aspects like CI/CD, automated builds, nuget releases?
