More Expressive Tests with Shouldly

There are a couple of key things all good Unit Tests should do:

  1. Tell you clearly what they’re testing.
  2. Tell you clearly why they’re failing.

Built-in assertion libraries don’t really help us to meet these two expectations. Take a simple boolean assertion in xUnit:

var response = new { IsSuccessStatusCode = false };

Assert.True(response.IsSuccessStatusCode);

/*
Assert.True() Failure
Expected: True
Actual:   False
*/

The Assert syntax is not very easy to read and the failure message doesn’t tell us what was being tested.

Things get worse when we need to compare values - see if you can spot the deliberate mistake.

var response = new { StatusCode = 404 };

Assert.Equal(response.StatusCode, 200);

/*
Assert.Equal() Failure
Expected: 404
Actual:   200
*/

Not only does this not tell me what was being tested, but due to a syntax error I’ve (deliberately!) mixed up the expected and actual values - I actually expected response.StatusCode to equal 200 but it’s trivially simple to get the values the wrong way round. In my experience as a developer, if you give someone the chance to make a mistake, they will make it!

nUnit has the Assert.That form that makes things slightly better:

var response = new { StatusCode = 404 };

Assert.That(response.StatusCode, Is.Not.EqualTo(404));

/*
Expected: not equal to 404
But was:  404
*/

It’s easier to get the actual and expected the right way round and both the code and the failure message are slightly easier to read, but it still doesn’t tell me what it expected not to equal 404. (and, honestly, who says “Assert.That …” in everyday speech?).

It’s also a lot of code to type. What we want is something more expressive and with a simpler syntax.

Shouldly (https://shouldly.io/) is a library that helps to solve all these issues (and more.)

We can add from the command line (or use the Visual Studio Package Manager.)

dotnet add package Shouldly

Our assertions now become:

var response = new { IsSuccessStatusCode = false };

response.IsSuccessStatusCode.ShouldBeTrue();

/*
response.IsSuccessStatusCode
    should be
True
    but was
False
*/

var response = new { StatusCode = 404 }; 

response.StatusCode.ShouldNotBe(404);

/*
response.StatusCode
    should not be
404
    but was
*/

Not only is it immediately clear from my tests what I’m testing and what I expect my results to be but the test failure messages tell me what is being tested, what results were expected and what the actual results were. In addition the syntax is much simpler to get right and it follows natural speech patterns so I don’t have to spend valuable time deciphering C# syntax when reading/debugging code.

There are additional advantages.

As you might expect, exceptions are handled (excuse the pun) gracefully:

class Throws
{
    public async Task GetResult() => throw new ArgumentOutOfRangeException();
}

var sut = new Throws();

var thrown = await Should.ThrowAsync<ArgumentNullException>(sut.GetResult);

thrown.ParamName.ShouldBe("expected");

/*
`sut.GetResult`
    should throw
System.ArgumentNullException
    but threw
System.ArgumentOutOfRangeException
*/

The Should.ThrowAsync method returns the thrown Exception so you can continue to test the message or any other properties of the Exception you wish (N.B. there is, of course a synchronous version).

As with any Assertion library there are useful assertions for Types, Collections, Strings and so forth, however there is one final killer feature which makes Shouldly stand out in my opinion.

Whilst in a Unit Test you should strive to only have one logical assertion per test, when you have something like a System Integration Test which has a lot of complex and expensive setup, you may wish to run a number of tests from a given setup, e.g. was the database updated correctly, did any logs get created, were APIs called correctly as well as checking the format and state of the returned HttpResponse.

The problem with a standard Assertion Library is that the first failed Assertion will cause the test to stop running and you won’t get feedback from any further tests.

Shouldly solves this problem with ShouldSatisfyAllConditions:

var response = new { StatusCode = 503 };
var repository = new {RecordCount = 7};
var logs = new {Entries = new[] {new {Message = "Failed"}}};

response.ShouldSatisfyAllConditions
(
    () => response.StatusCode.ShouldBe(200),
    () => repository.RecordCount.ShouldBe(2),
    () => logs.Entries.ShouldContain(e => e.Message == "Success")
);

/*
response
    should satisfy all the conditions specified, but does not.
The following errors were found ...
--------------- Error 1 ---------------
    response.StatusCode
        should be
    200
        but was
    503

--------------- Error 2 ---------------
    repository.RecordCount
        should be
    2
        but was
    7

--------------- Error 3 ---------------
    logs.Entries
        should contain an element satisfying the condition
    (e.Message == "Success")
        but does not

-----------------------------------------
*/

So you’ve got great feedback for all the failed assertions in this test and not just the first.

In this post I’ve given a very quick introduction to Shouldly's capabilities. I hope it helps you in your work.

Happy coding!

Sample code from the article and additional Shouldly examples available at https://github.com/paxdev/ShouldlyDemo

[ csharp  tdd  ]