Extensible Test Assertions With MSTest VSTS
If you have used more than one unit test framework (i.e. xUnit.NET, NUnit, MSTest/VSTS, etc.), chances are that you’ll miss features from one when using another.
In particular, MSTest/VSTS is lagging behind the crowd as it’s stuck in an NUnit circa 2002 (2.0) API for assertions and test attributes. This is what happens when you have a product that has to honor backwards compatibility even at the expense of usability and/or evolution of coding styles and API design.
Compare that with xUnit.NET, which had the luxury of starting from scratch just over a year ago, with all the accumulated learnings from NUnit and years of TDD work. The one assertion I missed the most from xUnit.NET was Throws:
Assert.Throws<ArgumentNullException>(() => service.Do(null));
Very explicit, highly precise check for an exception being thrown. MSTest, on the other hand, is still stuck in the (now even called a TDD anti-pattern, “The Secret Catcher”) attribute-based version:
[ExpectedException(typeof(ArgumentNullException))]
public void WhenDoReceivesNull_ThenThrowsArgumentNullException()
{
service.Do(null);
}
In this case, doesn’t look like a whole lot of difference, but if you have more than a couple lines of code in the test, chances are that the exception can be thrown by someone else in there and you would wrongly believe the test actually passed when it didn’t!
I also like xUnit.NET dropping the “Is” prefix on all assertions, which has always seemed redundant to me (i.e. Assert.NotNull vs Assert.IsNotNull).
Finally, with .NET 3.5 came along extension methods, essentially a way to infinitely extend any given interface as needs arise. Even xUnit.NET is lagging a bit there, but such is life for software: become partially obsolete on almost every rev of the underlying platform. Hence, leveraging now this open-ended extensibility mechanism in .NET seems like the obvious choice. And in the process, we would be able to bring in our beloved idioms from our favorite unit test framework.
Breeding New Life to MSTest APIs
In many projects you just don’t get to decide which test framework will be used. You’re given one. For many .NET developers working on Microsoft-only shops, that’s MSTest/VSTS and you have to live with it. Yes, things like TestDriven.NET can make it run fast (as it should natively) making it bearable. But you still have to use that old-looking API and there’s no easy way to extend it. Or is there?
As you probably know already, the key to extension methods is to have an actual instance of a type that we can use to hook the extension methods to. Hence, here is IAssertion:
public interface IAssertion
{
}
And a default implementation, which doesn’t need to provide any assertions, as we want all of them to be fully extensible and provided outside of these hook types:
public class Assertion : IAssertion
{
}
Now, let’s move on and define some basic assertions for IAssertion:
public static class BasicAssertions
{
public static void Equal<T>(this IAssertion assertion, T expected, T actual, string message = null, params object[] args)
{
Assert.AreEqual<T>(expected, actual, message, args);
}
... others
}
With that in place, we can easily enable this xUnit.NET style assertions quite easily:
[TestClass]
public class FooSpec
{
static readonly IAssertion Assert = new Assertion();
[Test]
public void WhenDo_ThenReturnsFoo()
{
var result = foo.Do();
Assert.Equal("foo", result);
}
}
Note how the test itself looks just like an xUnit.NET would. You can now get crazy and add whatever helper assertions you want via extension methods on IAssertion. Also, note how I’m mostly providing usability improvements on top of the underlying MSTest assertions, which is also nice. I’ve added the following two attributes to the assertion classes so that the debugging experience is nicer:
[DebuggerStepThrough]
[DebuggerNonUserCode]
How C# 4.0 Optional Simplify API Design
Current unit test frameworks provide multiple overloads for each assertion, so that you can provide an optional failure message to augment the default assertion failure message (i.e. “Account should be debited but it was not!”, rather than “Expected 0 but was 10”). Additionally, the message might even contain formatting arguments, such as "Should have been {0} but source account remained with a {1} balance", expected, account.Balance. With C# 4.0 optional parameters we can simplify what would be three overloads (one with no message, one with a fixed message, and one with a message plus formatting arguments) to just one:
public static void Equal<T>(this IAssertion assertion, T expected, T actual, string message = null, params object[] args)
In most cases where overloads just provide increasing amount of arguments that just enable overriding a default value, this new feature in C# 4.0 makes your code (the API but also the consuming code) much more explicit.
Get the assertions code from Clarius Labs.
