Writing unit tests in VSTT is great since it comes with a nice unit testing framework and the test tools are neatly integrated into Visual Studio. There is one exception though: Exception testing (yes, that pun was intended). Consider the following example:
public class Foo
{
private object[] _array;
public Foo(object[] array)
{
if (array == null)
throw new ArgumentNullException();
if (array.Length == 0)
throw new ArgumentException();
_array = array;
}
public object GetItem(int index)
{
if (index < 0 || index >= _array.Length)
throw new ArgumentOutOfRangeException();
return _array[index];
}
}
This contrived example contains three throw statements requiring four distinct checks in order to hit all code paths which will cause an exception to be thrown (the if statement in GetItem(int) contains two blocks due to short-circuit evaluation). Since the way to test for exceptions in VSTT is the ExpectedException attribute we need one test method for each code path:
[TestMethod]
[ExpectedException(typeof(ArgumentNullException))]
public void ctor_array_null()
{
Foo foo = new Foo(null);
}
[TestMethod]
[ExpectedException(typeof(ArgumentException))]
public void ctor_array_empty()
{
Foo foo = new Foo(new object[0]);
}
[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void GetItem_index_minus_one()
{
Foo foo = new Foo(new object[1]);
foo.GetItem(-1);
}
[TestMethod]
[ExpectedException(typeof(ArgumentOutOfRangeException))]
public void GetItem_index_array_length_plus_one()
{
Foo foo = new Foo(new object[1]);
foo.GetItem(1);
}
While there is nothing wrong with this in general it does bloat both the test code base as well as the test list a little bit. Also, if I need to go through a couple of complex initialization steps before actually triggering the exception I'll have to execute those for every single test method. Another more serious potential issue is that with this implementation I can only verify that an exception caused by the test method is of a certain type. But I can never know for sure that the statement I expected to throw actually did throw the exception without stepping through the code or paying close attention to my code coverage while writing the tests. The worst case scenario here is a bug in my test method that causes the expected exception to be thrown before the statement that was supposed to throw even gets executed. This could go unnoticed since the test would still pass.
Now, if you've worked with VSTT for a while you may have noticed that there are a couple of classes for assertions. The Assert class is probably the one that is most well-known. But there are also the classes CollectionAssert and StringAssert which complement the general purpose Assert class. What's missing is an ExceptionAssert class! So let's create one.
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
public static class ExceptionAssert
{
public static void Throws(Type exceptionType, Action action)
{
if (exceptionType == null || action == null)
throw new ArgumentNullException();
if (exceptionType != typeof(Exception) && !exceptionType.IsSubclassOf(typeof(Exception)))
throw new ArgumentException();
try
{
action();
}
catch (Exception e)
{
if (e.GetType() != exceptionType)
throw new AssertFailedException(String.Format(
"ExceptionAssert.Throws failed. Expected exception type: {0}. Actual exception type: {1}. Exception message: {2}",
exceptionType.FullName,
e.GetType().FullName,
e.Message));
return;
}
throw new AssertFailedException(String.Format(
"ExceptionAssert.Throws failed. No exception was thrown (expected exception type: {0}).",
exceptionType.FullName));
}
}
Parameter validation aside, Throws(Type, Action) takes the type of the expected exception and a delegate of the type Action pointing to the code that is supposed to throw. This delegate is called in a try block since we expect it to throw and the corresponding catch block catches all managed exceptions (this a rare instance in which it is actually okay to do so). If an exception is caught but it is not of the expected type an AssertFailedException is thrown to signal the test failure to the test runtime which is also what for example the Assert class that ships with VSTT does if the condition is not met. Finally, if calling the delegate did not cause an exception we also throw an AssertFailedException this time with a different message. Using this new assertion method simplifies our test code for the constructor of the Foo class to this:
[TestMethod]
public void Constructor()
{
ExceptionAssert.Throws(typeof(ArgumentNullException),
delegate { new Foo(null); });
ExceptionAssert.Throws(typeof(ArgumentException),
delegate { new Foo(new object[0]); });
}
You may argue that this alone is not that much of a simplification but it's a start. The Throws() method above only takes delegates that don't have parameters and return void. What we need to do is add overloads for different kinds of delegates. I just want to show one more example for delegates taking one parameter and returning void. While the remaining test code I haven't replaced yet could also be written using anonymous methods this overload will allow me to demonstrate the usage of lambda expressions in the test method (for all those programmers out there who love them - I think I'm slowly turning into one myself).
public static void Throws<T>(Type exceptionType, T target, Action<T> action)
{
if (exceptionType == null || target == null || action == null)
throw new ArgumentNullException();
if (exceptionType != typeof(Exception) && !exceptionType.IsSubclassOf(typeof(Exception)))
throw new ArgumentException();
try
{
action(target);
}
catch (Exception e)
{
if (e.GetType() != exceptionType)
throw new AssertFailedException(String.Format(
"ExceptionAssert.Throws failed. Expected exception type: {0}. Actual exception type: {1}. Exception message: {2}",
exceptionType.FullName,
e.GetType().FullName,
e.Message));
return;
}
throw new AssertFailedException(String.Format(
"ExceptionAssert.Throws failed. No exception was thrown (expected exception type: {0}).",
exceptionType.FullName));
}
As you can see most of the new overload is exactly the same as before (differences in bold) and it allows us to replace the remaining tests with the following single test method:
[TestMethod]
public void GetItem()
{
object content = new object();
Foo foo = new Foo(new object[] { content });
ExceptionAssert.Throws(typeof(ArgumentOutOfRangeException), foo, f => f.GetItem(-2));
ExceptionAssert.Throws(typeof(ArgumentOutOfRangeException), foo, f => f.GetItem(-1));
Assert.AreEqual(content, foo.GetItem(0));
ExceptionAssert.Throws(typeof(ArgumentOutOfRangeException), foo, f => f.GetItem(1));
ExceptionAssert.Throws(typeof(ArgumentOutOfRangeException), foo, f => f.GetItem(2));
}
This new test method does not only add a regular assert in-between exception checks but it also calls GetItem(int) with two more invalid values without the need for adding additional test methods. As cool as this may (or may not) be, I'd like to close this post with a little warning because there is something I failed to mention before which I consider to be an advantage of the attribute-based approach:
When in doubt you should assume that an exception is thrown because of literally exceptional circumstances which may leave your objects under test in an undefined state. Using an attribute and catching the expected exception in the test runtime makes it more difficult for you to shoot yourself in the foot by reusing objects in your test method which may be in a bad state (or looking at it from the other side, my ExceptionAssert class makes it easier since allowing you to keep going after an exception is exactly the point). That said, I only recommend this approach of reusing instances for multiple exception checks when you know for sure that the objects under test are not in an undefined state after throwing an exception because the exception is part of parameter validation before any state changes occur or the object can reliably recover from the error condition for example by rolling back changes.
This posting is provided "AS IS" with no warranties, and confers no rights.