Unit Testing Best Practices: Know Your Test’s Lifecycle, Part 2

Let’s take a look at what happens to our simple test fixture when we give it a base class:

using System;
using NUnit.Framework;

namespace NUnitLifecycle
{
    public abstract class TestBase : IDisposable
    {
        protected TestBase()
        {
            Console.WriteLine("Base Constructor");
        }

        [TestFixtureSetUp]
        public void BaseTestFixtureSetUp()
        {
            Console.WriteLine("Base TestFixtureSetup");
        }

        [TestFixtureTearDown]
        public void BaseTestFixtureTearDown()
        {
            Console.WriteLine("Base TestFixtureTearDown");
        }

        [SetUp]
        public void BaseTestSetUp()
        {
            Console.WriteLine("Base SetUp");
        }

        [TearDown]
        public void BaseTestTearDown()
        {
            Console.WriteLine("Base TearDown");
        }

        [Test]
        public void BaseFirstTest()
        {
            Console.WriteLine("Base First Test passes!");
            Assert.Pass();
        }

        [Test]
        public void BaseSecondTest()
        {
            Console.WriteLine("Base Second Test fails!");
            Assert.Fail();
        }

        public virtual void Dispose()
        {
            Console.WriteLine("Base Dispose");
        }
    }

    [TestFixture]
    public class TestClass : TestBase
    {
        public TestClass()
        {
            Console.WriteLine("Derived Constructor");
        }

        [TestFixtureSetUp]
        public void TestClassSetUp()
        {
            Console.WriteLine("Derived TestFixtureSetup");
        }

        [TestFixtureTearDown]
        public void TestClassTearDown()
        {
            Console.WriteLine("Derived TestFixtureTearDown");
        }

        [SetUp]
        public void TestSetUp()
        {
            Console.WriteLine("Derived SetUp");
        }

        [TearDown]
        public void TestTearDown()
        {
            Console.WriteLine("Derived TearDown");
        }

        [Test]
        public void FirstTest()
        {
            Console.WriteLine("Derived First Test passes!");
            Assert.Pass();
        }

        [Test]
        public void SecondTest()
        {
            Console.WriteLine("Derived Second Test fails!");
            Assert.Fail();
        }

        public override void Dispose()
        {
            Console.WriteLine("Derived Dispose");
            base.Dispose();
        }
    }
}

But now we can also see the following new behaviors:

1. TestFixture constructors and dispose methods are called in the same order as for any other class. This is one of the reasons I prefer using constructor/disposable semantics with test fixtures instead of relying on the special FixtureSetUp/FixtureTearDown attributes – we already know the rules for constructors and disposables.
2. The base class SetUp and TearDown methods run around those from the derived class. This is true for both Fixture- and Test- SetUp and TearDown methods. You can think of your base class wrapping your derived class, like Russian nested dolls. This pattern also extends as you create deeper inheritance hierarchies. SetUp is run from the outermost base class down to the leaf derived class, and TearDown is run in exactly the opposite order.
3. All Test-level SetUp and TearDown methods are run for every test. This behavior is a little surprising. Notice that even on the test methods defined in the base class, the derived class’ SetUp and TearDown methods are still getting run. If you start seeing base class tests failing because something they expected from SetUp is suddenly different, this is a likely culprit.
4. Tests are run in alphabetical order. Ok, you can’t infer this one from the output, you’re just going to have to take my word for it (or go play with the test names yourself and see what happens.) NUnit sorts all tests on the fixture by name, from A to Z, and then runs them in that order. Base class test methods are identified as <ClassName>.<TestMethodName> for purposes of sorting. This means that your base class tests will all be grouped together when the fixture is run, but they could all run right in the middle of your derived class’ tests.

Let’s add two more guidelines:

1. Don’t rely on test run order. Test method run order varies between different hosts. NUnit and ReSharper and Gallio rarely all agree on which order to run your tests. As a developer, you’re often not even running a full test suite -you’re running specific fixtures, or even just a few tests within a fixture. If your tests only pass when they’re run in a specific order, you’re going to waste a lot of time hunting down false failures. Remember, the closer you are to release, the more likely that your tests will start failing for “no reason at all.” Keep your tests independent of each other, and you’ll always be able to run them with confidence.
2. Limit test-level SetUp and TearDown semantics. It’s bad enough when a test fails. It’s even worse when you have to jump around between base and derived class code to stitch together what’s happening for each test. If at all possible, limit the definition of SetUp and TearDown methods to your base class. In recent years, I find that if I get my fixture initialization right in the constructor, then I rarely have to do much resetting around each individual test.

You’ve now got a clear understanding of how an individual test fixture is created, initialized, and executed, and how that changes when the fixture has a derived class with its own lifecycle. You’ve also got a few more guidelines to help you keep your unit tests easy to maintain and extend.