Sometimes, you need some code to run before or after a set of test fixtures. I tend to encounter this scenario frequently with integration and system tests. For example, if you’re using Selenium to run your full system tests through the Selenium Web Driver, you need to make sure that the Selenium server is running before you run any of your tests. You could do this on the build server by extending your build script with a “StartSelenium” task, and then remind all your developers to start the server themselves before they run the tests directly. But wouldn’t it be better to guarantee directly that, every time those ui tests get run, that the selenium server was active? AND, as a bonus, wouldn’t it be great if the selenium server was shut down when the tests finished?

Another case where this need shows up is in database-coupled integration tests. These kinds of tests often suffer from unintended order dependencies – one set of tests end up assuming that some data introduced by an earlier set of tests will be present in the database, and then those tests fail if you run them in isolation.

Another approach I often see people take is to have a “special” test fixture  that performs all their database initialization and initial data loading. That test fixture might look like this:

Dropping Database.
Rebuild Database with latest Schema.
Load basic test environment data.
Loading basic account data.
Opening outer Transaction scope
Executing test 1.
Executing test 2.
Roll back transaction scope.

using System;
using NUnit.Framework;

[SetUpFixture]
public class TestSuiteInitializer : IDisposable
{
    public TestSuiteInitializer()
    {
        Console.WriteLine("Constructing TestSuiteInitializer.");
    }

    [SetUp]
    public void SetUp()
    {
        Console.WriteLine("Setting up TestSuiteInitializer.");
    }

    [TearDown]
    public void TearDown()
    {
        Console.WriteLine("Tearing down TestSuiteInitializer.");
    }
    
    public void Dispose()
    {
        Console.WriteLine("Disposing TestSuiteInitializer.");
    }
}
namespace SetUpFixtureSample
{
    [SetUpFixture]
    public class NamespaceInitializer : IDisposable
    {
        public NamespaceInitializer()
        {
            Console.WriteLine("Constructing NamespaceInitializer .");
        }

        [SetUp]
        public void SetUp()
        {
            Console.WriteLine("Setting up NamespaceInitializer .");
        }

        [TearDown]
        public void TearDown()
        {
            Console.WriteLine("Tearing down NamespaceInitializer .");
        }

        public void Dispose()
        {
            Console.WriteLine("Disposing NamespaceInitializer .");
        }
    }

    [TestFixture]
    public class SimpleTests
    {
        [Test]
        public void Test()
        {
            Console.WriteLine("Executing test.");
        }
    }
}

Constructing TestSuiteInitializer.
Setting up TestSuiteInitializer.
Constructing NamespaceInitializer .
Setting up NamespaceInitializer .
Executing test.
Tearing down NamespaceInitializer .
Disposing NamespaceInitializer .
Tearing down TestSuiteInitializer.
Disposing TestSuiteInitializer.

using System;
using NUnit.Framework;

[SetUpFixture]
public class DatabaseInitializer
{
    public DatabaseInitializer()
    {
        Console.WriteLine("Dropping Database.");
        Console.WriteLine("Rebuild Database with latest Schema.");
        Console.WriteLine("Load basic test environment data.");
    }
}
namespace SetUpFixtureSample.DatabaseTests
{
    [SetUpFixture]
    public class DatabaseWithBasicAccountsInitializer: IDisposable
    {
        public DatabaseWithBasicAccountsInitializer()
        {
            Console.WriteLine("Loading basic account data.");
            Console.WriteLine("Opening outer Transaction scope");
        }

        public void Dispose()
        {
            Console.WriteLine("Roll back transaction scope.");
        }
    }

    [TestFixture]
    public class BasicAccountTester1
    {
        [Test]
        public void Test()
        {
            Console.WriteLine("Executing test 1.");
        }
    }

    [TestFixture]
    public class BasicAccountTester2
    {
        [Test]
        public void Test()
        {
            Console.WriteLine("Executing test 2.");
        }
    }
}

using NUnit.Framework;

namespace SetUpFixtureSample
{
    [TestFixture]
    [Explicit]
    [Category("DataLoader")]
    public class DataLoaderFixture
    {
        [Test]
        public void LoadData()
        {
            //Clean the database
            //Load test data into the database
        }
    }
}

The Explicit attribute means that the test fixture is not run if you execute all the tests in the assembly by default. Typically a test like this would be run by invoking the nunit console runner from an automated build to run only tests in the “DataLoader” category.

That’s not bad, but again – what happens on developer system when you forget to run the dataloader tests first? Or, what do you do when you’ve got different sets of tests that need different default data? You definitely don’t want each individual test or fixture going through an entire, scorched-earth style recreation of the entire database just to get to a clean state, do you?

Fortunately, NUnit provides a not-well-known, but powerful, soltuion for executing code before and after any tests run – the SetUpFixture. It doesn’t have the greatest name in the world, as it’s easily confused with TestFixtureSetup. The difference is that SetUpFixture runs around all the fixtures within the same namespace. SetUpFixture classes defined outside of a namespace will run before any tests in the entire assembly. Here’s a sample:

using System;
using NUnit.Framework;

[SetUpFixture]
public class TestSuiteInitializer : IDisposable
{
    public TestSuiteInitializer()
    {
        Console.WriteLine("Constructing TestSuiteInitializer.");
    }

    [SetUp]
    public void SetUp()
    {
        Console.WriteLine("Setting up TestSuiteInitializer.");
    }

    [TearDown]
    public void TearDown()
    {
        Console.WriteLine("Tearing down TestSuiteInitializer.");
    }
    
    public void Dispose()
    {
        Console.WriteLine("Disposing TestSuiteInitializer.");
    }
}
namespace SetUpFixtureSample
{
    [SetUpFixture]
    public class NamespaceInitializer : IDisposable
    {
        public NamespaceInitializer()
        {
            Console.WriteLine("Constructing NamespaceInitializer .");
        }

        [SetUp]
        public void SetUp()
        {
            Console.WriteLine("Setting up NamespaceInitializer .");
        }

        [TearDown]
        public void TearDown()
        {
            Console.WriteLine("Tearing down NamespaceInitializer .");
        }

        public void Dispose()
        {
            Console.WriteLine("Disposing NamespaceInitializer .");
        }
    }

    [TestFixture]
    public class SimpleTests
    {
        [Test]
        public void Test()
        {
            Console.WriteLine("Executing test.");
        }
    }
}

Which produces this output:

Constructing TestSuiteInitializer.
Setting up TestSuiteInitializer.
Constructing NamespaceInitializer .
Setting up NamespaceInitializer .
Executing test.
Tearing down NamespaceInitializer .
Disposing NamespaceInitializer .
Tearing down TestSuiteInitializer.
Disposing TestSuiteInitializer.

As you can see, the SetUpFixture classes exhibit the same nesting that we’ve come to expect from constructors, Dispose methods, SetUp, and TearDown atributes. As with basic test fixtures, I recommend that you use the construct/dispose pattern for SetUpFixture classes, and avoid the SetUp and TearDown-attributed methods. There’s no need for the extra layer of initialization.

Also note that SetUpFixture class construction and disposal is nested by namespace. If you have two of them in the same namespace though, order of execution between the two could be non-deterministic.

One of the great things about these SetUp fixture classes is that they run whenever tests in the same namespace are executed, regardless of how many tests within that namespace you happen to be running. The important thing is that your context of execution is always going to be firmly established, even if you’re only running one test. Andy, by establishing the right part of your entire testing context at the right leve, you can dramatically decrease the test time for expensive tests. For example, you could rebuild the whole test database in a SetUpFixture defined at the assembly level, and then establish and remove different sets of test data for related tests, organized by namespaces. This might look like this:

Dropping Database.
Rebuild Database with latest Schema.
Load basic test environment data.
Loading basic account data.
Opening outer Transaction scope
Executing test 1.
Executing test 2.
Roll back transaction scope.

using System;
using NUnit.Framework;

[SetUpFixture]
public class TestSuiteInitializer : IDisposable
{
    public TestSuiteInitializer()
    {
        Console.WriteLine("Constructing TestSuiteInitializer.");
    }

    [SetUp]
    public void SetUp()
    {
        Console.WriteLine("Setting up TestSuiteInitializer.");
    }

    [TearDown]
    public void TearDown()
    {
        Console.WriteLine("Tearing down TestSuiteInitializer.");
    }
    
    public void Dispose()
    {
        Console.WriteLine("Disposing TestSuiteInitializer.");
    }
}
namespace SetUpFixtureSample
{
    [SetUpFixture]
    public class NamespaceInitializer : IDisposable
    {
        public NamespaceInitializer()
        {
            Console.WriteLine("Constructing NamespaceInitializer .");
        }

        [SetUp]
        public void SetUp()
        {
            Console.WriteLine("Setting up NamespaceInitializer .");
        }

        [TearDown]
        public void TearDown()
        {
            Console.WriteLine("Tearing down NamespaceInitializer .");
        }

        public void Dispose()
        {
            Console.WriteLine("Disposing NamespaceInitializer .");
        }
    }

    [TestFixture]
    public class SimpleTests
    {
        [Test]
        public void Test()
        {
            Console.WriteLine("Executing test.");
        }
    }
}

Constructing TestSuiteInitializer.
Setting up TestSuiteInitializer.
Constructing NamespaceInitializer .
Setting up NamespaceInitializer .
Executing test.
Tearing down NamespaceInitializer .
Disposing NamespaceInitializer .
Tearing down TestSuiteInitializer.
Disposing TestSuiteInitializer.

using System;
using NUnit.Framework;

[SetUpFixture]
public class DatabaseInitializer
{
    public DatabaseInitializer()
    {
        Console.WriteLine("Dropping Database.");
        Console.WriteLine("Rebuild Database with latest Schema.");
        Console.WriteLine("Load basic test environment data.");
    }
}
namespace SetUpFixtureSample.DatabaseTests
{
    [SetUpFixture]
    public class DatabaseWithBasicAccountsInitializer: IDisposable
    {
        public DatabaseWithBasicAccountsInitializer()
        {
            Console.WriteLine("Loading basic account data.");
            Console.WriteLine("Opening outer Transaction scope");
        }

        public void Dispose()
        {
            Console.WriteLine("Roll back transaction scope.");
        }
    }

    [TestFixture]
    public class BasicAccountTester1
    {
        [Test]
        public void Test()
        {
            Console.WriteLine("Executing test 1.");
        }
    }

    [TestFixture]
    public class BasicAccountTester2
    {
        [Test]
        public void Test()
        {
            Console.WriteLine("Executing test 2.");
        }
    }
}

using NUnit.Framework;

namespace SetUpFixtureSample
{
    [TestFixture]
    [Explicit]
    [Category("DataLoader")]
    public class DataLoaderFixture
    {
        [Test]
        public void LoadData()
        {
            //Clean the database
            //Load test data into the database
        }
    }
}

Which of course produces the following output:

Dropping Database.
Rebuild Database with latest Schema.
Load basic test environment data.
Loading basic account data.
Opening outer Transaction scope
Executing test 1.
Executing test 2.
Roll back transaction scope.

The outermost SetUpFixture ensures that you always start from a known state, by dropping and rebuilding the test database. We open up a transaction within the namespace SetUpFixture’s constructor, and roll it back in the corresponding Dispose method. This technique can help ensure that your integration tests don’t develop unintended order dependencies by relying on data injected by other, previously run tests. Once all the tests in that namespace have run, all the database changes get rolled back. Now the next set of tests can start from a known data state again.

SetUpFixtures, by virtue of being run outside of, and around, all of your unit test fixtures, give you a lot of options in structuring the runtime context in which your unit tests execute. They can be a very powerful tool in your testing toolbelt. Hopefully this introduction to NUnit SetUpFixtures gives you some ideas on how you can re-structure and streamline your own test management – especially for those more challenging scenarios, like UI and integration tests.

1. You have lots of the same kind of tests. Maybe you’re writing integration tests for your persistence layer. Your tests are going to all be performing some similar operations, like creating a database connection. Maybe they all need to reference an Inversion of Control container like StructureMap or Autofac in order to retrieve repository instances. You don’t want to write that boilerplate code for every fixture, do you?
2. You have a hierarchy of classes that need testing. If you have a base class in your domain, you will probably want to set up a parallel type hierarchy of test fixtures. In this case, a base test fixture gives you one place to define tests that verify the base class invariants to which all your derived classes need to conform.

The Code

using System;
using NUnit.Framework;

namespace NUnitLifecycle
{
    public abstract class TestBase : IDisposable
    {
        protected TestBase()
        {
            Console.WriteLine("Base Constructor");
        }

        [TestFixtureSetUp]
        public void BaseTestFixtureSetUp()
        {
            Console.WriteLine("Base TestFixtureSetup");
        }

        [TestFixtureTearDown]
        public void BaseTestFixtureTearDown()
        {
            Console.WriteLine("Base TestFixtureTearDown");
        }

        [SetUp]
        public void BaseTestSetUp()
        {
            Console.WriteLine("Base SetUp");
        }

        [TearDown]
        public void BaseTestTearDown()
        {
            Console.WriteLine("Base TearDown");
        }

        [Test]
        public void BaseFirstTest()
        {
            Console.WriteLine("Base First Test passes!");
            Assert.Pass();
        }

        [Test]
        public void BaseSecondTest()
        {
            Console.WriteLine("Base Second Test fails!");
            Assert.Fail();
        }

        public virtual void Dispose()
        {
            Console.WriteLine("Base Dispose");
        }
    }

    [TestFixture]
    public class TestClass : TestBase
    {
        public TestClass()
        {
            Console.WriteLine("Derived Constructor");
        }

        [TestFixtureSetUp]
        public void TestClassSetUp()
        {
            Console.WriteLine("Derived TestFixtureSetup");
        }

        [TestFixtureTearDown]
        public void TestClassTearDown()
        {
            Console.WriteLine("Derived TestFixtureTearDown");
        }

        [SetUp]
        public void TestSetUp()
        {
            Console.WriteLine("Derived SetUp");
        }

        [TearDown]
        public void TestTearDown()
        {
            Console.WriteLine("Derived TearDown");
        }

        [Test]
        public void FirstTest()
        {
            Console.WriteLine("Derived First Test passes!");
            Assert.Pass();
        }

        [Test]
        public void SecondTest()
        {
            Console.WriteLine("Derived Second Test fails!");
            Assert.Fail();
        }

        public override void Dispose()
        {
            Console.WriteLine("Derived Dispose");
            base.Dispose();
        }
    }
}

Notice that the base class isn’t attributed with the [TestFixture] attribute. I only want this class to act as a base for my actual fixtures. I don’t intend for it to be run as a fixture in its own right.

Note too that I’m reinforcing the fact that this class doesn’t stand alone by making the base class abstract, and following standard practices for an abstract base class by making the constructor protected instead of public. We’ve also made the IDisposable.Dispose method implementation virtual so we can extend its behavior in derived classes.

Test code is still code. Follow the same design practices here that you would in your application code.

The Results
Now when we run our tests, we get the following output:

• Base Constructor
• Derived Constructor
• Base TestFixtureSetup
• Derived TestFixtureSetup
• Base SetUp
• Derived SetUp
• Derived First Test passes!
• Derived TearDown
• Base TearDown
• Base SetUp
• Derived SetUp
• Derived Second Test fails!
• Derived TearDown
• Base TearDown
• Base SetUp
• Derived SetUp
• Base First Test passes!
• Derived TearDown
• Base TearDown
• Base SetUp
• Derived SetUp
• Base Second Test fails!
• Derived TearDown
• Base TearDown
• Derived TestFixtureTearDown
• Base TestFixtureTearDown
• Derived Dispose
• Base Dispose

Observations
Note that even with a base class, all our observations from last time are still true:

1. The Test Fixture class is constructed once.
2. TestFixtureSetup and TestFixtureTeardDown run one time.
3. TestSetup and TestTearDown run around every test.
4. TestTearDown runs even when a test fails.
5. NUnit knows about IDisposable.

You can quickly see from the results above that our first guideline, Use constructors for class-wide setup. Use IDisposable for class-wide cleanup. Avoid TestFixtureSetup and TestFixtureTearDown, is even more relevant once we start creating a hierarchy of test fixtures.  Constructors and IDisposable keep our test fixtures working like any other class we’d write.

Next time, we’re going to see how to run our own initialization code before any of our test fixtures are even created. 

I’m with you. Personally, I’ve never sat around a meeting table with 15 other developers, all checking their email, while one poor sacrificial lamb walked us through a line-by-line explanation of code in “Representative changeset XYZ” the day after we put it into production. I’ve never been told (or said!) “That’s good feedback, but we don’t have time to re-test those changes before the release date.” And I’ve never, ever had an hours-long debate about where our braces should go.

Really.

But, for the benefit of that other team, I’ve put together some advice about  how to conduct effective code reviews. Some of the things I cover are:

• Understanding why we do code reviews in the first place
• Using code reviews to drive quality in your products early
• Getting a consistent code style across your team so you can focus on higher-level issues
• And more.

Of course, about the same time we’d gotten our homegrown framework running and built a couple thousand tests with it, NUnit 2.0 came out with a brand-new and shiny attribute-based metadata approach. We spent the next 3 years wishing we had the time to port all our tests to NUnit, and stealing features from it for our custom test runner.

One of the things I really envied about NUnit was its test method lifecycle. We support setup and teardown methods, but we never went beyond that. NUnit, by comparison has a rich set of events that occur for each test that gets run. Whenever I teach our Agile Boot Camp class, I always make sure to walk students through the finer points of a test fixture’s life, so they’ll know the best way to organize their test code. When you’re first writing unit tests, knowing whether to put something in the test method, the setup, the fixture setup, or somewhere else, just isn’t all that obvious. Oftentimes, I see people putting expensive initialization code into every single test method, which causes their tests to run slower than they need to.

Note: I’m using NUnit for these examples. Other frameworks have different semantics, and I recommend you write some similar code for whatever library you use. For me, NUnit’s always been the draft horse of .NET test frameworks – stable, steady, gets along with everybody, and does what I expect.

If you’re not already familiar with NUnit, take a look at their quick-start guide and then come back. I’ll wait.

Now, take a look at this simple test fixture class:

using System;
using NUnit.Framework;

namespace NUnitLifecycle
{
    [TestFixture]
    public class TestClass : IDisposable
    {
        public TestClass()
        {
            Console.WriteLine("Constructor");
        }

        [TestFixtureSetUp]
        public void TestClassSetUp()
        {
            Console.WriteLine("TestFixtureSetup");
        }

        [TestFixtureTearDown]
        public void TestClassTearDown()
        {
            Console.WriteLine("TestFixtureTearDown");
        }

        [SetUp]
        public void TestSetUp()
        {
            Console.WriteLine("SetUp");
        }

        [TearDown]
        public void TestTearDown()
        {
            Console.WriteLine("TearDown");
        }

        [Test]
        public void FirstTest()
        {
            Console.WriteLine("First Test passes!");
            Assert.Pass();
        }

        [Test]
        public void SecondTest()
        {
            Console.WriteLine("Second Test fails!");
            Assert.Fail();
        }

        public void Dispose()
        {
            Console.WriteLine("Dispose");
        }
    }
}

This fixture has two test methods, one which passes and one which fails. When we run the test framework in NUnit’s test runner, we get this output:

Constructor
TestFixtureSetup
SetUp
First Test passes!
TearDown
SetUp
Second Test fails!
TearDown
TestFixtureTearDown
Dispose

So, what does this tell us about the lifecycle of a test within a fixture?

1. The Test Fixture class is constructed once.  NUnit invokes all the test methods on the same fixture instance.
2. TestFixtureSetup and TestFixtureTeardDown run one time. TestFixtureSetup is run immediately after construction, and only runs once for the entire test run. TestFixtureTearDown is called immediately after all test methods have been run.
3. TestSetup and TestTearDown run around every test. These two methods execute immediately before, and immediately after, each test method.
4. TestTearDown runs even when a test fails. Noice that Even when the second test exits with a failed assertion, the TearDown method is still called for that method.
5. NUnit knows about IDisposable. Did you notice that our class implemented IDisposable? IDisposable is .NET’s canonical interface for when you have a resource whose scope needs to be managed. The IDisposable.Dispose method is the last method called on our fixture class.

//