Today a colleague was trying to diagnose why some long neglected UI tests were failing.  He quickly discovered that the tests weren’t failing because of changes to the code under test but because our UI test framework was misbehaving.  Since I’d been involved in developing parts of the framework he came to me to see what I remembered.

Our test framework is essentially a wrapper around WatiN that controls certain application specific tasks like logging in, navigating with our windowing system, or interacting with some of our more complex controls.  These tests were failing because the framework wasn’t able to get a reference to our main application window so a null reference exception was being thrown.  Now, we’ve made quite a few changes to our application but our windowing system certainly isn’t among them so why were the tests failing?

We re-ran the test using the script that invokes the NUnit console runner (2.5.3) after our build to confirm the failure.  Then we fired up the NUnit GUI test runner, loaded the assembly, found one of the failing tests, ran it, and watched it…succeed?  Yes, that’s right, it worked!  How could the same test code fail under the console runner but succeed under the GUI runner?

One of the first things we noticed was that when running under the GUI the browser (IE) opened but it didn’t open when running under the console.  I remembered that WatiN has a setting to control whether the browser is displayed but I couldn’t remember if we ever used it.  Some searching refreshed my memory about it and sure enough, we weren’t using it at all.  We pulled on a few other loose threads but to no avail.  Eventually we decided it was time to hook up the debugger.

Our script really doesn’t give us time to connect the debugger to the test runner, particularly since the tests were failing immediately so we added a call to Thread.Sleep() to give ourselves a small window of opportunity to get in.   Once connected it didn’t take long to find that the real error was masked by the framework but coming from WatiN itself:

The CurrentThread needs to have it’s ApartmentState set to ApartmentState.STA to be able to automate Internet Explorer.

Now we’re getting somewhere.  Of course, we’ve known this as long as we’ve been using WatiN.  That’s why we put the standard NUnit TestRunner section in the assembly’s configuration file when we started the project.  To confirm that the configuration file was being read we checked a few values and sure enough, they were being read correctly.

So, if the configuration file was being read, contained the correct information regarding apartment state, and the tests worked under the GUI runner, what could we do?  STAThreadAttribute is out of the question because we’re in a test rather than a standalone application.  The WatiN documentation also addresses setting the thread’s apartment state programmatically and basically says “don’t bother, it won’t work.”  Is all hope lost?

We spent some more time searching and found that NUnit 2.5 introduced a new attribute that looked promising.  RequiresSTAAttribute is used to indicate that tests should be run as single-threaded apartment.   A nice feature of this attribute is that it can be applied to assemblies, classes, and methods so we have better control over what is single-threaded.  As an aside, the documentation also states that STAThreadAttribute is treated as an alias so either can be used.

We added the attribute to our test fixture class and ran the script.  After a few seconds a new instance of IE opened and the test was underway, working as expected.  Has anyone else experienced this?  If so, how did you resolve it?