Going Underground: Microsoft Moles

3/27/2012 Update: According to the Moles page, the Moles framework has been integrated into Visual Studio 11 as the Fakes framework and Moles is no longer under active development.   After a quick review of the changes it appears that most of this guide still applies but there are a few changes to be aware of:

  • Mole types are now referred to as Shim types
  • The configuration file now has a .fakes extension
  • The generated types are now placed in a .Fakes namespace (i.e.: System.Fakes)

6/25/2012 Update: In preparation for my Faking It talk covering the Fakes framework I compiled a list of the notable differences between Moles and Fakes. There are quite a few more than I listed above so if you’re using this post as an introduction to either framework you’ll probably want to look them over.

Despite having been around for several years I hadn’t heard about Microsoft’s Moles framework until a few months ago when one of my teammates mentioned it during the 2011 Indy GiveCamp. I was interested in learning more about it at the time but given how we were running on caffeine I’m not surprised I forgot about it until he mentioned it again a few weeks ago. This time I was much more alert and started reading about it almost immediately. After seeing what Moles offered and not finding much in the way of resources for it on the Web I knew I needed to spread the word.

Note: For this article I used version 0.94 of the Moles Framework.

What is the Moles Framework?

Microsoft Moles is a project from Microsoft Research that came about because of work being done on Pex (another MS Research initiative). The Microsoft Moles Reference Manual describes Moles this way:

“The Microsoft Moles 2010 add-in for Microsoft Visual Studio 2010 is a lightweight framework for creating delegate-based test stubs and detours in .NET Framework applications.”

In other words, Moles is, in a sense, Microsoft’s answer to other mocking frameworks like Rhino Mocks or Moq but Moles differs from the other frameworks in a few ways.  One important distinction is that the Moles Framework does not provide any validation mechanisms of its own.  In that regard Moles is not so much a testing framework as it is an isolation framework intended for use in conjunction with other testing frameworks such as MSTest or NUnit.

Note: Although Moles started as part of Pex I’m not going to discuss Pex here.

Isolation Techniques

The Moles framework provides two isolation techniques:

  • Stub Types
  • Mole Types

Both types allow us to work around dependencies in different ways and have very different use cases.

Stub Types

Stub types are much like traditional stub or mock objects in that they allow us to test code that relies on some provided contract.  Stubs are based on inheritance so they can only be used with interfaces or non-sealed classes with virtual members.  Sometimes the code we want to test relies on static or non-overridable methods that can’t be stubbed.  In those scenarios we need to use mole types instead.

Mole Types

Mole types use a profiler to rewrite existing method bodies thereby allowing us to circumvent (or “detour” in Moles parlance) dependencies on static or non-overridable methods.  Mole types are generally most useful for isolating dependencies in third-party libraries.  Unfortunately, runtime rewriting does introduce some performance issues so moles should generally be used sparingly.

Note: At the time of this writing the Moles framework does not generate detour properties for auto-implemented properties although this could change in a future release (http://social.msdn.microsoft.com/Forums/en-IE/pex/thread/e5ce7b81-6b82-4e7c-bc99-2e7d81d9497d).

Choosing an Approach

With both stub types and mole types at our disposal it is important to understand when to use each.  In general Microsoft recommends that we favor stubs over moles whenever possible.  Stub types are much lighter than moles and operate over native language features.  Some instances where mole types are required are:

  • Private methods
  • Static methods
  • Sealed types
  • Static constructors/finalizers
  • 3rd party libraries without testable APIs

Getting Started

Installation

The Moles framework is available through the extension gallery as part of Pex (White box Unit Testing for .NET) or individually.  It can also be downloaded directly from the Microsoft Research site.  Regardless of which option you choose the functionality is made available to Visual Studio as an add-in.

Generating Stubs and Moles

As previously discussed, the Moles framework automatically generates the appropriate stub and mole types.  In order for it to generate the types though we need to first tell it which assemblies to inspect via a .moles file.

The quickest way to get started is to expand the References node in the test project, select the assembly being tested, and choose “Add Moles Assembly” from the context menu.  This will add a simple .moles file and automatically hook up the build action for the file.  When the project is built (or the .moles file is saved) Moles will generate an assembly with the appropriate isolation types and automatically reference it along with any other necessary assemblies.

Note: The Microsoft Moles Reference Manual describes adding the .moles file through the Add New Item dialog.  I couldn’t find these templates but did find the Add Moles Assembly context menu item on individual references. It’s also possible to manually add a .moles file and set the build action to Moles to achieve the same behavior.

Should it be necessary to generate isolation types for the types defined in mscorlib (i.e.: DateTime) we can add a moles assembly for it by right-clicking on the References folder in the test project and selecting the “Add Moles Assembly for mscorlib” option.

Basic Configuration

The .moles file is the heart of the Moles Framework configuration.  In its simplest form the .moles file merely identifies an assembly that will be inspected for type generation.

<Moles xmlns="http://schemas.microsoft.com/moles/2010/">
	<Assembly Name="Examples" />
</Moles>

In this example we’ve instructed the system to generate the isolation types for all types defined in the Examples assembly.

In addition to identifying the assembly the .moles file can also include additional configuration information for filtering which types will be stubbed and/or moled and strong name signing.  We’ll take a closer look at these topics later on in the Advanced Configuration section.

Naming Conventions

Once the types have been generated they can be referenced in your tests.  The Moles framework follows some basic naming conventions to make the types easier to find and identify in your code.  The full list of conventions is available in Appendix B of the Microsoft Moles Reference Manual so I’ll only note a few highlights here.

First, both of the isolation type implementations derive their names from their underlying types but use prefixes to distinguish them.  A prefix of “S” denotes a stub type whereas a prefix of “M” denotes a mole type.  For example, if you have a class named MyClass its corresponding stub will be named SMyClass and the mole type will be MMyClass.

Each of the isolation types are automatically placed in a .Moles sub-namespace under the namespace where the original types are located.  This means that if you have a User class in the Examples.Model namespace then the isolation types would be located in Examples.Model.Moles.

The exposed detour fields or properties on each isolation type also follow some basic naming conventions depending on what the underlying member is:

Convention Example(s)
Method Method name  with parameter type names appended LoadInt32
Property Property name with Get or Set FirstNameGetLastNameSet
Constructor “Constructor” with parameter type names appended ConstructorInt32

Delegation

One thing that stub and mole types have in common is that the framework generates both types for a given assembly.  By generating these types for us the framework saves us a great deal of time and allows us to focus on the problem at hand but if they are all generated how can we change their behavior?  The generator can’t possibly know how we intend to use them, can it?  Of course it can’t, but what it can do is expose fields that correspond to the various members and allow us to specify the alternate behavior with delegation.

Delegation is central to working with the Moles framework.  When the framework generates the stub and mole types it exposes the customizable pieces as properties.  One important thing to note about the delegate types Moles uses is that they’re not the standard Action<> or Func<> types that are provided by the .NET framework.  Instead, the Moles framework provides the static MolesDelegates class that exposes numerous generic types that represent functionality similar to the standard Action<> and Func<> types but expands upon them with additional overloads.  There are also some variations that accept some out or ref arguments as well.

Using Stub Types

As previously discussed, stub types should be used when we need to mock interfaces or virtual members of non-sealed classes.  Because stubs rely on inheritance they can be implicitly substituted for the stubbed type.

Methods

Methods can be stubbed by assigning a delegate with a matching signature to the corresponding field.  For example, if we have an interface:

public interface IMyInterface
{
	int DoSomething(int value1, int value2);
}

…and we want to stub the DoSomething method to return the sum of its two parameters we can easily set it up using a type initializer like so:

var stub = new SIMyInterface()
{
	DoSomethingInt32Int32 = (i1, i2) => i1 + i2
};

Generic methods are treated a bit differently than standard methods.  Rather than creating a delegate field for generic methods Moles generates an overload of the method that accepts a delegate matches that of the generic method.  To stub the generic method we pass a delegate for each type we want to stub to the overload and the stub class maintains an internal dictionary to track which generic instances are stubbed.

Properties

Because properties are really just syntactic sugar for accessor methods, stubbing properties is similar to stubbing methods.  Stub classes expose separate delegate fields for property getters and setters.  For instance, given a property on an interface:

public interface IMyInterface
{
	string MyProperty { get; set; }
}

…we can provide delegates for each accessor and simulate the property behavior in our test:

var value = default(string);

var stub = new SIMyInterface()
{
	MyPropertyGet = () => value,
	MyPropertySetString = v => value = v
};

Alternatively, Moles also generates a method to simplify this behavior:

var stub = new SIMyInterface();
stub.AttachBackingFieldToMyProperty();

Note: Each property with public get and set accessors will have a corresponding attach method.

Events

Stubs provide an easy way to test events by exposing a delegate field that can be invoked to raise the event.  This can be useful for ensuring that the correct handlers are hooked up correctly.  Stubbing an event is easiest when working with an interface since the event’s backing field will be exposed automatically but it’s also possible to stub an event on an existing class by making the event virtual.

To illustrate how an event is exposed through a stub consider the following:

public interface IMyInterface
{
	event EventHandler MyEvent;
}

When the Moles Framework builds the stub types it will include an EventHandler delegate field called MyEventEvent.

Partial Stubs

By default, when a stubbed member is invoked but doesn’t have an attached delegate a StubNotImplementedException is thrown.  This behavior makes sense when working with interfaces but what if we’re working with a class and want to use the base implementation rather than a stub?  The Moles framework lets us achieve this with partial stubs.  Activating partial stubs is simply a matter of setting the stub type’s CallBase property to true.

Note: The CallBase property is only generated for classes.

Behaviors

In the last section we saw that the default behavior for the Moles Framework is to throw a StubNotImplementedException when a stubbed member is invoked but doesn’t have an attached delegate.  This behavior is controlled through the aptly named IBehavior interface and can be set either globally or per stub type instance.

The Moles Framework includes a few built-in behaviors that are accessible through the static BehavedBehaviors class.  Of note are:

  • BehavedBehaviors.NotImplemented – The default behavior, throws a StubNotImplementedException
  • BehavedBehaviors.DefaultValue – Returns the default value: default(T), for the return type or does nothing for void

To set the behavior globally we just need to set the BehavedBehaviors.Current property:

BehavedBehaviors.Current = BehavedBehaviors.DefaultValue;

Alternatively, we can set the behavior for an instance by setting the instance’s InstanceBehavior property:

var stub = new SMyClass()
{
	InstanceBehavior = BehavedBehaviors.DefaultValue
};

We are by no means restricted to these behaviors either.  Should we need a custom behavior we just need to define a class that implements the IBehavior interface.

Using Mole Types

Although stub types are useful for interfaces and virtual members they can’t isolate everything and that’s where mole types come in.  As mentioned earlier, mole types allow us to detour virtually any .NET method.  The canonical example illustrating how handy mole types are involves a dependency most of us are all too familiar with: DateTime.Now.

Let’s assume we have some code to check if the end of the world is upon us:

public static class MayanApocalypse
{
	public static bool HasArrived()
	{
		return DateTime.Now >= DateTime.Parse("2012-12-21T00:00:00");
	}
}

Testing this code is a nightmare because DateTime.Now is non-deterministic due to its reliance on the system clock.  The fact that DateTime.Now is a static property doesn’t help matters any either.  Wouldn’t it be nice if we could force DateTime.Now to return some predetermined value?  Mole types let us do just that by injecting a callback into the MSIL body of DateTime.Now‘s get accessor and calling our custom implementation.

MDateTime.NowGet = () => DateTime.Parse("2012-12-21T00:00:00");

Assert.IsTrue(MayanApocalypse.HasArrived());

Accessing the Moled Type

Mole types derive from MoleBase<T>. This is an important distinction from stub types because it means that mole types don’t rely on inheritance. This has implications when we want to work with the moled type directly. Luckily MoleBase<T> provides a way to access the moled type through either the Instance property or an implicit conversion.

Instrumenting Mole Types

Due to how mole types work it’s necessary to run the tests under the Pex profiler.  MSTest handles this nicely through the HostTypeAttribute.  To instruct MSTest to run the tests under the Pex profiler simply decorate your tests:

[TestMethod]
[HostType("Moles")]
public void MyTest()
{
	// ...
}

It’s also possible to run moles under other test frameworks but additional setup work is required.  Refer to the Advanced Configuration section later in this guide for information about using Moles with NUnit.

One other consideration when using mole types is that it might be necessary to add an assembly attribute to allow some types to be instrumented, particularly for some classes in mscorlib.  For instance, moling the DateTime class requires us to add:

[assembly: MoledType(typeof(System.DateTime))] to our test project.

Methods

We already saw an example of moling a static method when we moled DateTime.NowGet. Moling instance methods is very similar but we have a bit more flexibility in that we can either provide delegates for individual instances or all instances of the moled type.

If we want to mole an instance method for all instances we hook it up through the nested AllInstances type. Each mole type has a nested AllInstances type that exposes write-only properties for setting the mole delegate.

Note: The Instance property is not available on abstract or static classes.

MMyClass.AllInstances.DoSomething = () => "Hello world";

Assert.AreEqual("Hello world", new MyClass().DoSomething());

Instance specific moles look much like their stub counterparts and we can set them in the same manner:

var mole1 = new MMyClass()
{
	DoSomething = () => "Hello world"
};

var mole2 = new MMyClass()
{
	DoSomething = () => "Goodbye"
};

Assert.AreEqual("Hello world", mole1.Instance.DoSomething());
Assert.AreEqual("Goodbye", ((MyClass)mole2).DoSomething());

Note: The Moles Framework will also provide properties for private methods provided that each type referenced in the method signature (parameter and return types) is accessible.

Events

Mole types handle events a bit differently than stubs.  Rather than exposing a field for the event Moles generates properties that represent the add and remove accessors.  For example, if you have the following event definition:

public event EventHandler MyEvent;

the mole type for the class will include properties named MyEventAddEventHandler and MyEventRemoveEventHandler.

Moling Members of Base Types

For this part of the discussion let’s assume a simple class hierarchy:

public abstract class MyClass
{
	public string DoSomething()
	{
		return String.Empty;
	}
}

public class MySubClass : MyClass
{
}

This structure poses an interesting problem for moles. The DoSomething method is defined on an abstract class and inherited by MySubClass but there is no DoSomething property generated for the MMySubClass mole so how can we mole DoSomething?

Note: The Moles reference manual recommends that this technique be reserved for constructor moles.

Each mole type actually has two constructors. While we primarily use the default constructor the other constructor lets us pass in another instance. In this case we can mole the DoSomething method by passing an instance of MMySubClass and passing it to the constructor for MMyClass:

var child = new MMySubClass();
var parent = new MMyClass(child)	// Note that child is implicitly converted here
{
	DoSomething = () => "Hello world"
};

Assert.AreEqual("Hello world", child.Instance.DoSomething());

Constructors

The Moles Framework also lets us mole constructors in a manner similar to how we mole static methods. Each constructor is exposed as a static write-only property named Constructor and following the naming conventions for parameter types as described above.

Note: There are no Constructor properties for static classes.

Static Constructors and Finalizers

Static constructors and finalizers can have some unexpected consequences when working with moles so there may be times that it’s necessary to work around them. Although we can’t mole them like we do with other members the Moles Framework provides a few attributes for simply erasing them.

To erase a static constructor use the following assembly attribute:

[assembly: MolesEraseStaticConstructor(typeof(MyClass))]

Similarly, to erase a finalizer use:

[assembly: MolesEraseFinalizer(typeof(MyClass))]

Note: Both of these attributes have overloads that allow specifying the assembly name and class name to identify the type.

Behaviors

Like their stub type counterparts, mole types also use behaviors to control what happens when a member hasn’t been moled. In many ways mole behaviors are just like stub behaviors in that they can be set either globally, per type, or per instance. The Moles Framework also provides several built-in behaviors for mole types. The default behavior even throws MoleNotImplementedException.

Despite their similarities though, mole and stub behaviors have very little in common. First, mole behaviors implement the IMoleBehavior interface. Additionally, the built-in behaviors and global behavior are all accessible through the static MoleBehaviors class.

Some of the notable built-in behaviors are:

  • MoleBehaviors.NotImplemented – The default behavior, throws MoleNotImplementedException
  • MoleBehaviors.DefaultValue – Returns the default value: default(T), for the return type or does nothing for void
  • MoleBehaviors.Fallthrough – Calls into the original (unmoled) implementation

The MoleBehaviors class also exposes some methods to set the default behavior for a given type. For instance, if we want all mole types for MyClass to use the DefaultValue behavior we can use some code like the following:

MoleBehaviors.BehaveAsDefaultValue(typeof(MyClass));

var mc = new MMyClass();
// Unmoled method returns "Hello world"
Assert.AreEqual(null, mc.Instance.DoSomething());

Advanced Configuration

So far we’ve covered the basics of using the Microsoft Moles Framework and largely stayed with the default configuration. Moles configuration provides options to restrict which isolation types are generated, provide a strong name, and control the target platform for the moles assemblies.

Note: The reference manual states that it is possible to disable compilation and add the generated code to the project but I haven’t been able to get that feature to work. If anyone has any insight on this I’d love to see an example of it working.

Controlling Type Generation

Generating the isolation types can be an expensive operation that greatly extends the amount of time needed to build the solution. You can control what is generated in the .moles file by including the StubGeneration and/or MoleGeneration elements within the Moles element. As their names imply, these elements control what will be generated for stubs and moles respectively.

The most basic way to restrict what is generated is to simply set the Disabled attribute to true on either element to prevent the framework from generating anything of that type.

<Moles xmlns="http://schemas.microsoft.com/moles/2010/" Diagnostic="true" DisableCache="true">
	<MoleGeneration Disable="true" />
</Moles>

Controlling isolation type generation doesn’t have to be an all-or-nothing approach though. Both elements also support a nested Types element that allows for more precise filtering based on a simple syntax. The Types element is constructed with the familiar combination of Clear, Add, and Remove elements that are frequently used in standard configuration files.

The Add and Remove elements define a number of common attributes that help refine which types will be included or excluded from the code generation.

Attribute Name Type Description
AbstractClasses Boolean Indicates whether to include or exclude abstract classes
Classes Boolean Indicates whether to include or exclude non-abstract classes
FullName String Identifies the full name of a type
Interfaces Boolean Indicates whether to include or exclude interfaces
Namespace String Identifies a namespace
Obsolete* Boolean Indicates whether to include or exclude types decorated with ObsoleteAttribute
TypeName String Identifies the name of a type

* – Only valid on Remove elements

The value of each of the string typed attribute above are actually filter pattern definitions. By default the patterns are used for case-insensitive partial matching but we can control that behavior with a few tokens.

  • Appending ! to the end of the pattern forces an exact, case-sensitive match
  • Appending * to the end of the pattern will match the beginning of the string

Multiple patterns can be combined using a semi-colon delimited list.

<Assembly Name="mscorlib"/>
	<StubGeneration>
		<Types>
			<Clear />
			<Add Namespace="System!" />
			<Add Namespace="System.IO!"/>
			<Remove TypeName="Handle" />
		<Types>
	</StubGeneration>
</Assembly>

Using the example above (taken from the reference manual), Moles would only generate stub types for the types in the System and System.IO namespaces and excludes anything containing “Handle” in the name.

Strong Name Signing

By default, the generated assembly will be signed with the same key as the moled assembly if it is strongly named. If it’s necessary to use a different key we just need to specify the path to the key file in the Compilation element’s KeyFile attribute in the .moles file.

<Moles xmlns="http://schemas.microsoft.com/moles/2010/">
	<!-- ... -->
	<Compilation KeyFile="key file path" >
</Moles>

Setting the Target Platform

Should it be necessary to force the moles profiler to run as a 32 or 64-bit process we can do so with the MolesAssemblySettingsAttribute found in the Microsoft.Moles.Framework namespace. We just need to include the attribute as an assembly attribute and set Bitness parameter accordingly. For instance, to use a 64-bit process we could use:

[assembly: MolesAssemblySettings(Bitness = MolesBitness.x64)]

Other valid options for Bitness are MolesBitness.x86 and MolesBitness.AnyCPU.

Using with NUnit

Being a project from Microsoft Research it’s no surprise that Moles is easiest to use with MSTest. As we discussed, to get Moles to work with MSTest we just need to decorate the test methods with [HostType("Moles")] and MSTest will set up the instrumentation accordingly.

Of course, not everyone is using MSTest so we need a way to run moled tests with other frameworks. The good news is that yes, Moles can be used with other frameworks. The bad news though is that it takes some extra effort to make it work. Although I’m focusing on using Moles with NUnit the process is similar for other frameworks like xUnit or MbUnit.

NUnit requires the Microsoft.Moles.NUnit assembly to be registered as an add-in. The Moles Framework ships with a copy of this assembly but it is built against a specific version of NUnit (2.5.2.922 in my case) so if you’re using a different version of NUnit (I’m on 2.5.10.11092) you’ll need to recompile the assembly with the correct NUnit assemblies. Once you have a version of the add-in assembly that corresponds to your NUnit version you can copy into the bin\addins folder to register it with NUnit.

Note: The source for the add-in assembly is available in the moles.samples.zip file located in the Documentation folder under the Moles installation folder.

The add-in assembly defines the MoledAttribute class that sets up the mole context in the NUnit runner. You’ll need to reference the assembly in your test project and import the Microsoft.Moles.Framework.NUnit namespace. For convenience I referenced the version of the DLL that I’d placed in my add-ins folder.

You might think that since you’ve registered the add-in and built the projects that you’re ready to run the tests. In a sense, you are but there are a few nuances to be aware of first.

  1. The GUI test runner doesn’t work with Moles
  2. The tests need to be run under the context of the Moles test runner

To run the tests with NUnit’s console runner under the context of the Moles runner you can use the following command:

moles.runner.exe /r:nunit-console.exe /args="/domain=None" [test assembly name]

Wrapping Up

So that’s Microsoft Moles in a nutshell. If you like what you see I encourage you to download it and give it a try. Of course, if you found any errors in what I’ve shown or think I missed something important please let me know in the comments.

Advertisements