ReSharper plugins for unit testing – MSpec, xUnit.net and Silverlight

ReSharper provides comprehensive support for running and debugging unit tests. Out of the box, it works with NUnit and Microsoft’s Visual Studio Unit Testing Framework (henceforth mstest for brevity) for .Net languages, and supports QUnit and Jasmine for JavaScript.

The same APIs that ReSharper uses to implement this support are fully available to third parties, and there are currently plugins to expand this support to include Machine.Specifications (MSpec), xUnit.net and support for Silverlight projects. More can be added at any time, through the use of the SDK.

The MSpec project is a “Context/Specification” testing framework, which is a form of BDD tool, focussed more on driving correct behaviour than on testing functionality – building the right thing, rather than building the thing right. It’s a significant change of style to anyone used to the NUnit/mstest style of attributed methods, but it’s not too difficult to follow – test methods are replaced with fields that take lambdas that arrange, act and assert. It’s worth checking out the example projects in the source repository – BankingSpecs is a good place to start.

Machine.Specifications ships as a NuGet package, which also includes the ReSharper runner. Simply run the InstallReSharperRunner batch file in your packagesMachine.Specificationstools folder (it includes support for ReSharper 6 and 7). Once installed, ReSharper now recognises MSpec tests in your source code, adding the unit test icons to the gutter in the editor:

mspec test recognised in the editor

One of MSpec’s nice features it that the names of the test and the information from the SubjectAttribute are used to generate a natural language description, which, when used with the different grouping options in the test runner can give very readable test results:

mspec results with natural language descriptions grouped by category

The xUnit.net framework is a more traditional unit testing framework, similar to NUnit and mstest, but arguably more idiomatic .Net. For example, it still uses attributes to mark methods as tests, but uses constructors and IDisposable instead of SetUp and TearDown attributes. Support is added by the xunitcontrib plugin, giving ReSharper the ability to locate and run xUnit.net tests. Like NUnit and mstest, it can handle parameterised row tests, dynamically adding rows into the results as the tests are run.

xUnit.net parameterised tests, using random data

It also works with the “Show Unit Test Usages” option in the filter drop down of the Find Results window. When viewing the results of a find usage search, any usage in a test method or test class is marked with a test icon, rather than an icon representing a method, class or property. Unchecking the “Show Unit Test Usages” option in the “Filter Usages” drop down will hide the test usages in the results, allowing you to focus on the usages in the production codebase.

Filtering xUnit.net test usages out of Find Usages results

Unit test providers don’t need to be source based. Most of them are, and display the test icon in the editor gutter, but they also examine the metadata of the compiled assembly, and can potentially find more tests here. Using this technique, it’s possible to run tests created in a source file that ReSharper doesn’t support. For example, we can create an xUnit.net test in F#, and while it’s not detected in the editor, it is still found and run when the assembly is compiled.

xUnit.net test in F#

Since the test hasn’t come from source code, you can’t double click the test in the test results to navigate to it. But a failing test will display a clickable stack trace that will navigate to the failure.

F# test results showing clickable call stack

This technique can be used to run F# tests using other F# test frameworks, as long as they are based on a unit test framework that ReSharper supports (and this technique can also be used to write a plugin that supports a framework that isn’t based on one of these frameworks).

Silverlight support is provided by AgUnit, which enables running and debugging your unit tests in Silverlight projects, and of course handles the Silverlight test framework’s AsynchronousAttribute.

AgUnit debugging an asynchronous Silverlight test

Perhaps surprisingly, a couple of our own products are also plugins for ReSharper’s unit test runner – dotTrace and dotCover. Although the products register themselves with ReSharper somewhat differently to normal plugins, these products use the same open APIs that the third party plugins use, albeit for a slightly different purpose. Instead of examining source code and metadata to find and run tests, they provide the environment in which tests are run; dotTrace collects profiling information, and dotCover collects coverage information. Both products extend the UI to provide means of running the tests, but dotCover also adds an output pane to the test runner to display the coverage information (and highlights covered and uncovered code in the editor), and of course, this works with third party test providers.

Unit test runner showing dotCover custom code coverage results pane

And, just to bring the extensibility full circle, dotCover includes ReSharper’s unit test runner, which is used if ReSharper isn’t installed. Again, NUnit and mstest support is provided out of the box, and there are separate dotCover plugins for xUnit.net and MSpec support (the dotCover MSpec runner is also distributed in the NuGet package, just use the InstallDotCoverRunner batch file in the tools folder).

ReSharper provides comprehensive unit testing support, in an environment that is very flexible. There is a lot of great support, provided in the box, and just a download away. And it’s all open and extensible – if you don’t see something you want, grab the SDK and get coding!

This entry was posted in ReSharper Tips&Tricks and tagged , , , , , . Bookmark the permalink.

5 Responses to ReSharper plugins for unit testing – MSpec, xUnit.net and Silverlight

  1. AllSpark says:

    I still couldn’t get the MSpec to work with Resharper 7. I see its installed in plugins but it doesn’t show up in unit tests. I installed using MySolutionFolder\packages\Machine.Specifications.0.7.0\tools\InstallResharperRunner.7.1.bat

    Please help thanks in advance

    • Hadi Hariri says:

      That normally indicates that you have a mismatch between versions. Can you check that please?

    • Dahdread says:

      You need to update the ReSharper plugin (Machine.Specifications Runner for ReSharper 7.1). If you get MSpec via nuget you can just close VS and run the bat file in the mspec directory for your version of resharper.

  2. Tom Cooley says:

    I got the latest plugin (0.9.0-Unstable0032) and now I get a MissingMethodException in Machine.Specifications.Runner.Impl.DefaultRunner whenever I try to run a unit test. At least it’s true to its name of being unstable, but I’d really like to be able run my tests in here.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">