ReSharper plugins for unit testing – MSpec, and Silverlight

ReSharper provides comprehensive support for running and debugging unit tests. Out of the box, it works with NUnit and Microsoft’s Visual Studio Unit Testing Framework (henceforth mstest for brevity) for .Net languages, and supports QUnit and Jasmine for JavaScript.

The same APIs that ReSharper uses to implement this support are fully available to third parties, and there are currently plugins to expand this support to include Machine.Specifications (MSpec), and support for Silverlight projects. More can be added at any time, through the use of the SDK.

The MSpec project is a “Context/Specification” testing framework, which is a form of BDD tool, focussed more on driving correct behaviour than on testing functionality – building the right thing, rather than building the thing right. It’s a significant change of style to anyone used to the NUnit/mstest style of attributed methods, but it’s not too difficult to follow – test methods are replaced with fields that take lambdas that arrange, act and assert. It’s worth checking out the example projects in the source repository – BankingSpecs is a good place to start.

Machine.Specifications ships as a NuGet package, which also includes the ReSharper runner. Simply run the InstallReSharperRunner batch file in your packagesMachine.Specificationstools folder (it includes support for ReSharper 6 and 7). Once installed, ReSharper now recognises MSpec tests in your source code, adding the unit test icons to the gutter in the editor:

mspec test recognised in the editor

One of MSpec’s nice features it that the names of the test and the information from the SubjectAttribute are used to generate a natural language description, which, when used with the different grouping options in the test runner can give very readable test results:

mspec results with natural language descriptions grouped by category

The framework is a more traditional unit testing framework, similar to NUnit and mstest, but arguably more idiomatic .Net. For example, it still uses attributes to mark methods as tests, but uses constructors and IDisposable instead of SetUp and TearDown attributes. Support is added by the xunitcontrib plugin, giving ReSharper the ability to locate and run tests. Like NUnit and mstest, it can handle parameterised row tests, dynamically adding rows into the results as the tests are run. parameterised tests, using random data

It also works with the “Show Unit Test Usages” option in the filter drop down of the Find Results window. When viewing the results of a find usage search, any usage in a test method or test class is marked with a test icon, rather than an icon representing a method, class or property. Unchecking the “Show Unit Test Usages” option in the “Filter Usages” drop down will hide the test usages in the results, allowing you to focus on the usages in the production codebase.

Filtering test usages out of Find Usages results

Unit test providers don’t need to be source based. Most of them are, and display the test icon in the editor gutter, but they also examine the metadata of the compiled assembly, and can potentially find more tests here. Using this technique, it’s possible to run tests created in a source file that ReSharper doesn’t support. For example, we can create an test in F#, and while it’s not detected in the editor, it is still found and run when the assembly is compiled. test in F#

Since the test hasn’t come from source code, you can’t double click the test in the test results to navigate to it. But a failing test will display a clickable stack trace that will navigate to the failure.

F# test results showing clickable call stack

This technique can be used to run F# tests using other F# test frameworks, as long as they are based on a unit test framework that ReSharper supports (and this technique can also be used to write a plugin that supports a framework that isn’t based on one of these frameworks).

Silverlight support is provided by AgUnit, which enables running and debugging your unit tests in Silverlight projects, and of course handles the Silverlight test framework’s AsynchronousAttribute.

AgUnit debugging an asynchronous Silverlight test

Perhaps surprisingly, a couple of our own products are also plugins for ReSharper’s unit test runner – dotTrace and dotCover. Although the products register themselves with ReSharper somewhat differently to normal plugins, these products use the same open APIs that the third party plugins use, albeit for a slightly different purpose. Instead of examining source code and metadata to find and run tests, they provide the environment in which tests are run; dotTrace collects profiling information, and dotCover collects coverage information. Both products extend the UI to provide means of running the tests, but dotCover also adds an output pane to the test runner to display the coverage information (and highlights covered and uncovered code in the editor), and of course, this works with third party test providers.

Unit test runner showing dotCover custom code coverage results pane

And, just to bring the extensibility full circle, dotCover includes ReSharper’s unit test runner, which is used if ReSharper isn’t installed. Again, NUnit and mstest support is provided out of the box, and there are separate dotCover plugins for and MSpec support (the dotCover MSpec runner is also distributed in the NuGet package, just use the InstallDotCoverRunner batch file in the tools folder).

ReSharper provides comprehensive unit testing support, in an environment that is very flexible. There is a lot of great support, provided in the box, and just a download away. And it’s all open and extensible – if you don’t see something you want, grab the SDK and get coding!

Comments below can no longer be edited.

8 Responses to ReSharper plugins for unit testing – MSpec, and Silverlight

  1. Avatar

    AllSpark says:

    February 16, 2014

    I still couldn’t get the MSpec to work with Resharper 7. I see its installed in plugins but it doesn’t show up in unit tests. I installed using MySolutionFolder\packages\Machine.Specifications.0.7.0\tools\InstallResharperRunner.7.1.bat

    Please help thanks in advance

    • Avatar

      Hadi Hariri says:

      February 22, 2014

      That normally indicates that you have a mismatch between versions. Can you check that please?

    • Avatar

      Dahdread says:

      April 4, 2014

      You need to update the ReSharper plugin (Machine.Specifications Runner for ReSharper 7.1). If you get MSpec via nuget you can just close VS and run the bat file in the mspec directory for your version of resharper.

  2. Avatar

    Tom Cooley says:

    July 25, 2014

    I got the latest plugin (0.9.0-Unstable0032) and now I get a MissingMethodException in Machine.Specifications.Runner.Impl.DefaultRunner whenever I try to run a unit test. At least it’s true to its name of being unstable, but I’d really like to be able run my tests in here.

    • Avatar

      Matt Ellis says:

      July 30, 2014

      You should follow up with the mspec team on GitHub – I suspect you’ll need to use the 0.9.0 nuget package in your project, as well as with the ReSharper runner.

    • Avatar

      Jørn H. Dalvik says:

      November 7, 2014

      Upgraded to 0.9.0 as suggested by Resharper extention manager, and now I get the excact same error. 🙁
      Did you find a solution?

  3. Avatar

    Rory says:

    November 20, 2014

    I hit this issue – ended up upgrading mspec, all the mspec dependencies, adding Machine.specifications.should and installing all the runner extensions for resharper 8.2 — these are inside resharpers own extension manager.

    All is running great now – except one developwithpassion module – as I think this is waiting for a compatible build to mspec 0.9.1. What a relief!

Discover more