Continuous Testing in dotCover and ReSharper Ultimate

Warning: This post is outdated. Continuous testing UX has been reworked. Find the up-to-date instructions in the dotCover documentation.
Over the last couple of years, continuous testing has been the top anticipated dotCover feature. Now, the wait is finally over as dotCover 10 supports continuous testing in Visual Studio! Before we move on to the nitty-gritty details to show you how it works, let’s try to recall what continuous testing actually is.

Continuous testing

In a nutshell, continuous testing means testing ‘without interruption.’ It implies that tests are run in the background, so you have actual test results instantly, as soon as you’ve changed your code. This dramatically speeds up a developer’s workflow as there’s no need to manually rebuild the project and re-run all tests after making changes: the impacted tests are run in the background automatically (the testing session is triggered by some explicit action, e.g., saving changes).

If you practice test-driven development, the benefit is even more pronounced: the Red-Green-Refactor workflow becomes virtually seamless. All the boring routines go away, leaving you only with the fun part:

1. Write a test.
2. Write a stub.
3. Build.
4. Run the tests (they must fail).
5. Update the stub to return the expected result.
6. Build.
7. Run the tests (they must pass).
8. Write the code.
9. Build.
10. Run the tests (they must pass).
11. Refactor the code.
12. Build.
13. Run the tests (they must pass).

So, how to use continuous testing in dotCover? It’s very easy, with only three steps to follow.

1. Open Continuous Testing Session window

Continuous testing is managed via the Continuous Testing Session window. To open it, use the menu ReSharper | Unit Tests | Show Continuous Testing Session.

Continuous testing is off by default

2. Select continuous testing mode

By default, continuous testing is switched off. To make it work, select one of the modes in Mode:

Select continuous testing mode

A wide variety of combinations are available. For example, you can tell dotCover to automatically build and run dirty tests after you save the project. Or, run dirty tests only after you build the project.

What does ‘dirty test’ mean? It is the test that was impacted by code changes. For example, you have a test that covers some code, you change the code, and the test result is now obsolete, i.e. the test becomes ‘dirty.’ In addition, dotCover considers as ‘dirty’ all new tests, tests that were aborted by user (aborted), and tests that failed to complete for some reason (inconclusive).

3. Just work

Once continuous testing is enabled, you can simply start working. Initially dotCover will mark all tests as dirty.

The current tests status can be checked on the continuous testing icon in the status bar (bottom right corner of the main Visual Studio window). This handy assistant shows you whether there are any failed tests or all of them pass successfully. The Continuous Testing Session window displays the results in more detail:

Continuous testing status

As we already mentioned, your workflow with the enabled continuous testing is almost seamless. Suppose you change code that is covered by some tests and save the project (and you’ve selected the On ‘Save’ Build and Run Dirty Tests mode). dotCover builds the project and re-runs (only) dirty tests in the background:

Working with continuous testing

To navigate to a failed test, click the test in Continuous Testing Session window or click the number next to the continuous testing icon.

Failed test icon

Another option to navigate to the covering tests is to do it right from the code. Simply click on the code and select Show covering tests in the context menu. This opens the list of all tests that cover this line of code. Here you navigate to the corresponding tests or force them to run:

Show covering tests

Next steps

If you’re willing to try continuous testing right now, download and install ReSharper Ultimate.

IMPORTANT! If your tests that use external files fail during continuous testing (while being successful in a traditional unit testing session), you should properly set dotCover’s Shadow-Copy option. The following post describes in details how this option works.

Should you have any questions, please feel free to ask in the comments to this post. Your feedback is greatly appreciated!

Comments below can no longer be edited.

24 Responses to Continuous Testing in dotCover and ReSharper Ultimate

  1. Avatar

    MichaelD! says:

    November 19, 2015

    Really awesome! I was about to say that this article needs a video, but webinar is just as good. 🙂 I have enrolled. This is a really awesome feature. Seems like what Visual Studio tried to do w/ its “Test on Build” which never seems to work. Once again ReSharper understanding MSFT’s technology better than MSFT itself.

    One suggestion: consider changing “dirty” to “modified.” Dirty is really an older term that is associated with the 90’s, and is not seen much of these days. Well, by me, at least. Personally, it makes me think something that is uncouth and/or needs to take a bath. 😛

    • Alexey Totin

      Alexey Totin says:

      November 20, 2015

      Thanks a lot for your involvement Michael! Actually, there was heated debate on how we should call these tests. The main objection against ‘modified’ (the other option was ‘affected’) was that these are not only tests modified by user or impacted by code changes but also aborted and inconclusive ones.

  2. Avatar

    Aleksandras says:

    November 19, 2015

    I believe it works really nicely in your demos with a solution from 1-2 projects and with just 2 tests, but.. did you consider how it should work in a bigger solution? E.g. starting from 30 projects (still there are even bigger solutions) and with 2-3 Tests projects.

    For now it totally didn’t work for me (well, my machine doesn’t have 16+ CPU cores). Something like having “Solution-wide code analysis” enabled. I appreciate it’s just a early version of this feature, but I have some serious doubts it can ever work for bigger solutions..
    Any comments/objections on it? Would be glad to hear, that I’m wrong.:)

    • Avatar

      Jura Gorohovsky says:

      November 20, 2015

      Aleksandras, as far as I know, QA tested continuous testing on a variety of solutions, with up to 50 projects.

      We perfectly accept that it might not work in all solutions, especially large solutions. If this is the case, then we’d be happy to see performance snapshots representing slowdowns related to continuous testing (can be made via ReSharper > Help > Profile Visual Studio). In cases where providing us solutions is possible (which is either if they’re available publicly or you’re willing to provide them under NDA), this would help as well. Based on what we can see in a performance snapshot, optimizations in terms of the way how dotCover’s continuous testing works might or might not be possible.

      In any case, as performance varies from solution to solution based on a myriad of factors, it makes sense to contact dotCover support to discuss what can be done to improve performance with continuous testing in each particular case.


      • Avatar

        Aleksandras says:

        November 26, 2015

        Thanks, Jura, really appreciate your input.
        I believe, that our solution might have it’s own issues and should be optimized, but it just raised overall concerns about Resharper and Continous Testing – if it can work at all.
        I wasn’t aware there is a possibility to create profile snapshot for you so easily. Currently I disabled and uninstalled dotCover (since it was just trial and unfortunately it didn’t convince me to buy it..), but when I have some spare time I can try produce such snapshot for you. If it really could help you making Resharper+dotCover better – it sounds really promising.:)

        It would be also very nice to try it live on our real solution, but yes – it’s not open-sourced and there might be “NDA-issues”. Any other ideas how it can be tested? Maybe some screen-sharing sessions?

        By the way – a brief introduction of my company, just to ensure you that we are not just using your trials, but also quite power-users of Resharper. Currently I’m working in a project with ~30 developers (one of the reasons the solution had grown so big), everyone is with Resharper Ultimate license. And it’s just in this project, while we have more projects in our company.:)

        • Avatar

          Jura Gorohovsky says:

          December 1, 2015


          Yes, please, we would appreciate a snapshot whenever you can provide one. In the best case, it would allow us to see immediately what can be optimized. If it doesn’t, then we’ll see what additional info we might need.

          Screen sharing is nice but it would probably only allow us to see the end-user effects rather than know what’s causing them. However, if ReSharper and dotCover are run with verbose logging, then we might be able to analyze the logs and see any evident issues that they might expose. Anyway, the best first step is a performance snapshot, and if it doesn’t help, then a discussion with the support team should be the best guide what actions to follow up with.

          Thanks a lot for your willingness to cooperate. Other than not being currently able to run Continuous Testing, I hope you’re enjoying a smooth ride with ReSharper Ultimate on your team.

  3. Avatar

    Todd Aspeotis says:

    November 21, 2015

    I hope one day ReSharper Ultimate might include something like

  4. Avatar

    David Hewson says:

    November 24, 2015

    We’re trying to use continuous testing but each continuous run has about 30-seconds to 1 minute of prep before the tests run and about 5 minutes afterwards which seems to be the coverage analysis. It spends most of that time writing a 500mb file into temp.
    Works fine on smaller projects though, just not this bigger one. To give you an idea of the size the total statements according to coverage is 689,939

    • Alexey Totin

      Alexey Totin says:

      November 26, 2015

      Hi, David
      This might be the result of how the caching is implemented in 10.0.0 and 10.0.1. If this is the case, 10.0.2 will solve the problem. If 10.0.2 (when it comes out) doesn’t help, we’ll really appreciate if you could take a performance snapshot when the slowdown takes place (can be made via ReSharper > Help > Profile Visual Studio) and send it to us.
      10.0.2 is currently available as EAP

  5. Avatar

    AnneTheAgile says:

    December 2, 2015

    I registered for the webinar but couldn’t make it. Is it recorded? Is there a youtube or other channel of your webinars? Today’s on JS was very helpful. ty!

    • Avatar

      Jura Gorohovsky says:

      December 2, 2015

      Yes, the webinar was recorded. We’ll publish the recording on YouTube within a week.

  6. Avatar

    Wes says:

    December 8, 2015

    This is a sweet feature and is the reason that I want to upgrade to Ultimate.
    However, I’m having some problems w/ some tests that pass if I use the normal unit test session, but fail whenever in the Continuous Testing Session. I have 2 such tests failing out of 2400.

    I am noticing that the Continuous mode seems to copy the assemblies to a random location for execution of the test, however, the tests which file depend on files which are copied or created in sub folders relative to the test assembly are failing.

    For example, there is a test that says to create a db in “.\DatabaseTests\Someplace\Something.mdf” (this is using SMO)
    This test normally passes, but in my Continuous Test Session, it fails w/ this message:

    “Directory lookup for the file “C:\Users\Wes\AppData\Local\Temp\dodu3uyj.vqz\duxuys2y.eve\DatabaseTests\Something\Somewhere.mdf” failed with the operating system error 3(The system cannot find the path specified.).”

    Error code 3 is ERROR_PATH_NOT_FOUND

    Is there any way to control this behavior?
    What is dotCover doing here? I feel if I know what it was doing and I could update my tests to get around it.

    • Alexey Totin

      Alexey Totin says:

      December 8, 2015

      The problem is in how the default shadow copying works in Visual Studio. Long story short, 10.0.2 fixes the issue. Please note that it’s still in EAP
      If you want more details on how you can reference test data, please take a look at this comment in our tracker.

      • Avatar

        Wes says:

        December 11, 2015

        I tried it but didn’t really get the result I wanted. I figured it’s easiest to remove these two tests from the scope.

        I reverted back to 10.0.1 (non-EAP) as my OCD was letting the exceptions submitter distract me.

        It seems that the continuous test session can get out of sync w/ regular unit tests. as an example, I made changes I expect to cause things to fail, but they report as passing in the continuous session, but will fail if I invoke it as a normal test, or vice versa. If I drop the session and create another it reports the right thing for a while.

        Sorta wondering why it does this.

        • Alexey Totin

          Alexey Totin says:

          December 14, 2015

          We’ve created an issue in our tracker. Could you please answer a number of questions in the comment to the issue so we could clarify what’s going wrong in your case.

          • Avatar

            Wes says:

            December 14, 2015

            Would be happy to.

  7. Avatar

    Evgeny says:

    December 15, 2015

    Is it possible to make coverage highlighting feature like NCrunch (continuous instead of manual and coloured markers instead of coloured background)?

    • Avatar

      Jura Gorohovsky says:

      December 15, 2015

      Currently there’s only colored background available as a way of highlighting; however, there’s a request for an alternative highlighting that you can vote up and/or comment: DCVR-7737.

  8. Avatar

    Jason Nappi says:

    December 21, 2015

    I have a large solution with lots of tests. When I open the solution it automatically considers all tests as dirty. Can I instead consider then ‘green’ at start up and only turn dirty as I start making changes.

    • Alexey Totin

      Alexey Totin says:

      December 22, 2015

      Hi, Jason
      No, it’s not possible. For continuous testing, just data on whether tests pass or fail is not enough. dotCover also requires coverage data which it can get only during a coverage session. That’s why initially all tests are considered dirty.

  9. Avatar

    Alin says:

    June 16, 2016


    I have Resharper Ultimate 2016.1.2.

    The .Net 4.6 solution has 90 projects, with 6400 MsUnit tests.

    Although, I’m very happy that netCover exists, the first main concern is speed execution of the Continous Testing Session. I’ve read this page and I see I’m not the only one.

    My machine is i7-3770 with 8 Gb of RAM, 256 Mb SSD.

    It takes tens of seconds to maybe 1 minute to execute and finnish 1 dirty test. I tasted NCrunch and the 2 are worlds apart. Still, I’d like to just use netCover Resharper Ultimate…

    I’m not sure how to give a profiling snapshot. Please guide me if possible.


  10. Avatar

    Vijay Venkatesh Prasad N says:

    October 21, 2016

    I have Resharper Ultimate 2016.1.2.

    I am trying to run the continuous testing session but it is failing because its not referring the Unit test sessions test settings.

    Could you please let me know how to update the test setting for continuous testing session?

  11. Avatar

    Jonathan says:

    November 29, 2019


    I am really having a hard time figuring this out, nothing seems to be where it is described here.

    I am using the latest ReSharper and DotCover but I can’t find the “Show Continuous Testing Session” option in the menu.

Discover more