dotCover 2018.1: Better continuous testing and more!

In this 2018.1 release, dotCover is by far the product that has received the most changes in the whole .NET tools lineup. These include:

  • Continuous testing with new modes available in any unit test session,
  • Coverage analysis improvements including a new Unit Test Coverage window, real-time filters, and others,
  • Console runner support to analyze coverage of web applications, and more.

We’ll leave the latter for a separate post, and focus on updated continuous testing and coverage analysis improvements.

Continuous testing. Now in any session!

So, what exactly has changed in continuous testing? First of all, there are no more separate continuous testing sessions. This is a big game changer: now, any session can be switched to the continuous testing mode.

This leads to some really positive consequences throughout:

  • To start using continuous testing, you no longer need to re-run all tests under coverage in a separate continuous testing session. This was a significant limitation that prevented many users from trying continuous testing.
  • In dotCover 2017.3 and earlier, your continuous testing scope was limited by a single continuous testing session. Now, you can have as many scopes for continuous testing as you need.

What the new workflow looks like:

  1. First of all, decide when you need dotCover to rerun tests: either after you save the solution, or after you build it (the default option). Select the preferred option in ReSharper | Options… | dotCover | Continuous Testing
    dotCover. Continuous testing settings
  2. Create a new unit testing session, or open one that has the scope you want to cover by continuous testing. If you don’t have any sessions yet, you can use the Unit Test Explorer window to create one.
    dotCover. Unit Test Explorer
  3. In the opened Unit Test Sessions window, select a continuous testing mode for your session. For example, if you want dotCover to autorun tests and get coverage each time you save or build the solution, select Autostart Tests on Build: Cover New and Outdated Tests.
    Select continuous testing mode
  4. If you don’t have any coverage data or it is outdated for tests in the scope, dotCover will ask you to perform initial coverage analysis.
    Initial coverage
  5. Coverage results will be shown in a separate Unit Test Coverage window (this is completely new – see the details below in this post).
    Continuous testing. Coverage results
  6. That’s it! Now, simply start working as usual: change code, build or save (as needed), and see test results in real time.
    Continuous testing workflow

New continuous testing modes

The next big thing in continuous testing is additional continuous testing modes. In dotCover 2017.3, you had no alternatives on how tests were auto-started during continuous testing. After you build or save your solution, dotCover started tests under coverage.

In 2018.1, coverage is no longer necessary: you can tell dotCover to simply run new and outdated tests (without covering them). This could be a great time saver if you already have an up-to-date coverage snapshot:

New continuous testing modes

Note that in this case, coverage results use pale highlighting to emphasize that they’re based on possibly outdated coverage information. Nevertheless, if you are sure that the information is still relevant, you may keep working and postpone coverage until it really becomes outdated.

Aggregated coverage results

In dotCover 2017.3 and earlier, coverage results were shown per-session in the Coverage tab of a unit testing session. Now, the results are shown in a separate Unit Test Coverage window. Some pros of this approach include:

  • The All Tests mode that aggregates coverage results from all sessions. It’s very cool, when, for example, you have some integration tests where continuous testing is not possible, and some simple unit tests where continuous testing is enabled. The window aggregates results from both sessions, allowing you to instantly see how close you are to covering all the code.
  • Simple to use: as a separate window, it can be easily moved, adjusted, docked, and so on.

Previously: Results were shown only for the current session (note there’s one more session opened):

Per-session coverage in 2017.3

Now:

Aggregated coverage results

Dynamic coverage data

The new Unit Test Coverage window has one more advantage over its predecessor. In 2017.3 and earlier, coverage info was static: to update it you had to explicitly restart tests. Now, the update is performed dynamically.

First, removing a test from the current session deletes the related coverage data from the coverage tree.

Dynamic coverage results. Remove test

If you add a test to the session, the Unit Test Coverage window will explicitly tell you that coverage data for some tests are outdated and will suggest running coverage analysis for this test.

Dynamic coverage. Add a test

Second, coverage filters are also applied instantly: if you apply a filter, the corresponding coverage data is removed from the tree. If you remove a filter, the Unit Test Coverage window explicitly tells you that coverage data is outdated.

Dynamic coverage. Add a filter

To try these upgraded continuous testing and other improvements, download and install the latest ReSharper Ultimate 2018.1 EAP. Should you have any questions, just ask us in the comments to this post. Your feedback is greatly appreciated!

P.S. No, we’re not done yet with all the great things dotCover 2018.1 has to offer! Stay tuned for an additional post on how to profile web apps running on IIS and IIS Express from the command line!

This entry was posted in How-To's and tagged , , . Bookmark the permalink.

20 Responses to dotCover 2018.1: Better continuous testing and more!

  1. Derek Price says:

    Are any of these features available in Rider?

  2. Look awesome! Did I get that right that you still need to press Save shortcut or build shortcut to run the tests?

  3. Mike-EEE says:

    WOW… alright… FINALLY got a chance to install 2018.1 and check this out. FWIW all 17 of my extensions were available for install with the exception of TaskObserve. Not bad. :)

    Anyways… Unit Testing does indeed look MUCH better from the outset. Everything looks as expected and RUNS SO MUCH FASTER. Sorry, excited, must talk in caps. :)

    My only nit thus far is the unit testing session windows. Why are they closed when the solution opens? The Build Results have the same problem. I have four monitors and have exact positioning of where I’d like my windows. When windows are closed they pop back in and give a very uneasy feeling to your product at best. Feels like a bug at worst. Or really, feels like a bug, period. 😛 I know you can configure Build Results to Always… but do you have to do that with every other window you offer? Not a consistent experience.

  4. Adrian says:

    Everything seems to be working great! I just have one issue, it seems the code highlighting will break pretty often with new or dirty tests. I have it set to Autostart Tests on Build: Cover New and Outdated Tests. Recovering the tests doesn’t seem to fix it. The code highlighting bar to the left seems to be absent entirely on these files while files from existing test work fine. My only remedy for the problem right now is close and re-opening visual studio, which is pretty annoying.

    Running Visual Studio Professional 15.7.1 and ReSharper Ultimate 2018.1
    This bug started occurring after updating.

    • Alexey Totin says:

      Hi Adrian,
      this is a known issue that will be fixed in 2018.1.1
      Currently, you can try to use a workaround:
      – close your solution
      – go to %temp%\JetBrains\ReSharperPlatformVs[N]\vAny_[XXX]\CoverageData folder
      – remove all subfolders
      – reopen the solution and repeat coverage analysis

  5. Eric Richards says:

    The Hot Spots coverage tree is really bad since I upgraded to 2018.1. It’s completely inconsistent from run to run, and very often the links within it will jump to some other arbitrary location in the code base from where it claims to be going. Not really useable at all.

    I’m kind of sad, that was one of my more favorite features.

    • Alexey Totin says:

      Hi Eric,
      yes, Hot Spots are not working as expected in 2018.1. This is a known issue that will be fixed in 2018.1.1
      Unfortunately, there is no workaround.

  6. LordWilmore says:

    I don’t want to be a killjoy, but is there any way to disable this feature, and still allow manual unit testing and coverage reports, as it worked in 2017?

    I’m not against the change, but some of our development machines struggle to cope with the size of some of our solutions (close to a thousand projects), and therefore unit testing is very much something that is done in a discrete batch. In short this is stopping me working as effectively as I was with 2017.

    I appreciate that the correct solution would be to use smaller solutions, but this is the situation that I find myself in, and if there was an option to turn off continuous testing then my troubles would go away.

    • Alexey Totin says:

      Hi,
      are we talking about continuous testing? If yes, you can disable it by toggling off the ‘Autostart Tests on Build’ button for a session. Or this is not your case and you mean some other feature?

  7. Ilya says:

    How to disable the Continuous Testing feature? I use R# together with dotCover and other dot* products. I had no other option except turning off dotCover completely to stop it!!!
    I don’t need it running in the background on every my action, OK?

Leave a Reply

Your email address will not be published. Required fields are marked *