Improved continuous testing performance in dotCover 2017.2

Continuous testing (CT) is a super-helpful feature but only under one condition: sufficient performance. Unfortunately, in dotCover 2017.1 and earlier, CT could be a real pain on large and very large solutions (30+ projects). The problem lies in three performance-critical phases of CT (actually, this is true not only for CT but for all tests run under coverage), which are:

  1. Coverage process startup.
  2. Test execution (could be much slower than usual due to the additional statement-level instrumentation).
  3. Post-processing of coverage snapshots.

In 2017.1 and earlier, (3) was the worst performing phrase because multiple coverage processes running in the background created as many coverage snapshots. Even though coverage time wasn’t too big, the merging of subsequent snapshots was very slow due to the snapshot format. On real solutions, post-processing could take up to several minutes (and prevented running tests during that time).

Our support team’s recommendation in all such cases was to limit the number of coverage processes. This could be done by setting Run up to N assemblies in parallel to 1 in ReSharper’s unit-testing options. Limiting the number of snapshots to just one would prevent snapshots from merging.

dotCover. Run N assemblies in parallel

dotCover 2017.2 has finally addressed the issue,* changing its snapshot format to reduce merging time. Reports from users who have tried the updated feature indicate that the coverage phase during continuous testing has decreased from several minutes to under 30 seconds.

*The slowdowns during phases 1 and 2 will be subject to optimization in future dotCover releases.

That’s why now, on the contrary, we can recommend that you set Run up to N assemblies in parallel so that it matches the number of logical processors of your computer. Of course, each user case is unique and on some solutions (e.g., if unit tests from different assemblies perform intensive disk operations) the performance increase will not be as noticeable. Nevertheless, in most cases, this setting should positively affect the overall performance of continuous testing.

Logical processors

To try accelerated continuous testing, download and install ReSharper Ultimate 2017.2. Should you have any questions, please feel free to ask in the comments to this post. Your feedback is greatly appreciated!

This entry was posted in How-To's and tagged , , . Bookmark the permalink.

4 Responses to Improved continuous testing performance in dotCover 2017.2

  1. George Cook says:

    Is it possible to change the default generate code behavior for fields? they are tucked away under the other section, it drives me crazy – the first option is generate var, which is 3 keystrokes and something I’d never ever want.
    generate field, which I want 90% of the time, and would be considerable more work to do than type the word var (I’d have to scroll up, lose my cursor, type the field delcaration, and return to where I was) – to do that I have to press alt enter start typing other, then type field

    I’m not raising a bug, as I assume that there must be a way to do this in the IDE that I don’t know.

  2. Paco says:

    Very glad to hear about it! Hope to see this and future improvements included also in dotCover’s version for TeamCity.

    By the way, in MS Windows you have the environment variable NUMBER_OF_PROCESSORS to get the number of logical processors of your computer.

Leave a Reply

Your email address will not be published. Required fields are marked *