Improved continuous testing performance in dotCover 2017.2

Alexey Totin

Continuous testing (CT) is a super-helpful feature but only under one condition: sufficient performance. Unfortunately, in dotCover 2017.1 and earlier, CT could be a real pain on large and very large solutions (30+ projects). The problem lies in three performance-critical phases of CT (actually, this is true not only for CT but for all tests run under coverage), which are:

1. Coverage process startup.
2. Test execution (could be much slower than usual due to the additional statement-level instrumentation).
3. Post-processing of coverage snapshots.

In 2017.1 and earlier, (3) was the worst performing phrase because multiple coverage processes running in the background created as many coverage snapshots. Even though coverage time wasn’t too big, the merging of subsequent snapshots was very slow due to the snapshot format. On real solutions, post-processing could take up to several minutes (and prevented running tests during that time).

Our support team’s recommendation in all such cases was to limit the number of coverage processes. This could be done by setting Run up to N assemblies in parallel to 1 in ReSharper’s unit-testing options. Limiting the number of snapshots to just one would prevent snapshots from merging.

dotCover 2017.2 has finally addressed the issue,* changing its snapshot format to reduce merging time. Reports from users who have tried the updated feature indicate that the coverage phase during continuous testing has decreased from several minutes to under 30 seconds.

*The slowdowns during phases 1 and 2 will be subject to optimization in future dotCover releases.

That’s why now, on the contrary, we can recommend that you set Run up to N assemblies in parallel so that it matches the number of logical processors of your computer. Of course, each user case is unique and on some solutions (e.g., if unit tests from different assemblies perform intensive disk operations) the performance increase will not be as noticeable. Nevertheless, in most cases, this setting should positively affect the overall performance of continuous testing.

To try accelerated continuous testing, download and install ReSharper Ultimate 2017.2. Should you have any questions, please feel free to ask in the comments to this post. Your feedback is greatly appreciated!

Subscribe to a monthly digest curated from the .NET Tools blog:

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!