Filtering with dotCover

dotCover allows us to run coverage analysis on our code. However, there are times when we do not want to perform an analysis on certain areas. This could be our test assemblies, certain third-party assemblies or even specific parts of our own project. Since the analysis has an impact on the overall statistics and potentially can take longer, it is often interesting for us to filter certain assemblies or classes out.

The figure below shows the coverage report of the default MVC project (yes, yet another MVC example, but it ships in the box and probably the ONLY template with VS that comes with tests). In this case we’re using xUnit for tests, which is mostly to demonstrate that dotCover works with other frameworks, not only MSTest (personally I’ve moved on to MSpec, which dotCover also supports).

SNAGHTMLfe87ba

If we focus on the Coverage results we can see that there are some areas in grey, which indicate that the corresponding PDB files were missing so coverage could not take place. We can also see other assemblies that have been included in the results yet might not be of interest, such as the test assemblies.

image 

In order to filter these out, we can use the Coverage Filters, which can be accessed via the dotCover menu in the IDE

SNAGHTML105d878

By default, everything is set to be covered, as shown by the Everything entry in the Allow Filters tab (which most likely will be renamed to Included). Examining this entry (clicking on Edit) we can see that it is composed of three values:

SNAGHTML107f54b

  • Module Mask indicates the project name
  • Class Mask indicates the class name
  • Function Mask indicates the method name

(All entries support wildcards as displayed by the * that appears)

We can now uses these patterns to exclude projects, assemblies, namespaces, classes and methods from out tests.

Excluding entire project

Click on Deny Filters tab (most likely will be renamed to Excluded) and click on Add, entering the following information:

SNAGHTML10eb8b9

Clicking OK  and re-running the tests, we now see that the entire test assembly has been excluded

Excluding an entire Namespace

We can exclude entire namespaces by setting it in the Class Mask as shown below:

SNAGHTML11da114

this will exclude all the classes inside the MvcApplication16.Models namespace. If we just wanted a specific class, we add it after Models instead of *.

Excluding a method

We can exclude a method, similar to how we’ve done it with the previous cases:

SNAGHTML12173ed

 

Adding a series of filters for test and auxiliary assemblies using the previous steps, we can end up with a cleaner and more accurate coverage report:

SNAGHTML1242f6e

produced by adding the following filters:

SNAGHTML1252438

Notice that by using the *.Tests (or alternatively as I call it *.Specifications), we can automatically exclude all our test projects from code coverage.

 

Beta Disclosure

Note 1: Currently these settings are Global, that is, they apply to all projects. Most likely we’ll be changing it so that they are Solution and Global scoped. This way, common libraries can be excluded for all solutions, and specific projects and/or namespaces can be solution based.

Note 2: We’re also looking at adding functionality to make it easier to define these exclusions (a la Right-Click on Folder, add to Excluded list)

As always, we’re still in beta and feedback is more than welcome!

This entry was posted in dotCover Tips&Tricks, How-To's and tagged , , . Bookmark the permalink.

20 Responses to Filtering with dotCover

  1. Is it possible to set these filters from the command line?

  2. hhariri says:

    @Scott

    Currently no but it will have these features.

  3. Brian says:

    Will there be exclusion by attribute? That’s a feature I’ve used with NCover and really like.

  4. hhariri says:

    @Brian,
    Not sure about that one but I’ll check (or someone else will follow-up). However, can I ask, why not use filters? It’s less intrusive and does not pollute your code with framework specific attributes.

  5. Simon Hargraves says:

    It would be awesome if we could right click on the assembly line in the Coverage result area and exclude it from there also.

  6. Hadi Hariri says:

    @Simon,

    That’s already on the list of features but most likely won’t be in the first version

  7. Steve Dunn says:

    I would really like to be able to use the [CoverageExclude] attribute. Without this, I probably wouldn’t recommend it for my team. We have a huge amount of code and use this attribute to signify to nCover to ignore the type.

    I’m not a great fan of attributes, but I find this particular attributes makes the intent of the code clearer, e.g. ‘this component physically touches external dependencies and can’t be tested’ (whether right or wrong!)

  8. Steve Dunn says:

    It looks like dotCover suffers the same problems with Lamdas that nCover does. For instance, I cannot write a unit test that shows 100% coverage for the following method:

    public static IEnumerable Pairwise(this IEnumerable source, Func resultSelector)
    {
    TSource previous = default( TSource ) ;

    using( var it = source.GetEnumerator( ) )
    {
    if( it.MoveNext( ) )
    previous = it.Current ;

    while( it.MoveNext( ) )
    yield return resultSelector( previous, previous = it.Current ) ;
    }
    }

  9. Hadi Hariri says:

    @Steve

    Coverage attribute will be added in future versions. The Lambda issue is a known, however we’re going to look into that particular code.

  10. Steve Dunn says:

    Hadi, thanks. I pasted the wrong snippet of code. The above code IS covered by my xUnit tests, but for some reason, when I run the whole test suite at once, it says it’s not covered.

    I think this is because I do quasi-bdd style tests, something like:
    public class FooSpecs
    {
    public class WhenFooIsConfigured
    {
    public class And_someone_calls_it_who_IS_permissioned
    {
    [Fact]
    public void It_should_do_something( )
    {}
    }

    public class And_someone_calls_it_who_IS_NOT_permissioned
    {
    [Fact]
    public void It_should_do_something( )
    {}
    }
    }
    }

    The R# test runner sometimes doesn’t run the test when I right click on FooSpecs, sometimes it does. Also, if I dotCover the tests manually and then dotCover the whole test suite, it says that previously covered code is no longer covered.

    I’ll shortly be seeing how it copes with mSpec.

  11. Hadi Hariri says:

    @Steve

    I use MSpec. Can’t get weirder than that :).

    Could you log it with an attachment project so the devs can take a look, at http://youtrack.jetbrains.net ?

  12. Michael says:

    Is it possible to exclude all files ending with one given pattern? For instance I would like to exclude from the coverage analysis all files in the solution matching the pattern *Specs.cs (i.e ending with Specs).

    The reason for that question is that our tests are defined in the project where the object being tested resides so that we do not have to duplicate the folder structure in a separate test project.

    Thanks

  13. Hadi Hariri says:

    @Michael,

    You’d need to do it for each of the folders with Folder.Specs.*. Not sure if *.Specs would work.

  14. Michael says:

    @Haidi,
    I had tried *.Specs, *.Specs.cs etc.. already but unfortunately this does not work.
    Filtering each folder would theoretically do the trick, but would be way to much work.
    Is this a filtering feature that you guys might consider implementing?

    Cheers,

  15. Hadi Hariri says:

    @Michael,

    Not sure if it’s on the backlog. Could you please add it as a feature request at http://youtrack.jetbrains.net/issues/DCVR

  16. Phil says:

    I have been using dotCover for about a month now.
    You mention that this is a beta product and that you aim to make it easier to manage filters in future releases so here are some of my suggestions and comments based on what I have played around with so far:
    I like the right-click folder, project, etc to add/remove from coverage filters.
    It’d be nice to have this in both the solution explorer and the coverage results window, this way I can manage down to the function level.
    Would like to see a folder/file/path name filter option as well. Using WCF and other MS tools that auto generate code with specific filenames can be filtered out. For example one could exclude all Reference.cs below the root solution folder.
    for reporting I think it could be useful to keep a histogram counting the number of times a method is called during the analysis. This way if I have a few methods that are not covered well, but one that is called more frequently I can concentrate effort there as it is likely a more significant point of potential failure.

    cheers,

    Phil

  17. Michael K. Campbell says:

    I’m using dotCover 2.0 and still don’t get why setting up filters/exclusions is so hard.

    The new UI where I can set up filters IS great. Fantastic even – as it’ll let me use wildcards and so on. So that’s a GREAT win.

    BUT, I still see a HUGE disconnect in the UI.

    Right now in the Coverage Browser, I can go in… right click on entire assemblies, namespaces, classes, methods, and so on – it’s INSANELY granular and VERY easy to use.

    Then I can save said ‘coverage’ to a .dcvr file.

    What I’d LOVE to do?

    Be able to LOAD that back in and have THAT be what is used for setting all of my filters. Or, maybe some other file/etc.

    But it seems to me that I should be able to just save a ‘MySolution.dcvr’ (or .whatever) and have THAT be the more-or-less ‘defacto’ list of coverage that I care about when I re-run coverage. Ideally, there’d be some sort of convention that would try to re-load that per solution… but even WITHOUT that option if I could just use the dotCover > Open Coverage Snapshot and have THAT remember all the crap I excluded before AND have that be what shows up in the Unit Tests window/Coverage tab?

    THEN this would be IDEAL for me.

    Because, as it is, I still find myself scratching my head each time I open this up, exclude a few classes/assemblies I don’t care about and then see the option to SAVE and then LOAD these ‘snapshots’ without ANY of the state (i.e. exclusions/filters) persisted.

  18. Jesse Jacob says:

    I second what Michael Campbell said. The disconnect between the test runner’s coverage tab’s filtering abilities and the dotCover 2.0 Edit Coverage Filters screen is just weird. I’ll add a feature request, but I’m sure there’s some reason you guys aren’t doing this because it seems really, really obvious that it’s a huge win for your users if both UIs can generate permanent filters.

  19. Jesse Jacob says:

    It’s already a feature: http://youtrack.jetbrains.com/issue/DCVR-3209

    Everyone who wants this, please log into youtrack and vote it up!

  20. Jalal Mohammad says:

    C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\VSTest.Console.exe
    /Platform:x64 UnitTestProject1.dll
    C:\Intel\dts_brita-repo\BritaAPI\BritaAPI_CS\UnitTestProject1\bin\Release
    C:\Intel\dts_brita-repo\BritaAPI\BritaAPI_CS\UnitTestProject1\bin\Release
    output_BRITAAPI.xml

    BritaAPI
    BritaAPI.TproveCSAPI.*
    *

    UnitTestProject1
    *
    *

    i am trying to exclude TPROVECSAPI namespace from the report but i still see it..

    Can anybody help?

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">