Shadow-copying in dotCover: if your NUnit tests fail during continuous testing

Posted on by Alexey Totin

When dotCover 10 came out, early users of continuous testing faced a problem: tests that used external files could fail in a continuous testing session while being successful in a ‘normal’ unit testing session. The issue was related to how dotCover worked with unit test assemblies.

To run a continuous testing session in the background, dotCover doesn’t use the original unit test assembly: instead, it creates a copy using the shadow-copying mechanism. Along with assemblies and symbol files, dotCover 10 and 10.0.1 also copied the entire content of the directory where the original assembly was located, be it a default location (e.g., …/bin/Release/) or a custom location specified by the option ReSharper | Options | Unit Testing | Run tests from.

If a unit test works with a file (using its relative path) that is located outside this directory, the test will fail as it looks for the file relative to the new temporary location. For example, the following test (used to process a file in some way) fails in a continuous testing session but passes in a ‘normal’ unit testing session.

       [Test]
       public static void JustAnotherTest()
       {
           var filepath =  "..\\..\\Test Files\\1.txt";

           Console.WriteLine(Path.GetFullPath(filepath));

           Assert.IsTrue(ProcessFile(filepath));
       }

Failed test in a continuous testing session

Successful test in a normal session

To address the issue, dotCover 10.0.2 introduced a new option, ReSharper | Options… | dotCover | Continuous Testing | Shadow-Copy.

Shadow-copy option

If this option is set to Assemblies and symbol files, dotCover will copy to a temporary location only assemblies and PDB files without touching other data. In this case, the working directory for tests will be the initial location of test assemblies (e.g., …/bin/Release/). Therefore, the sample test from above will pass successfully in both continuous testing sessions and traditional unit testing sessions.

Successful test in a continuous testing session

However, if All files from the tests start folder is selected, dotCover works as before. It copies the entire content of the tests’ start directory. In this case, the path ..\\..\\Test Files\\1.txt will refer to the new temporary location, causing the test to fail.

IMPORTANT! The Shadow-Copy option works as described ONLY with NUnit tests. All other unit testing frameworks use shadow-copying at their discretion depending on plugin implementation. For example, MSTest may decide to disable shadow-copying in a particular case (regardless of the Shadow-Copy option).

To simplify using continuous testing, you may want to change the way you reference files in tests. Here is a short table showing how you can build paths to the tests working directory. Source here refers to the initial directory where the unit test assembly is located; Temp means the temporary directory where the assembly copy is located.

Assemblies and symbol files All files from the tests start folder
Assembly.GetExecutingAssembly().Location Temp Temp
Assembly.GetExecutingAssembly().CodeBase Source Temp
Environment.CurrentDirectory Source Temp
AppDomain.CurrentDomain.BaseDirectory Source Temp
TestContext.CurrentContext.TestDirectory (NUnit 2.6) Source Temp
TestContext.CurrentContext.WorkDirectory (NUnit 2.6) Source Temp
Subscribe

Subscribe to .NET Tools updates