Unit Testing with CLion 2020.3: CTest Support
Our latest global developer survey shows that Google Test, Catch2, and Boost.Test are the three most popular frameworks for unit testing. CLion supports all of these, as well as doctest. We also know that CMake is the most trending project model in the C++ world. CTest, a testing tool distributed as part of CMake, is used to automate unit testing, which entails running tests and uploading the results to a dashboard. Looking at CTest as a CMake-specific test runner, you may think of it as something similar to CLion’s built-in test runner (even though it isn’t quite a straightforward comparison). Our users often ask us if there’s a chance CLion could support CTest. At long last, we can now answer “Yes!”.
We added support for CTest in CLion 2020.3 EAP. In this blog post, let’s look at how you can benefit from this support as you use CTest tests in CLion.
-
The list of tests in CTest is read and processed by CLion:
- The IDE automatically creates a Run/Debug configuration for them.
- The left-hand gutter displays icons to run the tests in your CMakeLists.txt files.
-
CLion can run the tests and present the results in its built-in test runner:
- You can review the test results In the test runner.
- With one click, you can navigate from the built-in test runner to the CMakeLists.txt.
- You can debug tests. When several tests are joined under one CTest entry, the dialog to select a final test to debug is shown. The debug in CLion omits the ctest executable launch – it launches and debugs the final test.
- All types of toolchains are supported, including remote toolchains and WSL.
- Running tests with Valgrind Memcheck, Sanitizers, Code Coverage, and Profiler is supported for CTest. However, tests are launched via the ctest executable, which affects the results.
Start working with CTest in CLion
When you first open a project that uses CTest, CLion detects the tests that are configured and automatically creates an All Tests configuration for running and debugging them. Let’s take the PEGTL project from GitHub for example and give it a try:
Alternatively, you can create a configuration manually if you select CTest Application type. In the configuration, you can tune the set of tests to run:
If your tests support that, you can run them in parallel. For this, specify a --parallel <N> value in the Program Arguments field.
Use the built-in test runner
You can launch tests via the configuration run, or use the run icon in the CMakeLists.txt file:
CLion’s built-in test runner shows the test results, which you can sort alphabetically or by duration. You can also show and hide successful and ignored tests, and rerun all failed tests in one click.
The console output is shown on the right, and the Jump to Source action will navigate you to the CMakeLists.txt file where the test is declared.
Debug tests configured via CTest
What if you decide to debug the test run via CTest? To avoid debugging the ctest child process, CLion runs the test itself under debug. If you’re using a multi-test configuration, the IDE will prompt you to select a single test from the list:
Run dynamic analysis for tests
CLion can dynamically analyze your test during execution to eliminate issues at runtime. This includes running under Valgrind Memcheck, with Google Sanitizers, under the profiler, or with code coverage. All of these will be applied to the execution under ctest, unlike when you debug.
Known issues and limitations
CTest support is implemented via a plugin:
If you don’t need this functionality, feel free to turn it off in the Plugins settings.
CLion relies on the ctest executable located in the same folder as the chosen cmake executable; there is currently no way to select an arbitrary ctest executable to use. The minimal supported version is 3.14 for now (CPP-22195)
See also the known issues linked to the parent ticket in YouTrack: CPP-905.
Do you use CTest in your projects? Then try CLion 2020.3 EAP and let us know how it works for you!
Your CLion team
JetBrains
The Drive to Develop
Subscribe to CLion Blog updates
