Doctest for unit testing in CLion

Anastasia Kazakova

In the C++ ecosystem there is more than one unit testing framework. Google Test is the one used the most. Boost.Test is obviously gaining in popularity among the Boost community. Catch(2) is relatively new but also growing impressively. Meanwhile, several years ago at CppCon there was an inspiring presentation by Viktor Kirilov of yet another framework called doctest. Viktor told our audience about it in the guest blog post in ReSharper C++ blog, once the support was added there.

In a nutshell, doctest is a fast single-header framework, with self-registering tests. It’s safe: uses thread-safe asserts and can be used without exceptions and RTTI. You can learn from the post in detail what makes doctest interesting.

The good news is – doctest is coming to CLion 2020.2 EAP! Grab the free EAP build and give it a try now.

DOWNLOAD CLION 2020.2 EAP

The corresponding plugin is bundled into the EAP build:
Doctest plugin
Both CLion and ReSharper C++, now provide support for 4 frameworks: Google Test, Boost.Test, Catch(2), and doctest.

Let’s take the doctest project from the GitHub and check out how the sample tests look in CLion:
Doctest run

If you already use Google Test, Boost.Test or Catch(2) integration in CLion, you are familiar with what’s available here:

  • Doctest tests are detected by CLion and Run/Debug configurations are created automatically when you run a test or all tests in a file.
  • Built-in test runner shows the results of the tests, along with the test output, test duration, and a bunch of controls that enable you to monitor the tests and analyze the results.
  • Run/Debug icons in the left gutter show the status of the tests.

That’s it! You can try out doctest for your project in CLion and benefit from the integration! Let me just add a few notes on the Run/Debug configurations here. In doctest there are suites, tests, and subtests. Run/Debug configurations in CLion use a Pattern field in the settings to define the set of tests to launch within this configuration.

  • To run all tests in a file use pattern [#*filename.cpp].
  • [suite_name] will run a suite with the corresponding name.
  • If you want to run a particular test from a given suite, you can point [suite_name][test_name]. Adding another level for subtest is also possible: [suite_name][test_name][subtest_name].
  • An anonymous suite is possible via []. However, not without a test name (so [] or [][] run nothing).
  • Patterns can be combined using commas as separators.

Doctest configs

DOWNLOAD CLION 2020.2 EAP

P.S. Many users ask us about CTest support in CLion (CPP-905) and we tried to address the point during the CLion AMA session in May. While another unit testing framework was taken on board, CTest is still not there. There is a workaround of course, but that’s definitely not an experience our users expect from us. We’ve recently conducted a small overview research trying to understand how CTest is usually used, what commands are regularly passed as a test script, what kind of support is provided for CTest by other IDEs. And while we see some challenges, like the very custom scripts used to run tests or deploy the environment, difficulties to detect the exact executable to debug, and others, we plan to prototype the support in the further CLion versions. Our call to action here would be – share your OS projects with us if you use CTest so that we can learn about more use cases! Thanks in advance.

Your CLion team
JetBrains
The Drive to Develop

Comments below can no longer be edited.

4 Responses to Doctest for unit testing in CLion

  1. Ernst says:

    June 18, 2020

    not about DocTest but Boost.Test :
    seems it’s available for cmake targets only?

    please confirm – this for my configuration:
    – 2 remote cmake’s “add_custom_targets” (scons based) – not available to add Boost.Test configuration
    – local (windows) pure custom target (scons as well) – not available for BOost.Test run configuration too..

    supposed to specify executable instead of build target there.
    can this be improved so?

  2. Алексей Виноградов says:

    June 18, 2020

    We use CTest as current backend to unify all our different kind of tests.
    Both, as simple test suite to run over build in-place, and also as full script which does all steps like configuring, building, running tests, collecting coverage and (in future) pass valgrind testing.
    Tests from google tests are also adopted and wrapped into individual Ctest each.
    Final results (it is what it is ‘as whole’ about) then pushed to CDash, and also converted into jtest format and pushed to gitlab CI.

    However as a note: there are no comprehensive materials about best practices with ctest, so everything in the project is just home-made bicycles; don’t thing for them as a good example.

    You can look at manticoresoftware/manticoresearch on github
    Start points from which it is invoked in CI are in .gitlab-ci.yml
    Start point for manual run the whole test suite is script smoke.sh in the root

    • Anastasia Kazakova says:

      June 18, 2020

      Thanks for the tips!
      Indeed, the main issue with CTest is it’s “free style” for everything. But I believe we can come up with some good solution finally, working for 80% cases at least.

Subscribe

Subscribe for updates