Features How-To's Tips & Tricks

Kotlin Configuration Scripts: Testing Configuration Scripts

IMPORTANT: See here for the updated version of the tutorial

This is part five of the five-part series on working with Kotlin to create Configuration Scripts for TeamCity.

  1. An Introduction to Configuration Scripts
  2. Working with Configuration Scripts
  3. Creating Configuration Scripts dynamically
  4. Extending the TeamCity DSL
  5. Testing Configuration Scripts

In this last part of the series we’re going to cover one aspect that using Kotlin Configurations Scripts enables, which is testing.

Given that the script is a programming language, we can simply add a dependency to a testing framework of our choice, set a few parameters and start writing tests for different aspects of our builds.

In our case, we’re going to use JUnit. For this, we add the JUnit dependency to the pom.xml file


We also need to define the test directory

In our case, this corresponds to the following directory layout

Directory Structure

Once we have this in place, we can write unit tests like in any other Kotlin or Java project, accessing the different components of our project, build types, etc. In our case we’re going to write a simple test that checks to see all build types start with a clean checkout

fun buildsHaveCleanCheckOut() {
    val project = Project
    project.buildTypes.forEach { bt ->
        assertTrue(bt.vcs.cleanCheckout ?: false, "BuildType '${bt.extId}' doesn't use clean checkout")


That additional layer of certainty

When we make changes to the Kotlin Configuration Script and check it in to source control, TeamCity synchronises the changes and it will report of an errors it encounters, which usually are related to compilation. The ability to now add tests allow us to add another extra layer of checks to make sure that beyond our build script containing any scripting errors, certain things are validated such as the correct VCS checkout as we’ve seen above, whether the appropriate number of build steps are being defined, etc. The possibilities are essentially endless given that once again, we’re just using a regular programming language.

Update: Important note

Given that currently TeamCity does not fetch third-party dependencies, the tests, which usually require JUnit or some other testing framework, will fail if places inside the .teamcity folder. This is due to TeamCity syncing the project and failing to compile unresolved references. For now the workaround is to place the tests outside of this folder.

Comments below can no longer be edited.

2 Responses to Kotlin Configuration Scripts: Testing Configuration Scripts

  1. Avatar

    Michal says:

    March 20, 2017

    Thank you for this series of posts. It forced us to start using Kotlin DSL for TeamCity builds.

    However, when such tests are added to the repo it’s failing for me on TeamCity:

    [02:38:46]: Failed to apply changes from VCS to project settings (revision 21dc088d2fd73cc2ba20decfff3c736bd81151ba): Kotlin DSL compilation errors. Please fix the errors in VCS and commit again.
    tests/ProjectTest.kt [2:12]: Unresolved reference: junit
    tests/ProjectTest.kt [6:6]: Unresolved reference: Test

    Is there a functional example somewhere on GitHub?

    • Avatar

      Hadi Hariri says:

      March 21, 2017

      Great to hear!

      Regarding the issue, can you please move the tests outside of the .teamcity folder? They can still belong to the same project, but place them outside, since currently TC does not grab third-party dependencies and that’s why it’s failing when trying to “compile” these on the server. I’ll update the post so that this is clear. Sorry about that.

Discover more