Fail build on metric change in TeamCity 7.0

Kirill Maximov

One of TeamCity’s remarkable features is a possibility to analyze the code of your application, find duplicates, show corresponding reports and charts. You can calculate code coverage metrics, run code inspections, and see results right in the TeamCity’s UI. And in most cases, these features work out of the box, without setting up additional external plugins.

All these code examining tools generate various numeric metrics, and you can see the dynamic of their change in the TeamCity UI.

What you could not do before (well, almost could not) is to specify metric thresholds, which would fail the build upon exceeding them. For instance, you couldn’t tell TeamCity that the build must fail if code coverage or code duplicates number is worse than in the previous build.

Corresponding requests stayed for a while in our tracker and collected quite a few number of votes:
– TW-2539 Inspection Build: fail if more inspection errors than in last successful build
– TW-3040 Make build fail if code coverage is below certain threshold
– TW-3041 Make build fail if code coverage is below previous code coverage value
– TW-4953 Setting to fail a build when number of found code duplicates exceeds some threshold
– TW-15679 Option to fail a build if a specific statistics value is below/greater then in last successful/tagged build

When we started pondering over the task, it became obvious, that it can be generalized. And besides code quality, we could consider other metrics, such as number of tests in a build or volume of published artifacts:
– TW-12449 Option to fail a build if test number drops
– TW-5451 Option to fail a build when not all artifacts are published (artifacts validation)

Moreover, if a build publishes its own numeric metric (and it is very easy to set up), this metric can also be used to determine build status.

So now you can specify several metric-based build failure rules for a build configuration:

or

As you see, you can create rules for both the absolute metric values, and for the difference with a previous build in the same build configuration. As a previous build anchor, you can use:
– last finished build
– last successful build
– last pinned build
– build with a specified build number
– last finished build with given tag

The default list of available metrics is:
– artifact bytes (you can fail a build, if not all artifacts were published, basing on their volume)
– number of tests in a build
– number of ignored tests
– code inspections: errors, warnings
– code duplicates count
– code coverage percentage – lines, blocks, methods, classes

To add your custom metric to this list, you have to change TeamCity’s configuration file main-config.xml (<TeamCity data directory>/config/main-config.xml) and add the following section there:

<build-metrics>
<statisticValue key="my_file_count" description="number of files"/>
</build-metrics>


So, if your build will publish value my_file_count, you can use it as a criteria for a build failure.

When build failure metrics are set up, the interface will look like:

When a build fails because of the bad metric value, it looks like this:

The feature is not completed yet, so your comments and feedback are very much welcome. To try it live, install TeamCity 7.0 EAP build, which is about to be released in the nearest future.

Subscribe to TeamCity Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!