Fail build on metric change in TeamCity 7.0
One of TeamCity’s remarkable features is a possibility to analyze the code of your application, find duplicates, show corresponding reports and charts. You can calculate code coverage metrics, run code inspections, and see results right in the TeamCity’s UI. And in most cases, these features work out of the box, without setting up additional external plugins.
All these code examining tools generate various numeric metrics, and you can see the dynamic of their change in the TeamCity UI.
What you could not do before (well, almost could not) is to specify metric thresholds, which would fail the build upon exceeding them. For instance, you couldn’t tell TeamCity that the build must fail if code coverage or code duplicates number is worse than in the previous build.
Corresponding requests stayed for a while in our tracker and collected quite a few number of votes:
– TW-2539 Inspection Build: fail if more inspection errors than in last successful build
– TW-3040 Make build fail if code coverage is below certain threshold
– TW-3041 Make build fail if code coverage is below previous code coverage value
– TW-4953 Setting to fail a build when number of found code duplicates exceeds some threshold
– TW-15679 Option to fail a build if a specific statistics value is below/greater then in last successful/tagged build
When we started pondering over the task, it became obvious, that it can be generalized. And besides code quality, we could consider other metrics, such as number of tests in a build or volume of published artifacts:
– TW-12449 Option to fail a build if test number drops
– TW-5451 Option to fail a build when not all artifacts are published (artifacts validation)
Moreover, if a build publishes its own numeric metric (and it is very easy to set up), this metric can also be used to determine build status.
So now you can specify several metric-based build failure rules for a build configuration:
As you see, you can create rules for both the absolute metric values, and for the difference with a previous build in the same build configuration. As a previous build anchor, you can use:
– last finished build
– last successful build
– last pinned build
– build with a specified build number
– last finished build with given tag
The default list of available metrics is:
– artifact bytes (you can fail a build, if not all artifacts were published, basing on their volume)
– number of tests in a build
– number of ignored tests
– code inspections: errors, warnings
– code duplicates count
– code coverage percentage – lines, blocks, methods, classes
To add your custom metric to this list, you have to change TeamCity’s configuration file main-config.xml (<TeamCity data directory>/config/main-config.xml) and add the following section there:
<statisticValue key="my_file_count" description="number of files"/>
So, if your build will publish value my_file_count, you can use it as a criteria for a build failure.
When build failure metrics are set up, the interface will look like:
The feature is not completed yet, so your comments and feedback are very much welcome. To try it live, install TeamCity 7.0 EAP build, which is about to be released in the nearest future.
Subscribe to Blog updates
Thanks, we've got you!
Introducing AWS Connection: A New Approach for Working With AWS in TeamCity
We’ve introduced a new feature called the AWS Connection, which makes it possible to work with AWS credentials more flexibly and safely.
TeamCity 2022.10: Sakura UI, Improved Integration With Amazon AWS, Out-of-the-Box HTTPS Support, Google OAuth Authentication, and More.
TeamCity 2022.10 comes with the Sakura UI enabled by default, introduces many cloud-related features including the AWS credentials management system and out-of-the-box terminal access to AWS EC2-based build agents, and adds important improvements to VCS integrations. Server administrators will love the new streamlined way to implement HTTPS for on-premises TeamCity installations. Also, a new authentication module for using Google accounts and a number of long-requested features have been implemented.
Transition to native Git in TeamCity 2022.04 brings 10x fetch time reduction to IntelliJ Platform
Starting from version 2022.04, TeamCity switched to native Git on the server side for Git VCS connections. The switch should positively impact both performance and overall experience of working with Git repositories on the TeamCity server side.In this blog post, we’ll talk about the reasons for the …
TeamCity 2022.04: Parallel Tests, Improved Integration with Amazon AWS, Built-in Code Quality Analysis, Updated VCS Integrations, and more
TeamCity 2022.04 introduces the new Parallel Tests feature that delivers an extraordinary speed boost when testing your projects. New powerful code analysis features will cut the time you spend on code reviews and increase the maintainability of your projects. Users who run their builds in the cloud…