When working with TeamCity, we get a lot of feedback on our continuous integration process. We can see when compilation started, which unit tests passed and which failed. But how much memory do we consume while compiling our software? What is the CPU usage percentage while running our unit tests? This is all important information for optimizing our build process.
TeamCity provides a build feature called Performance Monitor which gives us exactly that information: it gathers the build agent’s hardware usage statistics during the build and presents it on a separate tab of the build results page.
To enable Performance Monitor for a build configuration we have to go to the configuration’s build steps. Under Additional Build Features, we can use the Add build feature button, select Performance Monitor from the list and click Save to enable this feature.
Note that Performance Monitor reports the load of the entire operating system. This means that running more than one build agent on the host, running agent and server on the same host or running other workloads on the server will yield unreliable data.
We can enable Performance Monitor on build agents running Windows, Linux, Solaris and MacOS X operating systems. Once enabled, every new build that is triggered will produce an interactive chart with performance details under the new PerfMon tab for that build.
The data shown consists of 3 key metrics (CPU usage, disk load factor and memory usage) and can be related to an exact point in the build log. When we click on a CPU usage data point for example, we can clearly see what the build agent was executing at that point in time.
The information displayed can be used to find bottlenecks, understand possible hardware upgrade impact, discovering suspicious periods in the build when nothing happens and so on. For example, consider the following performance chart:
When we click on the “25s” next to the Run Tests build stage, a chart overlay appears which visualizes the duration of running unit tests. From the picture it is clear that at some point CPU and disk usage is very low, near zero in fact. This lasts for a number of seconds. It is worth analyzing the unit tests that were running during this period: they are probably blocked on some lock, waiting for an operation to return or something similar.
Consider another performance chart:
We can see from this chart that the CPU usage is 100% almost all the time. It’s probably worth investigating if the build agent could benefit from a CPU upgrade.
The Performance Monitor build feature can reveal high-level information that may hint an underlying issue in your build process.