TeamCity Build Dependencies
The subject of build dependencies is neither a trivial nor a minor one. Various build tools approach this subject from different perspectives contributing various solutions, each with its own strengths and weaknesses. Maven and Gradle users who are familiar with release and snapshot dependencies may not know about TeamCity artifact and snapshot dependencies or assume they’re somehow related to Maven (which isn’t true). So I have decided to provide an overview of how build dependencies work in TeamCity.
While Maven-based dependencies management and artifact repositories are very common and widespread in Java, there are cases where you may still find them insufficient or inadequate for your needs. For starters, you may not be developing in Java or perhaps your build tool is not providing built-in integration with Maven repositories, as is the case with Ant (or its Gant and NAnt spin-offs), SCons, Rake or MSBuild. Secondly, snapshot Maven dependencies provide their own set of challenges, making it harder to ensure correct snapshot dependency is used in a chain of builds.
TeamCity Artifact Dependencies
The idea of artifact dependencies in TeamCity is very simple: download the artifacts produced by another build before the current one begins. After the artifacts are downloaded to the folder specified (checkout directory by default), your build script can use them to achieve its goals. You can find configuration details in TeamCity documentation.
Naturally, this scheme is not suitable for build tools with automatic dependencies management, but it works well with build or shell scripts accepting and expecting local paths, relative to the checkout directory. Note that the copying works not only for the produced build binaries, but for any kind of binary or text files, like the TeamCity coverage report as demonstrated on the screenshot above.
There is one important detail about specifying artifact dependencies and that is “Get artifacts from” configuration where you specify what type of build should files be taken from. Possible values of this field are “last successful”, “finished”, “pinned”, or “tagged build”, as well as the build number or “Build from the same chain”. While most values should be trivial to understand with “Last successful build” being the default and generally suitable option, the definition of “same chain” build is directly related to TeamCity snapshot dependencies.
TeamCity Snapshot Dependencies
Imagine a monolithic multi-step build process (build, test, package, deploy) which you decide to split into multiple smaller builds, invoked sequentially, forming a chain of executions. Doing so allows one to configure or trigger every chain step separately and run certain steps in parallel in order to speedup the process (like executing tests or building independent components). Most of all, it makes the overall maintenance significantly easier. However, while doing so you need to ensure every chain step uses the same consistent set of sources pulled from VCS even if newer commits are made while chain steps are running. That’s what TeamCity snapshot dependencies are for: they connect several build configurations into a single chain of execution, called build chain, with every step using the same set of sources, regardless of VCS updates. Note that the TeamCity use of the term “snapshot dependencies” may confuse people familiar with Maven snapshot dependencies which are two unrelated concepts.
Snapshot dependencies are configured similarly to artifact dependencies. You can find configuration details in TeamCity documentation.
Using Artifact and Snapshot Dependencies Together
When applicable, it is recommended to define both kinds of dependencies between build configurations, as this ensures not only a consistent set of sources used throughout a chain steps but also a consistent flow of artifacts produced. Now the definition of “Build from the same chain” in artifact dependency mentioned above becomes clear, as this is the only meaningful option in this scenario.
In a way, you can think of build chain steps running in isolation from VCS updates after the first sources’ “snapshot” is taken. Chain artifacts are either re-created from the same sources or passed through chain steps with artifact dependencies. This makes chain steps consistent, reproducible and always up-to-date (when applied to using chain artifacts), something that can’t be easily achieved with Maven snapshot dependencies.
Build Chains Visibility in TeamCity 7.0
TeamCity 7.0 took the notion of build chains to a whole new level by providing build chains a new UI, making chain steps visible and re-runnable. Once you have snapshot dependencies defined, a new “Build Chains” tab appears in project reports, providing a visual representation of all related build chains and a way to re-run any chain step manually, using the same set of sources pulled originally.
Build Chain Triggering
Having build configurations connected with snapshot dependencies and, therefore, their builds grouped into build chains not only makes them more consistent regarding the sources used, it also impacts the way builds are added to the build queue: after a certain chain step is triggered, the default behavior is to add all preceding chain steps as well, keeping their respective order, in addition to the one that was triggered initially. Let me repeat it for more clarity: triggering certain chain configuration adds preceding (those to the left of it) and not subsequent (to the right of it) configurations to the build queue, although it may seem counterintuitive at first. The idea is to mark the location where chain execution stops, which is exactly the configuration that was triggered initially; it becomes the last execution step.
To trigger subsequent chain steps upon VCS changes found in a chain configuration, you can add a VCS trigger with the “Trigger on changes in snapshot dependencies” option to the configuration that would be the last execution step. This configuration is then triggered whenever any of the preceding chain steps is updated, which schedules the whole chain for execution.
Having this behavior in mind, you therefore need to decide which configurations are triggered automatically and which should be run manually. Usually, earlier chain steps having no impact on external environment can be triggered automatically by VCS trigger but final chain steps, potentially modifying external systems, are invoked manually after a human verification of the previous chain results. The process of running the final chain steps manually is usually referred to as “promoting” previously finished builds.
Sample Build Chain: Compile, Test, Deploy
Imagine three sample build configurations, “Compile”, “Test” and “Deploy” connected into a build chain: “Deploy” is snapshot dependent on “Test” which is snapshot dependent on “Compile”.
In this sample scenario the “Compile” and “Test” configurations are triggered automatically while “Deploy” is triggered manually, following the recommendations given above. VCS changes in “Compile” configuration only trigger an execution of this chain step, while VCS changes in “Test” configuration trigger “Compile” and “Test” execution (in that order).
Once a “Compile” configuration is added to the builds queue, its sources’ timestamp is recorded on the server to be used in all subsequent chain steps. If any of the chain steps is connected to a different VCS root, its sources are also pulled according to the same timestamp.
Promoting Finished Builds
As soon as the automatic chain execution stops (after running “Test”), you can continue it by clicking the corresponding “Run” button on the “Deploy” configuration that was not triggered (see the build chain screenshot above). Alternatively, it is possible to promote a finished “Test” build through its “Build Actions” and invoke configurations which are snapshot dependent on it – “Deploy” configuration in this case.
This article has provided an overview of TeamCity artifact and snapshot dependencies, build chains, how their steps are triggered and how finished builds are promoted. I hope you now have a good understanding of how it works and of when it is appropriate (or not) to use TeamCity build dependencies in addition to those provided by build tools such as Maven.
Please, refer to the TeamCity documentation for more information about this subject:our forum!
Subscribe to Blog updates
Kotlin DSL for Beginners: Recommended Refactorings
Imagine you have just switched your TeamCity project over to Kotlin DSL. Your builds run successfully, but what next? What small refactorings can you apply to your Kotlin DSL code to help keep it clean and tidy? In this article, we'll discuss the next steps you can take. 1. Remove disabled build st…
TeamCity integration with .NET, Part 3: Deploying projects
Today we are wrapping up our .NET integration demo. In this series, we've been explaining how TeamCity can integrate with .NET and walking you through a tutorial. To catch up with the demo, read the previous posts: Part 1: New approach and demo Part 2: Testing and building projects This project ha…
New TeamCity REST API documentation
TeamCity REST API is a powerful tool which lets you integrate external applications with the TeamCity server and create diverse script interactions. To make this instrument more useful and easier to understand, we have reworked the REST API documentation. Compared to a general guide we had in the p…
TeamCity integration with .NET, Part 2: Testing and building projects
In this blog series, we talk about the new approach we use to integrate TeamCity with .NET. Part 1 describes the .NET runner, that covers most of the core integration functionality, and introduces a demo project. In Part 2, we will dig deeper into the demo project and explore its Test and Build conf…