Explaining the Shared Resources plugin
This post describes one of the new features in the upcoming TeamCity 8.0
Using TeamCity, we can easily set up a build farm using a central TeamCity server and several connected build agents. By having multiple build agents, we can run builds and build configurations in parallel. Different branches of the same project can build simultaneously on different build agents, personal builds can be run on one build agent while the nightly build is running on another agent and so on.
But what if we have multiple build agents and we want to constrain parallelism for some projects? What if we do want to have parallelism but want to provide different build variables to each build? Good questions, and the Shared Resources plugin is the answer!
Limiting concurrent builds with simple named resources
The Shared Resources plugin works at the project level. Per project, we can specify one or more shared resources which we can use to decide on the number of concurrent builds for a given project and build configuration.
In the Project Home under the Shared Resources tab, we can specify the resources we want to be able to lock. A shared resource should be given a name which can be used later on in our builds. The resource type can be infinite (effectively not limiting build concurrency), a resource with quota and a resource with custom values. Let’s go with resource with quota.
The resource quota is the number of concurrent read locks that can be acquired on the resource. Or in other words: the maximum number of concurrent builds we want to run. Let’s be restrictive and set the quota to 1.
None of the Build Configurations will obey these quota: we have to add a Shared Resources Build Feature to every Build Configuration we want to subject to these quota. This can be done in the Build Configuration. Under Build Features, we can add a Shared Resources build feature which acquires a read or write lock on the shared resource created earlier.
Once added, the build configuration will only be run on one build agent at a time. Personal builds as well as regular builds will all be enqueued and wait for the lock on the shared resource to be released, even if one of the build agents is idle.
Note that it is possible to await multiple locks in a build configuration.
Working with custom resource values
While working with simple named resources may be useful, it may be worth having a custom resource value for every lock acquired. Imagine we have a build which has a custom build step running load tests on a server. Wouldn’t it be nice if we could limit the number of concurrent build based on the number of load test targets available and be able to get the URL to one of these servers in our build? We can using a set of custom resource values.
When creating a new Shared Resource, we can specify the type as a resource with custom values:
In our build, we have several options available to acquire a lock on this shared resource. We can lock on any value that is not locked, we can lock on all values or on a specific value. For our purpose of needing a dedicated load testing machine, we can opt to lock on any available value.
Once added, we will not be able to run more builds concurrently than the number of available values in our custom shared resource. If we need the value of the locked value during our build, we can read it using the %teamcity.locks.<lockType>.<lockName>% variable, %teamcity.locks.readLock.RunTestsAgainstServer% in our case. For example when working with build parameters, we can use the lock value as a custom parameter:
Where can I manage shared resources?
So far we’ve seen that shared resources can be added for every project separately and used in that project’s build configurations. With TeamCity 8.0, we’ve introduced project groups which allows us to group several projects under a project group. Shared resources can be added to both the project group as well as the individual projects. Locks can be acquired for every build configuration in a project and/or project group, making it possible to have shared resources across projects as well.
Do let us know your thoughts via the comments below. Dive in, experiment and… happy building!
Subscribe to Blog updates
Thanks, we've got you!
Introducing AWS Connection: A New Approach for Working With AWS in TeamCity
We’ve introduced a new feature called the AWS Connection, which makes it possible to work with AWS credentials more flexibly and safely.
TeamCity 2022.10: Sakura UI, Improved Integration With Amazon AWS, Out-of-the-Box HTTPS Support, Google OAuth Authentication, and More.
TeamCity 2022.10 comes with the Sakura UI enabled by default, introduces many cloud-related features including the AWS credentials management system and out-of-the-box terminal access to AWS EC2-based build agents, and adds important improvements to VCS integrations. Server administrators will love the new streamlined way to implement HTTPS for on-premises TeamCity installations. Also, a new authentication module for using Google accounts and a number of long-requested features have been implemented.
Transition to native Git in TeamCity 2022.04 brings 10x fetch time reduction to IntelliJ Platform
Starting from version 2022.04, TeamCity switched to native Git on the server side for Git VCS connections. The switch should positively impact both performance and overall experience of working with Git repositories on the TeamCity server side.In this blog post, we’ll talk about the reasons for the …
TeamCity 2022.04: Parallel Tests, Improved Integration with Amazon AWS, Built-in Code Quality Analysis, Updated VCS Integrations, and more
TeamCity 2022.04 introduces the new Parallel Tests feature that delivers an extraordinary speed boost when testing your projects. New powerful code analysis features will cut the time you spend on code reviews and increase the maintainability of your projects. Users who run their builds in the cloud…