TeamCity
Powerful CI/CD for DevOps-centric teams
TeamHackCity: What a bunch of TeamCity Developers got up to in 2 weeks
In September, after having released TeamCity 7.1 and before getting to discussing the plans for version 8.0 we took a pause and organized an internal TeamCity plugin contest for TeamCity team only. A two-week hackathon where all TeamCity developers could write anything TeamCity-related, try ideas they had in mind for a long time but were not able to spend time on them. It was not the first time we do such a contest in the team and we expected to see some really interesting ideas. Without a doubt, I can say that the results exceeded our expectations, in terms of quality of ideas and implementation. So let’s see what we were able to create during these two weeks of coding.
Remote Debugging of Tests on Agent
The idea was to simplify the process of remote debugging of tests running on an agent. First of all, it would be great to avoid changing any configuration settings, JVM arguments, etc., and start remote debug right from the IDE. Also it would be useful to have a quick restart of the debug session without waiting for the start of the next build (how often during a debug session we realize that we missed some stage and it is already too late?).
As a result of these thoughts an experimental feature emerged in TeamCity trunk (future version 8.0). It let’s you start debug session right from IntelliJ IDEA for the builds based on IntelliJ IDEA project build steps. Other build types are not supported now.
Here are some screenshots to demonstrate how it works.
Start a remote debug pretty much the same way you start a local debug session:
Select a build configuration where to start your build:
Debugging:
Debug finished, but we can restart it if we want:
Hope this gives you an idea of this feature. You’ll see it in the first EAP build of TeamCity 8.0 which should be available in a few weeks.
Author: Maxim Manuylov
Availability: the first TeamCity 8.0 EAP build.
Flaky Tests Analyzer
A “flaky test” is a one which fails and passes at seemingly random order. There can be some concurrency issues, or the test can be environment sensitive and its failures seem random because it starts on different agents. So the purpose of this plugin is to identify flaky tests in a project and then analyze their failures and detect cases when a test is environment specific. To identify a flaky test a number of heuristics can be used. For example, if a test failed in a build without changes, and it did not fail before, it is clear that the test is suspicious — either flaky or environment specific.
This plugin adds “Tests analysis” tab on the project page. Here is a screenshot to give you an idea:
In this specific case the analyzer diagnosed a test as flaky because it failed in a build without changes. It has also determined that the test fails on two agents only and only in one configuration, which runs tests against Oracle database. Not so bad for a start.
If you want to try this plugin, see how to install it in this comment.
Author: Maxim Podkolzine
Compatibility: TeamCity 7.1.x and newer.
Source code: link
VCS Users Sync
In order to start receiving notifications from TeamCity, user account needs to be created on the server. Wouldn’t it be great if a committer to some repository received a notification about build failure even if he/she does not have a TeamCity account or does not even know about the build server existence? So here comes another plugin — VCS users sync. It watches for VCS changes detected in TeamCity and creates a user account on TeamCity server. If committer username contains email (which is often the case for Git or Mercurial), this email will be set into the user profile. To match TeamCity user account to username in version control, plugin automatically configures VCS usernames for the user account. Then, once a build fails, user will receive a notification from the TeamCity server, although he/she might never seen the server before.
Author: Yegor Yarko
Compatibility: TeamCity 7.1.x and newer.
Source code: link
Artifacts Torrent Tracker
Some time ago we discussed how we could speedup downloading of artifacts from the TeamCity server, especially for the cases when the server is used from different locations (like at JetBrains). The slow artifact downloads is an often pain for our customers. So we discussed different approaches, like mirroring of some artifacts to different hosts located near the user who wants to download them, and came up with the idea to use BitTorrent protocol for this task. Let me explain it a bit in case you’re not familiar with BitTorrent.
With BitTorrent protocol there must be a tracker whose main purpose is to be a mediator between clients (peers) downloading and sharing some content. If a client has some file (such clients are called seeders) and wants to share it, it connects to the tracker and tells tracker some meta information (hash of the file, name, own IP, and other details). Then if another client wants to download file it comes to the same tracker and tells it the hash of the file it wants to download (such client is called leech). The tracker tells the leech addresses of all of the seeders, then leech connects directly to the seeder(s) and downloads the file. When the file is downloaded this leech becomes a seeder because it now also has a file to share. The meta information about the file as well as the address of the tracker is stored inside a special .torrent file. All of the parties like seeders, leechers and tracker must know about the torrent file to be able to share the content.
So we wrote a Artifacts mirror plugin which turns TeamCity into a torrent tracker for all the published artifacts. When an artifact is published, TeamCity creates a .torrent file and starts tracking all the seeders and leechers with this .torrent. But it would not make much sense to stop here, as we also need seeders to make the whole thing work efficiently. So not only TeamCity works as a torrent tracker, it also starts seeding all the artifacts with .torrent files. So now, when you open .torrent file downloaded from the TeamCity server in your BitTorrent client you’ll always see at least one seeder – TeamCity server.
Soon we realized that to speedup download we need more seeders. But who else could be a seeder for an artifact? The agent that produced it of course! So we turned agents to seeders for the artifacts they published and it brought the download speed to a comfortable level. If many users download same artifacts they quickly become seeders themselves and this improves download speed dramatically, and as a side effect lowers the TeamCity server load. Good news is that the plugin is compatible with TeamCity 7.1.2, so nothing should prevent you from trying it right away!
Authors:Pavel Sher, Maxim Podkolzine, Viktoriia Petrenko
Compatibility: TeamCity 7.1.2 and newer.
Source code: link
Deployer Plugin
The next one is Deployer plugin aimed to simplify publishing of artifacts from TeamCity agent to various places, like FTP, SMB share, Tomcat, SSH. Once installed the plugin adds several special types of build steps for different deployment targets.
This plugin should greatly simplify this tedious work of writing your own deployment tasks or scripts and it is compatible with the current TeamCity version – 7.1.x, so you can easily install it and try.
Author: Nikita Skvortsov
Compatibility: TeamCity 7.1.x and newer.
Source code: link
URL Build Trigger
URL build trigger plugin was already available for quite some time. It automatically adds new build to the build queue when the content returned by a specified URL changes. During the contest a number of issues was fixed in it. In particular it now supports ETag header. If you’re using this plugin consider upgrading to a newer version.
Author: Viktoriia Petrenko
Compatibility: TeamCity 6.0.x+
Source code: link
TeamCity Data Directory Browser
This feature is quite simple and will be bundled with TeamCity 8.0. It allows system administrator to browse files under the TeamCity data directory and edit them right from the web interface. It can be useful for modifying different settings that do not have user interface in the TeamCity.
Author: Maxim Podkolzine
Availability: TeamCity 8.0
That’s it! And your feedback is welcome!
Of course two weeks is not too much time for every idea to become something usable or even prove its validity. Among other things we also tried to implement a matrix builds feature, played with Gerrit code review system to try some things like displaying build status inside Gerrit interface, auto merging of commits on server based on builds status, and others. We hope that some of these ideas eventually materialize in one of the future versions of TeamCity.
Having tried writing plugins for TeamCity we’ve identified some pain-points that TeamCity plugin writers have: lack of documentation with samples and template projects often makes it difficult to start. So, expect to see improvements in this area in the nearest future.
As to the mentioned features and plugins, their future depends heavily on your feedback. If you decide to try some of these plugins and features, please do not hesitate to contact us with your thoughts. Whether you like them or not, your feedback is very much appreciated.
BTW, most of these plugins are open source, so contributions are welcome too!
Happy building!
Subscribe to TeamCity Blog updates
