TeamCity Torrent Plugin

Today we have some interesting news for you – the Torrent plugin for TeamCity!

With this plugin, users and build agents can download TeamCity build artifacts faster, especially in a distributed environment.

A bit of the BitTorrent protocol details

For those of you who are not familiar with BitTorrent – this is a peer-to-peer protocol. It means there is no single server for shared files. Instead, the files can be downloaded from peers that already have these files or parts of them.

Still, a server is required to find peers. This server is called a torrent tracker. To download a file, a peer must have a torrent file describing the required file. With this torrent file, the peer goes to the torrent tracker, and the tracker provides addresses of other peers in the network sharing the file.

A peer that does not have the file yet but wants to download it is called a leech. A peer that has the file and wants to share it is called a seeder.

Applying it to TeamCity

So what TeamCity Torrent plugin does – it turns TeamCity into a torrent tracker for all published artifacts greater than the specific size: by default, the artifacts of size less than 10 megabytes are not available via the BitTorrent protocol as it is not very efficient for small files. Additionally, the TeamCity server becomes a seeder for these artifacts. So for every artifact there is always one seeder in the network.

On top of that, when an agent decides to download an artifact dependency, it can choose to use the BitTorrent protocol instead of HTTP if there is more than one seeder available. Once the agent downloads the file, the agent itself automatically becomes a seeder for this file. When the agent publishes the artifact to the TeamCity server, both the server and agent start seeding it. So if you use TeamCity artifacts a lot, eventually you’ll end up with quite a number of agents seeding artifacts. Not only does this increase the speed of the artifacts delivery to agents, but it also reduces the load on the TeamCity server.

Finally, the users who have Torrent clients installed can also download files from TeamCity via the BitTorrent protocol.

Setting up the plugin

The plugin is compatible with version 8.1, so to try it, you need to install the latest TeamCity EAP build. Grab the plugin here and install it as usual.

Once you restart the server, a new link, Torrent Settings, will appear in the Administration area. The plugin is disabled by default. You can enable it on this page:

Verifying the plugin works

If the plugin works correctly and you checked both options on the Torrent settings page, then once a large enough artifact is published, you should see the following icon near the artifact name:

Clicking this icon should start your favorite torrent client. And this is what it looks like for some of the widely used artifacts on our server:

If a build has an artifact dependency on some artifact, you should see the attempts to download the artifact via torrents in the build log:

Please try this plugin and let us know what needs to be improved!
Happy building!

This entry was posted in EAP, Features, FYI, How-To's, News & Events and tagged , , , . Bookmark the permalink.

3 Responses to TeamCity Torrent Plugin

  1. Pingback: TeamCity 8.1 EAP (build 29581) is here! | JetBrains TeamCity Blog

  2. Igor Kalders says:

    What about a remote agent publishing build artifacts? Do they still need to be uploaded to TC first, or can they be seeded right away?

  3. Sergey Pak says:

    We don’t use BitTorrent protocol to publish artifacts to server.

    However, as of 8.1, all artifacts that agent produces during build execution are stored in the local (agent’s) artifacts cache. And agent seeds its artifacts directly from cache.

    In two words, artifacts can be available before they are uploaded to TC if you have a corresponding torrent file. But TC cannot reuse these artifacts before they are completely uploaded to server.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>