TeamCity Torrent Plugin

Sergey Pak

Today we have some interesting news for you – the Torrent plugin for TeamCity!

With this plugin, users and build agents can download TeamCity build artifacts faster, especially in a distributed environment.

A bit of the BitTorrent protocol details

For those of you who are not familiar with BitTorrent – this is a peer-to-peer protocol. It means there is no single server for shared files. Instead, the files can be downloaded from peers that already have these files or parts of them.

Still, a server is required to find peers. This server is called a torrent tracker. To download a file, a peer must have a torrent file describing the required file. With this torrent file, the peer goes to the torrent tracker, and the tracker provides addresses of other peers in the network sharing the file.

A peer that does not have the file yet but wants to download it is called a leech. A peer that has the file and wants to share it is called a seeder.

Applying it to TeamCity

So what TeamCity Torrent plugin does – it turns TeamCity into a torrent tracker for all published artifacts greater than the specific size: by default, the artifacts of size less than 10 megabytes are not available via the BitTorrent protocol as it is not very efficient for small files. Additionally, the TeamCity server becomes a seeder for these artifacts. So for every artifact there is always one seeder in the network.

On top of that, when an agent decides to download an artifact dependency, it can choose to use the BitTorrent protocol instead of HTTP if there is more than one seeder available. Once the agent downloads the file, the agent itself automatically becomes a seeder for this file. When the agent publishes the artifact to the TeamCity server, both the server and agent start seeding it. So if you use TeamCity artifacts a lot, eventually you’ll end up with quite a number of agents seeding artifacts. Not only does this increase the speed of the artifacts delivery to agents, but it also reduces the load on the TeamCity server.

Finally, the users who have Torrent clients installed can also download files from TeamCity via the BitTorrent protocol.

Setting up the plugin

The plugin is compatible with version 8.1, so to try it, you need to install the latest TeamCity EAP build. Grab the plugin here and install it as usual.

Once you restart the server, a new link, Torrent Settings, will appear in the Administration area. The plugin is disabled by default. You can enable it on this page:

Verifying the plugin works

If the plugin works correctly and you checked both options on the Torrent settings page, then once a large enough artifact is published, you should see the following icon near the artifact name:

Clicking this icon should start your favorite torrent client. And this is what it looks like for some of the widely used artifacts on our server:

If a build has an artifact dependency on some artifact, you should see the attempts to download the artifact via torrents in the build log:

Please try this plugin and let us know what needs to be improved!
Happy building!

Comments below can no longer be edited.

6 Responses to TeamCity Torrent Plugin

  1. TeamCity 8.1 EAP (build 29581) is here! | JetBrains TeamCity Blog says:

    December 6, 2013

    […] start looking for this build’s download link: we are happy to say that the new version of the Torrent Plugin comes with the considerably improved performance and stability! It is readily available for […]

  2. Igor Kalders says:

    December 13, 2013

    What about a remote agent publishing build artifacts? Do they still need to be uploaded to TC first, or can they be seeded right away?

  3. Sergey Pak says:

    January 31, 2014

    We don’t use BitTorrent protocol to publish artifacts to server.

    However, as of 8.1, all artifacts that agent produces during build execution are stored in the local (agent’s) artifacts cache. And agent seeds its artifacts directly from cache.

    In two words, artifacts can be available before they are uploaded to TC if you have a corresponding torrent file. But TC cannot reuse these artifacts before they are completely uploaded to server.

  4. foo says:

    September 26, 2014

    Could you specify which ports need to be reachable for this to work ? With my current firewall settings on the server, I can download the torrent file, but the peers are unreachable.

  5. Sergey Pak says:

    September 26, 2014

    Both agents and server bind on the first available port in 6881-6889 interval.

    • foo says:

      September 26, 2014

      Just TCP, or UDP as well ?
      What about the tracker ?

Subscribe

Subscribe for updates