TeamCity Take on Build Pipelines

Although it is possible to setup  build pipelines in TeamCity, you will not find the term “build pipeline” in the TeamCity web interface or even in our documentation. The reason is that we have our own term: “build chain“, which we feel describes TeamCity’s build-pipeline-related features better.

First of all, while the word “pipeline” does not impose sequential execution, it still implies it. At the same time, in TeamCity a build chain is a DAG – directed acyclic graph, and as such a traditional build pipeline is just a special case of the TeamCity build chain. But there are other differences as well.

Source code consistency

Source code consistency implies that all builds of a build pipeline will use the same source code revision. Traditionally, when build pipelines are described, not much attention is paid to this problem. After all, if someone needs consistency, it can be achieved by sharing data between builds using a file system or by tagging the source code in the repository and then fetching the same tag in all builds.

But from our experience in the majority of cases users need consistency for all of the builds participating in the pipeline. For instance, if you build parts of your project on different machines in parallel and then combine them in the last build, you must be sure that all of the parts have used the same source code. Actually, any time when you need to combine results of execution of several parallel builds, you need to be sure all these builds have used the same source code.

You can spend time and achieve this consistency somehow via build scripts, but since many developers face the same problem, obviously this is the area where CI tools should help. TeamCity provides source code consistency for all builds in the chain automatically. It does it even if the build chain uses different repositories and even if the builds use repositories of different types – Git, HG, SVN, Perforce, TFS.

Triggering

We also thought that traditional approach to triggering – when the next build of the pipeline is triggered after the previous one is finished –  is limiting, especially in terms of optimizations that CI system are capable of. If the whole build pipeline is known from the beginning, then it is possible to reduce build time by making some optimizations. But if the builds in the pipeline are triggered one by one, then it will always take the same time to finish completely.

For instance, say we have a pipeline C -> B -> A, where C is the first build and A is the last:

cba_chain

If we trigger the whole pipeline at once, all builds (C, B and A) will be added into the queue and, if the system already has a finished build which used the same source code revision as build C will, the build in the queue can be substituted with such finished build reducing the total build time of the pipeline. This is exactly what TeamCity does for all build chains while they sit in the queue.

But if we trigger the builds one by one, i.e C is the first, then upon finishing C we start B, and so on, then there are much fewer opportunities for a CI system to optimize something. If something triggered C, the CI system must obey it , because this is how the user configured it. Even if there are finished builds identical to C (run on the same sources revision), at this point  the CI system cannot avoid running C, as upon finishing C, build B and eventually A must be triggered as well. If we decide to not run C,  we will not run the whole pipeline, which is definitely not what the user expects.

So this is the reason why we always recommend to avoid triggering builds one by one. If the build chain is fully automated (which is what we all try to achieve these days), then ideally there should be only one trigger – at the very last build of the chain (in terms of TeamCity – the top build, build A in the example above), which will trigger all the builds down the pipeline. Fortunately, TeamCity triggers are quite advanced, you can configure them to trigger the whole chain if a change was detected in a part of it. This has an important benefit as the set of triggers that you need to maintain can be drastically decreased.

According to the data collected from our own build server, due to this and other optimizations performed in the build queue, TeamCity greatly reduces the amount of work performed by agents daily:
queue_stats

Note that our server produces about 2500-3000 build hours per day, so if there were no optimizations like this, we’d have to add more agents, or our builds would be delayed significantly.

Data sharing

Another important aspect is how you pass data from build to build in a build chain. Obviously you can use artifacts for this task: the previous build produces artifacts and the next one uses them as input. But note that in many cases you cannot rely on the fact that the next build will be executed on the same machine as the previous one. Even if it does, a few other builds could be executed on this machine and they could remove the results that you wanted to share. Fortunately, publishing artifacts to TeamCity and then using artifact dependencies to retrieve them solves all these problems.

Besides,  in TeamCity you can also share parameters between builds. All builds in TeamCity publish their parameters upon finishing: system properties, environment variables, the parameters set in their build configuration at the moment of the build start, the parameters produced by the build with the help of service messages, etc. And all of them can be used in the subsequent builds of the build chain. For instance, this makes it possible to use the same build number for the whole build chain.

Find out more on sharing parameters.

Build pipelines view

The traditional approach to build pipelines implies that there is a dashboard showing all pipelines and changes which triggered them. Given that some of our customers have build chains consisting of hundreds of builds (we’ve seen chains with up to 400 builds) and in large projects there can be hundreds or even thousands of source code changes per day, it is obvious that a simple dashboard will not work.

TeamCity has the Build Chains tab on the build configuration level displaying an aggregated view of all of the chains where builds of this build configuration participated. It can surely be used as a dashboard to some extent, but with large build chains it quickly becomes unusable.

build_chains

Fortunately in TeamCity each build of the chain also shows the state of all of the builds it depends on. So by opening the build results of the last build in the chain (the top build) you can see the state of the whole chain: which builds are running, which builds failed, which artifacts were produced by each build, which tests failed for the whole chain, etc.

build_results_deps

Summary

Hopefully this article sheds some lights on to how build pipelines can be configured and used in TeamCity and why they work this way. To sum up:

  • a build pipeline in TeamCity is called a build chain
  • source code synchronization in TeamCity comes for free, you don’t need to do anything to achieve it
  • the data from build to build can be passed with the help of artifacts or parameters
  • if possible, avoid one-by-one triggering: the bigger the parts you trigger, the better
  • monitor the build chain results using the results page of the last build in the chain or using the Build Chains tab
Posted in Blogroll, Features, FYI, How-To's, Tips&Tricks | 4 Comments

TeamCity 10 (code-named Indore) EAP is open

Greetings, Everyone!

The TeamCity development team is proud to announce the Early Access Program (EAP) for TeamCity 10.0. Following the tradition of naming our version after Indian cities, this one is code-named Indore.

The preliminary date for TeamCity 10.0 is June 2016. If you want to be the first to learn about new features in 10.0, keep watching for our EAP builds: we expect to publish them more often now.

Indore 10.0 EAP1 (build 40941) main features

  • TeamCity now can integrate with GitHub, GitHub Enterprise or BitBucket Cloud: with the integration configured, setting up projects which use GitHub or BitBucket Cloud is a piece of cake.
  • Build 40491 comes with cross-platform TFS integration: to work with TFS you no longer need to install a TeamCity server on a Windows machine.
  • This EAP comes with reworked user interface for Amazon EC2 cloud integration and some other new features and improvements in both Amazon and VMware vSphere plugins.
  • Those of you who use several TeamCity severs will appreciate the new TeamCity projects pop-up: written from scratch, it provides better performance and allows browsing projects and build configurations on multiple servers.
  • If you use Perforce with streams, try out our experimental feature branches support.
  • and much more!

As usual, our EAP builds come with 60-day Enterprise evaluation license.
See our release notes for details, and, if you want to test the newest TeamCity features, download this EAP version. We recommend installing the EAP version on a trial server.

Give this version a try and share your feedback, as we always listen to what you have to say!

Help us improve the new TeamCity and happy building!

Posted in Blogroll, Bugfix, EAP, Features, News & Events | 8 Comments

TeamCity Digest #4

teamcity-digest

In this fourth TeamCity Digest we want to draw your attention to several publications of the last couple of weeks. We’ve also decided to introduce a new section in our digests on the  TeamCity plugins recently added either by our users, or by our team members.

So read on and share your thoughts with us.

Matthew Adams of endjin shares his experience of using SpecFlow 2.0, NUnit3, Visual Studio and TeamCity.

The other blog post describes how TeamCity can be used to build and deploy the code and Sitecore items to a Sitecore instance when this instance and the TeamCity build server reside on different physical servers: Continuous Integration with Gulp and Sitecore Powershell Extensions Part 2 – TeamCity Integration.

Ahmed Gadir describes a process of using TeamCity to generate release notes on every build, with the user stories stored in Jira: How can TeamCity generate Release notes from JIRA during a build.

CTO of Keluro shares their story of setting up Jekyll website on Azure using TeamCity with triggers on Git repository branch changes: Hosting Jekyll website on Azure Web app using TeamCity automated deployment.

A couple of useful tips for those who use Powershell and TeamCity: TeamCity queries with powershell (for sake of healthy develop).

Recently added plugins:

  • TeamCity Theatre – a .NET application to monitor your TeamCity builds in your browser or on a TV
  • TravCorp radiator – JavaScript build radiator – improved fork of tc-radiate
  • SwiftCity – TeamCity REST API client in Swift
  • ILMerge Meta-Runner – meta-runner to run ILMerge (utility to merge multiple .NET assemblies into a single assembly)
  • TeamCityZen – provides a command line tool which can be run as a build step to send notifications based on @user mentions in change comments

Should you write or find a piece on TeamCity, or have a plugin which is not yet listed on our plugins page, feel free to share it with us.

P.S. No, we can’t resist sharing this cool hardware TeamCity ‘Deploy’ button with you: 

Read all TeamCity Digests.

Posted in Blogroll, Digest | Comments Off on TeamCity Digest #4

Webinar recording: Implementing Continuous Delivery with TeamCity at the Helm

The recording of our February 24 webinar, Implementing Continuous Delivery with TeamCity at the Helm, is now available on the JetBrains YouTube channel.

In this webinar Jeffrey Palermo of ClearMeasure goes over how to set up a continuous delivery pipeline managed by TeamCity, very similarly to how Microsoft’s NuGet team manages NuGet.org. The same chain of tools is used, and the same techniques. You will see, in detail, how to configure TeamCity to run a fully-capable deployment pipeline complete with unit tests, integration tests, acceptance tests, staging, deployment smoke tests, and gated & approved production deployments.

Below is the precise timeline of the webinar and select Q&A.

Continue reading

Posted in Partner, Webinar | Comments Off on Webinar recording: Implementing Continuous Delivery with TeamCity at the Helm

Implementing Continuous Delivery with JetBrains TeamCity at the Helm, February 24th Webinar

Join us Wednesday, February 24th, 15:00 – 16:00 GMT (10:00 AM – 11:00 AM EST) for our free live webinar, Implementing Continuous Delivery with JetBrains TeamCity at the Helm with Jeffrey Palermo of Clear Measure.

teamcity_webinar_Implementing_Continuous_Delivery_with_JetBrains_TeamCity_at_the_Helm

TeamCity has long managed Continuous Integration workflows. And it also plays well in the DevOps world. DevOps is a term representing the convergence of development and operations. Both the development and operations communities like to own the term, so it has accumulated multiple meanings. With the Agile Manifesto, developers have jettisoned annual software releases for processes that allow releasing software multiple times per day. To accomplish this, teams use rigorous suites of automated tests to ensure that every change to the software is a solid step forward rather than an accidental step backward.

In this webinar, you will learn how to set up a continuous delivery pipeline managed by TeamCity, very similarly to how Microsoft’s NuGet team manages NuGet.org. We will use the same chain of tools, and the same techniques. You will see, in detail, how to configure TeamCity to run a fully-capable deployment pipeline complete with unit tests, integration tests, acceptance tests, staging, deployment smoke tests, and gated & approved production deployments.

Space is limited, please register now. There will be an opportunity to ask questions during the webinar.

About the presenter:

Wes McClureJeffrey Palermo is a Managing Partner and CEO of Clear Measure, an expert software engineering firm based in Austin, TX with employees across North America. Jeffrey has been recognized as a Microsoft MVP for 10 consecutive years and has spoken at national conferences such as Tech Ed, VS Live, and DevTeach. He’s the purveyor of the popular Party with Palermo events and is the author of several books, articles, and hundreds of pithy Twitter quips. A graduate of Texas A&M University, an Eagle Scout, and an Iraq war veteran, Jeffrey likes to spend time with his family of five out at the motocross track.
Posted in Partner, Webinar | 2 Comments

Live webinar: YouTrack + Upsource + TeamCity Come Together

Notice: this post is originally published in the Hub blog. However, your comments are welcome here as well.

If you are using more than one team tool from JetBrains, or planning to start,  you definitely need to  join us Wednesday, February 17th, 15:00 – 16:00 GMT (10:00 AM – 11:00 AM EST) for our free live webinar with Dmitri Nesteruk.

hub_webinar_YouTrack+Upsource+TeamCity_Come_Together_2

In this webinar, we are going to demonstrate:

  • How our teamware tools can be connected to a single entry, user and permission management point (Hub).
  • How to manage users, groups and permissions in Hub.
  • How to connect different tools to one another.
  • The benefits that the various directions of integration offer to teams using YouTrack, Upsource and TeamCity together.

Space is limited, please register now.

This webinar is geared towards developers of different proficiency regardless of programming language of choice. During the webinar there will be an opportunity to ask questions. The recording will be available after the webinar.

Posted in Uncategorized | Comments Off on Live webinar: YouTrack + Upsource + TeamCity Come Together

TeamCity 9.1.6 is out now

Greetings, everyone!

Today we’ve published another bugfix release, TeamCity 9.1.6, resolving over 50 issues.

Build 37459 mostly fixes minor problems – the full list is available in our Release Notes. We hope that a number of enhancements in our NUnit 3 support and fixes for the irksome issues with NuGet should further improve TeamCity experience for .Net users.

As with any bugfix release, with TeamCity 9.1.x versions you can freely upgrade or downgrade, so be sure to check the Upgrade Notes, and download and install the latest TeamCity build!

Stay tuned for our latest news: we’ve been putting a lot of effort into the new TeamCity version, and it won’t be long before we announce the EAP for the jubilee TeamCity 10!

Happy upgrading and building!

Posted in Bugfix, EAP, News & Events | Comments Off on TeamCity 9.1.6 is out now

TeamCity Digest #3

We are continuing the series of our TeamCity Digests with this third compilation.

As before, we are glad to share some content that attracted our attention recently. Hope you find it useful as well.

Boris Modylevsky proposes to add some fun to the software development process by utilizing some of the TeamCity plugins: How to add some fun to software development?

Dan from Xamarin shares his recipe on how to achieve on-commit deployments for an iOS app, in just 1 day and 10 simple steps: Upgrading to continuous deployment with Xamarin, TeamCity, and HockeyApp.

Richard Macaskill of Redgate published a piece on using TeamCity for release management with a database deployment task: Wherever I Lay My Hat: Release Management in TeamCity with Redgate DLM.

Not a very fresh article, but something that caught our attention: A Slice of Groovy’s Hip Use of JFrog Artifactory & Bintray, and of Gradle and TeamCity. Describes how Groovy fully automates its release process using TeamCity as part of the solution.

Howard van Rooijen, our friend and partner, created an ILMerge Meta-Runner for TeamCity, which he described in his recent blog post.

Another example of TeamCity as an integral part of the continuous delivery process is described in this blog post: Continuous delivery with FAKE, Paket and TeamCity.

There is also this video on setting up a new continuous integration module within a development environment of Kentico, creators of the all-in-one CMS and e-commerce platform, with TeamCity in the mix: Kentico 9: Continuous Integration.

Ted Neward came up with a peculiar way of applying TeamCity to create new blog entries in his blog infrastructure, and he describes it in his DevOps-ing the Blog article. And in DevOps-ing the Blog, Part 2 he talks about using the TeamCity changes file to look up the commit message to post to Twitter and LinkedIn through the use of a Ruby script.

See all the TeamCity Digests.

Posted in Blogroll, Digest, FYI | Comments Off on TeamCity Digest #3

Brand new TeamCity 9.1.5 is here

Greetings, everyone!

We are happy to announce TeamCity 9.1.5 update! Have you noticed our new site featuring bright colors and an uncluttered design?

icon_TeamCity

We’re so excited about our new logo and icon and hope you are too! But new logo or old, TeamCity is still your 24/7 build engineer and will continue the drive to being the best product for you.

Build 37377 fixes over 80 problems, the most important being NuGet 3.2 compatibility fixes and bundled tools updates: TeamCity now comes with the latest IntelliJ IDEA 15.0.x and ReSharper 10.0.2.

Another change concerns the way the Gradle runner accesses TeamCity system properties: the System.properties syntax introduced in TeamCity 9.1.2 will no longer be functional; the new recommended way is to reference TeamCity properties similarly to Gradle properties. See details in our documentation.

We greatly appreciate our users reporting problems with TeamCity, fixing which helps us to make the product better. But at times the efforts of our users truly exceed our expectations! Just as we were about to release version 9.1.5, we were presented with a very subtle bug with notifications. Obviously, it had taken our users loads of time to reproduce the problem, so we decided we could not but fix it as a holiday gift to our users, all thanks to the guys from ok.ru!

Our release notes list all fixes, check them out and download TeamCity 9.1.5. now!

As usual you can freely upgrade-downgrade within all TeamCity 9.1.x releases, so check the upgrade notes and don’t hesitate to upgrade!

Happy building!

Posted in Blogroll, EAP, Features, News & Events, Tips&Tricks | 7 Comments

Webinar Recording: Continuous Integration with JavaScript

The recording of our December 2 webinar with Wesley Higbee, Continuous Integration with JavaScript, is now available on JetBrains YouTube channel.

In this webinar you will learn about building and deploying JavaScript applications with TeamCity. Wes explains why we need CI and then covers the whole workflow for the modern JavaScript application: building the app, linting, unit testing and deployment. He uses tools like Gulp, SystemJS, TSLint and Karma during his demo, but in the same way you can configure any command-line tools to run on TeamCity.

Continue reading

Posted in Webinar | Comments Off on Webinar Recording: Continuous Integration with JavaScript