Bica Studios Case Study

This guest post is brought to you by Bica Studios engineering team. It was initially set up as an interview, but it has outgrown the format to become a stand-alone blog post.

What does your company do?


BICA STUDIOS is a team that crafts the best gaming entertainment to bring enduring and inspiring new brands.

We develop great games as a service through a free-to-play approach, allowing millions of players to access new interactive experiences, exciting worlds and know charismatic characters with no initial monetary barriers. Our mission is to make a stand on the entertainment market by exploring current and alternative revenue models.

What obstacles or opportunities was your organization faced with?

From a business point of view, the company has to deal with market saturation and high UA costs. Another strong concern was the validation of proper use of strong engagement mechanics alongside meaningful and inspiring content.

As far as technology, our main concern is being able to deal with and react to change as quickly as possible.

What was your experience like before TC?

Our building pipeline is a complex one.

Having a mobile game as our main product, we obviously target all platforms: iOS, Android and Windows Phone. On the Android platform we target 3 different stores: Google Play Store, Amazon and Aptoide. Each build has a debug and store version.

So: 1 game, 3 platforms, 5 different builds, 10 different setups.

With manual specific configurations for every step, you can guess the amount of time that we take to handle this process and why we call it a ‘release day’ instead of ‘release morning’ or ‘release hour.’

Enter Unity 5 and build setup configuration through editor scripts. Several manual steps could be coded and automated on the build process, reducing a lot of human errors and saving a little bit of time. However, build time was still enormous and demanded at least one full-time dedicated individual to handle.

We decided it was time to a look into the world of Continuous Integration and try to lift the burden of building the project off our development team.

Continue reading

Posted in Blogroll, Case study, FYI | 4 Comments

TeamCity on Docker Hub – it’s official now!

As the popularity of Docker is growing exponentially, Docker Hub has become a huge software distribution platform. Users publish their own images, companies start their official repositories, and everyone is happy.

Up until now JetBrains wasn’t part of this. Then we noticed over 300 images related to TeamCity server, created by enthusiastic Docker Hub users. We decided to join the club and make support for this platform official.

We’ve published the latest version of our software as ready-to-use images and we are going to keep them updated as soon as new versions appear.

There are currently three official TeamCity images in Docker Hub:
1. jetbrains/teamcity-server
Use this image to run a fully functional, ready-to-use TeamCity server. There are multiple options and additional parameters available, which are summarized in the image description on its Docker Hub page.

Once the server is running, you will definitely need some build agents to do the actual work (building) for you. We’ve prepared two separate images for that.

2. jetbrains/teamcity-agent
This image is recommended for general Java development. Build agents created from this image will support agent-side checkout for Git, Mercurial, and SVN, any Java-related activity (as we provide OpenJDK installed), and a full set of features related to Java development.

3. jetbrains/teamcity-minimal-agent
As the requirements for build agents vary considerably, we have also created this minimalistic agent image which you can use for simple builds or as a base for your own images.

Take a look at the official JetBrains Docker Hub page for more details about our Docker images. Give them a try and let us know what you think!

Posted in Blogroll, FYI, How-To's | 12 Comments

TeamCity Digest #7

teamcity-digestWe are glad to present our seventh compilation of recent TeamCity-related articles and blog posts to you and we hope you will enjoy it. If you yourself stumble upon something interesting on the topic, or create your own piece on using TeamCity, please make sure to share it with us.

Darren Guy documents his experience setting up an automated build and deployment solution for Sitecore in a series of posts. Parts 1 and 6 focus specifically on TeamCity, other parts are linked to from each of these posts:
Building a Continuous Integration Environment for Sitecore Part 1 – Teamcity
Building a Continuous Integration Environment for Sitecore Part 6 – TeamCity project

Robert Firek from Codurance has put together this guide for creating a (somewhat simplified) continuous deployment pipeline based on Docker and TeamCity: Docker Meets Continuous Deployment.

Raygun’s Chief of Engineering describes their Continuous deployment process which lets them deploy 6 times per day. To achieve this, they use TeamCity and Octopus Deploy, among some other tools: Continuous Deployment: How To Make It Safer.

Visual Studio Team Services has released an update which can now consume TeamCity builds as artifacts in Release Management:

These two articles describe a Rails project continuous integration environment using TeamCity and Bitbucket. A curious note: the author focuses on using the default free edition of TeamCity which provides 20 build configurations and 3 build agents:
Deploy Your Rails Continuous Integration Environment With TeamCity and BitBucket (Part I: Initial Setup)
Deploy Your Rails Continuous Integration Environment With TeamCity and Bitbucket (Part II: Rails Integration)

Read this short post if you would like to avoid deployment on weekends and configure date/time triggers defined by a CRON expression in TeamCity accordingly: Run Web Application Deployment in TeamCity – Only on Weekdays.

See all TeamCity Digests.

Posted in Digest | Leave a comment

TeamCity 10 EAP 2 is out today

Greetings, everyone!

We are proud to announce that TeamCity 10.0 EAP 2 build 41463 has just been released!

One of the most exciting features of this EAP is the ability to create TeamCity projects and build configurations programmatically: using DSL based on the Kotlin language. You can now define TeamCity project settings using Kotlin, check them into your version control and TeamCity will execute your Kotlin script and produce projects and build configurations with the desired settings. See our Release notes for more information.

This EAP build also comes with improved algorithm for VCS changes polling. As you know, TeamCity polls repositories for changes according to the configured interval. Although this approach is quite reliable and easy to maintain, in case of large installations with thousands of VCS roots it imposes a great load on both the TeamCity server and VCS repository servers. To avoid this, you can set up a commit hook, but still you’d need to manually adjust checking-for-changes intervals in VCS roots, and to constantly monitor that the hook works. Fortunately, starting with this EAP, TeamCity will increase the checking-for-changes interval automatically, once it determines that changes were detected as a result of a commit hook. If TeamCity detects a non-working commit hook, the checking-for-changes interval will be decreased. Our Release Notes provide details.

Among other things, the latest TeamCity version is more pleasant to look at, as it comes with new refreshed icons in the Web UI! All the more reason to check out the full list of this EAP’s  great features, and give this version a test drive! Download  build 41463 and don’t hesitate to install it on a trial server.

All our EAP releases come with a  60-day Enterprise evaluation license, and we hope you’ll have something to say about the new version once you have tried this build.

Happy building!

Posted in Bugfix, EAP, Features, FYI, News & Events, Tips&Tricks | Leave a comment

TeamCity Digest #6

We continue to gatheteamcity-digestr various articles and blog posts related to TeamCity from social media and online resources. Thanks to those of you who send them to us via twitter and other channels.

Azure Application Insights has recently released support for Release Annotations, and Endjin – who are also our partners – have created a TeamCity MetaRunner which wraps Release Annotation Powershell Script and eventually allows passing build values from TeamCity into the script: TeamCity MetaRunner for creating Release Annotations in Azure Application Insights

Kalle Launiala from The Ball project shares their recent platform updates which also feature TeamCity in the heart of their automation process: Full Stack with Automated DevOps

Michael Hedgpeth describes the process of making the Chef pipeline a part of TeamCity build process: Chef Cookbook Builds in TeamCity.

Matt Richardson has published the New TeamCity Octopus Deploy Build Trigger Plugin. It allows you to trigger builds when a deployment to an environment is complete, a new release is created, or a new tentacle is added to the server.

A nice overview and step-by-step instructions for two ways of integrating TeamCity and Bitbucket server – the native Commit Status Publisher from JetBrains and TeamCity Stash Integration plugin: How to Integrate TeamCity and Bitbucket Server.

And here is a separate description of TeamCity-Stash integration from the author of the  TeamCity Stash plugin himself: TeamCity Stash Integration.

Sveinung Kval Bakken describes their setup of Google Test Lab in TeamCity, which allows them to automatically run fully parallelized tests on tens and tens of devices over a number of dimensions: Using Google Cloud Test Lab with TeamCity.

Microsoft announced their Visual Studio extension for deploying TeamCity build artifacts using Release Management in VS Team Services:

As it turns out, Stackoverflow also uses TeamCity, which they describe in ‘The Build System’ section of this post on the internals of Stackoverflow’s deployment process: Stack Overflow: How We Do Deployment – 2016 Edition.

This blog post goes over the steps to integrate TeamCity to Git feature branch workflow in Deveo, which is a code management and collaboration platform: TeamCity integration to feature branch workflow.

See all TeamCity Digests.

Posted in Blogroll, Digest | Leave a comment

TeamCity 9.1.7 Update

Greetings, everyone!

It’s been a while since our latest update, so today we are announcing TeamCity 9.1.7.

Build 37573 is a bugfix release addressing over 90 issues, see the full list in our Release notes.

As usual, TeamCity 9.1.7 Build 37573 uses the same data structure as the other 9.1.x builds enabling you to freely upgrade/downgrade within the latest released version, so no need to hesitate: download TeamCity 9.1.7 and upgrade.

Most likely this is the last update for TeamCity 9.1.

We are working on the next EAP for TeamCity 10, so watch for the new build!

Happy building!

Posted in Bugfix, EAP, News & Events, Tips&Tricks | 1 Comment

TeamCity brings Azure Resource Manager support


Microsoft has recently introduced a new way to deploy cloud resources, so currently there exist two Azure deployment models supported by TeamCity as two separate plugins.

The earlier Azure deployment model is in the feature freeze state and is now called Azure Classic. The same goes for the existing TeamCity Azure support, now called TeamCity Azure Classic plugin.

Azure Resource Manager is the new Azure deployment model, which has some advanced features and is now becoming popular. More and more users want to deploy cloud agents with it, so today we are glad to announce a new plugin, the Azure Resource Manager plugin for TeamCity.

Plugin Description

The plugin for TeamCity starts new cloud build agents from a generalized VHD image when all of the regular build agents are busy. It uses resource groups to deploy build agent resources and supports authentication with an Azure Active Directory (AD) application.

While starting a new virtual machine, the plugin creates a new resource group and deploys required resources into it. When a cloud agent is terminating, the resource group with the allocated resources is removed.

The latest plugin version is available for download to the TeamCity 9.1.x users as a separate plugin which can be installed manually.

Azure Authentication Settings

The plugin accesses the Azure Resource Manager by leveraging an Azure AD application. To create a new application, follow the steps described in the Azure documentation. Then open new portal, navigate to Subscriptions, select the target subscription and assign a Contributor role to your Azure AD application.

Note: To find your application, type the application name in the users search box, because originally that list contains only usernames, not applications.

Virtual Machines Preparation

The plugin supports generalized VHD images to start new TeamCity cloud build agents. Perform the steps below to prepare VHD images.

New Image Creation

Create a new virtual machine with the Resource Manager via the new portal or cli tools, and configure it properly. Then fulfill generalization according to the Azure documentation for Linux and Windows virtual machines. As a result, you will get a VHD image in your blob storage.

Migrating Specialized Classic VM Image

If you previously used a specialized image or Start/Stop behaviour for a cloud build agent, you must capture it now. To do that, prepare a virtual machine by removing temporary files and then make a new image capture. Please refer to the Azure documentation for Linux and Windows machines. Then follow the steps below to migrate a generalized classic VM image.

Migrating Generalized Classic VM Image

If you already have a generalized VHD image located in the classic storage account, just copy it to a storage account created with the Resource Manager. To do that, use Microsoft Azure Storage Explorer.

Network Settings

To start new virtual machines, ensure that you have a Virtual Network and Subnet created in the target Azure region. If you already have virtual machines deployed with the Resource Manager, you can use them; otherwise you must create a new one and configure it as follows:

  1. Open the Virtual Network creation widget on the new portal and press Create.
  2. Fill in the settings and select the Resource Group located in your target Azure region.
  3. Press Create and wait for the deployment completion.

Note: To access your virtual machines by a public IP address, please ensure that you have properly configured security group rules for your Subnet.

Configuring Agent Cloud Profile in TeamCity

Open TeamCity Agent Cloud configuration and create a new Azure Resource Manager profile. Fill in the AD application credentials from the classic portal: copy the tenant id, client id values, and create a new client secret. Then select your subscription and target region to deploy new TeamCity build agent resources:


Configuring Build Agent Images

Then add a new agent image in the cloud profile by pressing the Add image button and set configuration parameters:


Source image is a VHD blob URL of generalized Virtual Machine. Please note that currently the Azure Resource Manager allows creating OS disk images only in the original storage account where the source VHD image is stored. More details are available in the Azure forum request.

Name prefix will be used as a prefix for the resource group name. Note that you must use unique prefixes within your subscription.

Network settings allow using the preliminary created virtual networks in your target location and optionally creating a public IP address for build agent machines.

After the image is added and the cloud profile is saved, TeamCity does a test start for virtual agents to gather the agent configuration. Once the agents are connected, TeamCity stores their parameters to be able to correctly process build configurations-to-agents compatibility.


You are welcome to try the plugin for TeamCity, and your feedback will be really appreciated, so please share your thoughts in the comments to this blog post, in the forum or issue tracker.

Happy building in the cloud!

Posted in Features, FYI, How-To's, Tips&Tricks | Tagged , , , , | Leave a comment

AWS CodeDeploy Runner plugin

Good news for those of you who use both TeamCity and Amazon Web Services, and those who for now stick with only one of the above, but are looking for a good integration.

With the help of the new TeamCity AWS CodeDeploy Runner plugin, your application can now be automatically deployed directly to your fleet of EC2 instances, on-premises instances or both in the scope of the CodeDeploy application.

AWS CodeDeploy is a fully managed, high-scale deployment service that can work with any kind of deployable content like code, web and configuration files, executables, etc. and with almost any kind of target platforms.

Generally, any deployment consists of four main steps:

  • preparing deployable content
  • uploading it to a repository
  • triggering a deployment
  • viewing the deployment status

TeamCity AWS CodeDeploy Runner plugin helps to automate and simplify all the mentioned tasks by adding a build step which packages the build output as a revision, uploads it to an S3 bucket and registers it in CodeDeploy. For the registered revision, it can run a deployment and optionally monitor and show its status using the TeamCity progress and error reporting.

To set up a deployment, one basically needs to make the following five steps (assuming  the initial account set-up is complete):

  • set up target instances
  • plan a revision
  • set up a CodeDeploy application
  • add an AWS CodeDeploy step to a TeamCity build

Target instances

By default each deployment can target up to 50 instances which must be running an AWS CodeDeploy agent and either be tagged or belong to an Auto Scaling group. Amazon EC2 instances must be running with an IAM instance profile attached –  for the AWS CodeDeploy agent to verify the identity of the instance – and must belong to the same region as the CodeDeploy application.

AWS CloudFormation provides a template which creates and configures new, single Amazon EC2 instances based on Amazon Linux or Windows Server with the AWS CodeDeploy agent already installed. For more information on setting up both EC2 and on-premises instances, see Configure Instances.

Application revision

An application revision is an archive containing deployable files accompanied by the application specification file appspec.yml which is a YAML-formatted file used by CodeDeploy to determine where to place the application files on the target instance, and which lifecycle event hooks to run in response to deployment lifecycle events – that’s where most of the magic happens. The archive is uploaded to an existing S3 bucket and registered in a CodeDeploy application.

The TeamCity AWS CodeDeploy Runner accepts either a ready-made application revision (must contain appspec.yml) archive or a list of paths to package. For the latter Ant-style wildcards like dir/**/*.html are supported.

CodeDeploy application

A CodeDeploy application is a place where all the deployment resources are put together: the revision location, deployment group, deployment configuration and other settings.

An AWS CodeDeploy deployment group identifies a collection of instances based on their tags, Auto Scaling group names, or both. Multiple application revisions can be deployed to the same instance, and one and the same application revision can be deployed to multiple instances.

The deployment configuration specifies a set of deployment order rules and the success and failure conditions. It specifies the number (or percentage) of instances that must remain available at any time during a deployment. One may either use one of the default configurations, or create a custom one.

AWS CodeDeploy TeamCity build step

After one has a working CodeDeploy application,  he or she only needs to type in the settings such as  region, credentials, S3 bucket parameters, CodeDeploy application and deployment group name, and run a TeamCity build.

Screen Shot 2016-03-17 at 17.32.35

Posted in Blogroll, Features, FYI, How-To's | Tagged , , , | 4 Comments

TeamCity Digest #5

teamcity-digestPlease welcome the latest digest of new publications about TeamCity usage and different aspects of CI/CD processes that it covers.

We’ll start this one with the post by Paul Stovell in the official Octopus Deploy blog about how some of the new major changes in ASP.NET Core 1 and the dotnet CLI have impacted build and deployment pipelines for developers: ASP.NET Core 1 build & deployment pipelines with TeamCity and Octopus.

This post helps understand how to automate your .NET website deployment and relieve some of the stress induced by manual deployment. The toolset of the author is quite standard for the task: Visual Studio, TeamCity, and Octopus Deploy: How to level up your .NET website deployment process.

If you are a .NET developer who likes to obfuscate code and wants to automate the job, here is an example of how it can be done with the help of TeamCity: Using Dotfuscator to obfuscate code within TeamCity.

For a growing Docker users army out there, here is a detailed 3 piece guide for implementing Continuous Integration with TeamCity and Docker by Gabriel Schenker: Part 1, Part 2, Part 3.

A recent article in the Code Magazine by Jeffrey Palermo, who delivered a TeamCity webinar in February, covers many of the concepts of Continuous Delivery as a part of modern DevOps movement, and particularly focuses on how cloud technologies can help in the process: DevOps and Continuous Delivery: Made for a Cloud World.

To add some diversity to the .NET-biased content of this blog post, here is some practical information for PHP developers. If you are interested in getting code coverage out of the PHPUnit tests and being able to see them in your builds, you might find this post useful as it goes over some very basic build steps and configs needed to get PHPUnit tests run and code coverage generated: Setting up Code Coverage on TeamCity 9.1.

If you have published an article, a blog post, a video or some other form of content on the subject of using TeamCity, or know someone who did, we’ll  appreciate if you make sure it reaches our attention: such content will be promoted to a broader audience of TeamCity users.

Access all TeamCity Digests.

Posted in Blogroll, Digest, FYI | Leave a comment

TeamCity Take on Build Pipelines

Although it is possible to setup  build pipelines in TeamCity, you will not find the term “build pipeline” in the TeamCity web interface or even in our documentation. The reason is that we have our own term: “build chain“, which we feel describes TeamCity’s build-pipeline-related features better.

First of all, while the word “pipeline” does not impose sequential execution, it still implies it. At the same time, in TeamCity a build chain is a DAG – directed acyclic graph, and as such a traditional build pipeline is just a special case of the TeamCity build chain. But there are other differences as well.

Source code consistency

Source code consistency implies that all builds of a build pipeline will use the same source code revision. Traditionally, when build pipelines are described, not much attention is paid to this problem. After all, if someone needs consistency, it can be achieved by sharing data between builds using a file system or by tagging the source code in the repository and then fetching the same tag in all builds.

But from our experience in the majority of cases users need consistency for all of the builds participating in the pipeline. For instance, if you build parts of your project on different machines in parallel and then combine them in the last build, you must be sure that all of the parts have used the same source code. Actually, any time when you need to combine results of execution of several parallel builds, you need to be sure all these builds have used the same source code.

You can spend time and achieve this consistency somehow via build scripts, but since many developers face the same problem, obviously this is the area where CI tools should help. TeamCity provides source code consistency for all builds in the chain automatically. It does it even if the build chain uses different repositories and even if the builds use repositories of different types – Git, HG, SVN, Perforce, TFS.


We also thought that traditional approach to triggering – when the next build of the pipeline is triggered after the previous one is finished –  is limiting, especially in terms of optimizations that CI system are capable of. If the whole build pipeline is known from the beginning, then it is possible to reduce build time by making some optimizations. But if the builds in the pipeline are triggered one by one, then it will always take the same time to finish completely.

For instance, say we have a pipeline C -> B -> A, where C is the first build and A is the last:


If we trigger the whole pipeline at once, all builds (C, B and A) will be added into the queue and, if the system already has a finished build which used the same source code revision as build C will, the build in the queue can be substituted with such finished build reducing the total build time of the pipeline. This is exactly what TeamCity does for all build chains while they sit in the queue.

But if we trigger the builds one by one, i.e C is the first, then upon finishing C we start B, and so on, then there are much fewer opportunities for a CI system to optimize something. If something triggered C, the CI system must obey it , because this is how the user configured it. Even if there are finished builds identical to C (run on the same sources revision), at this point  the CI system cannot avoid running C, as upon finishing C, build B and eventually A must be triggered as well. If we decide to not run C,  we will not run the whole pipeline, which is definitely not what the user expects.

So this is the reason why we always recommend to avoid triggering builds one by one. If the build chain is fully automated (which is what we all try to achieve these days), then ideally there should be only one trigger – at the very last build of the chain (in terms of TeamCity – the top build, build A in the example above), which will trigger all the builds down the pipeline. Fortunately, TeamCity triggers are quite advanced, you can configure them to trigger the whole chain if a change was detected in a part of it. This has an important benefit as the set of triggers that you need to maintain can be drastically decreased.

According to the data collected from our own build server, due to this and other optimizations performed in the build queue, TeamCity greatly reduces the amount of work performed by agents daily:

Note that our server produces about 2500-3000 build hours per day, so if there were no optimizations like this, we’d have to add more agents, or our builds would be delayed significantly.

Data sharing

Another important aspect is how you pass data from build to build in a build chain. Obviously you can use artifacts for this task: the previous build produces artifacts and the next one uses them as input. But note that in many cases you cannot rely on the fact that the next build will be executed on the same machine as the previous one. Even if it does, a few other builds could be executed on this machine and they could remove the results that you wanted to share. Fortunately, publishing artifacts to TeamCity and then using artifact dependencies to retrieve them solves all these problems.

Besides,  in TeamCity you can also share parameters between builds. All builds in TeamCity publish their parameters upon finishing: system properties, environment variables, the parameters set in their build configuration at the moment of the build start, the parameters produced by the build with the help of service messages, etc. And all of them can be used in the subsequent builds of the build chain. For instance, this makes it possible to use the same build number for the whole build chain.

Find out more on sharing parameters.

Build pipelines view

The traditional approach to build pipelines implies that there is a dashboard showing all pipelines and changes which triggered them. Given that some of our customers have build chains consisting of hundreds of builds (we’ve seen chains with up to 400 builds) and in large projects there can be hundreds or even thousands of source code changes per day, it is obvious that a simple dashboard will not work.

TeamCity has the Build Chains tab on the build configuration level displaying an aggregated view of all of the chains where builds of this build configuration participated. It can surely be used as a dashboard to some extent, but with large build chains it quickly becomes unusable.


Fortunately in TeamCity each build of the chain also shows the state of all of the builds it depends on. So by opening the build results of the last build in the chain (the top build) you can see the state of the whole chain: which builds are running, which builds failed, which artifacts were produced by each build, which tests failed for the whole chain, etc.



Hopefully this article sheds some lights on to how build pipelines can be configured and used in TeamCity and why they work this way. To sum up:

  • a build pipeline in TeamCity is called a build chain
  • source code synchronization in TeamCity comes for free, you don’t need to do anything to achieve it
  • the data from build to build can be passed with the help of artifacts or parameters
  • if possible, avoid one-by-one triggering: the bigger the parts you trigger, the better
  • monitor the build chain results using the results page of the last build in the chain or using the Build Chains tab
Posted in Blogroll, Features, FYI, How-To's, Tips&Tricks | 6 Comments