The official TeamCity CloudFormation template

As you might have noticed, there was recently an option added to the Get TeamCity page of our website: AWS. This lets you run TeamCity in AWS using the official CloudFormation template.

get-teamcity-aws

In this post, we will go over what’s under the hood of the template, and why it may save you some time and effort.

Usually, installing TeamCity on top of AWS is quite a time-consuming task.
It requires the following steps:

  • Setting up an external database,
  • Configuring the EC2 instance to run a TeamCity server,
  • Configuring it to then connect to the database,
  • Installing the TeamCity server,
  • Installing a TeamCity agent.

And then making the whole installation secure requires even more effort.

We have tried to ease this process and created an official CloudFormation template to run the TeamCity stack in AWS. Using this template lets you run all the above steps with just a single click. And should you decide to destroy the stack, CloudFormation also provides a super simple way to do it with just one click.

The template is located in the S3 bucket. The stack can be launched via the ‘Run on AWS’ button available on the TeamCity site.

The template provides several parameters:

aws-1
It takes about 15 minutes for the template to deploy the whole stack, the most time-consuming task being the RDS Database instance roll up. Once the deployment is ready, you will see the TeamCity server endpoint in the Output section which points you to your TeamCity installation.

aws-2

Just generate the root account and it’s ready to use.

So what is under the hood?

The TeamCity server runs on an EC2 instance with CoreOS Container Linux. The default agent runs as a separate container on the same instance. The external database is provided by an RDS MySQL instance. We decided not to introduce a custom AMI with TeamCity. Instead, we use the official Docker images with the TeamCity server and build agent.

The server and the database are placed in their own VPC which is completely secure. The DB allows only internal connections within the VPC. It’s only possible to connect to the Server via HTTP(s) or SSH.

How the server is running

There are several systemd services that prepare the LVM on the EBS volume to persist your data, create the file system, and run the latest official TeamCity Server and TeamCity Build Agent from the DockerHub images. Those services are linked to each other and roll the whole system back after an instance reboot or failure.

To connect to the server’s console, you need to use your instance key:
ssh -i instance-key.pem core@[server IP]
To see the logs, just run the docker logs command for the desired container.

Once you have TeamCity up and running, there are a few more steps to consider:

Happy building with TeamCity on AWS!

Posted in Features, FYI, How-To's | Leave a comment

TeamCity 2017.1.5 is released

Today we are announcing another update for the latest version, TeamCity 2017.1.5.

This build fixes about 50 problems, including a few performance and security issues: the full list is available in our release notes.

Don’t hesitate to install the new version: TeamCity 2017.1.5. uses the same data format as the other 2017.1.x releases, so you can freely upgrade/downgrade within these versions.

Download the new version and upgrade your server.

Your feedback on the new version is welcome in our forum and tracker.

We are also cooking another EAP for TeamCity 2017.2, so follow us for the latest news!


Happy building!

The TeamCity Team

Posted in Bugfix, Release | Leave a comment

TeamCity Integration with IntelliJ-based IDEs

Starting from its very first version, TeamCity has provided support for IntelliJ platform-based IDEs; and since they are products from JetBrains, this integration is the most powerful one when compared to other IDE integrations.

The main idea behind the integration is the ability to take advantage of all the features provided by TeamCity as a continuous integration server without leaving the context of the IDE: using the integration, you can configure notifications, run builds and tests, get the build status, view your changes, assign investigations and perform other actions available in TeamCity right from the IntelliJ IDEA. Besides, the integration allows cross navigation between the IDE and build results in TeamCity; for non-DVCS it allows you to test your local changes and see how they integrate with the source code before the commit, which ensures your changes do not break anything.

In this blog post we’ll go over the setup of the TeamCity plugin with IntelliJ IDEA.
TeamCity provides the same level of support for both IntelliJ IDEA editions: the free community edition and the paid ultimate edition. All the IntelliJ platform-based IDEs are also supported: JetBrains RubyMine, JetBrains PyCharm, JetBrains PhpStorm/WebStorm, and AppCode. The versions compatibility information is provided in the TeamCity documentation.

So, first we need to install the TeamCity plugin.

Installing the Plugin

There are two ways to install a plugin.

Installing from the plugins repository

You can install the plugin right from the IDE (connection to the Internet is required) using File | Settings | Plugins. Click Browse JetBrains Plugins and search for TeamCity Integration. Double-click on the listed plugin to download and install it.

Alternatively, you can download the plugin directly from the TeamCity server, which is especially useful if the Internet access is restricted in your organization.

Installing from the TeamCity server

Log in to the TeamCity server, in the right top corner, click the arrow next to your username and select My Settings & Tools on the right of your name. On the right, you will see TeamCity Tools.

  1. Click on the download link near IntelliJ Platform plugin and download TeamCity-IDEAplugin.zip to your local machine.
  2. After this, in IDEA, go to File | Settings | Plugins | Install plugin from disk… and in the explorer window, select the downloaded ZIP archive.
  3. Click OK.

Next we’ll restart the IDE is required to activate the plugin.

After the restart, the TeamCity menu is available at the top of the screen.
1TCmenu

At the bottom on the left the TeamCity tool window 1aTeamCitybutton is visible, on the right we now have the TeamCity ‘Logged out’ status icon1bNotLogged.
Now we can connect to TeamCity.

Logging in to TeamCity

To log in to the TeamCity server, click TeamCity | Log in… and provide the required credentials:

1cLogin

After you log in, if you installed the plugin from the IDE, you might have to update the plugin to the version corresponding to the TeamCity server you are connected to using the TeamCity | Update plugin menu item.

Now let’s see the integration in action.

Configuring Notifications

Next we’ll configure notifications which the TeamCity server will send to our IDE. This can be done using TeamCity | Options | Edit Notification Rules or by clicking a link in the TeamCity tool window asking you the specify the projects you want to watch: either of the options redirects you to the Notification Rules page in the Web UI of the TeamCity server you are connected to. By default, TeamCity sends out IDE notifications to the user if a build containing their changes fails. You can add your own rules here:

2notifications

TeamCity will track the changes by the username specified in the VCS Usernames section.

Open Files from TeamCity in IDE

When looking at changes in the TeamCity Web UI, you can open changed files from in the IDE:

7viewinIDE

Clicking the IDE link opens up the specific file in the active IDE, i.e. the IDE we are currently connected to.

You can also work with all changes: in this case the patches are downloaded to the IDE, and they can be applied to the working directory as required.

Working with TeamCity from IDE

At the bottom on the left the TeamCity tool window has the Projects, My Changes, and My Investigations tabs, which correspond to the options available in the TeamCity Web UI.

The Projects tab displays projects and build configurations we decided to monitor. It also allows us to configure more visible projects by redirecting us to the TeamCity Web UI.

Let’s run a build from the IDE: right-clicking the build configuration name (Build in the image below) opens a menu allowing you to add a build to the queue, view pending changes, assign and investigation to any user registered on the TeamCity server you are connected to, mark an investigation as fixed, open the build configuration home page in the browser.3ProjectsBuildOpts

The My Changes tab displays all changes made by you and detected by TeamCity according to the specified username:

4changesUnfiltered

You can filter the changes and see only those related to the project you are currently working on:

4changeFiltered

It is easy to see the details of a specific commit in the IDE or quickly open the TeamCity change details page in the browser by right-clicking on the change and selecting the required option:

4changeView

If you are assigned to investigate problems (typically build failures caused by your changes) in TeamCity, you will your investigations in the My Investigations tab.
As you can see, the TeamCity plugin enables you to run builds, view change details, assign investigations and perform other actions available in TeamCity without leaving IntelliJ IDEA.

Committing Pre-Tested Changes to Non-DVCS

In our project we are using the most popular version control at the moment – Git, and we use Git branches to test how our changes integrate with the existing code. In this part we’ll talk about the feature that can help if your project uses centralized version controls (like Subversion (SVN) or Perforce).
Using the pre-tested commit feature, you can prevent committing defective code into a build, so the entire team’s process is not affected. First the developer runs a build in TeamCity with the local changes on the developer’s machine and if this remote run is successful, the changes are committed to the version control automatically.

“Remote” means that the build will not run on the developer machine, but on a TeamCity build agent and this build will include the changes on our developer’s machine and the code in the version control: this way we can see how our changes integrate with other changes in the repository and be sure that our changes will not break the code.

If the remote run (also called ‘Personal Build’) fails, we will get a notification and can roll the changes back or fix the issues.
To trigger a remote run, we’ll use the TeamCity | Remote Run menu item. In the dialog that is displayed, select all changes you made locally or cherry-pick just the changes you want to test on the build server like we did. It is possible to select when to commit the changes: our changes will be committed if no builds fail. All whats left is to click the OK button and the remote run starts:8pretestedCommit

Debugging Java Code Remotely

We already mentioned that TeamCity integration supports all IntelliJ-based IDEs: it is programming language agnostic. However, some of its features are limited to a particular technology, and the next feature we’ll talk about is Java-specific.

The TeamCity plugin for IntelliJ IDEA allows you to debug your tests remotely on the TeamCity agent machine from the IDE on the local developer machine. The remote debug feature is useful if the agent environment is unique in some aspect, which causes a test to fail, and it is difficult to reproduce the problem locally.

You can use this feature if the following requirements are met:

  1. an IntelliJ IDEA run configuration (Java application run configuration /JUnit run configuration /JTestNG run configuration) on the local developer machine with the TeamCity plugin for IntelliJ IDEA is installed
  2. a build configuration on the TeamCity Server with the IntelliJ IDEA Project runner as one of the build steps
  3. a remote TeamCity agent to run this build available to the local machine by socket connection

To start debugging, select the TeamCity | Remote Debug menu item. You will be asked to select the run configuration to debug.
(Alternatively, you can right-click tests and select the Debug Remotely on TeamCity Agent option from the context menu.)

DebugRunConfig

Now we click Debug Remotely on TeamCity agent. The build is added to the queue and the standard IntelliJ IDEA debug tool window appears.The debug tool window works in the listening mode, i.e. it waits for the agent to connect to it. Once the agent connects, the Java process on the agent is paused and the Agent Attached notification appears in the IDE. Now we can set some breakpoints, and actually start the debug session by clicking Start either in the notification popup or in the debug tool window.

6DebugStartButton

When the build is finished, you can view failed tests in the IDE:

6DebugLogIdea

Once the JVM process exits, another notification popup appears in the IDE:
6DebugAgDetached
The debug session is not finished yet, it is possible to either repeat or finish it. Selecting Repeat will rerun the same build step again, which is much faster than starting a new debug session.

Well, we’ve covered some of the most popular features of the TeamCity-IDEA integration and really hope we managed to persuade you to try the integration.

Your feedback is most welcome as usual herehere in the comments or ourissue tracker.


Happy integrating and building!

Posted in Features, FYI, Tips&Tricks | 2 Comments

TeamCity Docker Images for Windows

teamcity-docker-windowsSome time ago we’ve published official TeamCity docker images for Linux. Since then Microsoft, together with Docker released support for Windows containers. And now we’re glad to announce official TeamCity Docker images for Windows containers.

TeamCity Windows Docker images are based on windowsservercore and nanoserver images and have corresponding -windowsservercore and -nanoserver suffixes in tags.

For Windows we provide the same set of Docker images as for Linux containers:

For instance, to run a nanoserver based TeamCity server, you can execute the following command:

To connect TeamCity build agent to that server you could execute the following command:

Note: due to limitations of Docker on Windows, you may not be able to access TeamCity server on localhost via http://localhost:%port% address. If this is the case, you should use non localhost IP address of your machine.

Due to some other known issues, we’d not recommend to run a production TeamCity server in Windows containers right now. But it seems fine to run a TeamCity agent inside a Windows container.

Please try these images and let us know what you think.

Happy building!

TeamCity Team

Posted in News & Events | Tagged , , | Leave a comment

Docker integration plugin now available for 2017.1.x

Greetings everyone!

As you might already know, TeamCity 2017.2 EAP builds provide new Docker integration features. They are provided by the Docker Support plugin, which is bundled with TeamCity 2017.2 EAP by default.

Today we are making this plugin available to users of TeamCity 2017.1.x as well. If you are already on 2017.1.x and use Docker, we believe the Docker Support plugin will make your life simpler, and here is why:

  • The “Docker Build” runner helps create your own Docker images.
  • The “Docker Compose” runner allows starting services using Docker Compose during your build process. The TeamCity agent will stop these services automatically when the build finishes.
  • The Command Line, Gradle, and Maven runners are also extended with the ability to execute inside a Docker container. Check this blog post for a usage example.
  • Finally, the “Docker Support” build feature provides more convenience features like:
    • automatic login to a Docker registry before the build starts;
    • the “Docker Info” build tab with information about images published to the Docker registry during the build;
    • the option for automatic deletion of the associated images from the registry when the build is deleted during the TeamCity cleanup procedure.

For details, take a look at the integration documentation.

If you wish to try this plugin, there’s nothing to stop you:

Oh, wait, there is one more option. You can actually TestDrive it in the cloud!

Happy building!
The TeamCity Team

Posted in Uncategorized | 2 Comments

TeamCity 2017.2 EAP3 is released

Greetings everybody!

We continue working on the new TeamCity version and today we are announcing TeamCity 2017.2 EAP3.

This release brings you over 90 fixed issues, as well as a number of new features which we are sure you’ll find useful:

  • TeamCity now supports automatic upgrade which can be initiated from the Administration | Update page.
  • We introduce default templates, that can be defined in a project and used to collectively modify build configurations of this project.
  • Good news for Kotlin fans – now when Kotlin settings are enabled for a project, project and build configuration settings will be available for editing from the web UI.

Please refer to the Release notes for the full list of improvements including those in composite builds, Docker support and more.

Upgrade to the new EAP version either automatically or download TeamCity 2017.2 EAP3 build as usual. Remember to install it on a trial server as it changes the TeamCity data format.

We’ll be happy to get your feedback here in the comments, on our forum or tracker.

Happy building!
The TeamCity Team

Posted in EAP, Features | 2 Comments

TeamCity Plugin for HashiCorp Vault

When performing integration tests and deployments, build scripts need credentials to access external servers and services. Traditionally passwords are stored on the TeamCity server as secure parameters. But although secrets are masked in the UI, encrypted at rest, and protected from being exposed in the build log as plain text, this often does not provide a high enough level of security.

HashiCorp Vault offers a unified approach to managing secrets and credentials, allows auditing access, and helps with password rotation.

Integration with Vault

Today we are presenting a new plugin to help build scripts interact with Vault and obtain credentials dynamically.

The plugin allows connecting TeamCity to Vault, requesting new credentials when a build starts, passing them to the build script, and revoking them immediately when the build finishes.

Usage

We expect you have installed and configured:

  • a Vault server. Refer to the Getting Started guide to learn how to launch a testing Vault instance.
  • A TeamCity server. Refer to the Getting Started guide for details.

Perform the following:

  • Download the plugin from the plugin repository, and install it to TeamCity server.
  • Configure secret backends in Vault to obtain secrets, for example AWS credentials, or generic secrets.
  • Create an AppRole in Vault for the TeamCity server to access these backends.
  • Create a connection to Vault in your TeamCity project:1
  • Next in your build configuration declare new build parameters which refer to the Vault secrets to be used in this build:

2

The parameter value has a special syntax:%vault:PATH!KEY%where PATH is the secret name, and KEY is the specific value inside.
At the moment these values can be used in the build parameter declaration only, and cannot be specified in build steps.

  • Run a build step. Deployment scripts can read credentials from environment variables, and do not need to perform any Vault-related actions:3

Besides AWS, any other Vault backends can be used. In this case:

  • Declare a configuration parameter with a value from generic backend:4
  • In a build step refer to this  parameter explicitly:5

How It Works

  • The TeamCity server stores the AppRole ID and secret, and keeps the secret encrypted at rest. When required, it can be rotated manually in the UI, or via the REST API.
  • When a new build is started, the TeamCity server requests a one-time response wrappingtoken in Vault, and sends it to a build agent.
    The AppRole secret never leaves the TeamCity server machine.
  • The build agent requests the actual secrets from Vault.
    The actual credentials are obtained from Vault by a build agent machine directly, and never transferred to the TeamCity server.
  • The build parameters are resolved and the secrets are passed to environment variables and system properties.
  • The build steps are started, and they refer to these environment variables and system properties.
  • If secret values appear in a build log, they are automatically replaced by asterisks.
  • Right after the build finish, the credentials are explicitly revoked in Vault.

Security Considerations

It is advised that you limit access to TeamCity project settings. If a person can modify build configuration settings in a TeamCity project, they can get access to the secrets. Dynamic credentials are revoked after a build, but for generic secrets this may cause more risks. To minimize the risks, split build configurations for CI and deployments into separate projects with different permissions.

Make sure build agent machines are connected to the TeamCity server via HTTPS. Only one-time tokens are transferred over network, and if they are stolen, credentials will be immediately revoked. However, it is a good practice to encrypt all traffic between the server and agents.

Feedback Wanted

This is an initial release, and we continue to improve features and user experience, and encourage you to give it a try and tell us what do you think!

The plugin is open-sourced, and published on GitHub.

If you are attending HashiConf in Austin, TX, USA next week, visit the JetBrains sponsor booth for the demonstration of this plugin and our other plugins for integrating Packer and Terraform with TeamCity and IntelliJ-based IDEs.

Posted in Features, FYI, How-To's, Tips&Tricks, Uncategorized | 2 Comments

Introducing TeamCity RunAs Plugin

When building, testing, and deploying an application with TeamCity, you may need to run a build under a specific user account, different from the one used to run the build agent.

TeamCity RunAs Plugin Overview

The teamcity-runas plugin can be used to run TeamCity build steps under a specified user account on Windows or Linux.

The plugin is compatible with TeamCity 10.0 and later.

Installing Plugin

System Administrators can download the runAs.zip from here and install it on the TeamCity server as usual.

Configuring TeamCity Agents

On Windows

The teamcity-runas plugin requires the TeamCity agent’s account to have certain permissions. Depending on the way you run the agent, granting additional privileges may be required.

  • If you are running the TeamCity agent as a Windows service under the Local System account, it has all required permissions and the teamcity-runas plugin works out-of-the-box.
  • If you are starting the agent as a Windows service using a specified Windows account (for example AgentUser), it must have administrative privileges, must have the ability to replace a process level token (SeAssignPrimaryTokenPrivilege), and act as a part of the operating system (SeTcbPrivilege) or needs to work under the Local Service account. These privileges can be added via a security policy using Control Panel | Administrative Tools | Local Security Policy | Local Policies | User Rights Assignment | Replace a process level token. See Microsoft documentation for details.
    You can also do it using the provided scripts, e.g.: runGrantAccess.cmd BuildUser.
    The changes will not take effect until the next login, so you’ll have to restart the agent.
  • If you are starting the TeamCity agent as a Windows console application, your Windows user account must have administrative privileges.

On Linux

  • With Debian/Ubuntu/Linux, the recommended setup is to run the TeamCity agent as ROOT, as in this case the account has all the required privileges and the teamcity-runas plugin works out-of-the-box (see the related section below).
  • If you run the TeamCity agent as a user different from ROOT, you need to install socat using the following command:
    sudo apt-get update && sudo apt-get install socat

Using Plugin

After the server restart, the plugin provides the Run As build feature and you can add it to your build configuration specifying the username and password.

For Windows the following username formats are acceptable:〈username〉or〈username@domain〉or〈domain\username〉.

For Windows-based build agents it is possible to specify the Windows Integrity Level, the logging level, and additional arguments.

runAS

Once the build is run, the plugin will run all the commands under the specified user account.

Checking Account Used to Run Build

You can enable the teamcity.runAs.log.enabled= true parameter of a build configuration (set to “false” by default) and TeamCity will print the required information into the build log.

runASvalidCreds

If the credentials are invalid, the build fails with “Authentication failure” error:

runASinvalidCreds

Alternatively, to make sure the teamcity-runas plugin is configured properly, you can get the user whose account was used to run the build by checking the environment variables: USERNAME (for Windows) / USER, LOGNAME (for Linux).

For example, you can add a command line build step with the set USERNAME command on Windows / echo "$USER" echo “$LOGNAME” on Linux. This will print the user whose account was used to run the build into the log.

Accessing Files created by RunAs User

It is important to note that when running a build under a specified user account, all processes run in the environment specific for this user.

It means that all files created during the build (e.g. compiled classes, object files, executables, dlls, Gradle/Maven cache files, TeamCity artifacts, some test statistics and so on) will have the teamcity-runas user as the file owner.

The TeamCity agent must have the full control over the files created in the build checkout directory and ideally, over files in other locations, e.g. m2 or .gradle directories as well. Depending on the way you configured the agent (see Configuring TeamCity Agents above), the access level will differ and granting additional permissions may be required.

For Windows, the account with administrative permissions used to run the TeamCity build agent will have sufficient rights to manage the files.

For Linux, if the TeamCity agent is not running under ROOT, it may encounter problems accessing artifacts in the build checkout directory. In this case the necessary permissions can be granted by running the following command:
chmod -R a+rwX %teamcity.build.checkoutDir%.
It can be added as the last step of the build executed under the ‘Run As’ user.

Install the plugin on your TeamCity server, give it a try, and let us know what you think about it.

Happy building!

The TeamCity Team

Posted in Features, FYI, How-To's, Tips&Tricks | Leave a comment

TeamCity 2017.1.4 bugfix is released

Greetings, everyone!

Here is another update to our latest released version: please welcome the TeamCity 2017.1.4, a bugfix release addressing about 90 issues.

We recommend upgrading as this version contains a few performance and security fixes. The details are available in our release notes.

TeamCity 2017.4 data format is the same as that of all the 2017.1.x releases enabling you to easily downgrade if required, so just download the new version and install it on your server.

We’ll be happy to get your feedback: feel free to use our forum and issue tracker.

We are also working on the TeamCity 2017.2 EAP: follow our news for the latest features available in the preview version.

Happy building!
The TeamCity Team

Posted in Bugfix, Release | Leave a comment

TeamCity 2017.2 EAP2 is out

Greetings, everyone!

We are happy to present the new EAP version of TeamCity 2017.2 to you.

In TeamCity 2017.2 EAP2 we are introducing a new feature – composite builds allowing to aggregate results from several other builds connected by snapshot dependencies and presents the results in one place. A composite build can be viewed as a build which consists of a number of parts which can be executed in parallel on different agents. All these parts will have synchronized snapshot of the source code and the results can be seen in a single place. We would really like you to try this feature and share your feedback with us.

This EAP build also comes with improved Docker integration: you can now create a connection to a docker registry and TeamCity will be able to automatically log in to the specified registry before a build and log out of the registry after it. You can now instruct TeamCity to remove the published images during the build clean-up, and more.

Plugins management improvements include tracking newly uploaded plugins and disabling them via the web UI.

For the full list of improvements, see our release notes.

Download TeamCity 2017.2 EAP2 and make sure you install it on a trial server: this version is still in development; besides, it modifies the TeamCity data format making the downgrade impossible.

Try the new version and help us make it better: our forum and tracker are always at your disposal.

Happy building!
The TeamCity Team

Posted in EAP, Features, Uncategorized | Leave a comment