Run TeamCity CI builds in Google Cloud

teamcity-google-cloudWe are happy to announce that now TeamCity provides a tight integration with Google Cloud services. The Google Cloud Agents plugin allows using Google Compute Engine to start cloud instances on demand to scale the pool of cloud build agents and also supports using cost-efficient preemptible virtual machines. The Google Artifact Storage plugin provides the ability to use Google Cloud Storage to keep external build artifacts.

Installation

To enable these TeamCity integrations with the Google Cloud, you need to download the Google Cloud Agents and Google Artifact Storage from the plugins gallery and install them as additional TeamCity plugins. Remember to restart the TeamCity server.

Google Cloud Agents Configuration

Prepare Google Cloud Image

Now you need to create cloud images to start new cloud build agents from. To do that, create a new cloud instance from the available public boot disks and uncheck “Delete boot disk when instance is deleted” in the “Management, disk, networking, SSH keys” section. Then, after the start of the cloud instance, connect to it via SSH or RDP depending on the selected image and install a TeamCity build agent. You need to point the build agent to the TeamCity server to update plugins and set the agent to start automatically.

Create Google Cloud Instance

Then install required build tools, remove temporary files, remove the cloud instance and create a new custom image from the cloud instance boot disk.

Create Custom Google Image

Create Service Account and Key

Now we need to create a new service account with the Project Viewer and Compute Engine Instance Admin roles and a new JSON private key for that account.

google-cloud-credentials

Create Cloud Agent Profile

To create a Google agent cloud profile, navigate to the project where you want to set up the profile and select the Cloud Profiles link. Then click the “Create new profile” button and select “Google Compute” as the cloud type. Specify the profile name and the JSON private key value in the corresponding field.

To add a new image to the profile, press the “Add image” button, select the recently created cloud image, and fill in other properties. Next you need to save the image and then the profile settings. That’s all.

teamcity-google-profile

Google Artifact Storage Configuration

Create Service Account and Key

To access Google Cloud Storage, the plugin requires a service account with the Project Viewer, Storage Object Admin roles and a new JSON private key for that account.

teamcity-google-profile

Configure Artifact Storage

Navigate to the Artifacts Storage Project tab and click the Add new storage button. Select Google Storage type and fill in the name, JSON private key, bucket name and press Save. After that, activate this artifact storage in the TeamCity project by clicking the Make Active link.

teamcity-google-profile

That’s it! Your new builds in this project will store artifacts in the Google Cloud Storage.

Feedback

You are welcome to try these plugins for TeamCity, your feedback will be really appreciated: please share your thoughts in the comments to this blog post or in our issue tracker.

Happy building in the Google Cloud!

banner_blog@2x

Posted in Features, FYI, How-To's, Tips&Tricks | Tagged , , | Comments Off on Run TeamCity CI builds in Google Cloud

TeamCity to Access Private GitHub Repositories Securely


Introduction

One of the great features TeamCity has is its integration with GitHub. Authentication from TeamCity to GitHub should be configured for the integration to work. We consider TeamCity secure enough; however, stealing authentication data is a threat we all live with and the consequences of it should be clearly understood.

In this blog post we’ll go over the existing ways of configuring authentication in TeamCity VCS roots pointing to private GitHub repositories, discussing the advantages and disadvantages of each of them. We hope that TeamCity Server administrators and Project administrators will find this information useful when deciding on the approach to authentication from TeamCity to GitHub.

Username and Password Authentication

It is often the case when access to a repository from TeamCity is configured via a user’s GitHub username/password. All server administrators and administrators of the projects where the root is configured have access to this information.

If this data is stolen, it gives the intruder full access and the possibility to maliciously modify your GitHub repositories, including your GitHub profile and all your settings.

We do not recommend it as it seems to be the least secure approach.

Username and Generated OAuth Token

Not long ago TeamCity started to support OAuth for GitHub. In this case you use the username and a generated OAuth token.

If someone gains access to them, then all your repositories in all organizations where you have read rights will be at this user’s disposal as TeamCity uses the ‘repo’ token scope.
However, this authentication option gives access to repositories only and not your GitHub profile and settings; besides, it is easy to revoke this token. Besides, repositories cannot not be deleted, and although force-push may still be performed, using the protected branches feature of GitHub can help in this case. All in all, this way can be considered an improvement in comparison with the previous approach.

SSH Key

As an administrator, you can create an SSH key for your TeamCity server, with the public part of the SSH key uploaded to GitHub and the private part uploaded to the TeamCity server.

The advantages of this approach are the same as those of the one above.
As to the risks, it can be highly dangerous if you use this key in different servers and applications. This can be mitigated using the special SSH key added to the GitHub profile and TeamCity, which will be used for this integration exclusively.

‘Deploy Key’ GitHub Feature

For every GitHub repository that Teamcity has access to it is possible to generate an SSH key with the private part on the TeamCity and the public part added to the deploy keys of your repository using the repository settings page. GitHub administrator rights for this repository are required. The key can have either read-only or read-write permissions for the repository.

This seems to be the most secure approach, because an individual key can be added to each repository, which would make access revocation extremely easy in case of data loss.

Creating a key for every repository might be a nuisance for Windows users, this seems to be the main disadvantage (not affecting Linux or macOS users though).

Conclusion

From the approaches outlined above, using OAuth token authentication and Deploy keys are considered secure enough by the majority of people, with Deploy keys being more secure and therefore recommended by us. In the end, adopting this or that way is up to you.

Posted in How-To's, Tips&Tricks, Uncategorized | 3 Comments

TeamCity 2017.1.2 update is released

Greetings, everyone!

Here is another bugfix update for the latest TeamCity version, TeamCity 2017.1.2.

This release features about 90 fixes and improvements, including:
– a new health report displayed when incorrect Http proxy server configuration is detected
– several fixes related to password replacement in the build log
– the fix for the issue with deleting a queued build.

It is recommended that you upgrade to the latest version since this build also resolves several security and performance problems. For the full list of fixes, refer to our release notes.

As usual, you can freely upgrade/downgrade within 2017.1 release, so download the new version and install it on your server – you can always roll back to an earlier version if required.

Your feedback is always welcome in our forum or tracker.

Happy building!

Posted in Bugfix, Release | Comments Off on TeamCity 2017.1.2 update is released

Webinar recording: What’s New in TeamCity 2017.1

The recording of our May 3 webinar, What’s New in TeamCity 2017.1, is now available on the JetBrains YouTube channel.

In this webinar, Wes Higbee goes over the new and exciting features of the latest release of TeamCity, TeamCity 2017.1.

Q&A

Q: Do you support Artifactory by default?
A: There is a plugin from JFrog which provides support for Artifactory.

Q: How would this work on an in-house BitBucket instance?
A: TeamCity supports connections to Bitbucket cloud only, so there is no way to create connection to Bitbucket server. But other ways of creating projects are available. If you have URL to repository – you can just create project / build configuration from this URL.

Q: Can you have a manual build step with an approval button?
A: It is possible to configure build configuration to show some question when build of this configuration is triggered. this can be done with help of typed parameters: https://confluence.jetbrains.com/display/TCD10/Typed+Parameters

Q: Can we use SonarQube/CodeNarc metrics to take a call on whether the next step in a build pipeline should be executed or not?
A: This could be implemented as a plugin. Plugin can set some kind of a precondition for build start.

Q: Any plans for supporting Docker as a specific step in build procedure? E.g. using containers for building.
A: Yes, there are plans. There is a separate plugin in the works. But if you want you can always run a step in docker using command line step.

Q: Is there an option to move the artifacts from native to Azure on selected builds which is already built?
A: No, not yet. On the other hand if you enable Azure, artifacts of old builds will still be available.

Q: Do all builds go to the artifact path when building different branches? How to distinguish between what each branch is producing the artifacts?
A: Actually, each build has its own forlder for artficats, so there are no conflicts here.

Q: Would the server remember all of its artifcats storages? The ones that are active and the ones that are not active?
A: Yes it will. Unless you remove the settings, artifacts will still be available.

Q: How does the clean up would act on Azure/S3?
A: Should work the same way as usual cleanup. Artifact patterns should work too.

Q: Can a build agent retrieve an actual value behind a token and put it as an artifact?
A: Build will see the actual value. Tokens are only used for configuration.

Posted in Features, Release, Webinar | Comments Off on Webinar recording: What’s New in TeamCity 2017.1

TeamCity 2017.1.1 update is out

Greetings, everyone!

Today we are presenting TeamCity 2017.1.1 release. This is essentially a bugfix update
resolving a number of issues which prevented some of our users from upgrading to TeamCity 2017.1.

Besides these, this TeamCity build comes with the bundled IntelliJ IDEA and ReSharper Command Line Tools updated to version 2017.1.2 and contains a few performance and usability enhancements.

For the full list of fixes see our Release Notes.

Download the new build now, check the Upgrade Notes, and install the new TeamCity build.

Happy building!

Posted in Bugfix, Release | Comments Off on TeamCity 2017.1.1 update is out

What’s New in TeamCity 2017.1, May 3rd Webinar

Join us Wednesday, May 3rd, 17:00 – 18:00 CEST (11:00 AM – 12:00 PM EDT) for our free live webinar, What’s New in TeamCity 2017.1 with Wes Higbee.

join

In this webinar, Wes Higbee will go over some of the major features of the latest TeamCity release. He will demonstrate the configuration of agent cloud profiles, which was moved to the project level; new integration with Visual Studio Team Services; disabling builds in the default branch; and the new secure values storage setting.

He will also show how easy it is to upgrade from v10.0 to v2017.1 in Docker, and showcase the updated parts of the TeamCity UI.

Space is limited, please register now. There will be an opportunity to ask questions during the webinar.

About the presenter:

Wes McClureWes Higbee is passionate about helping companies achieve remarkable results with technology and software. He’s had extensive experience developing software and working with teams to improve how software is developed to meet business objectives. Wes launched Full City Tech to leverage his expertise to help companies rapidly deliver high quality software to delight customers. He has a strong background in using Continuous Integration with TeamCity to bring quality to the table.
Posted in Features, FYI, How-To's, Webinar | Comments Off on What’s New in TeamCity 2017.1, May 3rd Webinar

Test .NET Core projects with TeamCity

Greetings, everyone!

As many of you already know, TeamCity has a plugin to build .Net Core projects, which is basically a wrapper for the dotnet command. But if you try running dotnet test, you won’t see test results in TeamCity at the moment. As the test time can be significant, it is preferable to have fast feedback, so in this post we’ll explain how to integrate the dotnet test command with TeamCity and configure on-the-fly test reporting.

In short, you’ll need to perform the following:

  1. To run tests locally using the dotnet test command,  add references to the following packages to your test project: to Microsoft.NET.Test.Sdk, to the test framework, and the test adapter.
  2. To configure on-the-fly reporting, add a package reference to the TeamCity VSTest Adapter.
  3. To configure a build in TeamCity, use the TeamCity plugin for .NET Core projects or some other build runner.

Let’s go over these points in detail.

Run tests locally using dotnet test command

The approach suggested by Microsoft works fine for any target framework as well as for multiple frameworks at the same time, provided the test engine has a test adapter, e.g. MS tests, xunit tests, or some other test engine, for example.

That means that a test project needs to have at least three additional package references:

  • Microsoft.NET.Test.Sdk, which contains the MSBuild targets and properties for test projects and marks a project as a test project.
  • The selected Test Framework, e.g.  MSTest, or XUnit, or other
  • Test Adapter to discover and execute test framework based tests, e.g. the MSTest adapter, or the XUnit adapter, or other.

Configure on-the-fly reporting in TeamCity

Ideally, you should not need to do any additional preparation steps if you choose to run dotnet tests on TeamCity: TeamCity should pick up these tests automatically. For now this is not the case, so here is how you still can achieve this goal: use the TeamCity.VSTest.TestAdapter NuGet package containing a special logger which integrates with the test platform and sends service messages to a TeamCity server.

To turn on the integration, you’ll have to add a package reference to TeamCity VSTest Adapter. Note that this package does not impact the tests run locally, but only those run under TeamCity.

Here is an example of a project:

Visual Studio NuGet Dependencies

The project file for an MS test project can look as follows:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>    
    <TargetFrameworks>net45;netcoreapp1.0</TargetFrameworks>    
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="MSTest.TestAdapter" Version="1.1.14" />
    <PackageReference Include="MSTest.TestFramework" Version="1.1.14" />
    <PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.0" />    
  </ItemGroup>

  <ItemGroup>
    <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
  </ItemGroup>

</Project>

Configure build in TeamCity

To configure a build project in TeamCity, we used the TeamCity plugin for .NET Core projects, described here in detail.

You can see an example on the public TeamCity server, where a test project contains one class, UnitTests:

namespace MS.Tests
{
    using System;
    using Microsoft.VisualStudio.TestTools.UnitTesting;

    [TestClass]
    public class UnitTests
    {
        [TestMethod, Ignore]
        public void TestIgnored()
        {
        }

        [TestMethod]
        public void TestPassed()
        {
            Console.WriteLine("some text");
        }
    }
}

Here is an example of the TeamCity tests step:

.

After a build is run, TeamCity shows the test results:

This .Net project has two target platforms, net45 and netcoreapp1.0, that is why each test is run for both framework and you can see several results per test.

The proposed approach does require some extra effort, however, it is quite viable and useful for testing .Net Core projects. Currently we are working on simplifying running tests and configuring on-the-fly test reporting from TeamCity, which would not require modifying your project files, so stay tuned for the latest news.

As usual, feel free to share your feedback.

Happy building and testing!

banner_blog@2x

Posted in Features, How-To's, Uncategorized | 5 Comments

TeamCity 2017.1 aka 10.1 is released

Greetings, everyone!

We are proud to present the new TeamCity 2017.1 aka TeamCity 10.1.

650x170_product_header_JB_logo@2x_teamcity_2017_1

This release brings a lot of new exciting features, so hold your breath because here it comes:

  • Project administrators can now configure cloud profiles for their projects and build configurations saving time and effort for their system administrators. You can also define cloud profiles in Kotlin DSL now.
  • This version features improved Visual Studio Team Services integration: setting up a project or issue tracker has never been so easy.
  • We do hope you’ll like our considerably revamped UI, optimized for large-scale installations.
  • The new TeamCity release comes with the pluggable API enabling external artifacts storage. The S3 plugin from TeamCity is a good example of the new API implementation; you can also develop your own integration with a storage of your choice!

For the full list of features see our What’s New and Release NotesDownload TeamCity 2017.1 and remember to check the Upgrade Notes before you install the new version!

We’d love to hear from you, please share your feedback in our forum or tracker.

Happy building!

Posted in Features, Release | Comments Off on TeamCity 2017.1 aka 10.1 is released

TeamCity 2017.1 RC has arrived

Greetings, everyone!

We are happy to announce the release candidate for TeamCity 2017.1. This build fixes over 100 issues and includes a few performance improvements and security fixes.

We continue working on the TeamCity web UI and this version comes with the improved user profile page and the redesigned group editing page; besides, the new more performant selectors are now available in various places in the UI.

Other noteworthy changes are:

– the versioned settings now allow you to store security data outside the VCS
– cloud integration can be enabled/disabled for projects and subprojects
– you can now configure a proxy server for agent-to server connections.

See our Release Notes for the full list of fixes.

Please note that the TeamCity EAP license is not available for this build.

Though this is the release candidate, we would not recommend installing this version on the production server just yet,  please install it on a trial server. Download the new version, and please share your feedback in our forum or the tracker!

Stay tuned for the news on the TeamCity 2017.1 release which is coming soon!

Happy building!

Posted in Bugfix, EAP | 2 Comments

Here Comes EAP 2 for TeamCity 2017.1

Greetings, everyone!

We are happy to present TeamCity 2017.1 EAP 2!

This early preview build comes with several new features, the most important ones being:

– Agent cloud profile configuration has been moved to the project level
– Ability to exclude changes from default branch or hide default branch
– Web UI improvements: new breadcrumbs, new controls for build configurations selectors, build chains improvements
– Cross-Platform PowerShell is now supported

See Release Notes for details.

Download TeamCity 2017.1 EAP2 build, install it on a trial server as it changes the TeamCity data format, and share your feedback on our forum or tracker.

Happy building!
The TeamCity Team

Posted in EAP, Features | 4 Comments