TeamCity 2017.1.1 update is out

Greetings, everyone!

Today we are presenting TeamCity 2017.1.1 release. This is essentially a bugfix update
resolving a number of issues which prevented some of our users from upgrading to TeamCity 2017.1.

Besides these, this TeamCity build comes with the bundled IntelliJ IDEA and ReSharper Command Line Tools updated to version 2017.1.2 and contains a few performance and usability enhancements.

For the full list of fixes see our Release Notes.

Download the new build now, check the Upgrade Notes, and install the new TeamCity build.

Happy building!

Posted in Bugfix, Release | Leave a comment

What’s New in TeamCity 2017.1, May 3rd Webinar

Join us Wednesday, May 3rd, 17:00 – 18:00 CEST (11:00 AM – 12:00 PM EDT) for our free live webinar, What’s New in TeamCity 2017.1 with Wes Higbee.

join

In this webinar, Wes Higbee will go over some of the major features of the latest TeamCity release. He will demonstrate the configuration of agent cloud profiles, which was moved to the project level; new integration with Visual Studio Team Services; disabling builds in the default branch; and the new secure values storage setting.

He will also show how easy it is to upgrade from v10.0 to v2017.1 in Docker, and showcase the updated parts of the TeamCity UI.

Space is limited, please register now. There will be an opportunity to ask questions during the webinar.

About the presenter:

Wes McClureWes Higbee is passionate about helping companies achieve remarkable results with technology and software. He’s had extensive experience developing software and working with teams to improve how software is developed to meet business objectives. Wes launched Full City Tech to leverage his expertise to help companies rapidly deliver high quality software to delight customers. He has a strong background in using Continuous Integration with TeamCity to bring quality to the table.
Posted in Features, FYI, How-To's, Webinar | Leave a comment

TeamCity 2017.1 aka 10.1 is released

Greetings, everyone!

We are proud to present the new TeamCity 2017.1 aka TeamCity 10.1.

650x170_product_header_JB_logo@2x_teamcity_2017_1

This release brings a lot of new exciting features, so hold your breath because here it comes:

  • Project administrators can now configure cloud profiles for their projects and build configurations saving time and effort for their system administrators. You can also define cloud profiles in Kotlin DSL now.
  • This version features improved Visual Studio Team Services integration: setting up a project or issue tracker has never been so easy.
  • We do hope you’ll like our considerably revamped UI, optimized for large-scale installations.
  • The new TeamCity release comes with the pluggable API enabling external artifacts storage. The S3 plugin from TeamCity is a good example of the new API implementation; you can also develop your own integration with a storage of your choice!

For the full list of features see our What’s New and Release NotesDownload TeamCity 2017.1 and remember to check the Upgrade Notes before you install the new version!

We’d love to hear from you, please share your feedback in our forum or tracker.

Happy building!

Posted in Features, Release | Leave a comment

TeamCity 2017.1 RC has arrived

Greetings, everyone!

We are happy to announce the release candidate for TeamCity 2017.1. This build fixes over 100 issues and includes a few performance improvements and security fixes.

We continue working on the TeamCity web UI and this version comes with the improved user profile page and the redesigned group editing page; besides, the new more performant selectors are now available in various places in the UI.

Other noteworthy changes are:

– the versioned settings now allow you to store security data outside the VCS
– cloud integration can be enabled/disabled for projects and subprojects
– you can now configure a proxy server for agent-to server connections.

See our Release Notes for the full list of fixes.

Please note that the TeamCity EAP license is not available for this build.

Though this is the release candidate, we would not recommend installing this version on the production server just yet,  please install it on a trial server. Download the new version, and please share your feedback in our forum or the tracker!

Stay tuned for the news on the TeamCity 2017.1 release which is coming soon!

Happy building!

Posted in Bugfix, EAP | 2 Comments

Here Comes EAP 2 for TeamCity 2017.1

Greetings, everyone!

We are happy to present TeamCity 2017.1 EAP 2!

This early preview build comes with several new features, the most important ones being:

– Agent cloud profile configuration has been moved to the project level
– Ability to exclude changes from default branch or hide default branch
– Web UI improvements: new breadcrumbs, new controls for build configurations selectors, build chains improvements
– Cross-Platform PowerShell is now supported

See Release Notes for details.

Download TeamCity 2017.1 EAP2 build, install it on a trial server as it changes the TeamCity data format, and share your feedback on our forum or tracker.

Happy building!
The TeamCity Team

Posted in EAP, Features | 4 Comments

TeamCity 10.0.5 is here

Greetings everyone!

Please welcome TeamCity 10.0.5, most likely the last update for the 10.0.x version. This is a bugfix release, which addresses about 70 issues. See the full list of them in our Release Notes.

Please note if you’re using TFS with agent side checkout: due to the fix of TW-48555 TeamCity will have to recreate TFS workspaces, which can result in clean checkout on the agents after the upgrade.

This build does not change the data format, so in case there are any issues, you can easily revert to any of the earlier 10.0.x versions.

Happy building!

Posted in FYI, Release | 2 Comments

Kotlin Configuration Scripts: Testing Configuration Scripts

This is part five of the five-part series on working with Kotlin to create Configuration Scripts for TeamCity.

  1. An Introduction to Configuration Scripts
  2. Working with Configuration Scripts
  3. Creating Configuration Scripts dynamically
  4. Extending the TeamCity DSL
  5. Testing Configuration Scripts

In this last part of the series we’re going to cover one aspect that using Kotlin Configurations Scripts enables, which is testing.

Given that the script is a programming language, we can simply add a dependency to a testing framework of our choice, set a few parameters and start writing tests for different aspects of our builds.

In our case, we’re going to use JUnit. For this, we add the JUnit dependency to the pom.xml file

We also need to define the test directory

In our case, this corresponds to the following directory layout

Directory Structure

Once we have this in place, we can write unit tests like in any other Kotlin or Java project, accessing the different components of our project, build types, etc. In our case we’re going to write a simple test that checks to see all build types start with a clean checkout

 

That additional layer of certainty

When we make changes to the Kotlin Configuration Script and check it in to source control, TeamCity synchronises the changes and it will report of an errors it encounters, which usually are related to compilation. The ability to now add tests allow us to add another extra layer of checks to make sure that beyond our build script containing any scripting errors, certain things are validated such as the correct VCS checkout as we’ve seen above, whether the appropriate number of build steps are being defined, etc. The possibilities are essentially endless given that once again, we’re just using a regular programming language.

Update: Important note

Given that currently TeamCity does not fetch third-party dependencies, the tests, which usually require JUnit or some other testing framework, will fail if places inside the .teamcity folder. This is due to TeamCity syncing the project and failing to compile unresolved references. For now the workaround is to place the tests outside of this folder.

Posted in Features, How-To's, Tips&Tricks | 2 Comments

Welcome EAP1 for TeamCity 2017.1 (aka 10.1)

Greetings, everyone!

Today we are unveiling the Early Access Preview (EAP) for TeamCity 2017.1, formerly known as TeamCity 10.1! Following the JetBrains versioning scheme, our releases will be now identified by <year of the release>.<number of the feature release within the year>.<bugfix update number>.

Build 45965 comes with over 30 new features, and though most of them are minor, they provide a smoother experience with TeamCity.

The TeamCity Web UI has been given a facelift overall; we should mention that the build chains page has been reworked with chains being displayed in a more compact way with handy new options.

It’s worth noting that the server startup and the page load times have been decreased.

Besides the new features, this versions brings over 100 bugfixes – see the details in our Release Notes.

Don’t hesitate to get the new EAP build and share your feedback on our forum or tracker.

As usual, this new feature release changes the TeamCity data format, and given that version 2017.1 is still in development, we recommend you install it on a trial server.

Happy building!
The TeamCity Team

Posted in Bugfix, EAP, Features, FYI, Tips&Tricks | 4 Comments

TeamCity as Debian Package Repository

Recently we’ve been experimenting around using TeamCity as a Debian repository and we’d like to share some tips and tricks we’ve come up with in the process.

We used the TeamCity tcDebRepository plugin to build *.deb packages and serve package updates to Debian/Ubuntu servers. The plugin, the special prize winner of the 2016 TeamCity plugin contest, works like a charm, but we’ve encountered a few difficulties with the software used to build and package .deb files, hence this blog post.

It’s important to note that tcDebRepository is not capable of serving packages to recent Ubuntu versions due to the lack of package signing. This is planned to be resolved in version 1.1 of the plugin (the plugin compatibility details).

We do not intend to cover the basics of TeamCity and Debian GNU/Linux infrastructure. To get acquainted with TeamCity, this video might be helpful. Building Debian packages is concisely described in the Debian New Maintainers’ Guide.

Everything below applies to any other Debian GNU/Linux distribution, e.g. Astra Linux.

Prerequisites and Project Setup

We arbitrarily chose 4 packages to experiment with:

Our setup uses:

  1. The free Professional version of TeamCity 10 with 3 agents managed by Debian 8.0 (Jessie).
  2. The tcDebRepository plugin installed as usual.
  3. Besides the Teamcity agent software, each of the agents required installing:
    • the build-essential package as a common prerequisite
    • the dependencies (from the 01-build-depends build-depends and 02-build-depends-indep build-depends-indep categories) required for individual packages.

Next we created a project in TeamCity with four build configurations (one per package):
teamcity-build-configs

Configuring VCS Roots

When configuring VCS Roots in TeamCity, types of Debian GNU/Linux packages had to be taken into account.

Debian packages can be native and non-native (details).
The code of native packages (e.g. debhelper, dpkg), developed within the Debian project, contains all the meta-information required for building (the debian/ directory in the source code tree).
The source code of non-native packages (e.g. bash) is not related to Debian, and an additional tree of their source code and meta-information with patches (contained in the debian/ directory) has to be maintained.

Therefore, for native packages we configured one VCS Root (pointing to Debian); whereas non-native packages needed two roots.

Next we configured checkout rules: the difference from the usual workflow is that in our case the artifacts (Debian packages) will be built outside the source code, one directory higher. This means that the checkout rules in TeamCity should be configured so that the source code of the dpkg package is checked out not into the current working directory, but into a subdirectory with the same name as the package, i.e. pkgname/. This is done by adding the following line to the checkout rules:
+:.=>pkgname
Finally, a VCS trigger was added to to every configuration.

Artifact paths

In terms of TeamCity, the binary packages we are about to build are artifacts, specified using artifact paths in the General Settings of every build configuration:

03-teamcity-debian-general-artifact-paths

For native packages, the pkgname.orig.tar.{gz,bz2,xz,lzma} and pkgname.debian.tar.{gz,bz2,xz,lzma} will not be created.

Build Step 1: Integration with Bazaar

Building a package with Bazaar-hosted source code (bash in our case) requires integration with the Bazaar Version Control System not supported by TeamCity out-of-the box. There is an a 3rd party TeamCity plugin to support Bazaar, but it has at least two problems:

  1. agent-side checkout is not supported
  2. the plugin is based on bzr xmlls and bzr xmllog (the bzr commands invoked by the plugin), but these commands are provided by external modules not necessarily available in the default Bazaar installation

The TeamCity server we used is running Windows, so we decided not to opt for installing Bazaar on the server side. Instead, to build the bash package, we added a build step using the Command Line Runner and a shell script for Bazaar integration:

This approach will not allow us to see the changes in one of the two trees of the source codes and run build automatically on changes, but it is ok as the first try.

Build Step 2. Initial configuration

We used the Command Line Runner to invoke dpkg-buildpackage. The -uc and -us keys in the image below indicate that digital signatures are currently out of scope for our packages. If you wish to sign the packages, you’ll have to load the corresponding pair of GnuPG keys on each of the agents.

Note that the “Working directory” is not set to the current working directory, but to the subdirectory with the same name as the package (bash in this case) where the source code tree will be checked out and dpkg-buildpackage will be executed. If the VCS Root is configured, the “Working directory” field can be filled out using the TeamCity directory picker, without entering the directory name manually:

07-teamcity-debian-dpkg-buildpackage-step

After the first build was run, we encountered some problems.

Build problems resolution

Code quality

For some packages (bash in our case), two code trees were not synchronized, i.e. they corresponded to slightly different minor versions, which forced us to use tags rather than branches from the main code tree. Luckily, TeamCity allows building on a tag and the “Enable to use tags in the branch specification” option can be set in such case:

teamcity-vcs-tags-in-branch-specs-small

Dependencies

Here is another problem we faced. When running the first build, dpkg-buildpackage exits with code 3 even though all the required dependencies were installed (see the prereqs section above).

There are a few related nuances:

  • The most common case for continuous integration is building software from Debian Unstable or Debian Experimental. So, to meet all the dependencies required for the build, TeamCity agents themselves need to run under Debian Unstable or Debian Experimental (otherwise dpkg-buildpackage will be reporting that your stable dependencies versions are too old). To resolve the error, we added the -d parameter:
    dpkg-buildpackage -uc -us -d
  • A special case of old dependencies is the configure script, created by the GNU Autotools that are newer than those currently installed on your system. dpkg-buildpackage is unable to detect this, so if the build log shows messages about the missing m4 macros, you need to to re-generate the configure script using the of GNU Autotools currently installed on the agent. This was done by adding the following command as the build step executed before dpkg-buildpackage:
    autoreconf -i

Broken unit tests

When working with Debian Unstable or Debian Experimental, unit test may fail (as it happened in our case). We choose to ignore the failing test and still build the package by running dpkg-buildpackage in a modified environment:
DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -uc -us

Other build profiles are described here.

Final Stage

After tuning all of the build configurations, our builds finally yielded expected artifacts (the dpkg package is used as an example):
11-teamcity-debian-artifacts

If you do not want each incremental package version to increase the number of artifacts of a single build (giving you dpkg_1.18.16_i386.deb, dpkg_1.18.17_i386.deb, etc.), the working directory should be selectively cleaned up before starting a new build. This was done by adding the TeamCity Swabra build feature to our build configurations.

Tuning up Debian repository

The tcDebRepository plugin adds the “Debian Repositories” tab in the project settings screen, which allows you to create a Debian repository. Once the repository is created, artifact filters need to be configured: at the moment each filter is added manually. Filters can be edited, deleted or copied.
teamcity-adding-an-artifact-filter

The existing artifacts will not be indexed, therefore after the configuration of the Debian repository, at least one build must be run in every build configuration. Then we can view the available indexed packages:
16-teamcity-debian-debrepo-page-2

When adding the repository to the /etc/apt/sources.list, all these packages are visible on the client side. Note that the packages are not digitally signed, which prevents packages from being available on recent versions of Ubuntu. Package signing is planned for tcDebRepository version 1.1.

17-teamcity-debian-aptitude-1

(!) If you are compiling for several architectures (i386, x32, amd64, arm*), it is worth having several build configurations corresponding to the same package with different agent requirements, or, in addition to the VCS Trigger, using the Schedule Trigger with the “Trigger build on all enabled and compatible agents” option enabled.

20-teamcity-debian-change-log

Happy building!

* We’d like to thank netwolfuk , the author of the tcDebRepository  plugin, for contributing to this publication.
This article, based on the original post in Russian, was translated by Julia Alexandrova.

Posted in FYI, How-To's, Tips&Tricks, Uncategorized | Leave a comment

Kotlin Configuration Scripts: Extending the TeamCity DSL

This is part four of the five-part series on working with Kotlin to create Configuration Scripts for TeamCity.

  1. An Introduction to Configuration Scripts
  2. Working with Configuration Scripts
  3. Creating Configuration Scripts dynamically
  4. Extending the TeamCity DSL
  5. Testing Configuration Scripts

TeamCity allows us to create build configurations that are dependent on each other, with the dependency being either snapshots or artifacts. The configuration for defining dependencies is done at the build configuration level. For instance, assuming that we have a build type Publish that has a snapshot dependency on Prepare Artifacts, we would define this in the build type Publish in the following way

and in turn if Prepare Artifacts had dependencies on previous build configurations, we’d define these in the dependencies segment of its build configuration.

TeamCity then allows us to visually see this using the Build Chains tab in the user interface

Build Chains

Defining the pipeline in code

Pipeline is a sequence of phases, phase is a set of buildTypes, each buildType in a phase depends on all buildTypes from the previous phase. This can handle simple but common kind of build chains where some build produces an artifact, several builds test it in parallel and the final build deploys the result if all its dependencies are successful.

It would often be beneficial to be able to define this build chain (or build pipeline) in code, so that we could describe what’s going on from a single location. In essence, it would be nice to be able to define the above using the Kotlin DSL as so

Defining this at the Project.kt level in our DSL, would give us a good oversight of what’s taking place.

The issue is though that currently the TeamCity DSL does not provide this functionality. But that’s where Kotlin’s extensibility proves quite valuable as we’ll see.

Creating our own Pipeline definition

Kotlin allows us to create extension functions and properties, which are a means to extend a specific type with new functionality, without having to inherit from these. When passing extension functions as arguments to other functions (i.e. higher-order functions), we get what we call in Kotlin Lambdas with Receivers, something we saw in this series already when Generalising feature wrappers in the second part of this series. We apply the same concept here to create our pipeline DSL

What the code above is doing is define a series of new constructs, namely pipeline and phase.

The code then takes the contents of what’s passed into each of these and defines, under the covers, the dependencies. In essence, it’s doing the same thing we would do in each of the different build configurations, but from a global perspective defined at the Project level.

In order to pass in the configuration to the pipeline as we saw earlier, we merely reference the specific build type (HttpClient, Publish, etc.), assigning it to a variable

val HttpClient = BuildType(....)

Flexibility of defining our own pipeline constructs

It’s important to understand that this is just one of many ways in which we can define pipelines. We’ve used the terms pipeline and phase. We could just as well have used the term stage to refer to each phase, or buildchain to refer to the pipeline itself (and thus align it with the UI). In addition to how we’ve named the constructs, and more importantly, is how the definition is actually constructed. In our case, we were interested in having this present in the Project itself. But we could just as well define a different syntax that is used at the build type level. The ability to easily extend the TeamCity DSL with our own constructs, provides us with this flexibility.

 

 

 

Posted in Features, How-To's, Tips&Tricks | 1 Comment