TeamCity 8.0.2 (build 27482) is out

TeamCity 8.0.2 bugfix build is available now. It addresses several important issues and regressions, as well as some performance problems. We strongly recommend that you upgrade to this version, especially if you experienced problems with TFS, email notifications or slow builds finish. Additionally, there are general enhancements increasing the server performance.

If you upgrade from version 8.0, installing 8.0.2 build preserves the data format (this is usually the case with all our bugfix builds). In case you encounter any problems with build 8.0.2, you can safely revert to 8.0.1 or even 8.0 without the need to restore from backup.

Release notes
Download page

Happy building!

Posted in News & Events | Leave a comment

First-class PHP Continuous Integration using TeamCity

A while ago we blogged about using TeamCity to do Continuous Integration for PHP. TeamCity developers never seem to sleep: with TeamCity 8 a new feature called meta-runners was introduced. Meta runners allow us to reuse build steps by representing them as a native TeamCity runner. In other words, they allow us to make several build steps easier. Which is exactly what we decided to do: create some meta runners based on our previous blog post and make it really easy to run several PHP-related build steps.

image

The Meta-runner Power Pack for TeamCity 8 contains a set of PHP-related meta runners:

  • Run arbitrary PHP code using the PHP runner
  • Install a specific Composer dependency using the Install Composer package runner
  • Install/update packages from composer.json using the Run Composer runner
  • Run Phing build script using the Phing runner
  • Run PHPUnit tests (with code coverage reporting) using the PHPUnit runner

There is only one requirement: a PHP runtime should be installed on our build agent. The Phing, PHPUnit and Composer meta runners will automatically download the latest versions of the executables during the build! Full documentation for these PHP meta runners is available on GitHub.

Installing meta runners

After downloading the PHP meta-runners from GitHub, all files named MRPP_PHP_<name>.xml should be copied into the <TeamCity Data Directory>/projects/<Project ID>/pluginData/metaRunners/ directory on our TeamCity server.

Once this is done, the TeamCity server will list these new meta-runners under the administration page for the project in which the meta runners have been installed:

image

Using the PHPUnit runner

Since a big part of Continuous Integration is making sure the quality of our software is high, running unit tests is typically part of a build configuration. In the PHP world we typically use PHPUnit for this.

The PHPUnit runner allows running PHPUnit tests during the build and will optionally publish code coverage results to TeamCity. A build step using this runner will use a PHPUnit executable available on the build agent if specified, or automatically download the latest phpunit.phar from the PHPUnit.de website during the build.

To run unit tests, all we have to do is specify the path to our unit tests (but don’t worry, we can also tweak PHPUnit parameters):

image

This will run and report back unit test results to the TeamCity server.

image

And if we enable the Collect code coverage option, we get code coverage reports integrated with TeamCity as well:

image

Bugfixes and improvements for these meta runners (or your own, PHPLint and phpDocumentor runners would be nice!) are welcome through our GitHub repository.

Haven’t tried TeamCity? Give it a shot! It’s 100% free for up to 20 build configurations.

Let us know your thoughts through the comments below!

Happy building!
– The TeamCity Team

Posted in Features, How-To's, Tips&Tricks | Tagged , , , , , | 6 Comments

TeamCity 8.0.1 (build 27435) is available

The first bugfix update for version 8.0 is already available!

It has a fix for regression in the Powershell runner (TW-29803), a few performance improvements in the Git agent side checkout when SSH is used (TW-14598), fixes in custom charts (TW-29792), and a number of other important fixes.
Download the fix now and refer to the Release notes for the full list of resolved issues and improvements.

Happy building!
The TeamCity Team

Posted in Tips&Tricks | Leave a comment

TeamCity 8 — Welcome to The Light Side of Continuous Integration

It’s been a while since the last major release of TeamCity, and today we’re excited to announce that new TeamCity 8 is here and available for download. We carefully listened to everything you told us and did our best to give you the product which exceeds your expectations.

As with every release, the whole list of new features and improvements is very long and you can surely learn about all of them, but let’s take a look at some of the most noteworthy ones.

TeamCity is now more affordable for small teams

If you are using Professional Server, you can extend the number of build configurations by purchasing an additional Build Agent. Each purchased Build Agent gives you 10 more build configurations.

If you already have additional agents and your licenses maintenance period covers TeamCity 8.0 release date, good for you! Just upgrade to TeamCity 8 and start receiving profit.

New project hierarchy

The project hierarchy in TeamCity 8 allows you not to worry about the growing number of projects and configurations. You’re able to group several projects under a parent one and tune access, VCS settings, and other parameters for the whole projects tree.

Your server health reports

No matter how well you keep an eye on your build environment, there is always something out of your field of view: unused agents, huge build logs, redundant VCS roots and so on. The new Server Health page is here to show you these issues.

Other important features include:

  • a much faster (up to 5 times) build history cleanup, allowing to drastically reduce server maintenance time;

  • Meta-Runner allowing you to reuse build steps by representing them as a native TeamCity runner,

  • improved build problems reporting: detection of new problems, ability to assign an investigation or mute a build problem.

  • The native IntelliJ IDEA compiler is now bundled with TeamCity 8, which means that for IntelliJ IDEA projects you get much faster incremental compilation and support for Scala, Groovy, Clojure, Kotlin, Android, GWT and many other technologies supported by IntelliJ IDEA.

  • Feature branches building based on Mercurial bookmarks and Git tags.

  • TeamCity is able to detect and show changes made in Mercurial sub-repositories.

Check out more details and screenshots on the What’s New page and download TeamCity 8 now to try these great new features.

Happy building!
The TeamCity Team

Posted in News & Events | 5 Comments

TeamCity 8.0 RC is available (build 27390)

We’re glad to announce the availability of the TeamCity 8.0 release candidate. We worked hard polishing various rough edges and we hope you’ll appreciate our efforts.

A significant change in TeamCity licensing is worth mentioning. The change is targeted at the Professional Server license users. Starting from version 8.0, Professional Server users can increase the limit of build configurations by 10 with each additional Build Agent license. In other words, the maximum number of build configurations in a Professional Server is calculated as follows: the default 20 +10 * <N obtained Build Agent Licenses>.

Note that a Build Agent license does not give access to TeamCity Email support; for this you still need to buy the TeamCity Enterprise Server license.

Please try this RC and give us your feedback.

Happy building!

Posted in EAP | Leave a comment

New TeamCity EAP build (27323) is available

TeamCity 8.0 release is close and at the moment we’re mostly focused on enhancing the currently implemented functionality.
As a result, this EAP build does not have many new features but rather brings with it a number of bug fixes .

Please try it and tell us if some feature does not work for you the way it should or if something needs fixing.

Happy building!

Posted in EAP | Leave a comment

Improve Build Times with Compressed Artifacts

Philipp DolderThis guest blog post is from Philipp Dolder, Software Architect and Continuous Integration Expert at bbv Software Services AG, a new JetBrains Training and Consulting Partner. Philipp is an enthusiastic software architect, agile coach, continuous integration expert and joint-owner of open source projects FakeItEasy and Appccelerate. Follow him on Twitter @philippdolder.

The views expressed in this blog post are those of the individual, and do not necessarily represent the views of JetBrains.

Do you work with build chains and artifact dependencies? Do you wonder why it takes so much more build time when your artifacts get bigger and bigger?

You probably have the same potential for improvement as I had in my current project. Read on to learn how you can get your builds faster and waste less time waiting for your builds to complete. A fast feedback loop is of major importance, especially in agile software development.

The issue

In the project I’m currently working on, we have a fairly complex build process with many artifact dependencies between the various stages in our build chain. At some time we felt the build started slowing down. In the next retrospective meeting my team decided that I should analyze the build process and implement improvements to get us back to fast build times.

In this post I’ll only write about how I implemented the improvements. Let’s have a look at our build chain first:

BuildChain

In the above diagram you see the 4 stages in our build chain: build, test, installer, publish. You can see that the artifacts of the Integration-build (.NET) are used in four dependent builds: 3 test runs and the installer. These artifacts are about 1.2 GB in size and consist of thousands of dll’s and other small files. Now, keep in mind that the TeamCity system has to copy those artifacts 5 times per build (e.g. 6 GB in small files). Why 5? – You got it. Once from the Integration-build (.NET) build agent to the TeamCity server (artifact publishing) and 4 times to distribute them to the 3 test run and the create installer agents (artifact resolving).

Remember what you learned about reading and writing very small files from and to a hard disk? It is very slow compared to writing one large file with the same total size. Let’s have some numbers to see the impact on our build times: Resolving the build artifacts takes about 20 minutes for each of the 4 builds. So, it takes 80 minutes of CPU time just to resolve the artifacts in our build, which is triggered on every commit. As we are running these 4 builds in parallel, we actually “only” lose 20 minutes with artifact resolving, but it’s still too much. And it causes a higher CPU consumption on our virtualized server environment.

How can we improve and get our build faster?

The approach

Obviously we can save CPU time when we change our artifacts to a single large file. Don’t start changing your build script and use 7-zip command line to achieve that, yet!

MakeAZip

Let’s check out what TeamCity has to offer first. You probably read about it once you created your build configurations for the first time. At least that happened to me. I remember having read about zipping artifacts before but never gave a thought about it until now. Let’s get started.

The solution

In my project we build everything into a folder called results inside our working copy. For continuous integration builds we always build in Release mode; therefore, we also have a subfolder Release in our structure. Every C# project (.csproj) builds into its own subfolder which leads to the following folder structure:

resultsFolderStructure

How it was before

Our build artifacts for the Integration-build (.NET) should consist of everything inside the results folder. Initially our artifact paths in the General Settings looks like this:

artifactPathsBefore

That means everything inside the results folder and its subfolders will be published as build artifact recursively (excluding the results folder itself).

artifactDependencyBefore

That’s how my build configurations looked like before I started improving build times.

How it is now

We simply need to change the artifact paths of our Integration-build (.NET) build configuration to the following:

artifactPathsAfter

This will put everything inside the results folder into the results.zip file.

And in the build configurations depending on the artifacts we simply change the Artifacts rules to the following:

artifactDependencyAfter

And you’re done.

What is the impact?

When it comes to the build time, we improved the artifact resolving part of our build chain drastically. It was definitely more than I would have imagined. Having about 20 minutes to resolve artifacts for one dependent build before, we are down to about 6 to 7 minutes now. So we just cut the resolving time to a third by such a simple change.

ResolvingTimes

At the same time our artifact zip file shrank to a third of its initial size resulting in about 400 MB now. We also saved space and network traffic on our server.

ArtifactSize

Conclusion

We saved a lot of CPU time and network bandwidth with such a simple change in our build configuration without having to change our build script at all. But this is only the beginning of improving our continuous integration builds regarding the time it takes us for a complete cycle.

There is even more

This is just an incomplete list of additional options you should consider when further optimizing your build times.

Chain your build configurations properly…
to make sure you benefit from maximum parallelization of builds to have the shortest possible feedback time. Add additional build agents if necessary.

Filter unnecessary files from your artifacts…
when publishing them to minimize overall artifact size. TeamCity documentation provides more details for the configuration of artifact paths and dependencies.

Ensure good network connectivity and use fast hard disks…
to have shortest file transfer times possible.

Ensure fast unit and integration tests…
to avoid unnecessary long build times caused by long running tests that connect to real databases, unit tests that access the file system, etc.

Use TeamCity performance monitor…
to find bottlenecks, understand possible hardware upgrade impact, discover suspicious periods in the build when nothing happens and so on. See Maarten Balliauw’s blog post post for more details.

Posted in Partner, Tips&Tricks | Tagged , , | 4 Comments

TeamCity 8.0 EAP (build 27284) with new IDEA make

TeamCity 8.0 is coming and we’re thrilled to present the next EAP to you! Working hard, we have added several new features and we continue to make improvements of the existing ones.

New make in IDEA Project runner

We integrated a new IntelliJ IDEA external make bringing all of the goodies provided by this tool into TeamCity:

  • support for different languages like Groovy, Scala, Closure, etc.
  • ability to build Android projects and artifacts, as well as all other types of artifacts supported by IntelliJ IDEA itself
  • faster compilation thanks to the highly optimized IntelliJ IDEA make tool, plus the ability to build a project incrementally

Meta-Runner

All meta-runners are now stored on the project level. In other words, you can create a meta-runner which will be available in your project and its sub-projects only and won’t be visible outside. Also we have made several important fixes in the Meta-runner itself.

Other enhancements

There are other improvements including:

  • human assignable identifiers for VCS roots;
  • new icons in the web interface and IntelliJ plugin;
  • and so on.

You are most welcome to try new EAP and tell us about your experience. You can find the full list of changes in the release notes.

Happy building!

Posted in EAP | Leave a comment

Explaining the Shared Resources plugin

This post describes one of the new features in the upcoming TeamCity 8.0

Using TeamCity, we can easily set up a build farm using a central TeamCity server and several connected build agents. By having multiple build agents, we can run builds and build configurations in parallel. Different branches of the same project can build simultaneously on different build agents,  personal builds can be run on one build agent while the nightly build is running on another agent and so on.

But what if we have multiple build agents and we want to constrain parallelism for some projects? What if we do want to have parallelism but want to provide different build variables to each build? Good questions, and the Shared Resources plugin is the answer!

The Shared Resources plugin comes bundled with TeamCity 8.0 EAP. In TeamCity 7.1.x it has to be installed separately. Sources for this plugin are available on GitHub.

Limiting concurrent builds with simple named resources

The Shared Resources plugin works at the project level. Per project, we can specify one or more shared resources which we can use to decide on the number of concurrent builds for a given project and build configuration.

In the Project Home under the Shared Resources tab, we can specify the resources we want to be able to lock. A shared resource should be given a name which can be used later on in our builds. The resource type can be infinite (effectively not limiting build concurrency), a resource with quota and a resource with custom values. Let’s go with resource with quota.

The resource quota is the number of concurrent read locks that can be acquired on the resource. Or in other words: the maximum number of concurrent builds we want to run. Let’s be restrictive and set the quota to 1.

Add Shared Resource in TeamCity

None of the Build Configurations will obey these quota: we have to add a Shared Resources Build Feature to every Build Configuration we want to subject to these quota. This can be done in the Build Configuration. Under Build Features, we can add a Shared Resources build feature which acquires a read or write lock on the shared resource created earlier.

Add Build Feature

Once added, the build configuration will only be run on one build agent at a time. Personal builds as well as regular builds will all be enqueued and wait for the lock on the shared resource to be released, even if one of the build agents is idle.

Note that it is possible to await multiple locks in a build configuration.

Working with custom resource values

While working with simple named resources may be useful, it may be worth having a custom resource value for every lock acquired. Imagine we have a build which has a custom build step running load tests on a server. Wouldn’t it be nice if we could limit the number of concurrent build based on the number of load test targets available and be able to get the URL to one of these servers in our build? We can using a set of custom resource values.

When creating a new Shared Resource, we can specify the type as a resource with custom values:

Custom resource values

In our build, we have several options available to acquire a lock on this shared resource. We can lock on any value that is not locked, we can lock on all values or on a specific value. For our purpose of needing a dedicated load testing machine, we can opt to lock on any available value.

List of resources lock

Once added, we will not be able to run more builds concurrently than the number of available values in our custom shared resource. If we need the value of the locked value during our build, we can read it using the %teamcity.locks.<lockType>.<lockName>% variable, %teamcity.locks.readLock.RunTestsAgainstServer% in our case. For example when working with build parameters, we can use the lock value as a custom parameter:

Custom parameter based on shared resource lock value

Where can I manage shared resources?

So far we’ve seen that shared resources can be added for every project separately and used in that project’s build configurations. With TeamCity 8.0, we’ve introduced project groups which allows us to group several projects under a project group. Shared resources can be added to both the project group as well as the individual projects. Locks can be acquired for every build configuration in a project and/or project group, making it possible to have shared resources across projects as well.

Do let us know your thoughts via the comments below. Dive in, experiment and… happy building!

Posted in EAP, Features, Tips&Tricks | Tagged , , , , , | 3 Comments

TeamCity User Guide Courseware

We have created a TeamCity User Guide as a series of screencasts.

TeamCity User Guide

An outline of the table of contents is:

The entire course duration is under an hour. You can view it entirely for free either on JetBrainsTV or on our YouTube Channel. Enjoy!

Posted in How-To's | Leave a comment