TeamCity 8.0.3 (build 27540) is ready

Please welcome another bugfix release build with a bunch of improvements.

Let us mention some of the most important:
- The issue with caching server side checkout patches which caused significant performance problems in some installations is now fixed.
- We’ve found and fixed a defect with build triggering , thanks to Niek Bartholomeus.
- Some icons in the web interface were tuned to better suit red/green color-blind people, and we hope TeamCity is friendlier to you now!
- Dependencies of Ant network tasks (FTP, sshexec) are now bundled with TeamCity.

Refer to our release notes for the complete list of fixed issues.

Download page

Happy building!

The TeamCity Team

Posted in News & Events | 2 Comments

The Power of Meta-Runner – Custom runners with ease

TeamCity comes with a series of Runners out of the box which offer support for different functionalities such as MSBuild, Ant, Code Inspections, Gradle support, etc. In addition to these, there are a variety of runners that are supported via plugins, from FTP uploads to Grails and other things.

TeamCityDefaultRunners

What happens when you want to perform some operation that is not supported via a runner? We could use the Command Line runner or build or own plugin. Each have their own benefits and disadvantages. The first one is quite straightforward, but the second offers more flexibility and a better user experience. TeamCity 8 introduced a third option: Meta-Runners.

Meta runners allow us to reuse build steps by representing them as a native TeamCity runner. In other words: we can extract configured build steps into Meta-Runners and reuse them in other build configurations. A good example are the PHP Meta-Runners from the Meta-Runner Power Pack. While TeamCity doesn’t come with PHP-specific runners, we have extracted some build steps into Meta-Runners that do make it easy to do Continuous Integration with PHP.

The following video demonstrates creation of a Meta-Runner:

Voilà! With just some clicks, we’ve created a new runner that we can re-use in other projects and build configurations.

Assumed FAQ

Guessing some of the questions that might come up, let’s try and address them:

1. Do I need a script file to create Meta-Runners?

No. You can base a Meta-Runner off of any existing runner. The UI is defined by the Build Parameters you create. It is important to note however that anything the Meta-Runner uses (for example a specific runtime or executable) should be available on the agent. One way of doing this is adding a step in the build process to prepare the environment, downloading any external dependencies. The Meta-Runner Power Pack contains an example Meta-Runner which uses Ant to download a file from a given URL.

2. Do Meta-Runners belong to the Root Project or a specific Project?

They can belong to either of them. In this example, we created it at the project level, but you can also create it at the root project level.

3. How do I delete them?

They are located on a new tab under the Project Settings.

image

4. Are there any limitations?

Currently:

  • When extracted the runner, via the UI you cannot control which steps to extract. All steps are extracted.
  • Extract meta-runner UI does not allow to re-order the parameters. However, once meta-runner is extracted you can reorder them manually in the resulting xml file.

5. Sounds cool, but which Meta-Runners can I build?

Anything, really. Whenever it feels you’re duplicating build steps across build configurations, they are perfect candidates to extract a Meta-Runner. We already have some Meta-Runners that can be used in the Meta-Runner Power Pack. Feel free to contribute yours if they may be of use for others!

Try this and the other TeamCity 8 features and give us some feedback through the comments below!

Happy building!

Posted in Features, How-To's, Tips&Tricks | Tagged , , , , | 2 Comments

TeamCity 8.0.2 (build 27482) is out

TeamCity 8.0.2 bugfix build is available now. It addresses several important issues and regressions, as well as some performance problems. We strongly recommend that you upgrade to this version, especially if you experienced problems with TFS, email notifications or slow builds finish. Additionally, there are general enhancements increasing the server performance.

If you upgrade from version 8.0, installing 8.0.2 build preserves the data format (this is usually the case with all our bugfix builds). In case you encounter any problems with build 8.0.2, you can safely revert to 8.0.1 or even 8.0 without the need to restore from backup.

Release notes
Download page

Happy building!

Posted in News & Events | Leave a comment

First-class PHP Continuous Integration using TeamCity

A while ago we blogged about using TeamCity to do Continuous Integration for PHP. TeamCity developers never seem to sleep: with TeamCity 8 a new feature called meta-runners was introduced. Meta runners allow us to reuse build steps by representing them as a native TeamCity runner. In other words, they allow us to make several build steps easier. Which is exactly what we decided to do: create some meta runners based on our previous blog post and make it really easy to run several PHP-related build steps.

image

The Meta-runner Power Pack for TeamCity 8 contains a set of PHP-related meta runners:

  • Run arbitrary PHP code using the PHP runner
  • Install a specific Composer dependency using the Install Composer package runner
  • Install/update packages from composer.json using the Run Composer runner
  • Run Phing build script using the Phing runner
  • Run PHPUnit tests (with code coverage reporting) using the PHPUnit runner

There is only one requirement: a PHP runtime should be installed on our build agent. The Phing, PHPUnit and Composer meta runners will automatically download the latest versions of the executables during the build! Full documentation for these PHP meta runners is available on GitHub.

Installing meta runners

After downloading the PHP meta-runners from GitHub, all files named MRPP_PHP_<name>.xml should be copied into the <TeamCity Data Directory>/projects/<Project ID>/pluginData/metaRunners/ directory on our TeamCity server.

Once this is done, the TeamCity server will list these new meta-runners under the administration page for the project in which the meta runners have been installed:

image

Using the PHPUnit runner

Since a big part of Continuous Integration is making sure the quality of our software is high, running unit tests is typically part of a build configuration. In the PHP world we typically use PHPUnit for this.

The PHPUnit runner allows running PHPUnit tests during the build and will optionally publish code coverage results to TeamCity. A build step using this runner will use a PHPUnit executable available on the build agent if specified, or automatically download the latest phpunit.phar from the PHPUnit.de website during the build.

To run unit tests, all we have to do is specify the path to our unit tests (but don’t worry, we can also tweak PHPUnit parameters):

image

This will run and report back unit test results to the TeamCity server.

image

And if we enable the Collect code coverage option, we get code coverage reports integrated with TeamCity as well:

image

Bugfixes and improvements for these meta runners (or your own, PHPLint and phpDocumentor runners would be nice!) are welcome through our GitHub repository.

Haven’t tried TeamCity? Give it a shot! It’s 100% free for up to 20 build configurations.

Let us know your thoughts through the comments below!

Happy building!
– The TeamCity Team

Posted in Features, How-To's, Tips&Tricks | Tagged , , , , , | 7 Comments

TeamCity 8.0.1 (build 27435) is available

The first bugfix update for version 8.0 is already available!

It has a fix for regression in the Powershell runner (TW-29803), a few performance improvements in the Git agent side checkout when SSH is used (TW-14598), fixes in custom charts (TW-29792), and a number of other important fixes.
Download the fix now and refer to the Release notes for the full list of resolved issues and improvements.

Happy building!
The TeamCity Team

Posted in Tips&Tricks | Leave a comment

TeamCity 8 — Welcome to The Light Side of Continuous Integration

It’s been a while since the last major release of TeamCity, and today we’re excited to announce that new TeamCity 8 is here and available for download. We carefully listened to everything you told us and did our best to give you the product which exceeds your expectations.

As with every release, the whole list of new features and improvements is very long and you can surely learn about all of them, but let’s take a look at some of the most noteworthy ones.

TeamCity is now more affordable for small teams

If you are using Professional Server, you can extend the number of build configurations by purchasing an additional Build Agent. Each purchased Build Agent gives you 10 more build configurations.

If you already have additional agents and your licenses maintenance period covers TeamCity 8.0 release date, good for you! Just upgrade to TeamCity 8 and start receiving profit.

New project hierarchy

The project hierarchy in TeamCity 8 allows you not to worry about the growing number of projects and configurations. You’re able to group several projects under a parent one and tune access, VCS settings, and other parameters for the whole projects tree.

Your server health reports

No matter how well you keep an eye on your build environment, there is always something out of your field of view: unused agents, huge build logs, redundant VCS roots and so on. The new Server Health page is here to show you these issues.

Other important features include:

  • a much faster (up to 5 times) build history cleanup, allowing to drastically reduce server maintenance time;

  • Meta-Runner allowing you to reuse build steps by representing them as a native TeamCity runner,

  • improved build problems reporting: detection of new problems, ability to assign an investigation or mute a build problem.

  • The native IntelliJ IDEA compiler is now bundled with TeamCity 8, which means that for IntelliJ IDEA projects you get much faster incremental compilation and support for Scala, Groovy, Clojure, Kotlin, Android, GWT and many other technologies supported by IntelliJ IDEA.

  • Feature branches building based on Mercurial bookmarks and Git tags.

  • TeamCity is able to detect and show changes made in Mercurial sub-repositories.

Check out more details and screenshots on the What’s New page and download TeamCity 8 now to try these great new features.

Happy building!
The TeamCity Team

Posted in News & Events | 5 Comments

TeamCity 8.0 RC is available (build 27390)

We’re glad to announce the availability of the TeamCity 8.0 release candidate. We worked hard polishing various rough edges and we hope you’ll appreciate our efforts.

A significant change in TeamCity licensing is worth mentioning. The change is targeted at the Professional Server license users. Starting from version 8.0, Professional Server users can increase the limit of build configurations by 10 with each additional Build Agent license. In other words, the maximum number of build configurations in a Professional Server is calculated as follows: the default 20 +10 * <N obtained Build Agent Licenses>.

Note that a Build Agent license does not give access to TeamCity Email support; for this you still need to buy the TeamCity Enterprise Server license.

Please try this RC and give us your feedback.

Happy building!

Posted in EAP | Leave a comment

New TeamCity EAP build (27323) is available

TeamCity 8.0 release is close and at the moment we’re mostly focused on enhancing the currently implemented functionality.
As a result, this EAP build does not have many new features but rather brings with it a number of bug fixes .

Please try it and tell us if some feature does not work for you the way it should or if something needs fixing.

Happy building!

Posted in EAP | Leave a comment

Improve Build Times with Compressed Artifacts

Philipp DolderThis guest blog post is from Philipp Dolder, Software Architect and Continuous Integration Expert at bbv Software Services AG, a new JetBrains Training and Consulting Partner. Philipp is an enthusiastic software architect, agile coach, continuous integration expert and joint-owner of open source projects FakeItEasy and Appccelerate. Follow him on Twitter @philippdolder.

The views expressed in this blog post are those of the individual, and do not necessarily represent the views of JetBrains.

Do you work with build chains and artifact dependencies? Do you wonder why it takes so much more build time when your artifacts get bigger and bigger?

You probably have the same potential for improvement as I had in my current project. Read on to learn how you can get your builds faster and waste less time waiting for your builds to complete. A fast feedback loop is of major importance, especially in agile software development.

The issue

In the project I’m currently working on, we have a fairly complex build process with many artifact dependencies between the various stages in our build chain. At some time we felt the build started slowing down. In the next retrospective meeting my team decided that I should analyze the build process and implement improvements to get us back to fast build times.

In this post I’ll only write about how I implemented the improvements. Let’s have a look at our build chain first:

BuildChain

In the above diagram you see the 4 stages in our build chain: build, test, installer, publish. You can see that the artifacts of the Integration-build (.NET) are used in four dependent builds: 3 test runs and the installer. These artifacts are about 1.2 GB in size and consist of thousands of dll’s and other small files. Now, keep in mind that the TeamCity system has to copy those artifacts 5 times per build (e.g. 6 GB in small files). Why 5? – You got it. Once from the Integration-build (.NET) build agent to the TeamCity server (artifact publishing) and 4 times to distribute them to the 3 test run and the create installer agents (artifact resolving).

Remember what you learned about reading and writing very small files from and to a hard disk? It is very slow compared to writing one large file with the same total size. Let’s have some numbers to see the impact on our build times: Resolving the build artifacts takes about 20 minutes for each of the 4 builds. So, it takes 80 minutes of CPU time just to resolve the artifacts in our build, which is triggered on every commit. As we are running these 4 builds in parallel, we actually “only” lose 20 minutes with artifact resolving, but it’s still too much. And it causes a higher CPU consumption on our virtualized server environment.

How can we improve and get our build faster?

The approach

Obviously we can save CPU time when we change our artifacts to a single large file. Don’t start changing your build script and use 7-zip command line to achieve that, yet!

MakeAZip

Let’s check out what TeamCity has to offer first. You probably read about it once you created your build configurations for the first time. At least that happened to me. I remember having read about zipping artifacts before but never gave a thought about it until now. Let’s get started.

The solution

In my project we build everything into a folder called results inside our working copy. For continuous integration builds we always build in Release mode; therefore, we also have a subfolder Release in our structure. Every C# project (.csproj) builds into its own subfolder which leads to the following folder structure:

resultsFolderStructure

How it was before

Our build artifacts for the Integration-build (.NET) should consist of everything inside the results folder. Initially our artifact paths in the General Settings looks like this:

artifactPathsBefore

That means everything inside the results folder and its subfolders will be published as build artifact recursively (excluding the results folder itself).

artifactDependencyBefore

That’s how my build configurations looked like before I started improving build times.

How it is now

We simply need to change the artifact paths of our Integration-build (.NET) build configuration to the following:

artifactPathsAfter

This will put everything inside the results folder into the results.zip file.

And in the build configurations depending on the artifacts we simply change the Artifacts rules to the following:

artifactDependencyAfter

And you’re done.

What is the impact?

When it comes to the build time, we improved the artifact resolving part of our build chain drastically. It was definitely more than I would have imagined. Having about 20 minutes to resolve artifacts for one dependent build before, we are down to about 6 to 7 minutes now. So we just cut the resolving time to a third by such a simple change.

ResolvingTimes

At the same time our artifact zip file shrank to a third of its initial size resulting in about 400 MB now. We also saved space and network traffic on our server.

ArtifactSize

Conclusion

We saved a lot of CPU time and network bandwidth with such a simple change in our build configuration without having to change our build script at all. But this is only the beginning of improving our continuous integration builds regarding the time it takes us for a complete cycle.

There is even more

This is just an incomplete list of additional options you should consider when further optimizing your build times.

Chain your build configurations properly…
to make sure you benefit from maximum parallelization of builds to have the shortest possible feedback time. Add additional build agents if necessary.

Filter unnecessary files from your artifacts…
when publishing them to minimize overall artifact size. TeamCity documentation provides more details for the configuration of artifact paths and dependencies.

Ensure good network connectivity and use fast hard disks…
to have shortest file transfer times possible.

Ensure fast unit and integration tests…
to avoid unnecessary long build times caused by long running tests that connect to real databases, unit tests that access the file system, etc.

Use TeamCity performance monitor…
to find bottlenecks, understand possible hardware upgrade impact, discover suspicious periods in the build when nothing happens and so on. See Maarten Balliauw’s blog post post for more details.

Posted in Partner, Tips&Tricks | Tagged , , | 4 Comments

TeamCity 8.0 EAP (build 27284) with new IDEA make

TeamCity 8.0 is coming and we’re thrilled to present the next EAP to you! Working hard, we have added several new features and we continue to make improvements of the existing ones.

New make in IDEA Project runner

We integrated a new IntelliJ IDEA external make bringing all of the goodies provided by this tool into TeamCity:

  • support for different languages like Groovy, Scala, Closure, etc.
  • ability to build Android projects and artifacts, as well as all other types of artifacts supported by IntelliJ IDEA itself
  • faster compilation thanks to the highly optimized IntelliJ IDEA make tool, plus the ability to build a project incrementally

Meta-Runner

All meta-runners are now stored on the project level. In other words, you can create a meta-runner which will be available in your project and its sub-projects only and won’t be visible outside. Also we have made several important fixes in the Meta-runner itself.

Other enhancements

There are other improvements including:

  • human assignable identifiers for VCS roots;
  • new icons in the web interface and IntelliJ plugin;
  • and so on.

You are most welcome to try new EAP and tell us about your experience. You can find the full list of changes in the release notes.

Happy building!

Posted in EAP | Leave a comment