2015 TeamCity Plugin Contest: Don’t Hesitate to Participate

We are announcing the first ever TeamCity Plugin Contest! Whether you are new to TeamCity or a devoted fan, we believe you should be limited by your creativity alone and not by the tool you use, so how would you like to win up to $7000 in Amazon certificates, enterprise licenses for team tools, personal licenses for IDEs, or even a trip to one of JetBrains’ offices?

It’s easy – develop your own plugin to TeamCity! TeamCity has a flexible architecture which can be extended in a number of ways (runners for new build tools and testing frameworks, notifiers, etc.) and the contest is straightforward: create an innovative plugin to TeamCity or add some significant features to one of the existing plugins, and take a prize home!

Submissions start today and close at 11:59 AM (EDT) on June 15, 2015! Don’t hesitate to enter: winner or not, everyone who submits a valid plugin gets a free personal license for any JetBrains IDE!

See the full information on the prizes and conditions on our site and register to enter the contest!

The TeamCity documentation provides lots of information to help you write a plugin, and with our Getting Started guide you can create a simple plugin in a matter of minutes!

If you need inspiration, there are also a bunch of possible plugin ideas in our tracker for you.

We are sure you’ll have plenty of feedback while writing your plugin, and you are most welcome to post comments or issues to our forum or issue tracker!

Good luck and happy building!

Posted in Blogroll, Contest, FYI, News & Events | 3 Comments

Case Study: TeamCity in Clueda

This guest post is brought to you by Clueda engineering team.

cluedaClueda is a cognitive computing solutions provider. We extract knowledge and market moving events from infinite amounts of unstructured data in real-time. The Best in Big Data 2013 award-winning technology is based on over 100 man-years of research in systems biology and cognitive computing. The successful implementation of our software helps traders work more efficiently and increase trading profits on a daily basis. Founded in 2012 as spin-off from the elite research institution Helmholtz Zentrum Munich, Clueda currently has 25 employees.

The Pain

Within two years, Clueda has experienced significant growth, both in the size of our engineering team and complexity of our products. While the previously used development infrastructure worked well to build a product that our customers praised, we were looking for tools options that could sustain our growth and maintain our products at a high quality.

In the original small-team workflow, developers built projects on their own machine and manually deployed artifacts to an Artifactory repository. This is simple and straightforward. However, with the number of repositories reaching over 250 at a team size of about 20 developers, keeping track of the status and dependencies of the myriad of experimental and productive projects in development at any point in time became a prime issue.

Best practices suggest a continuous integration workflow: Instead of manually deploying builds to an artifact storage or even testing or productive system, we wanted to automate as many of such error-prone tasks as possible. Ideally, a testing system will always be updated with the most current state of the code in source control. But let’s be humble: an obvious starting point is centralizing compilation, testing and packaging of software artifacts.

Solution Space

There are many tools designed to help in the task of automated building and deploying, the most prominent ones being Jenkins, TeamCity and Bamboo. But before looking at the contenders, we need to identify the key points to be addressed.

Most of our development is JVM-centric, with Scala as the primary programming language. We use git for source control, sbt and Maven as build tools. A good solution would make it easy to run both and report results in a uniform and easily comprehended fashion.

Configuring build processes is not a one-off task. Requirements evolve, new steps are added, configuration changes. With dozens of projects at any one time, it is absolutely necessary that a solution be able to centrally manage build configurations and apply them to multiple projects at once, while still allowing individual projects to adapt and override these settings.

A preliminary survey showed that only the three “big guns” would fit our overall requirement for flexibility and compatibility with all our projects.

Looking at them in detail, we found:

  • Bamboo was a pleasurable experience in regards to usability, but it lacked the key features we needed in the first place.

  • Jenkins certainly has the largest ecosystem and rudimentarily supports sbt as a build tool. There is a Jobs DSL plugin available that may support scripting more complex tasks and some level of abstraction on the build process.

  • TeamCity is the only tool we found that has first-class support for build templates, which were straightforward to setup.

Decisions

TeamCity quickly turned out to be the option that promised to save the most time. During evaluation, we had direct interaction with TeamCity developers, who were very responsive to our questions. This made the final decision very easy.

Relief

After 2 months of using TeamCity productively, we have automated many parts of building and deployment of all active projects. Code quality checks, such as Scalastyle and a self-built configuration verification, were easy to introduce into all configurations and have greatly simplified maintaining and improving our coding standards.

Whenever a developer pushes changes to their git repository, TeamCity pulls these changes and triggers the build process. After a successful build, which includes compiling, tests and code quality checks, development artifacts are deployed directly to Artifactory. Release builds are triggered manually and run through the sbt-release plugin, which ensures correct versioning and disables use of snapshot artifacts in a release, while TeamCity keeps track of the changes since the last release version. Nightly test runs ensure that our artifacts are always in a compatible state.

Outlook

Because it is simple to improve the process, we are not content with the status quo. There’s plenty of ideas on how to improve processes. Here are some of our ideas.

Documentation

TeamCity already enables a direct view on the documentation generated for each build. We will consolidate generated docs and automatically deploy them to a searchable website.

Dependency Management

Dependency Hell is cause for a large share of the swearwords uttered in software development offices. Centrally detecting such errors should lead to a more tranquil environment.

Deployment

Continuous integration is seen by some as an old hat. Continuous deployment involves bringing the code from source control to production fully automated. TeamCity does not offer specific features for this, but it allows arbitrary deploy processes to be integrated into the environment, thus simplifying tracking the success of such operations.

Visualization

Visualization can improve team engagement. There are a variety of plugins available for this task. Unfortunetely, none of them really fit what we had in mind. Luckily, TeamCity offers an extensive REST API

Posted in Blogroll, Case study, Features, FYI, Guest post, News & Events, Partner | Leave a comment

TeamCity 9.0.3 is available

Please welcome TeamCity 9.0.3 (build #32334) with more than 150 bugfixes and improvements.

We added a couple of new options to projects import to control how import of users and groups should be performed in case of conflicts.

With this release we bundled new version of ReSharper Command Line Tools. This should fix a number of issues found in .NET Inspections runner. TeamCity 9.0.3 also adds support for NUnit 2.6.4.

Versioned settings feature have got several important stability fixes. Also a number of fixes have been done in WebSockets support. Finally, we fixed several performance problems mainly affecting large installations. See our release notes for the full list.

Given the number of fixed problems in this release, upgrade to 9.0.3 is highly recommended. Please download the build and follow the upgrade instructions. As usual upgrade to this bugfix release from another 9.0 version does not change data format, so in case of any problems downgrade will be simple.

As the team is switching their efforts to TeamCity 9.1 (which is planned for the end of Spring), this is probably the last 9.0.x bugfix release with so many fixes. We hope to publish the first 9.1 EAP build in a matter of weeks.

Stay tuned and happy building!

Posted in Bugfix, Uncategorized | Comments Off

Setting up TeamCity as Symbol and Source Server

When debugging applications with various Microsoft tools, you need to have symbol and source information, provided by the Microsoft Symbol Server and Source Server technology.

It’s been some time since we created a plugin which allows using TeamCity as a symbol and source server, and now, after some bug-fixing has been done, we decided we are ready to announce it officially.

So please welcome the TeamCity Symbol Server plugin! With just a few mouse clicks you can set up TeamCity to index the .PDB files and binaries for all your .Net applications and, once the plugin is enabled, you can easily debug applications or analyse crash dumps: your debugger will be able to automatically retrieve the required symbol files and to download the sources of the appropriate version for shared libraries referenced by you application, if, say, these modules are developed by a different team in your company and shared between several projects.

Prerequisites

To set up TeamCity as a Symbol and Source server, we assume you have the following:

  1. TeamCity server 8.0.3 or later with a project and build configuration to build your application.
  2. Debugging Tools for Windows on the agent machine(s) where your application is built (can be installed using this link).

Now all you need to do is enable the TeamCity Symbol Server plugin in TeamCity and set a number of options in your debugger.

Download and install the plugin

  1. Since the plugin is not bundled with TeamCity, download the required version from here and install the plugin as usual.
  2. Now enable the symbol server on the Administration| Integrations| Symbol Server page. You will see that it is referenced by the following URL < TEAMCITY_SERVER_URL>/app/symbols. This is the URL to be used in the Visual Studio or windbg settings.

The permissions required to access symbols and sources via the TeamCity server are default for the project developer role, and the indexed data can be accessed in both public (not requiring authorization) and authorized modes.

Tune the build configuration settings

Next we need to add the Symbol Files indexer build feature to our build configuration settings:

AddBuildFeature

Once you add the feature and save the settings, all PDB files published as build artifacts of this configuration and all related sources will be indexed.

Next let’s take a look at the files we are publishing as build artifacts. In this setup we are building an application referencing a dll and we are publishing a .zip archive which includes the application along with the PDB file and the library:

Artifacts

The plugin can handle the PDB files published separately or packed using the TeamCity artifact publishing rules.

That’s it! Now we can run the build. When the build is finished, the build log shows that source indexing was performed and symbols files signatures were collected and written to an xml file:

buildLog

The file containing symbol files signatures is published internally and is available in the TeamCity internal artifacts under .teamcity\symbols:

publishedArtifacts

Set the debugger options

Now we need to set a few options in the debugger, Visual Studio in this case:

  1. In the Debug menu, select Options and Settings, in the General section uncheck the default Enable Just My Code and Require source files to exactly match the original version options.
    If you do not have the source code on the local machine or the .pdb file does not match the source code, you can use Source Server to find and download the sources: select Enable source server support and check all the options in that section.VSDebugOptions
  2. During a debugging session in Visual Studio, the debugger automatically loads symbol files that are located in the project directory. To specify TeamCity as a symbol server, select the Symbols section on the left and add the TeamCity symbol server URL ( from the Administration| Integrations| Symbol Server page) to the list of symbol file locations.
    On the Symbols page you can also filter out or specify modules that you want the debugger to automatically load symbols for. These settings can be changed manually when you are debugging.VSSymbolSources
  3. Save your changes.

Now your debugger will load the required symbol and source files from TeamCity enabling you to successfully debug applications and crash dumps!

Try out the TeamCity Symbol Server plugin and let us know what you think!

Happy building and debugging!

Posted in Features, FYI, How-To's, News & Events, Tips&Tricks | Comments Off

Dashboarding with TeamCity

Data visualization has never been so popular as it is today. The human eye and brain are so powerful that they catch information and detect patterns on-the-fly: just in a blink of an eye you can extract a lot of information and make a decision on what to do next. Dashboards are everywhere for business and fun.

Here at JetBrains we have a ‘KitchenTV’ project – a set of pages rotated on several large screens in the office. In the beginning they were charts with data from our issue tracker, lists of tweets, nice widgets with weather information for all JetBrains office locations. Then the real-time visualization of the daily purchases was added – every time one of our products is bought, a ball of the corresponding color falls down forming several nice different-sized and -colored mountains at the end of the day, which are a lot of fun to look at.

Now we decided to contribute to this project and add some useful information from our internal TeamCity server for the common benefit, and we thought you might be interested too, so here it is.

At the moment we have two widgets. The first one shows the list of assigned investigations – so your colleagues can see your involvement in problem digging and fixing.

Screen Shot 2015-02-09 at 14.35.37

The next widget displays the list of most recent VCS commits. It means that anyone can quickly understand what their teammates are working on.

Screen Shot 2015-02-13 at 14.45.16

The technologies behind these widgets are simple: REST + AngularJS, so it is basically the static content. If you want to use the widgets with your own TeamCity, you can choose one of two options:

  • install the TeamCity.Widgets plugin to your TeamCity or

  • install the TeamCity.StaticUIExtention plugin and add widgets content as a folder under the StaticUIExtention config root. Once you restart your server to make TeamCity.StaticUIExtention work, you can update the static content without further restarts.

The code of the plugin is open and available on GitHub.

Your feedback is welcome: in the comments here or in our issue tracker.

Enjoy the widgets and happy building!

Posted in Uncategorized | 3 Comments

TeamCity 9.0.2 is here

Greetings, everyone!

Today we are announcing the release of TeamCity 9.0.2!

This bugfix is rather important as it addresses about 100 issues. The fixes affect:

  • backup performance (TW-39612)
  • LDAP synchronization (TW-39589)
  • mechanism of applying changes in versioned settings (TW-39519, TW-39553)
  • recognition of Visual Studio Online URLs on the “create project from URL” page (TW-39730)

In addition, we updated the bundled JetBrains dotCover to version 3.0, and added the ability to reorder custom charts on the “Project Statistics” page. See our Release Notes for the full list of fixes.

As usual, bugfix releases do not change data format, so do not hesitate to upgrade if you are already running TeamCity 9.0!

Download the build and follow the upgrade instructions!

Your feedback will be greatly appreciated!

Happy building!

Posted in EAP, Features, News & Events, Tips&Tricks, Uncategorized | 2 Comments

What’s New in TeamCity 9, Webinar recording

The recording of our January 21st webinar with Wes Higbee, What’s New in TeamCity 9, is available on JetBrains YouTube channel.

In this webinar, Wes goes over the new features of TeamCity 9, namely: rearranging projects with Project Import; storing settings in VCS; creating and editing Custom Charts; running builds in Cloud Agents; as well as some other improvements.

We were glad to see tons of questions from the attendees. Below are select Q&A from the webinar.

Continue reading

Posted in Features, How-To's, Webinar | Comments Off

How Groovy uses TeamCity

This guest post is brought to you by Cédric Champeau, a Senior Software Engineer working on the Groovy language at SpringSource/Pivotal.

NB: starting today Groovy is looking for a new home.

Upgrading the continuous integration chain

For several months now, the Groovy language has been building a new continuous integration architecture based on TeamCity. We’re running it on a dedicated server sponsored by Jetbrains. One of the key reasons for moving to a dedicated server was the long feedback loop that we had on the Codehaus infrastructure and that classic open-source project CI infrastructures like Travis, based on slow cloud based instances offer: as an example, a minimal build on Travis took as much as 45 minutes. It may sound reasonable for a language like Groovy, but considering that the same build takes around 5 minutes on a recent laptop, there’s something very wrong. And there’s more than a single build to do:

The Groovy project has for example very strict requirements in terms of compatibility with several JDKs. We are testing Groovy against multiple versions of the JDK:

  • OpenJDK 5, for any Groovy version before 2.2
  • OpenJDK 6, the minimal JDK version for Groovy 2.2+
  • OpenJDK 7, with two different runtime versions: legacy and invoke dynamic
  • OpenJDK 8, with the same runtime versions as JDK 7

In addition, we test Groovy against early release versions of OpenJDK 7, 8 and even 9! We do that to report bugs to the OpenJDK team before new versions of the JDK go out in the wild. It has become more and more important lately since JDK 7 and the advent of invokedynamic: we have seen many buggy releases of the JDK breaking Groovy, so we really had to find a solution to avoid this as much as possible.

teamcity1

In addition to the combination of JDKs to be tested, Groovy is a widely used language, meaning that we have to support multiple versions. In particular, bugfixes are often backported on multiple branches. This means that we have more than one branch alive. Those days, we maintain Groovy 2.2.x, Groovy 2.3.x, Groovy 2.4.x and 2.5 is coming.

Last but not least, pull requests from the team or the community need to be pre-tested, increasing the combination of builds that we need.

For all those reasons, running TeamCity on a dedicated server looked like the best solution we found. It was also in particular important because TeamCity has the concept of a build agent. Build agents are important because they allow increasing the build capabilities externally. We haven’t done it yet, but one of the key aspects that made us lean towards TeamCity was the ability to build a group of “community driven” agents that would let us test Groovy on a wider variety of environments. In particular, the current builds are all based on a Linux x64 box, but Groovy needs to run on more than that: Windows, Mac OS, …​ All those environments need to be tested, but are not, because of the lack of build resources for that. This means that in the current architecture, we only rely on some community members who build on Windows or Mac OS to tell us if a build works or not. But we know that with TeamCity, we have a way to improve the situation in the future.

teamcity2

So imagine that each commit on the Groovy project needs to be tested on a combination of JDK, branch, environment, plus pull requests. This requires a lot of work, but TeamCity greatly simplifies this. In particular, the branch combination is reduced to simple configuration : on a build plan for JDK 7 for example, we can choose which branches need to be built. That dramatically reduces the effort needed to have a proper CI toolchain working.

From building to releasing

Another aspect of choosing TeamCity was the ability to simplify the release process. Before the new CI infrastructure was setup, releasing a new Groovy version was a bit complex. It involved building locally, checking the build, creating a tag, pushing the tags on VCS, uploading the distribution through WebDAV, or even the documentation through WebDAV. Last but not least, we also had to take care of synchronizing with Central through the Codehaus repository. Literally, it took hours. Releasing a Groovy version could take as long as a full day.

Since we migrated to TeamCity, we took advantage of using Gradle and Bintray to simplify the process. Using the Artifactory plugin and some configuration, releasing a new version of Groovy is as easy as filling a form in TeamCity.

teamcity3

Everything is built on the CI server, packaged, uploaded to Bintray and synchronized with Central using a single click. Lately we even added support for GVM directly in the release process on TeamCity, meaning that a new Groovy release is automatically promoted on GVM! There is still some manual process involved (for example, the CI server will not decide for itself if a build is considered a major version or not, nor will it automatically update the website to announce a release, or close JIRA versions for us). But what is interesting is that the overall process can be completed in less than an hour, including building, testing, and uploading. It’s a huge win.

teamcity4

A Groovy plugin for TeamCity

Interestingly, automating the release process soon required capabilities that TeamCity didn’t have out of the box. For example, notifying GVM, cleaning artifacts produced by pull requests, uploading the distribution to Bintray, calling the Artifactory REST API, …​ All those required scripting capabilities. For that reason, we developped a plugin for TeamCity which allows writing a build step as a Groovy script. The plugin gives access to a lot of information available in the build, like the build number, the branch name, etc…​ allowing us to trigger specific behavior based on that contextual information. For example, building all those branches on multiple JDKs is very nice, you soon realize that you are filling the disk with artifacts that you don’t want to keep. Unfortunately TeamCity will not let you have custom cleanup rules based on particular variables of a single build configuration. In our case, we wanted to remove the artifacts only for branches built from pull requests, and that is done through a Groovy script used as a build step.

Conclusion

In conclusion, we can say that we are very happy with the current status of our continous integration toolchain. We went as far as being able to release directly from the CI server, allowing us to dramatically reduce the errors related to a long, unfriendly human process, as well as dramatically improving our feedback loop (multiple JDKs, pull requests, …).

While the process can certainly be improved, for example by adding support for adding community build agents, doing it with TeamCity was very easy, so we’re very happy that we chose it!

Posted in Blogroll, Guest post, Partner | Comments Off

What’s New in TeamCity 9, January 21st Webinar

Join us Wednesday, January 21st, 12:00 PM – 1:00 PM EST (17:00 – 18:00 GMT) for our free live webinar, What’s New in TeamCity 9 with Wes Higbee.

In this webinar, Wesley Higbee will go over some of the new and exciting features of TeamCity 9.0. He will demonstrate rearranging projects among servers with Project Import; working with Versioned Settings; creating and configuring Custom Charts; running builds with Cloud Agents, and other improvements in TeamCity.

Space is limited, please register now. There will be an opportunity to ask questions during the webinar.

About the presenter:

Wes McClureWes Higbee is passionate about helping companies achieve remarkable results with technology and software. He’s had extensive experience developing software and working with teams to improve how software is developed to meet business objectives. Wes launched Full City Tech to leverage his expertise to help companies rapidly deliver high quality software to delight customers. He has a strong background in using Continuous Integration with TeamCity to bring quality to the table.
Posted in Uncategorized | Comments Off

TeamCity 9.0.1 update is out

Greetings everyone!

TeamCity 9.0 is your 24/7 build engineer and our developers are unstoppable as well even during the holiday time! We thought we’d make you a small gift – a new bugfix release just before the New Year’s Day – and here it is: TeamCity 9.0.1 is available!

Thank you for taking time to report issues to us! Build 32116 addresses over 70 problems related to different functionality, including some improvements in project import and the fix for the issue with Groovy plug. For the full list of fixes take a look at the Release Notes.

Download the distribution from the JetBrains site. We recommend upgrading your server to this version: as usual, this update does not modify the data format, so rolling back to 9.0 is possible at any time!

Happy building and happy New Year!

Posted in Bugfix, EAP, Features, FYI, News & Events, Tips&Tricks | Comments Off