Webinar Recording: Getting Started with TeamCity Plugins

The recording of our April 30 webinar, Getting Started with TeamCity Plugins is now available on the JetBrains YouTube channel.

In this webinar, we developed a custom runner plugin for TeamCity. The plugin adds a new build step for the echo command and displays the result of the execution in a custom tab for the completed build. You can find the source code of the plugin on GitHub.

The timeline:

4:23 – Generate project with Maven archetype
8:23 – Create server-side part of the plugin
18:31 – Create agent-side part of the plugin
24:47 – Configure TeamCity SDK Maven plugin
31:13 – Adding configuration UI to the server plugin
33:49 – Adding input validation
54:29 – Display results in a custom tab for the build

Posted in Plugin development, Webinar | Tagged , | Comments Off on Webinar Recording: Getting Started with TeamCity Plugins

Webinar: Turbocharging TeamCity with Octopus Deploy

Join us May 14, 2019, 16:00 CEST (10:00 AM EDT) for our free live webinar, Turbocharging TeamCity with Octopus Deploy.

webinar-14 (2)

TeamCity and Octopus Deploy go together like cheese and crackers, or peanut butter and jelly for our American friends. In this webinar, Derek will start with a blank TeamCity and Octopus installation and show you how to integrate TeamCity and Octopus Deploy, build a solution and take you through deploying to Development, Test and finally to Production in under an hour to both Azure and AWS.

Space is limited, so please register now. There will be an opportunity to ask questions during the webinar.

Register for the webinar

DerekDerek Campbell has worked across the world in London, Melbourne, Singapore and now from home in Glasgow, Scotland, for Octopus Deploy as a CI/CD Architect in the Customer Success team. Derek works with and helps Octopus Deploy’s biggest clients like ASOS, AON and L’Oreal with their CI/CD pipeline and helps enable their success with Octopus Deploy.
Posted in Webinar | Tagged | Comments Off on Webinar: Turbocharging TeamCity with Octopus Deploy

TeamCity Kanpur 2019.1 EAP3 is here

We are glad to present you the third and final Early Access Preview (EAP) version of TeamCity 2019.1.

In this EAP, we are introducing the following features:

  • Improvements for Amazon spot instances
  • Build artifacts publishing options
  • Docker requirement for build agents
  • Parsing of Go tests
  • Token-based authentication
  • New documentation website

Improvements for Amazon spot instances

Many of our users benefit from launching cloud images on Amazon spot instances. The spot fleet functionality allows you to use instances even more efficiently.

With a spot fleet option enabled for an image, you can apply the capacity allocation strategy which suits your project the best.

spot-fleet-option

Build artifacts publishing options

Build artifacts publishing is now more flexible. Previously, the artifacts were published only for completed builds, either successful or failed. We have added two new options:

  • Publish artifacts only if a build is completed successfully. This allows saving disk space by restricting publishing only to cases when all build steps are successful.
  • Publish artifacts for all builds, even for interrupted ones. This helps investigate an interruption problem by inspecting the artifacts.

Docker requirement for build agents

The bundled Docker Support plugin now sets requirements for build agents. If you add the ‘Docker Support’ feature to a build configuration, TeamCity will check all available agents and put those with installed Docker to the ‘Compatible agents’ list.

docker-requirement

Parsing of Go tests

TeamCity now can parse Go tests. To enable the Go processor for a project, add the ‘Golang’ feature to a build configuration. To generate tests:

  • either append the ‘-json’ flag to the ‘go test’ command in the script of a Command Line build runner
  • or add the ‘env.GOFLAGS = -json’ parameter to the build configuration.

Token-based authentication

In addition to basic HTTP authentication, TeamCity now supports authentication based on permanent access tokens. With tokens, you don’t need to expose a user login and password in scripts. Instead, you can send a token in an authentication header when querying the TeamCity API. You can generate new tokens in TeamCity or delete them any time.

New documentation website

To create a better experience and ensure a common look and feel across the documentation of all JetBrains products, we moved TeamCity documentation to a new platform. The documentation for previous versions stays in Confluence.

The Plugin Development Help is now residing on a separate site. Its source files are moved to the public GitHub repository so our community can contribute.

The detailed feature description and the list of fixes are available in our Release Notes.

As usual, the new release changes the TeamCity data format and, since the version 2019.1 is still in development, we recommend installing it on a trial server.

You are welcome to try the new EAP build and share your feedback on our forum or tracker.

Happy building!

Posted in EAP | 5 Comments

Streamline your CI/CD pipeline with TeamCity and CloudShell Colony’s Environment as a Service

This gallery contains 7 photos.

This guest post is brought to you by Meni Besso, Product Manager at Quali, the creators of CloudShell Colony CloudShell Colony is an Environment as a Service platform. It connects to your cloud providers including AWS, Azure, and Kubernetes and … Continue reading

More Galleries | Comments Off on Streamline your CI/CD pipeline with TeamCity and CloudShell Colony’s Environment as a Service

TeamCity Plugin Contest 2019: Create Plugins for TeamCity and Win Prizes

We are thrilled to announce the TeamCity Plugin Contest 2019 – a competition among TeamCity plugin developers where everyone wins!

Screenshot 2019-04-24 at 15.52.02

Every contestant who submits a plugin will receive a 1-year subscription to one of the JetBrains IDEs. The Contest starts today, April 24, and will last until June 24. The results will be announced on July 17.

Register for the Contest

What the prizes are

Everyone who submits a plugin will be awarded a 1-year personal product subscription to a JetBrains IDE of their choice. The first three places will also receive Amazon gift cards of $3,000, $2,000, and $1,000 value, respectively.

There will also be a Special Prize from the TeamCity development team – a trip to a JetBrains office in St. Petersburg or Munich. Finally, the People’s Choice Award winner will receive a 1-year All Products Pack subscription as well as a TeamCity Build Agent license.

Learn about plugin development

If you are new to TeamCity plugin development, or looking for additional guidance, there are several opportunities available:

  • Sign up for the upcoming webinar on “How to build plugins for TeamCity” on April 30, where Anton Arhipov will take you step by step through the plugin development process.
  • See the plugin development documentation or get help through the plugin development forum.
  • Join the Slack channel to chat with other TeamCity plugin developers.
  • Learn from Snyk’s experience. Snyk offers a developer-first solution for open source security. They are developing a TeamCity plugin to help users with automated vulnerability scanning. Check out the Snyk’s blog post series documenting their story of building the plugin and tips to help others learn from their experience.

Get inspiration

You are of course very welcome to come up with any plugin ideas on your own. However, if you need some inspiration, we’ve prepared this list of the most often requested features and integrations by the community. Feel free to draw your inspiration from them:

  • Additional authentication options (OAuth, SAML, etc.) TW-34709
  • Manual approval for build start TW-26623
  • Integrations with continuous delivery platforms (such as Spinnaker) TW-49297
  • Server-wide announcements TW-3737
  • Automatic build tagging based on predefined rules TW-16385
  • Conditional build steps TW-17939
  • Comparing builds TW-60097
  • Find usages for build configuration parameters TW-18543
  • Release downloads page TW-2963
  • Integrations with code review tools (Gerrit, GitHub) TW-19102

How the winners will be picked

Once the submissions close on June 24, the judges will get to work. This year we have three amazing judges joining us:

ixchelruizIx-chel Ruiz
Ix-chel has developed software applications & tools since 2000. Her research interests include Java, dynamic languages, client-side technologies, and testing. She is a Java Champion, Oracle Developer Champion, hackergarten enthusiast, Open Source advocate, public speaker, and mentor.
AndresAlmirayAndres Almiray
Andres is a Java/Groovy developer and a Java Champion with more than 20 years of experience in software design and development. He has been involved in web and desktop application development since the early days of Java. Andres is a true believer in open source and has participated in popular projects like Groovy, Griffon, and DbUnit, as well as his own projects (Json-lib, EZMorph, GraphicsBuilder, and JideBuilder). Founding member of the Griffon framework and the Hackergarten community event.
igalIgal Tabachnik
Igal is a software developer at Wix.com, working on build infrastructure and internal developer tools. His passion for developer productivity allowed Igal to work on such tools as debuggers, unit testing frameworks, plugins for Visual Studio, and IDEs from JetBrains, during his 15-year career. Igal is also the creator of several ReSharper plugins as well as a TeamCity plugin which took 2nd place in the previous JetBrains TeamCity Plugin Contest.

These judges will join the TeamCity development team in reviewing all the submitted plugins and rate them according to the criteria. The best three will get the main prizes. There will also be a special prize from the TeamCity team. All the plugins will become available for community voting, and the top voted plugin will receive the People’s Choice Award.

The countdown has started! Follow the news via our blog and Twitter, with the #TeamCityContest2019 hashtag.

We are looking forward to your entries. Happy developing and building!

Posted in Contest, Plugin development | Tagged | Comments Off on TeamCity Plugin Contest 2019: Create Plugins for TeamCity and Win Prizes

Building Open Source Security Into The TeamCity Workflow With Snyk

This guest blog post is brought to you by Snyk.

snyk-logoSnyk offers a developer-first solution for open source security. Snyk is developing a TeamCity plugin to help users with automated vulnerability scanning. Snyk’s blog post series documents their story of building the plugin to help others learn from their experience.

As part of the effort to “shift-left”, the DevOps movement has mainly focused on controls or gating mechanisms within the pipeline. Normally defined by platform/architecture leadership but used by developers, these gates can make sure that whatever doesn’t measure up to policy doesn’t get shipped. Security is not an exception to that, as security teams attempt to ensure that critical vulnerabilities don’t get through to the infrastructure.

This focus on gates in the DevOps pipeline is efficient in a narrow sense for operators, but not so much in the bigger context of software development and delivery. Its binary nature doesn’t lend itself to insights about what was wrong, how to fix it, how to automate that fix, or how to prioritize between tasks.

In contrast, Snyk’s approach is to catch vulnerabilities in open source dependencies even earlier: at the IDE stage (as we’ve shown recently with our IntelliJ IDEA integration) and, even more powerfully, by using tight integration with Git (where users can fix vulnerabilities with a simple pull request, and monitor for any new commit).

Using a tool like Snyk allows a development team to make the most of the capability at the development stage—in this case, the capability is to find and fix vulnerabilities with detailed and actionable insights (or choose to ignore vulnerabilities of a certain level; users appreciate the choice). Once that happens, it is absolutely fine for the DevOps gates further down the pipe to be binary, as they are in this case a secondary control mechanism; they come into play after developers have already gained visibility into the issues and already fixed those that they deemed most critical.

Integrating With TeamCity

At Snyk, we have been embedding open source vulnerability scanning inside many third-party products and applications through available APIs. As we grow our language support and our user base, integrating with tools like TeamCity becomes an important enabler.

Here’s how it works: after putting my Snyk token into TeamCity, my build makes sure that it is in line with my Snyk org’s rules. In the example below, I have failed the build due to an app that is vulnerable beyond what my policy allows.

teamcity_project_log-output

At this stage, I can know more before I go back into Snyk and start fixing vulnerabilities, as shown in the metadata below, which is embedded into the TeamCity UI.

teamcity_project_report

With simple integration into Snyk, I have not only gated the build to keep safe from vulnerabilities, but I have also opened a door for myself to go back and use these insights for catching and fixing issues much earlier.

In our next post, we will go through the process of building this TeamCity plugin and share what we’ve learned along the way.

Read the next post: Building the Snyk Plugin for TeamCity.

Posted in Contest, Guest post, Plugin development | Tagged , | Comments Off on Building Open Source Security Into The TeamCity Workflow With Snyk

Webinar: Getting Started With Building Plugins For Teamcity

Missing a feature in TeamCity? Build your own plugin! To learn how, join us Tuesday, April 30th, 16:00 CEST (11:00 AM EDT) for the Getting Started with TeamCity Plugins webinar.

webinar-14 (1)

The webinar introduces you to the ins and outs of plugin development for TeamCity. How to get started, where to find the docs and samples, what are the typical use cases, and even more importantly – where to ask the questions! We will develop a new plugin for TeamCity from scratch, explore the possible extension points, and discuss the essential concepts.

In this webinar:

  • Use the Maven archetype for TeamCity plugins
  • Applying TeamCity SDK for Maven
  • Overview of the typical plugins for TeamCity
  • TeamCity OpenAPI overview
  • Updating plugins with no server restarts

Space is limited, so please register now. There will be an opportunity to ask questions during the webinar.

Register for the webinar

Anton ArhipovAnton Arhipov is a Developer Advocate for JetBrains TeamCity. His professional interests include everything Java, but also other programming languages, middleware, and developer tooling. A Java Champion since 2014, Anton is also a co-organizer of DevClub.eu, a local developer community in Tallinn, Estonia.
Posted in How-To's, Webinar | Tagged | Comments Off on Webinar: Getting Started With Building Plugins For Teamcity

TeamCity 2018.2.4 is out now

We’re announcing TeamCity 2018.2.4 today!

This is essentially a bugfix update, which addresses over 40 issues and includes a few security and performance fixes. See the full list in our Release Notes.

We recommend you install the new version to benefit from the latest improvements. Like any bugfix update, 2018.2.4 has the same data format as all 2018.2.x versions, allowing you to upgrade/downgrade within these builds easily.

Check our Upgrade Notes and use one of the upgrade options:

Help us improve TeamCity and submit your feedback to our forum or tracker.

Happy building!

Posted in Bugfix, Release | Tagged | Comments Off on TeamCity 2018.2.4 is out now

Configuration as Code, Part 5: Using DSL extensions as a library

  1. Getting started with Kotlin DSL
  2. Working with configuration scripts
  3. Creating build configurations dynamically
  4. Extending Kotlin DSL
  5. Using libraries
  6. Testing configuration scripts

In the previous post, we have seen how to extend TeamCity’s Kotlin DSL by adding new abstractions. If the new abstraction is generic enough, it would make sense to reuse it in different projects. In this post, we are going to look at how to extract the common code into a library. We will then use this library as a dependency in a TeamCity project.

Maven project and dependencies

In the first post of this series, we started out by creating an empty project in TeamCity. We then instructed the server to generate the configuration settings in Kotlin format.

The generated pom.xml file pointed at two repositories and a few dependencies. This pom.xml is a little excessive for our next goal, but we can use it as a base, and remove the parts that we don’t need for the DSL library.

The two repositories in the pom.xml file are jetbrains-all, the public JetBrains repository, and teamcity-server that points to the TeamCity server where we generated the settings. The reason why the TeamCity server is used as a repository for the Maven project is that there may be some plugins installed that extend the TeamCity Kotlin DSL. And we may want to use those extensions for configuring the builds.

However, for a library, it makes sense to rely on a minimal set of dependencies to ensure portability. Hence, we keep only those dependencies that are downloaded from the public JetBrains Maven repository and remove all the others. The resulting pom.xml lists only 3 libraries: configs-dsl-kotlin-{version}.jar, kotlin-stdlib-jdk8-{version}.jar, and kotlin-script-runtime-{version}.jar.

The code

It’s time to write some code! In fact, it’s already written. In the previous post, we have introduced the new abstraction, the sequence, to automatically configure the snapshot dependencies for the build configurations. We only need to put this code into a *.kt file in our new Maven project.

teamcity-pipelines-dsl-lib

We have published the example project on GitHub. Pipelines.kt lists all the extensions to the TeamCity DSL. That’s it! We now can build the library, publish it, and use it as a dependency in any TeamCity project with Kotlin DSL.

Using the library

The new library project is on GitHub, but we haven’t published it to any Maven repository yet. To add it as a dependency to any other Maven project we can use the awesome jitpack.io. The demo project demonstrates how the DSL library is applied.

Here’s how we can use the library:

1. Add the JitPack repository to the pom.xml file:

2. Add the dependency to the dependent DSL project’s pom.xml:

The version is equal to a tag in the GitHub repository:

teamcity-pipelines-dsl-tags

Once the IDE has downloaded the dependencies, we are able to use the DSL extensions provided by the library. See settings.kts of the demo project for an example.

using-the-library

Summary

TeamCity allows adding 3rd-party libraries as Maven dependencies. In this post, we have demonstrated how to add a dependency on the library that adds extensions to the TeamCity Kotlin DSL.

Posted in Features, How-To's, Tips&Tricks | Tagged | 6 Comments

Configuration as Code, Part 4: Extending the TeamCity DSL

  1. Getting started with Kotlin DSL
  2. Working with configuration scripts
  3. Creating build configurations dynamically
  4. Extending Kotlin DSL
  5. Using libraries
  6. Testing configuration scripts

TeamCity allows us to create build configurations that are dependent on one another, with the dependency being either snapshots or artifacts. The configuration for defining dependencies is done at the build configuration level. For instance, assuming that we have a build type Publish, that has a snapshot and artifact dependencies on Package, we would define this in the build type Publish in the following way:

and in turn, if Package had dependencies on previous build configurations, we’d define these in the dependencies segment of its build configuration.

TeamCity then allows us to visually see this using the Build Chains tab in the user interface:

build chains

The canonical approach to defining build chains in TeamCity is when we declare the individual dependencies in the build configuration. The approach is simple but as the number of build configurations in the build chain grows it becomes harder to maintain the configurations.

Imagine there’s a large number of build configurations in the chain, and we want to add one more somewhere in the middle of the workflow. For this to work, we have to configure the correct dependencies in the new build configuration. But we also need to update the dependencies in the existing build configurations to point at the new one. This approach does not seem to scale well.

But we can work around this problem by introducing our own abstractions in TeamCity’s DSL.

Defining the pipeline in code

What if we had a way to describe the pipeline on top of the build configurations that we define separately in the project? The pipeline abstraction is something we need to create ourselves. The goal of this abstraction is to allow us to omit specifying the snapshot dependencies in the build configurations that we want to combine into a build chain.

Assume that we have a few build configurations: Compile, Test, Package, and Publish. Test needs a snapshot dependency on Compile, Package depends on Test, Publish depends on Package, and so on. So these build configurations compose a build chain.

Let’s define, how the new abstraction would look. We think of the build chain described above as of a “sequence of builds”. So why not to describe it as follows:

Almost immediately, we could think of a case where we need to be able to run some builds in parallel.

In the example above, Test1 and Test2 are defined in the parallel block, both depend on Compile. Package depends on both, Test1 and Test2. This can handle simple but common kinds of build chains where a build produces an artifact, several builds test it in parallel and the final build deploys the result if all its dependencies are successful.

For our new abstraction, we need to define, what sequence, parallel, and build are. Currently, the TeamCity DSL does not provide this functionality. But that’s where Kotlin’s extensibility proves quite valuable, as we’ll now see.

Creating our own DSL definitions

Kotlin allows us to create extension functions and properties, which are the means to extend a specific type with new functionality, without having to inherit from them. When passing extension functions as arguments to other functions (i.e. higher-order functions), we get what we call in Kotlin Lambdas with Receivers, something we’ve seen already in this series when Generalising feature wrappers in the second part of this series. We will apply the same concept here to create our DSL.

The code above adds an extension function to the Project class and allow us to declare the sequence. Using the aforementioned Lambda with Receivers feature we declare that the block used as a parameter to the sequence function will provide the context of the Sequence class. Hence, we will be able to call the build function directly within that block:

Adding parallel blocks

To support the parallel block we need to extend our abstraction a little bit. There will be a serial stage that consists of a single build type and a parallel stage that may include many build types.

To support the parallel blocks we will need to write slightly more code. Every build type defined in the parallel block will have a dependency on the build type which was declared before the parallel block. And the build type declared after the parallel block will depend on all the build types declared in the block. We’ll make the assumption that a parallel block cannot follow a parallel block, though it’s not a big problem to support this feature.

The DSL now supports parallel blocks in the sequence:

basic-parallel-blocks

We could extend the DSL even further to support nesting of the blocks by allowing defining the sequence inside the parallel blocks.

sequence-in-parallel

Nesting the blocks allows us to create build chains of almost any complexity. However, our example only covers snapshot dependencies. We haven’t covered artifact dependencies here yet and these would be nice to see in the sequence definition as well.

Adding artifact dependencies

For passing an artifact dependency from Compile to Test, simply specify that Compile produces the artifact and Test requires the same artifact.

produces and requires are the new extension functions for the BuildType:

We also need to provide a way to execute these new functions in the context of BuildType. For this, we can override the build() function of the Sequence and Parallel classes to accept the corresponding block by using Lambda with Receivers declaration:

As a result, we can define a more complex sequence with our brand new DSL:

Summary

It’s important to understand that this is just one of many ways in which we can define pipelines. We’ve used the terms sequence, parallel and build. We could just as well have used the term buildchain to align it better with the UI. We also added the convenience methods to the BuildType to work with the artifacts.

The ability to easily extend the TeamCity DSL with our own constructs, provides us with flexibility. We can create custom abstractions on top of the existing DSL to better reflect how we reason about our build workflow.

In the next post, we’ll see how to extract our DSL extensions into a library for further re-use.

Posted in Features, How-To's, Tips&Tricks | Tagged | Comments Off on Configuration as Code, Part 4: Extending the TeamCity DSL