Webinar Recording: Turbocharging TeamCity with Octopus Deploy

The recording of our May 14 webinar, Turbocharging TeamCity with Octopus Deploy, is available on the JetBrains YouTube channel.

In this webinar, Derek Campbell showcases how to integrate TeamCity and Octopus Deploy. He starts off with building a solution in TeamCity and takes you through deploying it to Development, Test, and finally to Production environments.

DerekDerek Campbell has worked across the world in London, Melbourne, Singapore and now from home in Glasgow, Scotland, for Octopus Deploy as a CI/CD Architect in the Customer Success team. Derek works with and helps Octopus Deploy’s biggest clients like ASOS, AON and L’Oreal with their CI/CD pipeline and helps enable their success with Octopus Deploy.
Posted in Webinar | Tagged | Leave a comment

Configuration as Code, Part 6: Testing Configuration Scripts

In this blog post, we are going to look at how to test TeamCity configuration scripts.

  1. Getting started with Kotlin DSL
  2. Working with configuration scripts
  3. Creating build configurations dynamically
  4. Extending Kotlin DSL
  5. Using libraries
  6. Testing configuration scripts

Given that the script is implemented with Kotlin, we can simply add a dependency to a testing framework of our choice, set a few parameters and start writing tests for different aspects of our builds.

In our case, we’re going to use JUnit. For this, we need to add the JUnit dependency to the pom.xml file

We also need to define the test directory.

In this example, we have redefined the source directory as well, so it corresponds with the following directory layout.

Once we have this in place, we can write unit tests as we would in any other Kotlin or Java project, accessing the different components of our project, build types, etc.

However, before we can start writing any code we need to make a few adjustments to the script. The reason is that our code for the configuration resides in settings.kts file. The objects that we declared in the kts file are not visible in the other files. Hence, to make these objects visible, we have to extract them into a file (or multiple files) with a kt file extension.

First, instead of declaring the project definition as a block of code in the settings.kts file, we can extract it into an object:

The SpringPetclinic object then refers to the build types, VCS roots, etc.

Next, to make this new object visible to the test code, we need to move this declaration into a file with a kt extension:

kotlin-dsl-test-code-in-files

settings.kts now serves as an entry point for the configuration where the project { } function is called. Everything else can be declared in the other *.kt files and referred to from the main script.

After the adjustments, we can add some tests. For instance, we could validate if all the build types start with a clean checkout:

Configuration checks as part of the CI pipeline

Running the tests locally is just one part of the story. Wouldn’t it be nice to run validation before the build starts?

When we make changes to the Kotlin configuration and check it into the source control, TeamCity synchronizes the changes and it will report any errors it encounters. The ability to now add tests allows us to add another extra layer of checks to make sure that our build script doesn’t contain any scripting errors and that certain things are validated such as the correct VCS checkout, as we’ve seen above, and the appropriate number of build steps are being defined, etc.

We can define a build configuration in TeamCity that will execute the tests for our Kotlin scripts prior to the actual build. Since it is a Maven project, we can apply Maven build step – we just need to specify the correct path to pom.xml, i.e. .teamcity/pom.xml.

kotlin-dsl-code-in-ci-pipeline

The successful run of the new build configuration is a prerequisite for the rest of the build chain. Meaning, if there are any JUnit test failures, then the rest of the chain will not be able to start.

Posted in Features, How-To's, Tips&Tricks | Tagged , | Leave a comment

Building the Snyk Plugin for TeamCity

This guest blog post is brought to you by Brian Vermeer, Developer Advocate at Snyk.

snyk-logo
Snyk offers a developer-first solution for open source security. Snyk is developing a TeamCity plugin to help users with automated vulnerability scanning. Snyk’s blog post series documents their story of building the plugin to help others learn from their experience.

In our previous post (Building open source security into the TeamCity flow with Snyk), we explained our motivation for building the Snyk plugin for TeamCity, and what we aim to achieve with it. As we explained, security is mainly used as a gating mechanism to make sure that whatever doesn’t measure up to policy doesn’t get shipped—but that isn’t always efficient. With Snyk’s plugin for TeamCity, we enable development teams to make the most of security analysis during the development stage, by helping to find and fix vulnerabilities with detailed and actionable insights.

In this follow-up post, we will highlight the Snyk features that we took advantage of when creating this plugin.

Taking advantage of the Snyk CLI

Finding and fixing vulnerabilities in your project’s dependencies is the core business of Snyk. With the Snyk CLI you can run scans on your projects from any machine—even your local development machine. The CLI automatically detects the type of project you have and creates a dependency tree. Snyk validates these dependencies against our proprietary vulnerability database. However, we do not only show whether a particular direct or transitive dependency contains a vulnerability, but also whether there is a possible solution.

For the TeamCity plugin, we use the same existing CLI capabilities to provide you these valuable insights.

We bundled the latest version of our CLI with the TeamCity plugin, which works by calling the CLI binary (embedded in the plugin) for scanning and then returning all key information in a structured JSON file. By default, we provide binaries for macOS, Linux, and Windows so, whatever system your TeamCity instance runs on, there is no need to separately download the Snyk CLI.

Additional capabilities: reporting and monitoring

To present this useful data in a report, we used yet another Snyk tool: the latest snyk-to-html binaries also ship with the plugin for macOS, Linux and Windows. Calling this tool with the Snyk CLI JSON output creates a fancy HTML report displaying all the insights you need to pinpoint and remediate vulnerable dependencies.

The Snyk CLI also enables monitoring your project from the Snyk interface: a snapshot of your dependency tree is uploaded to your Snyk account and then all your dependencies for that specific build are monitored on a regular basis. Additionally, you can enable active alerting (for example by email) so that every time a new vulnerability in your dependencies is found, you receive a notification. This monitoring capability is now also available in the TeamCity Plugin.

Screenshot 2019-04-25 at 14.00.45

How we brought it all together

The plugin introduces a “Snyk Security” build runner, which can be added as a build step in TeamCity.

screencapture-teamcity-demo-tretyakov-me-admin-editRunType-html-2019-05-05-00_14_40

The new TeamCity plugin executes multiple commands and pushes the result forward. With the plugin, users can run snyk test and snyk monitor, ultimately, pushing the test output to snyk-to-html.

We implemented this in the build agent part of the plugin by using a MultiCommandBuildSessionFactory to chain all these commands to the shipped binaries. If everything succeeds then an HTML artifact is produced, which we then made available to the user. The plugin adds the “Snyk Security Report” tab to a finished build page.

Screenshot 2019-05-04 at 19.46.06

A good way to use this step is to run it before deployment and add a snapshot dependency from the deployment build configuration to a configuration which contains the Snyk Security step. You can set it up in a way that if a security scan does not pass a specified threshold, the deployment won’t be started.

Screenshot 2019-05-04 at 20.05.27

Once you fix all the vulnerabilities with the remediation advice that Snyk provides in the report, you are good to go. Make sure that before deploying to production you run a test including snyk monitor. That way, when you ship and deploy that artifact, Snyk continues to monitor it as long as it is in production, and alerts you if new vulnerabilities are found in that build.

Happy coding!

Posted in Contest, Guest post, Plugin development | Tagged , | Leave a comment

Webinar Recording: Getting Started with TeamCity Plugins

The recording of our April 30 webinar, Getting Started with TeamCity Plugins is now available on the JetBrains YouTube channel.

In this webinar, we developed a custom runner plugin for TeamCity. The plugin adds a new build step for the echo command and displays the result of the execution in a custom tab for the completed build. You can find the source code of the plugin on GitHub.

The timeline:

4:23 – Generate project with Maven archetype
8:23 – Create server-side part of the plugin
18:31 – Create agent-side part of the plugin
24:47 – Configure TeamCity SDK Maven plugin
31:13 – Adding configuration UI to the server plugin
33:49 – Adding input validation
54:29 – Display results in a custom tab for the build

Posted in Plugin development, Webinar | Tagged , | Leave a comment

Webinar: Turbocharging TeamCity with Octopus Deploy

Join us May 14, 2019, 16:00 CEST (10:00 AM EDT) for our free live webinar, Turbocharging TeamCity with Octopus Deploy.

webinar-14 (2)

TeamCity and Octopus Deploy go together like cheese and crackers, or peanut butter and jelly for our American friends. In this webinar, Derek will start with a blank TeamCity and Octopus installation and show you how to integrate TeamCity and Octopus Deploy, build a solution and take you through deploying to Development, Test and finally to Production in under an hour to both Azure and AWS.

Space is limited, so please register now. There will be an opportunity to ask questions during the webinar.

Register for the webinar

DerekDerek Campbell has worked across the world in London, Melbourne, Singapore and now from home in Glasgow, Scotland, for Octopus Deploy as a CI/CD Architect in the Customer Success team. Derek works with and helps Octopus Deploy’s biggest clients like ASOS, AON and L’Oreal with their CI/CD pipeline and helps enable their success with Octopus Deploy.
Posted in Webinar | Tagged | Leave a comment

TeamCity Kanpur 2019.1 EAP3 is here

We are glad to present you the third and final Early Access Preview (EAP) version of TeamCity 2019.1.

In this EAP, we are introducing the following features:

  • Improvements for Amazon spot instances
  • Build artifacts publishing options
  • Docker requirement for build agents
  • Parsing of Go tests
  • Token-based authentication
  • New documentation website

Improvements for Amazon spot instances

Many of our users benefit from launching cloud images on Amazon spot instances. The spot fleet functionality allows you to use instances even more efficiently.

With a spot fleet option enabled for an image, you can apply the capacity allocation strategy which suits your project the best.

spot-fleet-option

Build artifacts publishing options

Build artifacts publishing is now more flexible. Previously, the artifacts were published only for completed builds, either successful or failed. We have added two new options:

  • Publish artifacts only if a build is completed successfully. This allows saving disk space by restricting publishing only to cases when all build steps are successful.
  • Publish artifacts for all builds, even for interrupted ones. This helps investigate an interruption problem by inspecting the artifacts.

Docker requirement for build agents

The bundled Docker Support plugin now sets requirements for build agents. If you add the ‘Docker Support’ feature to a build configuration, TeamCity will check all available agents and put those with installed Docker to the ‘Compatible agents’ list.

docker-requirement

Parsing of Go tests

TeamCity now can parse Go tests. To enable the Go processor for a project, add the ‘Golang’ feature to a build configuration. To generate tests:

  • either append the ‘-json’ flag to the ‘go test’ command in the script of a Command Line build runner
  • or add the ‘env.GOFLAGS = -json’ parameter to the build configuration.

Token-based authentication

In addition to basic HTTP authentication, TeamCity now supports authentication based on permanent access tokens. With tokens, you don’t need to expose a user login and password in scripts. Instead, you can send a token in an authentication header when querying the TeamCity API. You can generate new tokens in TeamCity or delete them any time.

New documentation website

To create a better experience and ensure a common look and feel across the documentation of all JetBrains products, we moved TeamCity documentation to a new platform. The documentation for previous versions stays in Confluence.

The Plugin Development Help is now residing on a separate site. Its source files are moved to the public GitHub repository so our community can contribute.

The detailed feature description and the list of fixes are available in our Release Notes.

As usual, the new release changes the TeamCity data format and, since the version 2019.1 is still in development, we recommend installing it on a trial server.

You are welcome to try the new EAP build and share your feedback on our forum or tracker.

Happy building!

Posted in EAP | 4 Comments

Streamline your CI/CD pipeline with TeamCity and CloudShell Colony’s Environment as a Service

This gallery contains 7 photos.

This guest post is brought to you by Meni Besso, Product Manager at Quali, the creators of CloudShell Colony CloudShell Colony is an Environment as a Service platform. It connects to your cloud providers including AWS, Azure, and Kubernetes and … Continue reading

More Galleries | Leave a comment

TeamCity Plugin Contest 2019: Create Plugins for TeamCity and Win Prizes

We are thrilled to announce the TeamCity Plugin Contest 2019 – a competition among TeamCity plugin developers where everyone wins!

Screenshot 2019-04-24 at 15.52.02

Every contestant who submits a plugin will receive a 1-year subscription to one of the JetBrains IDEs. The Contest starts today, April 24, and will last until June 24. The results will be announced on July 17.

Register for the Contest

What the prizes are

Everyone who submits a plugin will be awarded a 1-year personal product subscription to a JetBrains IDE of their choice. The first three places will also receive Amazon gift cards of $3,000, $2,000, and $1,000 value, respectively.

There will also be a Special Prize from the TeamCity development team – a trip to a JetBrains office in St. Petersburg or Munich. Finally, the People’s Choice Award winner will receive a 1-year All Products Pack subscription as well as a TeamCity Build Agent license.

Learn about plugin development

If you are new to TeamCity plugin development, or looking for additional guidance, there are several opportunities available:

  • Sign up for the upcoming webinar on “How to build plugins for TeamCity” on April 30, where Anton Arhipov will take you step by step through the plugin development process.
  • See the plugin development documentation or get help through the plugin development forum.
  • Join the Slack channel to chat with other TeamCity plugin developers.
  • Learn from Snyk’s experience. Snyk offers a developer-first solution for open source security. They are developing a TeamCity plugin to help users with automated vulnerability scanning. Check out the Snyk’s blog post series documenting their story of building the plugin and tips to help others learn from their experience.

Get inspiration

You are of course very welcome to come up with any plugin ideas on your own. However, if you need some inspiration, we’ve prepared this list of the most often requested features and integrations by the community. Feel free to draw your inspiration from them:

  • Additional authentication options (OAuth, SAML, etc.) TW-34709
  • Manual approval for build start TW-26623
  • Integrations with continuous delivery platforms (such as Spinnaker) TW-49297
  • Server-wide announcements TW-3737
  • Automatic build tagging based on predefined rules TW-16385
  • Conditional build steps TW-17939
  • Comparing builds TW-60097
  • Find usages for build configuration parameters TW-18543
  • Release downloads page TW-2963
  • Integrations with code review tools (Gerrit, GitHub) TW-19102

How the winners will be picked

Once the submissions close on June 24, the judges will get to work. This year we have three amazing judges joining us:

ixchelruizIx-chel Ruiz
Ix-chel has developed software applications & tools since 2000. Her research interests include Java, dynamic languages, client-side technologies, and testing. She is a Java Champion, Oracle Developer Champion, hackergarten enthusiast, Open Source advocate, public speaker, and mentor.
AndresAlmirayAndres Almiray
Andres is a Java/Groovy developer and a Java Champion with more than 20 years of experience in software design and development. He has been involved in web and desktop application development since the early days of Java. Andres is a true believer in open source and has participated in popular projects like Groovy, Griffon, and DbUnit, as well as his own projects (Json-lib, EZMorph, GraphicsBuilder, and JideBuilder). Founding member of the Griffon framework and the Hackergarten community event.
igalIgal Tabachnik
Igal is a software developer at Wix.com, working on build infrastructure and internal developer tools. His passion for developer productivity allowed Igal to work on such tools as debuggers, unit testing frameworks, plugins for Visual Studio, and IDEs from JetBrains, during his 15-year career. Igal is also the creator of several ReSharper plugins as well as a TeamCity plugin which took 2nd place in the previous JetBrains TeamCity Plugin Contest.

These judges will join the TeamCity development team in reviewing all the submitted plugins and rate them according to the criteria. The best three will get the main prizes. There will also be a special prize from the TeamCity team. All the plugins will become available for community voting, and the top voted plugin will receive the People’s Choice Award.

The countdown has started! Follow the news via our blog and Twitter, with the #TeamCityContest2019 hashtag.

We are looking forward to your entries. Happy developing and building!

Posted in Contest, Plugin development | Tagged | Leave a comment

Building Open Source Security Into The TeamCity Workflow With Snyk

This guest blog post is brought to you by Snyk.

snyk-logoSnyk offers a developer-first solution for open source security. Snyk is developing a TeamCity plugin to help users with automated vulnerability scanning. Snyk’s blog post series documents their story of building the plugin to help others learn from their experience.

As part of the effort to “shift-left”, the DevOps movement has mainly focused on controls or gating mechanisms within the pipeline. Normally defined by platform/architecture leadership but used by developers, these gates can make sure that whatever doesn’t measure up to policy doesn’t get shipped. Security is not an exception to that, as security teams attempt to ensure that critical vulnerabilities don’t get through to the infrastructure.

This focus on gates in the DevOps pipeline is efficient in a narrow sense for operators, but not so much in the bigger context of software development and delivery. Its binary nature doesn’t lend itself to insights about what was wrong, how to fix it, how to automate that fix, or how to prioritize between tasks.

In contrast, Snyk’s approach is to catch vulnerabilities in open source dependencies even earlier: at the IDE stage (as we’ve shown recently with our IntelliJ IDEA integration) and, even more powerfully, by using tight integration with Git (where users can fix vulnerabilities with a simple pull request, and monitor for any new commit).

Using a tool like Snyk allows a development team to make the most of the capability at the development stage—in this case, the capability is to find and fix vulnerabilities with detailed and actionable insights (or choose to ignore vulnerabilities of a certain level; users appreciate the choice). Once that happens, it is absolutely fine for the DevOps gates further down the pipe to be binary, as they are in this case a secondary control mechanism; they come into play after developers have already gained visibility into the issues and already fixed those that they deemed most critical.

Integrating With TeamCity

At Snyk, we have been embedding open source vulnerability scanning inside many third-party products and applications through available APIs. As we grow our language support and our user base, integrating with tools like TeamCity becomes an important enabler.

Here’s how it works: after putting my Snyk token into TeamCity, my build makes sure that it is in line with my Snyk org’s rules. In the example below, I have failed the build due to an app that is vulnerable beyond what my policy allows.

teamcity_project_log-output

At this stage, I can know more before I go back into Snyk and start fixing vulnerabilities, as shown in the metadata below, which is embedded into the TeamCity UI.

teamcity_project_report

With simple integration into Snyk, I have not only gated the build to keep safe from vulnerabilities, but I have also opened a door for myself to go back and use these insights for catching and fixing issues much earlier.

In our next post, we will go through the process of building this TeamCity plugin and share what we’ve learned along the way.

Read the next post: Building the Snyk Plugin for TeamCity.

Posted in Contest, Guest post, Plugin development | Tagged , | Leave a comment

Webinar: Getting Started With Building Plugins For Teamcity

Missing a feature in TeamCity? Build your own plugin! To learn how, join us Tuesday, April 30th, 16:00 CEST (11:00 AM EDT) for the Getting Started with TeamCity Plugins webinar.

webinar-14 (1)

The webinar introduces you to the ins and outs of plugin development for TeamCity. How to get started, where to find the docs and samples, what are the typical use cases, and even more importantly – where to ask the questions! We will develop a new plugin for TeamCity from scratch, explore the possible extension points, and discuss the essential concepts.

In this webinar:

  • Use the Maven archetype for TeamCity plugins
  • Applying TeamCity SDK for Maven
  • Overview of the typical plugins for TeamCity
  • TeamCity OpenAPI overview
  • Updating plugins with no server restarts

Space is limited, so please register now. There will be an opportunity to ask questions during the webinar.

Register for the webinar

Anton ArhipovAnton Arhipov is a Developer Advocate for JetBrains TeamCity. His professional interests include everything Java, but also other programming languages, middleware, and developer tooling. A Java Champion since 2014, Anton is also a co-organizer of DevClub.eu, a local developer community in Tallinn, Estonia.
Posted in How-To's, Webinar | Tagged | Leave a comment