Test .NET Core projects with TeamCity

Greetings, everyone!

As many of you already know, TeamCity has a plugin to build .Net Core projects, which is basically a wrapper for the dotnet command. But if you try running dotnet test, you won’t see test results in TeamCity at the moment. As the test time can be significant, it is preferable to have fast feedback, so in this post we’ll explain how to integrate the dotnet test command with TeamCity and configure on-the-fly test reporting.

In short, you’ll need to perform the following:

  1. To run tests locally using the dotnet test command,  add references to the following packages to your test project: to Microsoft.NET.Test.Sdk, to the test framework, and the test adapter.
  2. To configure on-the-fly reporting, add a package reference to the TeamCity VSTest Adapter.
  3. To configure a build in TeamCity, use the TeamCity plugin for .NET Core projects or some other build runner.

Let’s go over these points in detail.

Run tests locally using dotnet test command

The approach suggested by Microsoft works fine for any target framework as well as for multiple frameworks at the same time, provided the test engine has a test adapter, e.g. MS tests, xunit tests, or some other test engine, for example.

That means that a test project needs to have at least three additional package references:

  • Microsoft.NET.Test.Sdk, which contains the MSBuild targets and properties for test projects and marks a project as a test project.
  • The selected Test Framework, e.g.  MSTest, or XUnit, or other
  • Test Adapter to discover and execute test framework based tests, e.g. the MSTest adapter, or the XUnit adapter, or other.

Configure on-the-fly reporting in TeamCity

Ideally, you should not need to do any additional preparation steps if you choose to run dotnet tests on TeamCity: TeamCity should pick up these tests automatically. For now this is not the case, so here is how you still can achieve this goal: use the TeamCity.VSTest.TestAdapter NuGet package containing a special logger which integrates with the test platform and sends service messages to a TeamCity server.

To turn on the integration, you’ll have to add a package reference to TeamCity VSTest Adapter. Note that this package does not impact the tests run locally, but only those run under TeamCity.

Here is an example of a project:

Visual Studio NuGet Dependencies

The project file for an MS test project can look as follows:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>    
    <TargetFrameworks>net45;netcoreapp1.0</TargetFrameworks>    
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.0.0" />
    <PackageReference Include="MSTest.TestAdapter" Version="1.1.14" />
    <PackageReference Include="MSTest.TestFramework" Version="1.1.14" />
    <PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.0" />    
  </ItemGroup>

  <ItemGroup>
    <Service Include="{82a7f48d-3b50-4b1e-b82e-3ada8210c358}" />
  </ItemGroup>

</Project>

Configure build in TeamCity

To configure a build project in TeamCity, we used the TeamCity plugin for .NET Core projects, described here in detail.

You can see an example on the public TeamCity server, where a test project contains one class, UnitTests:

namespace MS.Tests
{
    using System;
    using Microsoft.VisualStudio.TestTools.UnitTesting;

    [TestClass]
    public class UnitTests
    {
        [TestMethod, Ignore]
        public void TestIgnored()
        {
        }

        [TestMethod]
        public void TestPassed()
        {
            Console.WriteLine("some text");
        }
    }
}

Here is an example of the TeamCity tests step:

.

After a build is run, TeamCity shows the test results:

This .Net project has two target platforms, net45 and netcoreapp1.0, that is why each test is run for both framework and you can see several results per test.

The proposed approach does require some extra effort, however, it is quite viable and useful for testing .Net Core projects. Currently we are working on simplifying running tests and configuring on-the-fly test reporting from TeamCity, which would not require modifying your project files, so stay tuned for the latest news.

As usual, feel free to share your feedback.

Happy building and testing!

Posted in Features, How-To's, Uncategorized | 3 Comments

TeamCity 2017.1 aka 10.1 is released

Greetings, everyone!

We are proud to present the new TeamCity 2017.1 aka TeamCity 10.1.

650x170_product_header_JB_logo@2x_teamcity_2017_1

This release brings a lot of new exciting features, so hold your breath because here it comes:

  • Project administrators can now configure cloud profiles for their projects and build configurations saving time and effort for their system administrators. You can also define cloud profiles in Kotlin DSL now.
  • This version features improved Visual Studio Team Services integration: setting up a project or issue tracker has never been so easy.
  • We do hope you’ll like our considerably revamped UI, optimized for large-scale installations.
  • The new TeamCity release comes with the pluggable API enabling external artifacts storage. The S3 plugin from TeamCity is a good example of the new API implementation; you can also develop your own integration with a storage of your choice!

For the full list of features see our What’s New and Release NotesDownload TeamCity 2017.1 and remember to check the Upgrade Notes before you install the new version!

We’d love to hear from you, please share your feedback in our forum or tracker.

Happy building!

Posted in Features, Release | Leave a comment

TeamCity 2017.1 RC has arrived

Greetings, everyone!

We are happy to announce the release candidate for TeamCity 2017.1. This build fixes over 100 issues and includes a few performance improvements and security fixes.

We continue working on the TeamCity web UI and this version comes with the improved user profile page and the redesigned group editing page; besides, the new more performant selectors are now available in various places in the UI.

Other noteworthy changes are:

– the versioned settings now allow you to store security data outside the VCS
– cloud integration can be enabled/disabled for projects and subprojects
– you can now configure a proxy server for agent-to server connections.

See our Release Notes for the full list of fixes.

Please note that the TeamCity EAP license is not available for this build.

Though this is the release candidate, we would not recommend installing this version on the production server just yet,  please install it on a trial server. Download the new version, and please share your feedback in our forum or the tracker!

Stay tuned for the news on the TeamCity 2017.1 release which is coming soon!

Happy building!

Posted in Bugfix, EAP | 2 Comments

Here Comes EAP 2 for TeamCity 2017.1

Greetings, everyone!

We are happy to present TeamCity 2017.1 EAP 2!

This early preview build comes with several new features, the most important ones being:

– Agent cloud profile configuration has been moved to the project level
– Ability to exclude changes from default branch or hide default branch
– Web UI improvements: new breadcrumbs, new controls for build configurations selectors, build chains improvements
– Cross-Platform PowerShell is now supported

See Release Notes for details.

Download TeamCity 2017.1 EAP2 build, install it on a trial server as it changes the TeamCity data format, and share your feedback on our forum or tracker.

Happy building!
The TeamCity Team

Posted in EAP, Features | 4 Comments

TeamCity 10.0.5 is here

Greetings everyone!

Please welcome TeamCity 10.0.5, most likely the last update for the 10.0.x version. This is a bugfix release, which addresses about 70 issues. See the full list of them in our Release Notes.

Please note if you’re using TFS with agent side checkout: due to the fix of TW-48555 TeamCity will have to recreate TFS workspaces, which can result in clean checkout on the agents after the upgrade.

This build does not change the data format, so in case there are any issues, you can easily revert to any of the earlier 10.0.x versions.

Happy building!

Posted in FYI, Release | 2 Comments

Kotlin Configuration Scripts: Testing Configuration Scripts

This is part five of the five-part series on working with Kotlin to create Configuration Scripts for TeamCity.

  1. An Introduction to Configuration Scripts
  2. Working with Configuration Scripts
  3. Creating Configuration Scripts dynamically
  4. Extending the TeamCity DSL
  5. Testing Configuration Scripts

In this last part of the series we’re going to cover one aspect that using Kotlin Configurations Scripts enables, which is testing.

Given that the script is a programming language, we can simply add a dependency to a testing framework of our choice, set a few parameters and start writing tests for different aspects of our builds.

In our case, we’re going to use JUnit. For this, we add the JUnit dependency to the pom.xml file

We also need to define the test directory

In our case, this corresponds to the following directory layout

Directory Structure

Once we have this in place, we can write unit tests like in any other Kotlin or Java project, accessing the different components of our project, build types, etc. In our case we’re going to write a simple test that checks to see all build types start with a clean checkout

 

That additional layer of certainty

When we make changes to the Kotlin Configuration Script and check it in to source control, TeamCity synchronises the changes and it will report of an errors it encounters, which usually are related to compilation. The ability to now add tests allow us to add another extra layer of checks to make sure that beyond our build script containing any scripting errors, certain things are validated such as the correct VCS checkout as we’ve seen above, whether the appropriate number of build steps are being defined, etc. The possibilities are essentially endless given that once again, we’re just using a regular programming language.

Update: Important note

Given that currently TeamCity does not fetch third-party dependencies, the tests, which usually require JUnit or some other testing framework, will fail if places inside the .teamcity folder. This is due to TeamCity syncing the project and failing to compile unresolved references. For now the workaround is to place the tests outside of this folder.

Posted in Features, How-To's, Tips&Tricks | 2 Comments

Welcome EAP1 for TeamCity 2017.1 (aka 10.1)

Greetings, everyone!

Today we are unveiling the Early Access Preview (EAP) for TeamCity 2017.1, formerly known as TeamCity 10.1! Following the JetBrains versioning scheme, our releases will be now identified by <year of the release>.<number of the feature release within the year>.<bugfix update number>.

Build 45965 comes with over 30 new features, and though most of them are minor, they provide a smoother experience with TeamCity.

The TeamCity Web UI has been given a facelift overall; we should mention that the build chains page has been reworked with chains being displayed in a more compact way with handy new options.

It’s worth noting that the server startup and the page load times have been decreased.

Besides the new features, this versions brings over 100 bugfixes – see the details in our Release Notes.

Don’t hesitate to get the new EAP build and share your feedback on our forum or tracker.

As usual, this new feature release changes the TeamCity data format, and given that version 2017.1 is still in development, we recommend you install it on a trial server.

Happy building!
The TeamCity Team

Posted in Bugfix, EAP, Features, FYI, Tips&Tricks | 4 Comments

TeamCity as Debian Package Repository

Recently we’ve been experimenting around using TeamCity as a Debian repository and we’d like to share some tips and tricks we’ve come up with in the process.

We used the TeamCity tcDebRepository plugin to build *.deb packages and serve package updates to Debian/Ubuntu servers. The plugin, the special prize winner of the 2016 TeamCity plugin contest, works like a charm, but we’ve encountered a few difficulties with the software used to build and package .deb files, hence this blog post.

It’s important to note that tcDebRepository is not capable of serving packages to recent Ubuntu versions due to the lack of package signing. This is planned to be resolved in version 1.1 of the plugin (the plugin compatibility details).

We do not intend to cover the basics of TeamCity and Debian GNU/Linux infrastructure. To get acquainted with TeamCity, this video might be helpful. Building Debian packages is concisely described in the Debian New Maintainers’ Guide.

Everything below applies to any other Debian GNU/Linux distribution, e.g. Astra Linux.

Prerequisites and Project Setup

We arbitrarily chose 4 packages to experiment with:

Our setup uses:

  1. The free Professional version of TeamCity 10 with 3 agents managed by Debian 8.0 (Jessie).
  2. The tcDebRepository plugin installed as usual.
  3. Besides the Teamcity agent software, each of the agents required installing:
    • the build-essential package as a common prerequisite
    • the dependencies (from the 01-build-depends build-depends and 02-build-depends-indep build-depends-indep categories) required for individual packages.

Next we created a project in TeamCity with four build configurations (one per package):
teamcity-build-configs

Configuring VCS Roots

When configuring VCS Roots in TeamCity, types of Debian GNU/Linux packages had to be taken into account.

Debian packages can be native and non-native (details).
The code of native packages (e.g. debhelper, dpkg), developed within the Debian project, contains all the meta-information required for building (the debian/ directory in the source code tree).
The source code of non-native packages (e.g. bash) is not related to Debian, and an additional tree of their source code and meta-information with patches (contained in the debian/ directory) has to be maintained.

Therefore, for native packages we configured one VCS Root (pointing to Debian); whereas non-native packages needed two roots.

Next we configured checkout rules: the difference from the usual workflow is that in our case the artifacts (Debian packages) will be built outside the source code, one directory higher. This means that the checkout rules in TeamCity should be configured so that the source code of the dpkg package is checked out not into the current working directory, but into a subdirectory with the same name as the package, i.e. pkgname/. This is done by adding the following line to the checkout rules:
+:.=>pkgname
Finally, a VCS trigger was added to to every configuration.

Artifact paths

In terms of TeamCity, the binary packages we are about to build are artifacts, specified using artifact paths in the General Settings of every build configuration:

03-teamcity-debian-general-artifact-paths

For native packages, the pkgname.orig.tar.{gz,bz2,xz,lzma} and pkgname.debian.tar.{gz,bz2,xz,lzma} will not be created.

Build Step 1: Integration with Bazaar

Building a package with Bazaar-hosted source code (bash in our case) requires integration with the Bazaar Version Control System not supported by TeamCity out-of-the box. There is an a 3rd party TeamCity plugin to support Bazaar, but it has at least two problems:

  1. agent-side checkout is not supported
  2. the plugin is based on bzr xmlls and bzr xmllog (the bzr commands invoked by the plugin), but these commands are provided by external modules not necessarily available in the default Bazaar installation

The TeamCity server we used is running Windows, so we decided not to opt for installing Bazaar on the server side. Instead, to build the bash package, we added a build step using the Command Line Runner and a shell script for Bazaar integration:

This approach will not allow us to see the changes in one of the two trees of the source codes and run build automatically on changes, but it is ok as the first try.

Build Step 2. Initial configuration

We used the Command Line Runner to invoke dpkg-buildpackage. The -uc and -us keys in the image below indicate that digital signatures are currently out of scope for our packages. If you wish to sign the packages, you’ll have to load the corresponding pair of GnuPG keys on each of the agents.

Note that the “Working directory” is not set to the current working directory, but to the subdirectory with the same name as the package (bash in this case) where the source code tree will be checked out and dpkg-buildpackage will be executed. If the VCS Root is configured, the “Working directory” field can be filled out using the TeamCity directory picker, without entering the directory name manually:

07-teamcity-debian-dpkg-buildpackage-step

After the first build was run, we encountered some problems.

Build problems resolution

Code quality

For some packages (bash in our case), two code trees were not synchronized, i.e. they corresponded to slightly different minor versions, which forced us to use tags rather than branches from the main code tree. Luckily, TeamCity allows building on a tag and the “Enable to use tags in the branch specification” option can be set in such case:

teamcity-vcs-tags-in-branch-specs-small

Dependencies

Here is another problem we faced. When running the first build, dpkg-buildpackage exits with code 3 even though all the required dependencies were installed (see the prereqs section above).

There are a few related nuances:

  • The most common case for continuous integration is building software from Debian Unstable or Debian Experimental. So, to meet all the dependencies required for the build, TeamCity agents themselves need to run under Debian Unstable or Debian Experimental (otherwise dpkg-buildpackage will be reporting that your stable dependencies versions are too old). To resolve the error, we added the -d parameter:
    dpkg-buildpackage -uc -us -d
  • A special case of old dependencies is the configure script, created by the GNU Autotools that are newer than those currently installed on your system. dpkg-buildpackage is unable to detect this, so if the build log shows messages about the missing m4 macros, you need to to re-generate the configure script using the of GNU Autotools currently installed on the agent. This was done by adding the following command as the build step executed before dpkg-buildpackage:
    autoreconf -i

Broken unit tests

When working with Debian Unstable or Debian Experimental, unit test may fail (as it happened in our case). We choose to ignore the failing test and still build the package by running dpkg-buildpackage in a modified environment:
DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -uc -us

Other build profiles are described here.

Final Stage

After tuning all of the build configurations, our builds finally yielded expected artifacts (the dpkg package is used as an example):
11-teamcity-debian-artifacts

If you do not want each incremental package version to increase the number of artifacts of a single build (giving you dpkg_1.18.16_i386.deb, dpkg_1.18.17_i386.deb, etc.), the working directory should be selectively cleaned up before starting a new build. This was done by adding the TeamCity Swabra build feature to our build configurations.

Tuning up Debian repository

The tcDebRepository plugin adds the “Debian Repositories” tab in the project settings screen, which allows you to create a Debian repository. Once the repository is created, artifact filters need to be configured: at the moment each filter is added manually. Filters can be edited, deleted or copied.
teamcity-adding-an-artifact-filter

The existing artifacts will not be indexed, therefore after the configuration of the Debian repository, at least one build must be run in every build configuration. Then we can view the available indexed packages:
16-teamcity-debian-debrepo-page-2

When adding the repository to the /etc/apt/sources.list, all these packages are visible on the client side. Note that the packages are not digitally signed, which prevents packages from being available on recent versions of Ubuntu. Package signing is planned for tcDebRepository version 1.1.

17-teamcity-debian-aptitude-1

(!) If you are compiling for several architectures (i386, x32, amd64, arm*), it is worth having several build configurations corresponding to the same package with different agent requirements, or, in addition to the VCS Trigger, using the Schedule Trigger with the “Trigger build on all enabled and compatible agents” option enabled.

20-teamcity-debian-change-log

Happy building!

* We’d like to thank netwolfuk , the author of the tcDebRepository  plugin, for contributing to this publication.
This article, based on the original post in Russian, was translated by Julia Alexandrova.

Posted in FYI, How-To's, Tips&Tricks, Uncategorized | Leave a comment

Kotlin Configuration Scripts: Extending the TeamCity DSL

This is part four of the five-part series on working with Kotlin to create Configuration Scripts for TeamCity.

  1. An Introduction to Configuration Scripts
  2. Working with Configuration Scripts
  3. Creating Configuration Scripts dynamically
  4. Extending the TeamCity DSL
  5. Testing Configuration Scripts

TeamCity allows us to create build configurations that are dependent on each other, with the dependency being either snapshots or artifacts. The configuration for defining dependencies is done at the build configuration level. For instance, assuming that we have a build type Publish that has a snapshot dependency on Prepare Artifacts, we would define this in the build type Publish in the following way

and in turn if Prepare Artifacts had dependencies on previous build configurations, we’d define these in the dependencies segment of its build configuration.

TeamCity then allows us to visually see this using the Build Chains tab in the user interface

Build Chains

Defining the pipeline in code

Pipeline is a sequence of phases, phase is a set of buildTypes, each buildType in a phase depends on all buildTypes from the previous phase. This can handle simple but common kind of build chains where some build produces an artifact, several builds test it in parallel and the final build deploys the result if all its dependencies are successful.

It would often be beneficial to be able to define this build chain (or build pipeline) in code, so that we could describe what’s going on from a single location. In essence, it would be nice to be able to define the above using the Kotlin DSL as so

Defining this at the Project.kt level in our DSL, would give us a good oversight of what’s taking place.

The issue is though that currently the TeamCity DSL does not provide this functionality. But that’s where Kotlin’s extensibility proves quite valuable as we’ll see.

Creating our own Pipeline definition

Kotlin allows us to create extension functions and properties, which are a means to extend a specific type with new functionality, without having to inherit from these. When passing extension functions as arguments to other functions (i.e. higher-order functions), we get what we call in Kotlin Lambdas with Receivers, something we saw in this series already when Generalising feature wrappers in the second part of this series. We apply the same concept here to create our pipeline DSL

What the code above is doing is define a series of new constructs, namely pipeline and phase.

The code then takes the contents of what’s passed into each of these and defines, under the covers, the dependencies. In essence, it’s doing the same thing we would do in each of the different build configurations, but from a global perspective defined at the Project level.

In order to pass in the configuration to the pipeline as we saw earlier, we merely reference the specific build type (HttpClient, Publish, etc.), assigning it to a variable

val HttpClient = BuildType(....)

Flexibility of defining our own pipeline constructs

It’s important to understand that this is just one of many ways in which we can define pipelines. We’ve used the terms pipeline and phase. We could just as well have used the term stage to refer to each phase, or buildchain to refer to the pipeline itself (and thus align it with the UI). In addition to how we’ve named the constructs, and more importantly, is how the definition is actually constructed. In our case, we were interested in having this present in the Project itself. But we could just as well define a different syntax that is used at the build type level. The ability to easily extend the TeamCity DSL with our own constructs, provides us with this flexibility.

 

 

 

Posted in Features, How-To's, Tips&Tricks | 2 Comments

Kotlin Configuration Scripts: Creating Configuration Scripts Dynamically

This is part three of the five-part series on working with Kotlin to create Configuration Scripts for TeamCity.

  1. An Introduction to Configuration Scripts
  2. Working with Configuration Scripts
  3. Creating Configuration Scripts dynamically
  4. Extending the TeamCity DSL
  5. Testing Configuration Scripts

We saw in the previous post how we could leverage some of Kotlin’s language features to reuse code. In this part, we’re going to take advantage of the fact that we are in fact dealing with a full programming language and not just a limited DSL, to create a dynamic build configuration.

Build Configurations based on parameters

The scenario is the following: we have an HTTP server that we need to test on different platforms with a range of concurrent connections on each one. This generates potentially a lot of different build configurations that we’d need to create and maintain.

Instead of doing this manually, what we can do is write some code to have our Kotlin DSL Configuration Script generate all the different build configurations for us.

Let’s say that we have a list of Operating Systems and a range of concurrent connections for each one. The first thing to do is to create a data class that represents this information

which in essence is like a regular class but provides a series of benefits such as a nicer string representation, equality and copy operations amongst other things.

Now let’s imagine we have the different platforms we want to test on represented as a list of BuildParameters

what we’d like to do, is iterate over this list and create a new build type for each combination. In essence, if our standard Project is

what we want to do is create a new buildType for each entry in the list

Creating a base Build Type

Any parameter passed into buildType needs to be of the type BuildType. This means we’d need to inherit from this class and at the same time provide some parameters to it (BuildParameters)

The code is actually a pretty standard Configuration Script that defines a Gradle build step, has a VCS definition and defines a VCS trigger. What we’ve done is just enhance it somewhat.

To begin with,  we’re using the BuildParameters name property to suffix it to the uuid, extId and name. This guarantees a unique ID per build configuration as well as provides us a name to identify it easily. To make sure the values are properly formatted, we use the TeamCity DSL defined extension function to String, named toExtId() which takes care of removing any forbidden characters.

In the steps definition, we pass in certain parameters to Gradle, which in our case is the number of concurrent connections we want to test with. Obviously, this is just a sample, and the actual data being passed in can be anything and used anywhere in the script.

Finally, we also use the BuildParameters operatingSystem property to define Agent requirements.

Summary

The above is just a sample of what can be done when creating dynamic build scripts. In this case, we created multiple build configurations, but we could just as well have created multiple steps, certain VCS triggers, or whatever else could come in useful. The important thing to understand is that at the end of the day, Kotlin Configuration Script isn’t just merely a DSL but a fully fledged programming language.

In the next part of this series, we’ll see how we can extend the Kotlin Configuration Scripts (hint – it’s going to involve extension functions).

 

 

Posted in Features, How-To's, Tips&Tricks, Uncategorized | 2 Comments