Automating Deployments with TeamCity and Octopus Deploy

This guest post is brought to you by Paul Stovell, creator of Octopus Deploy.

octopusdeploy_logoOctopus is a deployment automation server for .NET developers. Octopus takes over where your build server finishes. The build server builds the code and runs tests, while Octopus takes care of deployment.​

TeamCity is a powerful continuous integration and build automation tool. Scroll through the list of built-in build runners and build features in TeamCity, and you’ll find almost any activity that you could want to perform during a build. It can restore NuGet packages, compile solutions, run unit tests, patch AssemblyInfo files, inspect code quality, and capture code coverage metrics, just to name a few. You can also combine multiple builds together to form a chain of builds, which is a powerful way to orchestrate an entire continuous delivery solution.

TeamCity can also be used to deploy your application during the build. This is useful if you want to deploy it to a test environment for users to test, or even automate deployments straight to production. You can even chain builds in TeamCity to make promoting between Test and Production environments possible. You can do this by telling TeamCity to call your script or command-line deployment tool during the build:

octopus_post1 Unfortunately, while TeamCity provides some high-level build activities like compiling with MSBuild or running NUnit tests, it doesn’t provide any high level deployment related tasks (“Configure IIS”, “Install a Windows Service”, “Update this configuration file”, etc.). “Run a script” is where TeamCity begins and ends; it’s up to you to write that script. Most teams solve these problems themselves, by writing their own deployment automation scripts, by combining tools like WebDeploy and PowerShell Remoting. Along the deployment automation journey, teams often encounter similar problems:

Problems/stumbling blocks
Possible solutions
Binary distribution
Many files are needed to deploy an application. Binaries (DLLs, EXEs), CSS and JavaScript files, images and other assets, configuration files, and so on. How do we distribute these files to remote machines? How do we do this securely and ensure no tampering when transmitting over insecure networks?
FTP(S); rsync; file shares; WebDeploy
Remote execution
Some tasks must execute directly on the remote machine (e.g., configuring IIS sites and Windows Services, installing COM components, making registry changes). How will we invoke these tasks? Is it easy to read the output? How will we detect failures? How do we deal with authentication?
SSH; PowerShell Remoting; WinRM; WebDeploy; PSEXEC; Remote Desktop
Configuration differences
Each environment is different. Different connection strings, different network paths, different credentials, etc. How will configuration files be modified?
Multiple copies of files renamed during deployment; XPath transforms; .NET Configuration transform files; PowerShell
Parallelism
Deployments often involve many machines. How do we distribute files and install applications in parallel to keep downtime to a minimum? What if each server has different settings?
Custom logic in scripts. Be careful to maintain it and not treat it like throwaway code.
Visibility and security
If different steps in the deployment run on different machines, how will we handle authentication/authorization? Can we allow testers to deploy to Test but restrict who can deploy to Production? Are changes audited and easy to roll back? Can anyone on the team easily see what is deployed where?
TeamCity dashboards; chained builds; build permissions

Four years ago, these were the kinds of problems we faced when it came to writing our own deployment scripts for client projects. We wanted something that elegantly solved the problems above, and which could work in the most secure environments, and yet would be as easy to use as the high-level build runner tasks in TeamCity. That’s why we built Octopus Deploy. Where TeamCity is a build automation server, Octopus is a deployment automation server. Like TeamCity, Octopus is free for small teams. When you use TeamCity and Octopus together, the goal is to let TeamCity do what it does best – build – and let Octopus take care of deployments. octopus_post2 TeamCity is responsible for:

  • Detecting source control changes
  • Compiling the solution
  • Running unit tests
  • Inspecting code, gathering build metrics, etc.
  • Packaging the application ready for deployment
  • Using our TeamCity plugin, notify Octopus that a new release is available

While Octopus is responsible for:

  • Distributing the packaged applications to the remote web/application servers, all in a secure way
  • Extracting and installing them
  • Modifying configuration files with environment-specific variables
  • Configure IIS, install and update Windows Services, etc.
  • Invoking any custom deployment actions on the remote machines
  • Gathering all the deployment logs into a central place

And just as the dashboard in TeamCity gives you an overview of your builds, the dashboard in Octopus gives an overview of your deployments: octopus_post3 If you have a build pipeline in TeamCity, extending it to deployment can do wonders in giving your team a tighter feedback loop. Writing your own custom scripts and invoking them from TeamCity isn’t the only option; the next time you find yourself automating deployments from TeamCity, have a look at Octopus Deploy.

banner_blog@2x

This entry was posted in Blogroll, Guest post. Bookmark the permalink.

7 Responses to Automating Deployments with TeamCity and Octopus Deploy

  1. D.R. says:

    We have considered using Octopus Deploy, however, decided against it due to too many stuff we still need to do in PowerShell (SQL server database management, NTFS privileges, Windows event source management, scheduled task management, client certificate management, etc.). Too much, to really see Octopus Deploy as an all-in-one-place solution. Is there a roadmap for such features?

    On another point: is Octopus Deploy also thinking about “undeployment”? Like installers are able to remove everything they installed, even in case of in-the-midst-of-deployment errors?

    Thank you for any additional information on those two topics!

    • trailmax says:

      Octopus is really good with orchestrating PowerShell. We have a lot of PS scripts in deployment and these are all executed by Octopus Deploy. No need for roadmap – it is already doing PS.

  2. Erik A. Brandstadmoen says:

    I have used Octopus Deploy before, and was very happy with it, however, now I’m only using TeamCity. The nice thing about using one tool, is that you get dependency chains and version numbers all the way from compiling to deployment, and you can also run tasks AFTER deployment, e.g. run integration tests. I struggle a bit finding where to run integration tests in the pipeline if using another tool than TeamCity for the deployment part.

    Any ideas?

  3. Paul Stovell says:

    @D.R. Indeed, some tasks do just come down to writing PowerShell, and ultimately there’s a lot of flexibility there. If PowerShell isn’t your cup of tea, you can write scripts using C# with ScriptCS inside Octopus instead.

    That said there are an increasing number of tasks appearing in our library that might cover your needs or provide a starting point. You can take these and copy them into your Octopus server to use in your deployment processes. I see items there for SQL, granting permissions, creating event sources, creating and scheduled tasks.

    In the next 6 months making more of these tasks built-in to Octopus (less PowerShell) is one of our major goals.

    Undeployment, not so much. Some aspects of that are easy, but some are hard, especially when it comes to databases. It’s one of those problems that’s extremely difficult to get 100% right and yet isn’t used all that frequently, so it hasn’t made it on to our roadmap.

    Hope this helps!

    Paul

  4. Paul Stovell says:

    @Erik: You can definitely mix Octopus steps within a TeamCity build chain:

    1. TeamCity builds and runs unit tests, tells Octopus to create a release and deploy to a Test environment. TeamCity waits for Octopus to finish the deployment. If the deployment fails, the build will go red and you can view the build log to see the details from Octopus.
    3. A chained build in TeamCity then runs to perform the integration tests

    Presumably somewhere in your single TeamCity build chain you need to push artifacts around, configure them, deploy them, etc. – just imagine replacing those steps with a call to Octopus to do the heavy lifting, and wait for it to complete. The rest of the build chain can then continue as normal.

  5. axiomme says:

    Undeployment means completely different things to different people, which is what makes it complicated. The best approach is to go with the approach of just deploying; in other words, an undeployment should normally be “deploy my previous artifact”.

  6. Atul Shirke says:

    So here is a wiered question to you all..

    I am creating the final package using batch scripts [Which I hate it]. Now I have the package ready with me. I need a tool which will pick up the changes from this package and deploy it on the server at client’s end. Clients wont have any access to any tools other than one single tool which will use my ready made package and deploy.

    Can you guys please suggest me some? Currently we are using our own in house built tool to deploy the package.

    Please help. Thanks

Comments are closed.