Support for Multi-Repository Builds in Space Automation

Alexey Totin

Read this post in other languages:

• Français
,
• Türkçe

Space Automation (a part of JetBrains Space responsible for CI/CD) recently gained a couple of cool new features that let your Automation scripts work with several Git repositories.

Here’s what’s changed, in a nutshell:

1. A job can now check out any repository within the project (not just the one that contains the running Automation script).

2. Automation can now trigger a job run on changes in a certain repository, branch, directory, or file.

Where this might be helpful? For example, you have a project based on microservices (each in a separate repo) and you have a separate repo with integration tests that run against the project as a whole. Another real-life example: a project consists of multiple repos and each repo has its own doc/ folder with documentation. Using Automation, you can set up a job that tracks changes in the doc/ folders of all these repositories and builds and deploys an internal documentation website.

Read on for more details!

Checking out source code

Normally, in Space Automation you don’t need to bother about the "check out the source code" step. Every time a job is started, Automation clones the current branch from the current repository (the one that contains the running script).
Now, if a job requires the contents of another repository within a project, you can check out this repository in addition to the main one. Just specify the name of the second repo in the git block:

job("check out second repo") {
    // check out second-repo to /mnt/space/work/repo-2
    git("second-repository") {
        // the default path is /mnt/space/work/$repoName
        // but you can change it with cloneDir
        cloneDir = "repo-2"
    }

    container("ubuntu") {
        shellScript {
            content = """
                echo Directory structure
                ls -R /mnt
                echo Working dir is
                pwd
            """
        }
    }
}
// the /mnt/space/work dir will contain the current and the second repo
// /mnt/space/work/main-repo ; /mnt/space/work/repo-2
// Working dir is /mnt/space/work/main-repo

That’s not all! Now, you also have a choice of what data you want to fetch from the repository: you can fetch tags, particular branches, or revisions. Learn more in the Automation documentation

Triggering a job run on changes in a branch, directory, or file

By default, when a commit is pushed to the repository to a particular branch, Automation triggers a script run in this branch. Obviously, this works for all branches in the repository. To run the script only in particular repositories, use the branchFilter parameter inside the gitPush block.

For example, to trigger jobs only in the my-feature branch:

job("Run on git push") {
    startOn {
        gitPush {
            branchFilter {
                +"refs/heads/my-feature"
            }
        }
    }
}

Note that branchFilter supports Regex, exclude and include rules, and wildcards.

To trigger a job run on changes in files and directories, use the pathFilter block (as well as branсhFilter, it supports creating complex filters):

job("Run on git push") {
    startOn {
        gitPush {
            pathFilter {
                // include all from 'targets' directory
                +"targets/**"
                // exclude 'targets/main' directory
                -"targets/main/**"
                // include all 'Main.kt' files in the project
                // As this rule is more specific,
                // 'Main.kt' will be included even if
                // it's located in 'targets/main'
                +"**/Main.kt"
            }
        }
    }
}

Triggering a job run on changes in another repository

Sometimes, you may need to configure a build that uses source code from several project repositories. For example, you can create a separate repository that will contain the build script for the whole project while other repositories will have no .space.kts files at all.

Let’s take a look at the simplest possible example. Say, you have a project with two repos: first-repository and second-repository. You add the following .space.kts file to the first-repository:

job("Run on change in second-repository") {
    startOn {
       gitPush {
            repository = "second-repository"
        }
    }

    // don't forget to check out 
    // the content of second-repo
    git("second-repository")

    container("alpine") {
        shellScript {
            content = "ls /mnt/space/work"
        }
    }
}

How does it look in the UI? If you open the Jobs page for first-repository, you will see nothing new, just a normal job run:

But if you now switch the page to the list of jobs for second-repository, you will see that it includes cross-referenced jobs – jobs from other repositories that have references to the currently selected repository. In our case that one job is our job from first-repository:

Of course, you can go deeper and add one more level to the repository trigger: filter by a branch or path. Find the details in the Automation documentation.

That concludes today’s little announcement. We kindly ask you to leave your comments below and to try these new features out!

Subscribe to Space Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!