Using Gradle build cache with Kotlin
This is a guest blog post from Eric Wendelin
software engineer for Gradle
A build cache allows Gradle to reuse task output from any previous invocation, including those from other machines. Kotlin 1.2.21 allows Kotlin projects to make use of build caching.
The build cache works by storing compiled classes, test outputs, and other build artifacts in a cache, taking into account all task inputs, including input file contents, relevant classpaths, and task configuration.
This frequently results in faster builds. The following chart shows aggregated build time with and without the build cache for part of Gradle’s CI:
In this post, we’ll explain how you can use Gradle’s build cache to avoid unnecessary Kotlin compilation to speed up your builds.
Quick demo with Spek
You can try out build caching with Gradle right now. Just follow these steps:
Clone Spek
git clone https://github.com/spekframework/spek.git cd spek
The Spek 2.x branch (which is the default) already has all prerequisites for build caching that we’ll describe later.
Build and populate cache
The following command builds Spek and populates the local build cache.
❯ ./gradlew assemble --build-cache BUILD SUCCESSFUL in 10s 21 actionable tasks: 21 executed
Using the --build-cache flag is one way to tell Gradle to store outputs in a separate task output cache.
Remove/change build outputs
This simulates being on another machine or perhaps making a change and stashing it. The quickest way to demonstrate is use the clean task.
❯ ./gradlew clean
Re-build and resolve from build cache
This time when we re-build, all Kotlin compiled sources are pulled from the build cache.
❯ ./gradlew assemble --build-cache BUILD SUCCESSFUL in 2s 21 actionable tasks: 11 executed, 10 from cache
Voilà! You just used Gradle’s build cache to reuse Kotlin compiled classes instead of recompiling again! The build was about 5x faster!
You can see from this build scan that Kotlin compile tasks were pulled from the build cache; :jar and :processResources tasks were not because it’s faster to generate JARs and copy files locally than pull from a cache. Note that caching :test tasks is also supported.
The Gradle build cache is particularly effective when a CI instance populates a shared build cache, which developers can pull from. Links to more resources for achieving this are listed below.
Enabling the build cache for your projects
I hope you’re excited to try this out on your project — you can follow these steps to enable the build cache.
First, you need to ensure you’re using Gradle 4.3 or above, so the Kotlin Gradle Plugin can opt-in to using new APIs in Gradle. You can upgrade Gradle easily using the Gradle wrapper.
Next, we need to ensure we are compiling with Kotlin version 1.2.20 or above. You might have something like this declared in your buildscript {} block in build.gradle:
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:1.2.21"
}
Next, we need to tell Gradle to use the build cache. There are 3 ways to do this:
- Enable for the current build using
--build-cacheon the command-line. - Enable for the project by adding
org.gradle.caching=trueto$PROJECT_ROOT/gradle.properties. - Enable for all builds for the current user by adding
org.gradle.caching=trueto$GRADLE_HOME/gradle.properties.
NOTE: Android developers still need to do this even if android.enableBuildCache=true is set, because Gradle’s build cache is separate from the Android build cache.
We can optionally take advantage of the build cache from IDEs by delegating run and test actions to Gradle.
Enabling build caching in IntelliJ
If you use IntelliJ to execute Gradle actions, you will need to “Delegate IDE build/run actions to Gradle” in your IDE settings to take advantage of build caching when building and running tests from IntelliJ.
NOTE: Android Studio does this by default.
Caching kapt tasks
Caching for kapt is currently disabled by default, even with --build-cache, because Gradle does not yet have a way to map inputs and outputs for annotation processors. You can explicitly enable use of the build cache for Kotlin annotation processing tasks by setting useBuildCache to true in kapt configuration.
kapt {
useBuildCache = true
}
Further reading
You can learn more about leveraging the Gradle build cache through these resources:
- Configuring the build cache
- Setting up a shared, remote build cache ⚡️
- Debugging build cache misses
- Developing cacheable custom tasks
Conclusion
Compiling Kotlin code using kotlin-gradle-plugin version 1.2.20 and above can take advantage of Gradle’s --build-cache feature to speed up your development cycle. Work continues to expand the set of tasks that support build caching.
Onward!
Brian Stokes says:
February 6, 2018Does anybody know which version of Android Studio the “Delegate IDE build/run actions to Gradle” setting appears in? I don’t see it in 3.0 or either of the 3.1 and 3.2 previews. I can’t wait to try it out.
Sergey Igushkin says:
February 6, 2018This step is not needed with Android Studio, since it runs Android project builds with Gradle by default.
Scott says:
February 6, 2018I don’t see any such “Delegate IDE build/run actions to Gradle” in Android Studio
Sergey Igushkin says:
February 6, 2018Android Studio does that by default, all Android project builds are run with Gradle. This step is therefore not needed with Android Studio.
Louis CAD says:
February 8, 2018Why is it disabled by default for kapt? Are there any risks or drawbacks enabling it?
Sergey Igushkin says:
February 8, 2018We disabled it for kapt by default because annotation processors are basically arbitrary third party code, which may not necessarily only transform the inputs into the outputs but also read and write files that are not tracked by Gradle or take some system state like date & time into account.
If you are sure that the annotation processors that you are using only access the sources, classpath (and, possibly, Android resources) and produce only Java/Kotlin sources or class files, and that they are also pure (produce the same outputs given same inputs), then it’s fine to cache them. Many annotation processors fall into this category, sou you can just check how caching works with kapt in your projects.
Some annotation processors may use or modify additional files or other system state and are not pure. If you use them, it’s up to you to declare all of their inputs and outputs as, respectively, inputs and outputs of the kapt tasks. If you don’t, you may get false positive cache hits with caching enabled for kapt, or some of the outputs might not be stored in the cache.
If there appears a mechanism in Gradle for tracking annotation processor inputs and outputs, we may enable kapt caching by default for builds that only use annotation processors that correctly declare their inputs/outputs.
Svyatoslav Chatchenko says:
February 8, 2018What kind of issues should we expect from enabling caching for
kapt? How should we handle them? Executegradle cleanorgradle cleanBuildCacheor something else?Sergey Igushkin says:
February 8, 2018Basically, I can imagine three kinds of issues that can appear:
1) An annotation processor uses some data that is not tracked by Gradle as inputs of the kapt task. In this case, the cache key for kapt tasks in your project won’t include that untracked data, and you will get false cache hits. For example, if there are some special local configuration files for the annotation processor, you can still get a FROM-CACHE result and stale outputs unpacked after having modifed the configuration files. You can handle this sort if issues by adding the additional data as inputs to the kapt tasks in your Gradle build script. By default, the tracked data includes sources, compilation classpath, compiler options, annotation processors and kapt options (plus, Android layouts, if present). If an annotation processor does not use anything beside that, you won’t face this kind of problem.
2) An annotation processor produces something that is not tracked as an output of the kapt task. This will lead to those outputs not being packed into the cache and, consequently, not being unpacked on cache hit. It’s relatively easy to detect, since the build will likely fail in this case. Likewise, you can handle this by registering the locations of those special outputs as the kapt task outputs. By default, the tracked outputs are Java and Kotlin generated sources and class files, all in the pre-defined locations.
3) An annotation processor is not pure, i.e. given the same inputs it may produce different outputs depending on some additional state (e.g. system date and time). Similarly to the first case, you will get false positive cache hits, but if the state that the annotation processor takes into account is as transient as system time, you should not use caching for annotation processing.
Many annotation processors can be cached just fine, though. You can try to enable caching for kapt in your projects and check how it works.
Nirmesh says:
February 12, 2018Fantastic article.
Tim Kist says:
February 13, 2018Great article and thanks for the demo!
I think there’s a typo where the build cache is supported in Kotlin from 1.2.20 rather than 1.2.21. The link at the top goes to a release page for 1.2.20, but the text says 1.2.21.
Sergey Igushkin says:
February 20, 2018Thanks for the feedback! In fact, the 1.2.21 release is a hotfix release for 1.2.20, and it does not have its own blog post. We recommend 1.2.21 over 1.2.20, so the version in the article is intended.
Tomek says:
February 13, 2018Thanks for the article.
In my Android codebase I do not see a significant performance improvement when using build cache.
In what kind of projects should it improve the performance the most? Those who have multiple small modules? That use java only? Or any other?
Cheers
Vitaly says:
February 27, 2018I’ve added org.gradle.caching=true to gradle.properties and enabled build/run task delegation to Gradle in IDEA. But now when I run the app with Ctrl+Shift+R after making some changes, I still see the same output, as if changes haven’t been made.