Faster Groovy Compilation in IntelliJ IDEA 14.1

Compiling Groovy code is typically slower than compiling Java code, but in 14.1 we took a swing at speeding it up.

First, we’re now supporting Groovy-Eclipse compiler that handles Java and Groovy simultaneously, which brings a noticeable performance gain. So, if you’re compiling your projects in Eclipse, you might want to give this new integration a try. Just go to the Java Compiler settings, select Groovy-Eclipse, and point to the groovy-eclipse-batch.jar. Remember that is should be the same version as your Groovy distribution.

GroovyEclipse setup

Even if you’re not into using the Eclipse compiler, you will notice improved performance, because we’ve optimized the way IntelliJ IDEA invokes groovyc. We’ve seen about 2-3 times faster Groovy compilation with its own sources (where many tests are written in Groovy).

Here are some technical details.

For years the following had been IntelliJ IDEA’s routine for handling modules that contain Groovy files:

  1. Run external groovyc process to generate Java stubs from Groovy files.
  2. Run Java compiler (in-process) and compile all the Java files in the module, along with the generated stubs, so that references from Java to Groovy code resolve without errors.
  3. Run another external groovyc process to generate actual non-stub code from the Groovy sources.

Well, this isn’t the very best scenario: creating processes takes time, and each of them will subsequently need to load all the Groovy compiler classes along with project itself and library classes to be able to resolve references. And the icing on the cake is that the Groovy files are actually parsed twice in each of the processes: first for the stub generation, and then for actual compilation.

Unlike Eclipse developers, we don’t have the luxury of our own compiler and cannot directly integrate it with Groovy. IntelliJ IDEA has to be able to work with just any Java compiler, be it javac, ecj or aspectj, and with any version of Groovy too.

We probably could not do without the stub generation. Moreover, our compiler architecture suggests that Groovy and Java compilation be performed in different subsequent steps, which is not compatible with the groovyc API for joint compilation.

Despite all these limitations, we managed to find a way around this predicament with the following improvements:

  • Both groovyc and Java compilers are executed in the same build process.
  • After stubs are generated, groovyc is paused, waits for Java compiler to do its job, and then resumes. Thus we still have stubs, java compilation and groovyc run sequentially, but groovyc is not interrupted in between and restarted from scratch.
  • We now use a special class loader that caches directory contents and thus reduces the number of disk accesses to find a class or resource. In multi-module projects, when compilation classpath is comprised of many directories from different module dependencies that often intersect (meaning that one directory is used in more than one module), this cache gives a good performance boost (especially on Windows with its slow file system.)
  • If you have groovy-all*.jar in your dependencies, IntelliJ IDEA will reuse its Groovy compiler classes for compiling different modules.
  • Optimizations from our side end here. But you can have a faster Groovy jar! Build it yourself a from a branch where it doesn’t use classloading for resolving code references, and get even better speed (and help us with testing it.)

These improvements are all available in the EAP version, and we recommend that you try it with your Groovy projects so we can fix any oversights. Any feedback you have is welcome, as always, in our issue tracker.

Develop with Pleasure!

This entry was posted in New Features and tagged , , , , . Bookmark the permalink.

18 Responses to Faster Groovy Compilation in IntelliJ IDEA 14.1

  1. Paolo says:

    That’s cool, but where to find the Groovy-Eclipse compiler? Is it embedded with eclipse distribution ?

    p

  2. Jesper Thuun-Petersen says:

    I’ve done a couple of compiles (rebuilds) with the eclipse-groovy plugin. I did two consecutive rebuilds, and they timed in at 2.07m and 1.26m. I then changed back to the old javac and did a rebuild. It timed in at 1.14…

  3. Pingback: Diario di Grails (settimana 11 del 2015) | BME

  4. Mikhail says:

    Is there way to disable the new groovy compilation method and return back to the old one?
    I start getting OOM when compiling my project with 14.1. Increasing heap size for groovyc does not help.

    • Peter Gromov says:

      Yes, you can add -Dgroovyc.in.process=false as an additional build process VM option in “File | Settings | Build, Execution, Deployment | Compiler”. But we’d prefer to fix this. Could you please try increasing build process head size on the same settings page first? If this doesn’t help, could you please file a YouTrack issue and attach your log there (Help | Show log)?

      • Mikhail says:

        Okay, I got it compiled successfully increasing the “build process heap size” setting.
        So I was just confused by the “maximum heap size” setting for groovyc. Does this setting make any sense when using the in-process compiling on a groovy project?

        • Peter Gromov says:

          Indeed, I somehow missed that, thanks for noticing. Filed as https://youtrack.jetbrains.com/issue/IDEA-138136

          Out of curiosity, what are your groovyc heap size setting, your previous build process xmx and your current build process xmx?

          • Mikhail says:

            it was 1200M for groovyc and it was enough.
            it failed with 700M for build process on 14.1
            And 1200M is enough for the build process as well. Actually, first I tried 1500M but build failed due to restrictions of 32-bit VM that is configured for my project.

            • Peter Gromov says:

              That’s quite a lot. Either your project contains lots of Groovy files, or groovyc takes unnecessarily much memory. That’s most likely Groovy core issue, but I could look at a memory snapshot (e.g. taken with jmap) and check if there’s something easy that can be done. If you feel like capturing snapshots, you can write me (peter@jetbrains.com).

  5. Gary Shurgin says:

    I switch to use groovy-eclipse-batch/2.3.7-01 and got 804 errors and 46 warnings.

    The errors appear to be when attempting to reference a java class from groovy or a groovy class from java.

    My project is setup for maven, with groovy files in src/main/groovy and java files in src/main/java.

    Is there any other configuration that I need to make this work.

    • Peter Gromov says:

      That’s not expected. Did you try full rebuild? If yes, it would be great if you file a YouTrack issue (https://youtrack.jetbrains.com), attach idea.log and build-log/build.log there (Help | Show log) and, if possible, a sample project that we can reproduce the issue on. In the meantime you can use the default Groovy compiler.

Leave a Reply

Your email address will not be published. Required fields are marked *