Brand New Compiler Mode in IntelliJ IDEA 12 Leda

One of the major features in the upcoming release of IntelliJ IDEA 12 Leda is a completely revised approach to compilation of a project. We have rebuilt it from the ground up to bring a new user experience and a better performance.

There are following key principles for this new compilation approach:

  • All compilation tasks are performed in a separate process, fully independent from the main IDE’s process, to get rid out of memory and other resources contention.
  • Java compiler is used “in-process” via Java API for better performance and more flexible annotation processors management.
  • Java classes dependency management completely rewritten for faster and more accurate search of dependent classes.
  • Compilation is automatically triggered (“auto-make”) by events from the file system of the IDE if such option is enabled.

As a result of these improvements we have:

  • Significantly faster rebuilds.
  • Lower memory requirements for the IDE.
  • Global error highlighting without explicit compilation invocation.

We’re still working on the feature and its performance, so we will provide the benchmarks later.

There are also other things coming soon:

  • Eclipse Java compiler support.
  • Ability to use the feature from the command-line and and Ant/shell scripts.

Integration with Eclipse Java compiler will enable better errors highlighting for the project and even running Java applications containing compilation errors.

Expect more details in the upcoming EAP releases!

Since the feature is still under development, it has constraints, which we hope will be removed soon:

  • Enabled only for Java and Groovy sources, resource files, UI designer forms, artifacts, Android classes and resources, annotation processors and not-null instrumentation.
  • Not supported (yet) Eclipse-based and Grails projects.

This external build feature is currently disabled by default. You can enable it in Preferences → Compiler settings:

The checkbox Use out-of-process build for project compilation enables the new compilation approach. If you enable Make project automatically checkbox, the compilation will be automatically triggered once you make any changes in the sources.

We very much hope to receive a constructive and fair feedback from you. Please share your impressions and ideas on our EAP discussion forum or report any found issues in YouTrack.

News from our neighbor teams (non-paid)

YouTrackOur great keyboard-oriented issue tracker YouTrack has been just updated to the latest version.
YouTrack 4.0 introduces a completely new Agile project management module. Now Scrum teams can plan and manage their work during a series of consecutive sprints, while teams that follow Kanban process can manage and measure their work flow on the board.

Check out the YouTrack 4 What’s New page for details.

Develop with pleasure!
IntelliJ IDEA Team

About Andrey Cheptsov

Andrey Cheptsov is IntelliJ IDEA product marketing manager at JetBrains. He's passionate about productivity, programming languages and tools.
This entry was posted in New Features and tagged , , . Bookmark the permalink.

40 Responses to Brand New Compiler Mode in IntelliJ IDEA 12 Leda

  1. B. K. Oxley (binkley) says:

    Fantastic news. This was one of the largest complaints from my Eclipse-using friends — no automated compilation as they edited.

  2. Laurent Kempé says:

    Great!

    Do you also plan to have automated parallel continuous testing tool?

  3. fcamblor says:

    When I try to check the “Allow auto make while a run configuration is active”, the “apply” button is activated then really fastly disabled :-(

  4. Tim Freeman says:

    Will there be any support for remote compilation, distcc style?

  5. Eugene Zhuravlev says:

    fcamblor, please leave this option (“Allow auto make while a run configuration is active”) off. We’ll remove it later. To enable auto-make, just use the corresponding checkbox

  6. Laurent, we have TeamCity for automated continuous testing – http://www.jetbrains.com/teamcity

  7. Eugene Zhuravlev says:

    Tim, as for distcc, we did not consider it for IDEA yet. Could you please write some details why do you think it will be useful and what particular problems that are present now, this approach will solve?

  8. Steven Willems says:

    U know this is a bit off topic, but would like to react on the continuous testing / team ity comments:
    TeamCity is great and I use it for a long time now. But being able to continuously running all unit tests in the background would give one even faster direct feedback. Dev machines are getting powerful enough to do this quietly in the background..
    And it can be done smart, e.g. first run the tests in the package you are working, or only the ones in the current project. A simple config panel would suffice.
    Just suggesting.. it would make my favourite IDE even better! No more manual test running!

  9. BenP says:

    I am really glad you are adding incremental background compilation and Eclipse-style ‘run with errors’ support. Excellent news!

  10. Ivan says:

    This is a real killer feature. I really love Idea for many years, but I’m sick and tired of performance problems of 11th version, so everything that makes Idea faster makes me very excited.

  11. Eugene Zhuravlev says:

    BenP, we’ve had incremental compilation run in background since version 1.0. This feature is about moving all the stuff out of IDEA process + adding some additional modes, like ‘auto’-compilation.

  12. Andreas Guther says:

    What is the license upgrade policy? We just go IntelliJ 11 in January.

  13. Graham Nash says:

    I agree with Laurent Kempé & Steven Williams re: continuous testing. We also use Team City, but it would tighten the feedback loop if IntelliJ automatically ran outdated tests after automatic compile. Another task that would be useful to run on successful compile would be automatic SCM sync (i.e. git pull –rebase).

    See:

    http://www.amazon.com/Continuous-Testing-Ruby-Rails-JavaScript/dp/1934356700/ref=sr_1_1?ie=UTF8&qid=1340908927&sr=8-1&keywords=continuous+testing

  14. yole says:

    Andreas,
    As usual, major versions of IntelliJ IDEA are paid upgrades. IntelliJ IDEA 12 is coming out in late November/early December, so you should get a lot of value from your January investment.

  15. Meo says:

    And what about parallel java compilation? It would be huge performance improvement. Not sure if single module could be parallelized, but multi module project could, just like maven does it.

  16. MuppetGate says:

    This is pretty damn fast! Amazing!

  17. Stephan says:

    I hope that the tight Maven integration is still fully supported like today?!

  18. BenP says:

    Eugene, I found the existing background compile to be very hard to use.

    Eclipse has a dedicated ‘problems’ window that updates nicely with changes based on the background compilation, whereas IDEA has issues where the Make window jumps in and out of existence, steals focus and so on — it’s just not as smooth. I hope that the work you’re doing now will make it as nice to use as Eclipse is in this area.

  19. Eugene Zhuravlev says:

    BenP, automake does not show any windows at all. All errors reported by compiler are highlighted in ProjectView and Editor (provided you have these files open).
    The current error window is opened on explicit compilation request and shows compile errors from this particular compile session (like it has always been).
    Later we’ll probably add a toolwindow containing all errors from automake, but this is a separate feature that will augment and improve particularly the ‘automake’ mode of compiler.

  20. yole says:

    Ben,
    We have an equivalent view too: select “Scopes | Problems” in the Project view. It will likely become a separate toolwindow in IDEA 12.

  21. Dave Griffith says:

    Please, Scala support!

  22. Tim Freeman says:

    Well what I meant was “in the style of” distcc. Where you can have a thin client such as a weak laptop and have something more powerful on the same LAN doing the heavy lifting (with a lot more RAM than in a laptop). It’s helpful for bigger projects. Not saying it makes sense in your architecture, just curious if that is a long term plan.

  23. Laurent Kempé says:

    Eugene, I don’t really talk about that, I know quite well TeamCity, I am talking about what Steven is clarifying; running unit test continuously as you are doing with the compilation

    Steven, Thanks for clarifying!

  24. Jon Freedman says:

    Regarding continuous testing, take a look at the Infinitest plugin: http://infinitest.github.com/

  25. Lukasz says:

    One thing:
    - when I remove setter that is used in other classes, nothing happen, I must switch windows and move back to see the red underscore on the affected classes
    - but when I’ve added the setter back, the underscore disappears in few seconds without windows switching

    is it a bug ?

  26. Eugene Zhuravlev says:

    >- when I remove setter that is used in other classes, nothing happen
    > is it a bug ?

    what is your configuration and what do you expect? If you run compiler and it succeeds without errors, then yes, it is a bug.

  27. Geoffrey De Smet says:

    Will we able to run a project of which some classes don’t compile? This is very handy in some situations: as long as the program blows up as soon the run uses any of the uncompileable classes.

  28. Nick Hristov says:

    I feel that this is taking the IDE completely in the wrong direction. I think you will take a Mercedes Benz and turn it into a Volkswagen.

    We already have “incremental” build feature within intelliJ: I never have to rebuild the project from scratch, and I have never had a dependency problem (i.e. I modify one class and something breaks because the compiler did not figure out that a dependent class had to be re-compiled).

    “Running with errors” feature is stupid. If the code does not run, it should be fixed; there is no point in being “intelligent” and running in half-broken way, wasting developer time by testing code which, for all practical purposes, runs non-deterministically, because it has been “patched” by the IDE.

    The beautiful thing about intelliJ is that it does not try to be too smart. For 99% of the time, it is a very effective, highly efficient text editor with syntax completion. It does not run background jobs because it thinks that “task X” must be done ["task X" being completely moronic]. It does not block the UI and force you to wait because it feels that “task X” must be complete, and your operation must block until “task X” needs to complete, ala idiotic Eclipse…

    There is no point in making IntelliJ more like Eclipse. There is already one Eclipse, its free, and its price reflects its design.

  29. Eugene Zhuravlev says:

    Geoffrey,

    > Will we able to run a project of which some classes don’t compile?

    This is mostly a feature of compiler used to compile classes. With javac it is not possible for obvious reason. With Eclipse compiler it will be possible. Eclipse compiler integration has been already added.

    Nick, thanks for your comment, I agree with most points.
    I’ll post here my comments taken from the thread in our discussion forum:

    The whole point of the “automake” feature, is to compile in background as sources change on disk. This may reduce work to the “non-auto” make invoked by you on demand and thus reduce the time needed to compile when you start the application or a test. Another side-effect of this feature that might be usefull will be showing errors from the automake in project view (this works now parttially, we need to make some changes to our error highlighter).
    Note, however, that the new features are not the replacement for the usual make, but rather the features that augment it. This is also not the exact copy of Eclipse’s background compilation just because the goals of running compiler in IDEA and Eclipse differ. Eclipse needs background compilation because it uses compiled classes to build the AST. This is also the reason why Eclipse needs its own compiler for the work: because standard javac cannot generate class file for sources with errors. So the ability to “run non-compilable code” is an implication of using Eclipse compiler in special “proceedOnError” mode.
    IDEA does not need compiled classes to build the AST. Compiler is used just to produce executable classes. IDEA can use either standard javac or Eclipse compiler for this job. Using Eclipse compiler is an advantage to those who always want to see full list of compilation errors in the project. In combination with automake this feature will also be supported.

  30. Lukasz says:

    Eugene I don’t see my answer I’ve posted last week (don’t know why :-( ), so again:

    I’ve enabled “automake” option and what I expect is to see errors when a class cannot be compiled (like the above example with setter). Right now I have to switch to other application (another window) and switch back to see red underscore on the affected classes. It looks like to invoke autocompile I must take an action (like it was with autosaving sometime ago).

  31. batto says:

    @Nick

    > “Running with errors” feature is stupid. If the code does not run, it should be fixed; there is no point in being “intelligent” and running in half-broken way, wasting developer time by testing code which, for all practical purposes, runs non-deterministically, because it has been “patched” by the IDE.

    Because some unfinished and UNREALTED code in your project doesn not compile doesn’t mean that you can’t compile & run COMPILABLE code. I like IDEA for being good all-purpose Java IDE that lets me write code how I like. On the other hand I hate doctrine tellers like you.

    I’m working on some algorithm that depends on some common code. Friend sends me his version with tests in main() that depends on same common code. So I create a class in IDEA and paste it there. I try to run it but I can’t because my algorithm code is unfinished. So I add dumb things like “return null”, comment that, comment whole file, …

    You enterprise folks and your doctrines…

  32. Stephane Nicoll says:

    The rewrite of the annotation processor management in Leda still misses a very important point: you may want to trigger annotation processor in production code and in test code.

    Right now, the UI allows you to specify ONE directory where the generated source files will be stored. That’s obviously not going to work in case you are in this case with a build tool, like Maven who generates this by default in two separate locations. Worse, the user experience is pretty bad since you easily get an error because you have a duplicated source file (idea generating in its own directory, maven in another).

    There are also weird issue related to annotation processor. There is a sample project that reproduces these issues, please have a look at http://youtrack.jetbrains.com/issue/IDEA-60774#comment=27-366379

    Thanks!

  33. Christian says:

    Hey guys,
    I’m running 122.395 and I’m getting OOMEs from the new compilation process because it is always using the following (hardcoded?) settings

    -XX:MaxPermSize=150m -XX:ReservedCodeCacheSize=64m -Xms350m -Xmx700m

    The memory settings for javac at
    Settings > Compiler > Java Compiler > javac > Maximum Heap Size (MB)
    seem to be ignored by the new compilation process as I’ve set this to 2048.

    is there any other way to overwrite the default memory settings?

    Kind regards,
    Christian

  34. Dave says:

    I’d like to second the request for a remote compiler – being able to use the resources of a fast machine for compilation is a big benefit for us road warrior/telecommute types.

    In addition, parallelizing the build (maven style or better) would be fantastic. Parallelizing the build across arbitrary remote backends would be bliss! (far quicker compile times, much better utilization of hardware in a dev team).

  35. Joakim says:

    Wow, I’m late to the party as I just found this. Sounds like great news, though!

    How do I hook up the AspectJ bytecode weaver to this new compiler mode?

  36. Wesslan says:

    I have version 122.694 but when I go to preferences->Compiler I can’t see Use out-of-process build for project compilation, just Use external build.

  37. Sebastiano Pilla says:

    @batto: I second Nick’s point, if some code is broken then the only sensible thing to do is to remove it or to fix it. The compiler/IDE should not make this decision for the developer.

  38. stevo says:

    hello, i have the same issue as Lukasz.
    when I remove setter that is used in other classes, nothing happens. no red underscore on the affected classes / tree. but when I’ve added the setter back, the underscore disappears in few seconds..

    (IDEA: 122.639)

    cheers, stevo

  39. Jim W says:

    For projects with multiple modules containing source, the display of errors in the “Problems” tab populated by automake, and those files outlined in red in the project pane, only appear to take into account errors in the module where the error was originated.

    For instance, if module A contains an interface that contains implementors in modules A, B, and C, and I change the interface in A to force an error with automake options turned on, the errors reported by the automake and those files outlined in red in the project pane only include the implementation in A. However, if I open the implementors in B or C, then those errors are immediately shown in the project pane. It’s as if the automake did compile the code in modules B and C, but fails to render them to the project pane or to the errors reported by the automake.

    This of course makes determining quickly all the errors produced by a non-automated change (one not created by a refactoring) more difficult to determine.

    Is there some view option I’m missing, or is this a feature that is coming soon?

  40. Neil says:

    @batto: @Nick

    There is at least one perfectly good reason you may want to run with broken code. Let’s say you have code with a few classes A, B and C which don’t compile. You want to fix one class at a time and run the unit tests as you go. While fixing class A, you want to be able to run ATest.java without first having to go and sort classes B and C.

    In intellij, you can’t do this as after fixing class A you’ll get compile errors on B and C even though you are only trying to run the tests for class A.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>