IntelliJ IDEA 2017.1 EAP Extends Debugger with Async Stacktraces

Following the reactive programming trend, our code is getting more asynchronous. Earlier Java 8 introduced CompletableFuture (adopted from Guava’s ListenableFuture). Akka, Ratpack, Reactor, RxJava, Vert.x and other libraries implement Reactive Streams, Scala offers Future, Kotlin is adding Coroutines, and finally, Java 9 is about to bring us the Flow. Well, reactive programming helps us build efficient applications, but boy are they difficult to write and debug.

Consider this sample (courtesy of github.com/nurkiewicz):

If we put a breakpoint here, the stack trace will look somewhat like this:

Instead of the place where we invoke CompletableFuture.supplyAsync (S2_Creating.supplyAsync), we’re looking at ForkJoinPool.runWorker (executor service that runs code asynchronously) and all of its internals. This makes it harder to understand the data flow and also makes it almost impossible to navigate to relevant frames.

IntellIJ IDEA 2017.1 helps us here with its a new Debugger feature called Capture, which alters the stacktrace by substituting its parts related to the asynchronous code execution (receiver) with the corresponding parts of the stacktrace captured from where the asynchronous code is passed (sender).

For this feature to work, IntelliJ IDEA needs to know exact signatures of methods used for sending and receiving data (asynchronous code), along with expression that represents the data being sent.

IntelliJ IDEA 2017.1 EAP offers the following settings to configure this (Settings > Build, Execution, Deployment > Debugger > Capture):

Right now you have to manually configure this, but later we may provide a predefined configuration for the most popular libraries. Please let us know which libraries you would like to have on this list.

Once all the requirements have been met, IntelliJ IDEA will display the adapted stacktraces:

For better clarity you can filter out the library frames:

idea_2017_1_capture_captured_stack_filtered

The substituted parts of stack trace display local variables (without object fields, though):

It’s important to know that this feature may affect performance because it collects additional data during execution.

This feature is already available in the newly-released IntelliJ IDEA 2017.1 EAP build.

The way the feature works as well as the way it’s configured are in active development. We appreciate any feedback that may help us improve this feature. Share your ideas as well as bug reports here in the comments or directly in the issue tracker.

In other news, we’ve added a new option to the Diff dialog: Ignore imports and formatting and, as its name says, it ignores changes within import statements and whitespaces (at the same time it respects whitespaces within String literals):

Hope you’ll find this useful.

UPDATE: For more technical details about Async Stacktraces, read this comment.

UPDATE 2: Please submit your ideas of particular capture points to our GitHub repository via pull requests. The definitions from this repository will be bundled with the IDE.

Develop with Pleasure!

About Andrey Cheptsov

Part-time dreamer. JVM/JavaScript geek. IntelliJIDEA Marketing Manager at JetBrains.
This entry was posted in EAP Releases, New Features and tagged , , . Bookmark the permalink.

22 Responses to IntelliJ IDEA 2017.1 EAP Extends Debugger with Async Stacktraces

  1. Jan says:

    I thinks this EAP broke the JVM Debugger Memory View – as my Event log is saying:

    Plugin Error
    Problems found loading plugins:
    Following plugins are incompatible with current IDE build: JVM Debugger Memory View

    • Andrey Cheptsov says:

      This is because JVM Debugger Memory View is not a plugin anymore. It’s now built-in into the IDE. You can remove the plugin now completely.

  2. Alex says:

    Would this work with Scala’s Futures out of the box?

  3. Dimitar Dimitrov says:

    Both features are very exciting!

    The diff is immediately useful and becomes my new default setting.

    I wonder if we can use the async stacktraces in interation tests for distributed applications, talking over network transport. In a typical case, one actor would send a message over reliable multicast and any number of actors may act on it and send their own messages, etc. until the system stabilizes.

    Seeing the chain of events is great, seeing the tree of events is even more awesome (but here we venture into “tracing” grounds).

    I got a few questions related to this:

    1. Can we specify a correlation expression (i.e. message.getSender() + ":" + message.getId())
    2. Can we have more than one child stacktrace for a parent? (i.e. one to many communication, rather than a single job hopping between threads).
    3. Can we get more details about the resource management scheme? I.e. when is the captured stacktrace purged?

    • Dimitar Dimitrov says:

      I just tried setting up a stacktrace capture and it doesn’t seem to work.
      Overall we could use a bit of documentation.

      Here are a few questions from the 15 mins I spent trying to make it work:

      – For the capture and insert classes
      — does this work across the class hierarchy, or do we need to specify concrete classes to instrument?
      — can we specify more than one capture/insert points
      — What is the class name syntax (i.e. $ vs . for inner classes)
      — Would it work for lambdas (i.e. insertion point can be a lambda implementing listener interface)

      – For the key and capture param
      — How is the capture param used? Do we apply the key expression to it, or is it compared to the key expression? What happens if it is a mutable buffer (i.e. flyweight over recycled byte array).
      — What is the “key expression”? Is it a field path? Can it be a method call? Can we do concatenation? What is the evaluation context, how do we refer to things?
      — Would be useful to be able to specify different key expressions for the capture and insert points.

      – The example:
      — The AsyncSupply::run in Java 8_112 does not have anything called “f” – the insert key. Makes it difficult to understand the setup in the screenshot.
      — All CompletableFuture code does not line up – the stack trace line numbers do not correspond to the code in the last few versions of the JDK.

      • egor.ushakov says:

        Hi Dimitar, thanks for your interest. Will try to provide more details on this.

        Currently it captures stacks in a place described with the method fqn (settings columns 1 and 2).
        For this it sets an emulated method breakpoint (fast version of method breakpoints introduced in IDEA 2017.1) at the entry of the method.
        Any method overrides in the class inheritors should work.
        It stores the stack and variables info in a map, where the key is a parameter with the specified number ((settings columns 3)).
        Currently the map holds only the last 1000 stacks.
        That’s it for capture part. Now to the insertion.
        For every frame in a stack displayed we check:
        1. insert class name equals to the current (it may also be empty)
        2. insert method name equals to the current
        3. if 1 and 2 it true, we evaluate the insert key expression and check that the value object is a key in the map with the captured stacks.
        If all 3 is true, we replace the rest of the stack with the captured stack.
        The insertion process also happens when we capture a stack, making “nested” stacks capturing possible.

        Current restrictions:
        – the stacks map is one for all capture points, there could be only one captured stack per the key object.
        – key object could only be a parameter of the method specified for capture
        – insertion point could take any captured stack, so you can specify any number of fake capture points with different insertion points to see the captured stack in different places (broadcast situation), just enter anything in the capture method class (do not leave it blank)
        – capture class name should look the same as in debugger (with $ for inner classes)
        – lambdas as capture points are not supported yet

        More answers:
        – there is “Supplier f” local variable in AsyncSupply.run which is an instance of what was passed into supplyAsync, if you do not have jdk sources, or they are different, it may not work

        Once again, thanks for your interst, I’m looking forward for more ideas and feedback!

        • Andrey says:

          >It stores the stack and variables info in a map, where the key is a parameter with the specified number ((settings columns 3))

          >CompletableFuture supplyAsync(Supplier supplier)

          In post example: key of map will be supplier, right?

        • Andrey says:

          Can constructor be used as method name?

          Look at this example:
          CompletableFuture future = new CompletableFuture();
          executor.submit(() -> {
          try {
          Thread.sleep(3000);
          } catch (InterruptedException ignored) {
          }
          future.complete(123);
          });

          future.whenComplete((result, throwable) -> {
          System.out.println(“Task completed”);
          System.out.println(“result = ” + result);
          });

          I tries to use as method name with 0 parameter. But looks it doesn’t work

        • Dimitar Dimitrov says:

          In my case, we have an inter-thread messaging API, serializing events between channels.

          Our events objects are just flyweights over byte buffers, so there is no point of putting the event into a map – it will always be the same.

          The way to corellate stacks would be to call event.getId() at the capture point and compare that to event.getId() at the insertion point.

          I understand the current implementation does not handle this case, but it would be nice if it can be accommodated. I have seen this approach to ITC/IPC in multiple codebases.

      • Dimitar Dimitrov says:

        Another benefit of having real key expressions for both sides is that we can do namespacing. I.e. we can have "ipc." + event.getId() and separate "workerPool." + job.getId() and both would never clash even if a job has the same ID as an event.

        In that case, it would also be useful if in the debugger we can browse the stack cache.

  4. Dan O'Reilly says:

    I’d love to see RxJava (both 1.x and 2.x) and Vert.x support included out of the box!

  5. @JetBrains: Are you guys aware of the Node.js/Dart zone concept? It might be a useful architectural/UX metaphor if you want to take this further…

    * https://github.com/strongloop/zone (JS server/node.js)
    * https://github.com/angular/zone.js (JS client/browser)
    * https://www.dartlang.org/articles/libraries/zones (dart)

    • …and Async stack traces in Chrome Dev Tools. For us fullstack devs it would be great to get a similar UX in both cases to ease switching between dev envs.

      • Ekaterina Prigara says:

        IntelliJ IDEA’s JavaScript debugger that works Google Chrome has support for debugging async stack traces. You can enable that right on the JS debugging tool window. Please give it a try and let us know what you think. Thanks!

Leave a Reply

Your email address will not be published. Required fields are marked *