IntelliJ Rust 0.3: New Macro Expansion Engine

Marina Kalashina

IntelliJ Rust has reached a major milestone: the new macro expansion engine has moved out of the experimental stage and is now enabled by default. In this post, we’ll take a closer look at its implementation details and the features it brings for your code.

• How the plugin expands declarative macros
• The new engine status
• Code insight provided by the new engine

How the plugin expands declarative macros

To begin with, let’s see how IntelliJ Rust handles declarative macros in your code. When the plugin meets a macro call, it searches for the corresponding pattern in macro_rules!, substitutes the arguments, and gets the first step of the expansion, which is basically a piece of Rust code. If there are more calls inside, the process repeats until it reaches full expansion or hits the limit of 64 steps.

Both the old engine and the new one share this first part of the workflow. The differences emerge when it comes to storing expansion results.

Although the old engine was able to handle macro-generated structures, functions, and enums, it was unaware of module declarations and impl blocks. The old engine stored expansion results in RAM, making it impossible for the IDE to index the expanded code. Without indexes, the plugin couldn’t perform name resolution for mod-s and impl-s properly. This limitation could affect your code even if there were no custom macros, since a lot of methods from the standard library, like those for the primitive types, are macro-generated as well.

The new engine solves this problem by using disk memory instead. It stores the results in compressed binaries which the IDE can index, allowing the plugin to properly handle the generated mod-s and impl-s.

As a bonus, the reduction in RAM interactions makes the new engine much more cache-friendly, which has a positive impact on the overall performance.

Another notable detail about the new engine is that it processes macros in a separate phase. You may notice that initial project loading slows down, but as you type, the updates happen incrementally in the background without affecting your work in most cases.

Also, keeping the process in a separate phase allows the new engine to order the expansion steps more accurately. This is essential for handling crates with nested module declarations, like tokio.

The new engine status

Macro expansion can be extremely time-consuming. To give you an example, a simple ‘Hello, world’ project involves around 9,000 macro expansions due to its use of the standard library! So the algorithms used by the new engine required a lot of tricky optimizations along the way, and there’s still much room for further improvement.

However, over the last year, the engine gradually became faster and more stable until it could handle big projects like the Rust compiler with sufficient speed and quality. At that point, we decided it was time to turn it on by default.

Nonetheless, you can always switch back to the old engine in case the new engine fails for your code at some point. Use the same ‘Expand declarative macros’ switcher in Settings / Preferences | Languages & Frameworks | Rust:

Code insight provided by the new engine

The key feature of the new engine is its ability to handle macro-generated impl blocks and module declarations. This brings proper type inference to the entirety of your code and affects many inspections, eliminating previous false-positive results. In fact, enabling the new engine has let us close an entire list of various issues.

Now that macro-generated modules are supported, you can work to the full extent with crates like tokio, async-std, reqwest, and others, which typically include nested mod declarations:

Methods inside impl blocks generated by macros get full code insight too:

And there are yet more features available for your code:

• Highlighting now works for all code elements inside a macro call body.

• Code completion is available inside macro calls.

• You can use actions like Go to Declaration or Usages (Ctrl+Click / Ctrl+B on Linux/Windows, ⌘Click / ⌘B / force-touch on macOS) to navigate through the generated items.

• Other features, such as Rename refactoring (Shift+F6 on Windows/Linux, ⇧F6 on macOS) or Type Info (Ctrl+Shift+P on Windows/Linux, ^⇧P on macOS), also work for your code with macro calls the same way you would expect them to perform on any other Rust code.
  

Are you intrigued by these new capabilities? Update the plugin to version 0.3, try them out, and let us know what you think. Feel free to leave your feedback in the comments below or use the plugin’s issue tracker.

What’s next? We’re continuing to tune the new engine, and one of the big features on our long-term roadmap is support for procedural macros.

Thank you for reading, and stay tuned!

Your Rust team
JetBrains
The Drive to Develop

Subscribe to CLion Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!