Revamping Plugins #1 – .ignore

Jakub Chrzanowski

The IntelliJ Platform SDK changes dynamically, and each major release of JetBrains IDEs fixes bugs, adds new features, or introduces breaking changes in the SDK. Because of that, it’s essential to keep track of the Known Breaking Changes list in the IntelliJ Platform SDK Documentation. The plugins that we’re working on can sometimes become outdated, and to catch up with all the changes required to get them up-to-date and compatible with the latest IDE releases is difficult.

The Revamping Plugins series is about sharing the experience of updating outdated plugins to align with the latest IntelliJ Platform SDK guidelines. We hope this series will help you to understand the process of doing this and the tools JetBrains provide to make it easier.

About .ignore

The main idea of the .ignore plugin is to provide an easy way for creating .gitignore files using predefined templates provided by the official GitHub repository – github/gitignore. The first version of it was released in June 2014 and initially supported only .gitignore files. Further releases introduced new features, including syntax highlighting, custom user templates, and other ignore-like file support.

IntelliJ IDEA 2019.2 introduced native support for the dot-ignore files handling of Git (.gitignore), Mercurial (.hgignore), and Perforce (.p4ignore). As a result, some parts of the implementation were disabled, and the plugin was put into maintenance.

Why did we decide to revamp the .ignore plugin? We were working on the 2020.3 versions of our IDEs which were due to be released in late 2020, and the plugin was missing support for them. That is why we applied the changes described below to make the adjustment process for the latest IDE versions easier. As a bonus, removing dead code and introducing GitHub Actions as CI/CD has improved the plugin’s stability and quality.

The revamping process

README file

The .ignore plugin repository is located in the JetBrains GitHub account, JetBrains/idea-gitignore. We can see that the README file is descriptive and provides a brief introduction to the plugin, installation guidance, and a few details about contributing to the project.

We have introduced a few enhancements:

• The text was checked over using Grammarly
• Added a couple of badges such as build status
• Performed a general cleanup
• Added new demo recordings

Build configuration

The .ignore plugin is built on the Gradle build system which is written in Groovy, and uses four plugins:

• jacoco – provides code coverage metrics for Java code via integration with JaCoCo.
• org.jetbrains.intellij, v0.4.21 – supports building plugins for IntelliJ-based IDEs.
• de.undercouch.download, v4.0.0 – adds a download task to Gradle that displays progress information.
• com.github.kt3k.coveralls, v2.8.4 – sends coverage data to coveralls.io.

The plugin provides custom BNF and Lexer definitions for describing the language of .gitignore and other ignore-like file types to allow for syntax highlighting, navigation, and refactoring. In the resources/bnf directory, we can find Ignore.bnf used by the Grammar-Kit to generate the parser. The JFlex file is located in src/mobi/hsz/idea/gitignore/lexer/Ignore.flex, which doesn’t look like an appropriate place for a resource file.

The Gradle build file defines 7 tasks in 65 lines of code that are responsible for the process of fetching Grammar-Kit and JFlex outdated binaries (Grammar-Kit v1.2.0.1 was released in November 2014). Over the last 6 years, Grammar-Kit provided a Gradle plugin that simplifies that process to the following form:

val generateLexer = task<GenerateLexer>("generateLexer") {
    source = "src/main/grammars/Ignore.flex"
    targetDir = "src/main/gen/mobi/hsz/idea/gitignore/lexer/"
    targetClass = "_IgnoreLexer"
    purgeOldFiles = true
}

val generateParser = task<GenerateParser>("generateParser") {
    source = "src/main/grammars/Ignore.bnf"
    targetRoot = "src/main/gen"
    pathToParser = "/mobi/hsz/idea/gitignore/IgnoreParser.java"
    pathToPsiRoot = "/mobi/hsz/idea/gitignore/psi"
    purgeOldFiles = true
}

As you may notice, both resource files have been moved to the src/main/grammars directory to separate them from actual sources.

Below is the sourceSets section that redefines our project directory structure:

sourceSets {
    main {
        java.srcDirs 'src', 'gen'
        resources.srcDir 'resources'
    }
    test {
        java.srcDir 'tests'
        resources.srcDir 'testData'
    }
}

Thanks to that definition, our project tree looks like so:

├── build
├── gradle
├── resources
├── src
│   └── mobi.hsz.idea.gitignore
├── testData
│   └── inspections
└── tests
    └── mobi.hsz.idea.gitignore

Plugin sources, resources, tests, and testData are all located right in the root directory. That approach may be confusing for contributors because of its structure, which is different from the default Gradle one. By removing the above sourceSets definition, we can achieve something like:

├── build
├── gradle
└── src
    ├── main
    │   ├── gen
    |   │   └── mobi.hsz.idea.gitignore
    │   ├── grammars
    │   ├── java
    |   │   └── mobi.hsz.idea.gitignore
    │   └── resources
    └── test
        ├── java
        │   └── mobi.hsz.idea.gitignore
        └── resources
            └── inspections

A bit more nested, but now we can start migrating our plugin sources from Java to Kotlin with ease, file by file, introducing new content in src/[main|test]/kotlin directories.

The last step of the Gradle script enhancements is to apply all the tools already provided by the IntelliJ Platform Plugin Template:

• Rewrite existing parts of the Gradle configuration to Kotlin language, and rename the file to gradle.build.kts.
• Adjust the CHANGELOG.md file to match the common pattern and integrate it with the Gradle Changelog Plugin.

Continuous Integration

The Plugin’s CI depends on the external TravisCI service, whose setup is defined within the .travis.yml configuration file, but the test phase was recently disabled. Since CI doesn’t even verify the plugin quality or deploy new releases to Marketplace, it’s safe to remove it.

Looking closer at the old gradle.build script, the publishPlugin section of the Gradle IntelliJ Plugin is present. The following publishPlugin configuration requires values to be provided in the gradle.properties file:

intellij {
    publishPlugin {
        username publishUsername
        token publishToken
        channels publishChannel
    }
}

A similar approach is already covered in the mentioned Plugin Template that provides a quick-to-adapt GitHub Actions integration:

publishPlugin {
    token(System.getenv("PUBLISH_TOKEN"))
    channels(pluginVersion.split('-').getOrElse(1) { "default" }.split('.').first())
}

First of all, providing a token, it is not necessary to define a username anymore. The token is set using environment variable – thanks to this mechanism, we can store our secret value in GitHub Secrets storage and provide it within the GitHub Actions workflow step using:

# Publish the plugin to the Marketplace
- name: Publish Plugin
env:
  PUBLISH_TOKEN: ${{ secrets.PUBLISH_TOKEN }}
run: ./gradlew publishPlugin

The value of Channels is now set automatically based on the SemVer-like version value, that is, v1.0.0-beta will publish our plugin within the beta channel.

Both of the GitHub Actions workflow files are documented and prepared ready for use in your project:

• .github/workflows/build.yml
• .github/workflows/release.yml

Tests

As the .ignore project already contains some test classes, we should start by migrating them to ensure that further modifications performed in the actual plugin sources will not cause any side effects. Of course, it still depends on the complexity of our tests, and we can’t be 100% sure that the automated conversion will go smoothly, but it’s at least something we can start with.

The first step is to create a kotlin directory with all subdirectories reflecting the current structure in the /src/tests/ next to the java to move packages quickly.

Migrating Java source code to Kotlin is a straightforward operation when you use IntelliJ IDEA. We simply call up the built-in Convert Java File to Kotlin File action (⌥⇧⌘K / Ctrl+Alt+Shift+K).

During the test verification, most tests seemed to fail. Further investigation reveals that because of the native support for .gitignore and .hgignore, which was introduced in IntelliJ IDEA 2019.2, the tests still invoked inspections and actions using GitFileType and GitLanguage references.

By swapping GitFileType and GitLanguage classes with generic ones – IgnoreFileType and IgnoreLanguage – we could fix the issue. As a result, all features provided by plugins can be tested again without conflicting file type support.

Fixing deprecated APIs

The IDE actually notifies us about usages of any deprecated APIs, but we’ll use the Plugin Verifier integrated within the Gradle IntelliJ Plugin in order to fully understand the current compatibility status.

The IntelliJ Plugin Verifier tool, which is also used by Marketplace to verify the plugins against specified IDE versions, provides a full list of the potential issues found in the current implementation, like:

• compatibility problems
• experimental API usages
• internal API usages, and so on

The .ignore plugin after the refreshment process will match the latest stable IDE version (2020.3 at the time we’re writing this). The verification process should include the current target IDE and others that may introduce changes in the IntelliJ SDK API. You can obtain the available build versions here. Our task configuration will look like this:

intellij {
    runPluginVerifier {
        ideVersions("2020.3, …")
    }
}

After running the runPluginVerifier task, we can collect the following result:

IC-203.5784.10 against mobi.hsz.idea.gitignore:4.0.0: Compatible. 25 usages of deprecated API. 1 usage of internal API
Plugin mobi.hsz.idea.gitignore:4.0.0 against IC-203.5784.10: Compatible. 25 usages of deprecated API. 1 usage of internal API
Deprecated API usages (25): 
    #Deprecated method com.intellij.openapi.vfs.VirtualFileManager.addVirtualFileListener(VirtualFileListener) invocation
        Deprecated method com.intellij.openapi.vfs.VirtualFileManager.addVirtualFileListener(com.intellij.openapi.vfs.VirtualFileListener arg0) : void is invoked in mobi.hsz.idea.gitignore.IgnoreManager.enable() : void
    #Deprecated class com.intellij.openapi.fileTypes.FileTypeFactory reference
        Deprecated class com.intellij.openapi.fileTypes.FileTypeFactory is referenced in mobi.hsz.idea.gitignore.file.IgnoreFileTypeFactory
        Deprecated class com.intellij.openapi.fileTypes.FileTypeFactory is referenced in mobi.hsz.idea.gitignore.file.IgnoreFileTypeFactory.<init>()
    #Deprecated method com.intellij.openapi.components.BaseComponent.initComponent() is overridden
        Deprecated method com.intellij.openapi.components.BaseComponent.initComponent() : void is overridden in class mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler
        Deprecated method com.intellij.openapi.components.BaseComponent.initComponent() : void is overridden in class mobi.hsz.idea.gitignore.outer.OuterIgnoreLoaderComponent
    #Deprecated constructor com.intellij.codeInsight.daemon.LineMarkerInfo.<init>(T, TextRange, Icon, Function, GutterIconNavigationHandler, GutterIconRenderer.Alignment) invocation
        Deprecated constructor com.intellij.codeInsight.daemon.LineMarkerInfo.<init>(T element, com.intellij.openapi.util.TextRange range, javax.swing.Icon icon, com.intellij.util.Function tooltipProvider, com.intellij.codeInsight.daemon.GutterIconNavigationHandler navHandler, com.intellij.openapi.editor.markup.GutterIconRenderer.Alignment alignment) is invoked in mobi.hsz.idea.gitignore.daemon.IgnoreDirectoryMarkerProvider.getLineMarkerInfo(PsiElement) : LineMarkerInfo
    #Deprecated interface com.intellij.openapi.components.ProjectComponent reference
        Deprecated interface com.intellij.openapi.components.ProjectComponent is referenced in mobi.hsz.idea.gitignore.IgnoreManager
        Deprecated interface com.intellij.openapi.components.ProjectComponent is referenced in mobi.hsz.idea.gitignore.FilesIndexCacheProjectComponent
        Deprecated interface com.intellij.openapi.components.ProjectComponent is referenced in mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler
        Deprecated interface com.intellij.openapi.components.ProjectComponent is referenced in mobi.hsz.idea.gitignore.outer.OuterIgnoreLoaderComponent
    #Deprecated method com.intellij.openapi.util.IconLoader.getIcon(String) invocation
        Deprecated method com.intellij.openapi.util.IconLoader.getIcon(java.lang.String path) : javax.swing.Icon is invoked in mobi.hsz.idea.gitignore.util.Icons.<clinit>() : void
    #Deprecated method com.intellij.openapi.components.BaseComponent.disposeComponent() is overridden
        Deprecated method com.intellij.openapi.components.BaseComponent.disposeComponent() : void is overridden in class mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler
        Deprecated method com.intellij.openapi.components.BaseComponent.disposeComponent() : void is overridden in class mobi.hsz.idea.gitignore.IgnoreManager
        Deprecated method com.intellij.openapi.components.BaseComponent.disposeComponent() : void is overridden in class mobi.hsz.idea.gitignore.outer.OuterIgnoreLoaderComponent
    #Deprecated method com.intellij.openapi.vfs.VirtualFileManager.removeVirtualFileListener(VirtualFileListener) invocation
        Deprecated method com.intellij.openapi.vfs.VirtualFileManager.removeVirtualFileListener(com.intellij.openapi.vfs.VirtualFileListener arg0) : void is invoked in mobi.hsz.idea.gitignore.IgnoreManager.disable() : void
    #Deprecated method com.intellij.util.indexing.FileBasedIndex.registerIndexableSet(IndexableFileSet, Project) invocation
        Deprecated method com.intellij.util.indexing.FileBasedIndex.registerIndexableSet(com.intellij.util.indexing.IndexableFileSet arg0, com.intellij.openapi.project.Project arg1) : void is invoked in mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler$1.run() : void
    #Deprecated method com.intellij.util.containers.ContainerUtil.newConcurrentMap() invocation
        Deprecated method com.intellij.util.containers.ContainerUtil.newConcurrentMap() : java.util.concurrent.ConcurrentMap is invoked in mobi.hsz.idea.gitignore.IgnoreManager.<clinit>() : void
        Deprecated method com.intellij.util.containers.ContainerUtil.newConcurrentMap() : java.util.concurrent.ConcurrentMap is invoked in mobi.hsz.idea.gitignore.IgnoreManager.RefreshTrackedIgnoredRunnable.run(boolean) : void
    #Deprecated class com.intellij.openapi.fileTypes.StdFileTypes reference
        Deprecated class com.intellij.openapi.fileTypes.StdFileTypes is referenced in mobi.hsz.idea.gitignore.ui.IgnoreSettingsPanel.TemplatesListPanel.customizeDecorator$1.actionPerformed.descriptor$1.isFileSelectable(VirtualFile) : boolean
    #Deprecated constructor com.intellij.notification.NotificationGroup.<init>(String, NotificationDisplayType, boolean, String, Icon, int, DefaultConstructorMarker) invocation
        Deprecated constructor com.intellij.notification.NotificationGroup.<init>(java.lang.String arg0, com.intellij.notification.NotificationDisplayType arg1, boolean arg2, java.lang.String arg3, javax.swing.Icon arg4, int arg5, kotlin.jvm.internal.DefaultConstructorMarker arg6) is invoked in mobi.hsz.idea.gitignore.util.Notify.<clinit>() : void
    #Deprecated method com.intellij.util.indexing.IndexableFileSet.iterateIndexableFilesIn(VirtualFile, ContentIterator) is overridden
        Deprecated method com.intellij.util.indexing.IndexableFileSet.iterateIndexableFilesIn(com.intellij.openapi.vfs.VirtualFile file, com.intellij.openapi.roots.ContentIterator iterator) : void is overridden in class mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler
    #Deprecated method com.intellij.util.indexing.FileBasedIndex.removeIndexableSet(IndexableFileSet) invocation
        Deprecated method com.intellij.util.indexing.FileBasedIndex.removeIndexableSet(com.intellij.util.indexing.IndexableFileSet arg0) : void is invoked in mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler.projectListener$1.projectClosing(Project) : void
    #Deprecated method com.intellij.ide.plugins.PluginManager.getPlugin(PluginId) invocation
        Deprecated method com.intellij.ide.plugins.PluginManager.getPlugin(com.intellij.openapi.extensions.PluginId id) : com.intellij.ide.plugins.IdeaPluginDescriptor is invoked in mobi.hsz.idea.gitignore.util.Utils.isPluginEnabled(String) : boolean
    #Deprecated method com.intellij.openapi.startup.StartupManager.registerPreStartupActivity(Runnable) invocation
        Deprecated method com.intellij.openapi.startup.StartupManager.registerPreStartupActivity(java.lang.Runnable runnable) : void is invoked in mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler.<init>(Project, ProjectManager, FileBasedIndex)
    #Deprecated field com.intellij.openapi.fileTypes.StdFileTypes.XML access
        Deprecated field com.intellij.openapi.fileTypes.StdFileTypes.XML : com.intellij.openapi.fileTypes.LanguageFileType is accessed in mobi.hsz.idea.gitignore.ui.IgnoreSettingsPanel.TemplatesListPanel.customizeDecorator$1.actionPerformed.descriptor$1.isFileSelectable(VirtualFile) : boolean
Internal API usages (1): 
    #Internal class com.intellij.ide.plugins.IdeaPluginDescriptorImpl reference
        Internal class com.intellij.ide.plugins.IdeaPluginDescriptorImpl is referenced in mobi.hsz.idea.gitignore.util.Utils.isPluginEnabled(String) : boolean. This class is marked with @org.jetbrains.annotations.ApiStatus.Internal annotation and indicates that the class is not supposed to be used in client code.
    Plugin cannot be loaded/unloaded without IDE restart: Plugin cannot be loaded/unloaded without IDE restart because it declares project components: `mobi.hsz.idea.gitignore.FilesIndexCacheProjectComponent`, `mobi.hsz.idea.gitignore.IgnoreFileBasedIndexProjectHandler`, `mobi.hsz.idea.gitignore.IgnoreManager`, `mobi.hsz.idea.gitignore.outer.OuterIgnoreLoaderComponent`

Deprecated interface ProjectComponent reference

Plugins that use Plugin Components do not support dynamic loading. While some plugins may require an IDE restart, this is not something we wanted for .ignore. Instead, users of this plugin should be able to install/update/enable/disable without needing to restart their IDE – a smooth process every user would surely enjoy.

ProjectComponent interface usages were reported for five components in .ignore:

1. IgnoreFileBasedIndexProjectHandler – where the component was refactored to a ProjectManagerListener.
   c24dec2
2. OuterIgnoreLoaderComponent – which was simplified from using MessageBus to using projectListeners.
   cc4dff9
3. FilesIndexCacheProjectComponent – registered BulkFileListener for the VirtualFileManager.VFS_CHANGES on the application level, which is an expensive operation that caches too much data. Since the plugin doesn’t support Git and Mercurial, we moved the getFilesForPattern method to use utils with no caching enabled (but also with less overhead for this scenario).
   18c8a83
4. IgnoreManager – a component with exposed getInstance method and matcher instance shared across the whole application could be easily transformed into the projectService class together with a part of matcher functionality extracted as IgnoreMatcher.
   5471f6f

Deprecated class com.intellij.openapi.fileTypes.FileTypeFactory reference

The .ignore plugin registers multiple file types using the fileTypeFactory extension point through the looping of the IgnoreBundle.LANGUAGES supported languages collection. That method was convenient, but it was deprecated in 2019.2. See the Registering a File Type article for more details.

A fix for that was fairly easy and repetitive: <fileType /> entries were added for each of the supported languages explicitly, with all the information such as language name and associated file extension provided.
d9ebe89

Deprecated method invocation: VirtualFileManager.addVirtualFileListener(VirtualFileListener)

The IgnoreManager.enable() method adds a new listener to watch for changes in the files tree and clears the cache of calculated entries.
VirtualFileManager.addVirtualFileListener(VirtualFileListener) and VirtualFileManager.removeVirtualFileListener(VirtualFileListener) were deprecated in 2019.3, so a MessageBus with the VFS_CHANGES topic should be used instead.
ef4e22d

Deprecated constructor com.intellij.codeInsight.daemon.LineMarkerInfo invocation

IgnoreDirectoryMarkerProvider provides LineMarkerInfo to mark lines with entries recognized as directories with a folder icon. It also requires passing accessibleNameProvider, a callback that provides a localized accessible name for the icon (for use by a screen reader).
46ad2dd

Deprecated method com.intellij.openapi.util.IconLoader.getIcon(String) invocation

A quick-fix that requires passing the current Icons class as a second argument.
50c3fe3

Deprecated method com.intellij.ide.plugins.PluginManager.getPlugin(PluginId) invocation

PluginManager.getPlugin invokes PluginManagerCore.getPlugin – we just have to call the core class directly.
b60999e

Dynamic Plugin

SeveritiesProvider

The .ignore plugin implements custom localInspections – one of them inspects and highlights unused entries. If the provided rule is not applied to any file (i.e., ignore non-existing file), it is highlighted with a custom severity level, UNUSED ENTRY. Such level is provided explicitly by the <severityProvider> extension point, which is not marked with dynamic=”true” in its definition. Unfortunately, it would prevent us from making the plugin dynamic, and the only option is removing it and setting the localInspection level as WEAK WARNING.
e257ed13

Cancel Runnable

IgnoreManager service initiates two objects:

• debouncedStatusesChanged: Debounced
• refreshTrackedIgnoredFeature: InterruptibleScheduledFuture

There’s a chance that Runnable or ScheduledFuture may not be finished before unloading the plugin. Making both Disposable and registering their instances with Disposer.register(Disposable parent, Disposable child) will let us properly cancel ongoing tasks.
1b0bbc76

Conclusion

The .ignore plugin took a lot of time to refactor, and describing the whole process would span much more than one blog post. You’re welcome to review this pull request for more details. To summarize, here are all of the actions we performed:

• Migrating the code to Kotlin and removing the verbose license headers from the source files led to a 60% decrease in code volume, from 644 KB to 262 KB.
• All of the compatibility issues were adequately addressed and resolved.
• Introducing detekt as a static code analysis tool allowed us to keep the code in good shape, which is very important for future project contributions.
• Setting up CI based on the IntelliJ Platform Plugin Template enhanced the testing and deployment process.

And that concludes our first episode of the Revamping Plugins series, and since it’s marked with “#1”, you can expect further articles covering our further endeavors. It is hard to say what we might pick for the next episode, but we’ll try to find something rusty and creaky to provide you with as applicable information as possible.

Stay tuned and follow us on Twitter!

Jakub Chrzanowski and JetBrains Platform

Subscribe to JetBrains Platform updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!