How to Write Custom Language Support Plugins

Andrey Cheptsov

Today we would like to share with you a simple tutorial how to write a plugin with custom language support for IntelliJ IDEA and IntelliJ Platform.

As you know IntelliJ IDEA provide powerful facilities for developers to implement advanced code assistance for custom languages and frameworks. In this step-by-step tutorials you will learn how to use Grammar-Kit to generate a parser and PSI elements, how to use JFlex to generate a lexer, how to implement custom formatting, refactoring support, etc.

The sample code for this tutorial can be found on GitHub.

More steps with other aspects of plugin development are coming soon. In the meanwhile don’t miss the opportunity to register for the second live coding webinar about plugin development, which will take place on Tuesday, January 22.

Develop with Pleasure!

Comments below can no longer be edited.

37 Responses to How to Write Custom Language Support Plugins

  1. Ladislav Thon says:

    January 16, 2013

    This is great! If only this was available a month ago, when I started playing with this. By now, I’ve figured out almost everything of this by myself, using the Erlang plugin and the Developing Custom Language Plugins for IntelliJ IDEA page ( I haven’t needed the reference contributor, though. What is it needed for?

    Also, what I’m playing with now involves navigating across files. Stub indexes etc. Cool stuff, although I’m not sure I fully understand it. The Indexing and PSI Stubs in IntelliJ IDEA page ( only provides a very basic introduction.

  2. Ladislav Thon says:

    January 16, 2013

    Oh, and reference resolution. That’s pretty important topic, I’d suggest covering the scope processors stuff as well. It isn’t needed for such extremely simple language, but it’s absolutely essential once you start playing with something real. Thank you!

  3. Konrad says:

    January 16, 2013

    Please share tutorial how to write a complete plugin for PyCharm;) or API Reference

  4. Alberto says:

    January 16, 2013

    Will such a custom language plugin only work in IntelliJ IDEA or also with the other IDE’s, like PHPStorm or PyCharm?

  5. Luke says:

    January 16, 2013

    Could you please also explain, on an example, how the pin and recoverUntil work, as well as how to simplify the AST trees by using the extends?

  6. Cemo Koç says:

    January 16, 2013


  7. Tobias Lidskog says:

    January 16, 2013

    This is great! I noted that the “Lexer and Parser Definition” page is missing a step to create the SimpleFile class that is used in the parser definition. Looking forward to writing my first plugin now.

  8. Joe says:

    January 17, 2013

    @Alberto: It usually works in all IDEs, depends on how you define the metadata.

  9. Yann Cebron says:

    January 17, 2013

    @Alberto: please see for more information about making your plugin compatible with other IntelliJ based products

  10. Andrey Cheptsov says:

    January 17, 2013

    @Tobias You are right, thanks. Fixed.

  11. Ladislav Thon says:

    January 17, 2013

    +∞ to Luke. The GrammarKit documentation altogether would need some love, but the only important thing I really didn’t understand is pin/recoverUntil. Or actually, I think that I understand them (when a pinned element of the rule is successfully parsed, the rule itself will be considered successfully parsed, and the parser will eat all the following tokens until it reaches some tokens from the recoverUntil set), but I have a really hard time using them correctly. Basically, since I have a C-like syntax, I just settled with a single pinned element in a grammar and a single recoverUntil on ‘;’ or ‘}’. Everytime I try to add some more, the parser stops working and I have no idea why 🙂

  12. yole says:

    January 17, 2013

    If you have any specific questions regarding the stubs/indexes, please ask, and we’ll update the document to make sure they’re covered.

  13. yole says:

    January 17, 2013

    Check out lib/src/ in the PyCharm distribution and

  14. thomas says:

    January 18, 2013

    Great product! This time its mine 

  15. Sergey says:

    January 18, 2013

    @Ladislav, @Luke Please check out and the 5 lines below. This is a perfectly working example of recoverUntil concept.

  16. Ladislav Thon says:

    January 20, 2013

    @Sergey Thanks, but I was able to get to something very similar. I have a working rule with pin/recoverUntil in my grammar, just by following the documentation. That, however, isn’t enough to really understand. Once I get back to it, I will probably end up reverse engineering the concept from differences in generated code.

    Oh, and one thing I found just before few minutes. GrammarKit doesn’t support generating classes needed for stub indexes. It looks like it is possible to write all that manually, but it requires working around some GrammarKit bugs: 1. GrammarKit doesn’t like generics (extra interface for each StubBasedPsiElement needed), 2. GrammarKit doesn’t generate proper imports in generated visitor (and for this, there is a pull request with fix for more than 2 months!). And that’s only a beginning for me.

  17. Gregory Shrago says:

    February 8, 2013 may shed more light on grammar-kit error recovery

  18. Sven says:

    February 14, 2013

    The FindUsagesProvider-section uses an static word-scanner-instance.
    public WordsScanner getWordsScanner() {
    return WORDS_SCANNER;
    As far as I know this will cause threading issues as the lexer is not thread safe.

  19. Andrey Gayvoronsky says:

    February 21, 2013

    Is any way to extend existing ‘language plugin’ with minimal efforts?

    For example, i need plugin for Squirrel language:

    Its like slightly simplifier and abit extended version of Java.

  20. Prasan says:

    March 14, 2013

    Thanks for the tutorial, but IntelliJ doesn’t seem to associate the file type specified with my language after following step 2: Language and file type.

  21. Scott Wells says:

    March 19, 2013

    Totally agreed that this is an outstanding tutorial! However, like Luke and Ladislav, I still don’t quite understand how best to use pin and recoverUntil, even after looking at the Erlang grammar. The language I’m implementing is very Java-like, and I’ve made some progress with pins so that I now get “foo expected, found bar” most of the time, but I have a feeling I’m abusing it rather than using it. Also, every time I try to use recoverUntil, primarily by using an expression such as “!(SEMICOLON | RBRACE)”, I just end up tanking the parser.

    I’d be happy to share a few simple rules from my grammar as concrete examples of those help the discussion.

  22. Noah says:

    December 15, 2013

    I’m sorry but while this “tutorial” may be a good start to see what operations must be done in which order, it’s not even close to being a tutorial. It’s a recipe, there is almost no explanation for most of the steps and the reader is just led blindly through it.

    And the example is not practical either. I understand it has to be simple, but like so many examples it just lacks the fundamental links people need in order to build an actual case.

    What for example if I want to build a real parser, it will have to let the grammar get some information from what the lexer is sending (number value, identifier string, …). The JFlex documentation shows how to define a type that contains this information but here it’s all really fuzzy, I gather that somehow it has to be based on IElementType but then what? If I scan a string, where do I put the contents? If it’s an integer, where should the value go?

    This could be a basic grammar, no need for complex stuff, but at least it would show the reader how to do practical things.

    Is there any good example of a language plugin out there which is based on a JFlex lexer and a .bnf grammar definitions? The only ones I found did their own recipe for the grammar part, either with Scala routines or home-made code which seems to amount to ten times the effort (or even trying to squeeze part of the grammar in the lexer, which makes it even worse).

    Thanks for all the work done so far anyway 🙂

    • vishnu says:

      February 8, 2014

      @Noah I’ve just started with plugin development for a custom language and even I have the same feeling did you get to find any resources or tutorials which could have helped you? Thanks in advance

  23. Noah says:

    December 22, 2013

    And actually it doesn’t work.

    If you take the project from Git, it compiles the first time. Try and generate the parser from the BNF file and it all goes haywire.

    Generated parser:
    import static com.simpleplugin.parser.GeneratedParserUtilBase.*;
    builder_ = adapt_builder_(root_, builder_, this, null);

    In the file, you realize that the static constructor has a different signature:

    public static PsiBuilder adapt_builder_(IElementType root, PsiBuilder builder, PsiParser parser)

    And by the way, the file you provide as a separate link is completely different and also incompatible. It already starts with this:

    package org.intellij.grammar.parser;

    which is actually a different package! What a mess. Why isn’t that file part of the SDK in the first place, as should the GrammarKit, it’s the best way to ensure consistency between those parts.

    • Noah says:

      December 22, 2013

      Trying to change the package to match the com.simpleplugin.parser is not enough. Now the PsiBuilder type is missing the .rawTokenIndex() method, PsiBuilderImpl.ProductionMarker is missing the .getStartIndex() method.

      Trying to change the SDK from 129.1359 to 133.286, regenerating parser. Nope, now PsiBuilder.rawTokenIndex() is there, but PsiBuilderImpl.ProductionMarker is still missing getStartIndex(), plus a new getEndIndex() method…

      I’m giving up for now, the lack of documentation or at least of a relevant example, the inconsistency between those parts, make it too much trouble to use. Back to Eclipse and Xtext I guess, a shame.

      • Noah says:

        December 22, 2013

        Found the bug, apparently this line has to be changed in the BNF, since this file IS indeed part of the SDK and doesn’t need to be included from elsewhere:


        It only works with SDK 133.286 though, with the other ones it produces more errors that I will care to report.

        So it finally compiles. Then all the unit tests but one fail in SimpleCodeInsight. The SimpleParsingTest simply crashes…

        • Vektah says:

          April 12, 2014

          For anyone encountering this issue I can run with the current master by NOT using the supplied ‘light-psi-all.jar’ and instead using my current install of intellij.

          java -cp ‘/path/to/grammar-kit.jar:/path/to/intellij/lib/*’ org.intellij.grammar.Main gen src/bnf/MyGrammar.bnf

  24. Shishir says:

    August 26, 2014

    I want to navigate from variable to its declaration etc, but i dint find any anything related to navigation in the tutorial. can somebody please help me out for the same.

  25. Adam says:

    July 26, 2015

    Got all of this compiling and working in debug mode, but when you deploy the plugin and try it for real it loads with no errors, but none of the functionality is there.

    Any ideas why ?



  26. Paul Preibisch says:

    March 23, 2016

    the github link is 404

  27. Paweł Bogdan says:

    January 18, 2017

    I am trying to build similar plugin, is there any possiblity to create such a plugin without BNF file? We cannot find the proper file in the Internet and the language is much more complicated than than example one.

  28. Dnltsk says:

    March 30, 2017

    I figured out that the DSL definition (syntax) and Editor styles are much easier configure in MPS as to code it. But unfortunately its not possible to export it as a plain Custom Language Support Plugin. Thats really a pitty.
    Or do I understand something wrong here/their?

  29. David says:

    June 10, 2018

    very old post but still I have to ask.
    Is there any real tutorial with explanations for writing a language plugin?
    Because based on what JetBrains offers I can get a working example , but with literally 0 understanding on whats going on.


Subscribe for updates