New Products News

Nitra goes Open Source!

We’ve been rather quiet about Nitra since our introductory blog post, back in November of last year. But we haven’t been sitting still. We’ve been busy building the foundations to a project that we want to see as the basis for a lot of future work here at JetBrains. Now that we feel we’ve reached a minimum viable feature set, as well as a good architectural base for future development, we’re very happy to take the next steps on our roadmap, and release Nitra as an Open Source project.

The source is currently available on GitHub, released under the Simplified BSD license.

What is Nitra?

As a reminder, Nitra is a framework for the next generation of language tooling.

Traditionally, languages don’t ship with any kind of IDE support. It is up to tools vendors to provide syntax highlighting, outlining, code completion, navigation, find usages, refactorings and so on. Some of these features, such as syntax highlighting and outlining can be crudely implemented in a manner similar to regular expressions, but the more powerful features require a deeper understanding of the language.

Ideally, a language service should be able to parse a file into a form that it can reason about – an abstract syntax tree (AST). With such a deep knowledge of the underlying language, it knows what nodes in the tree are keywords for syntax highlighting, knows where blocks start and end for outlining, can pull out identifiers and their usages for code completion and navigation. Manipulating the tree can reformat or even rewrite the source code, enabling refactorings.

This is what JetBrains does right now, and, as a company, we’re very good at it – ReSharper ships with parsers and ASTs for approximately 20 file formats, and the IntelliJ ecosystem supports many more. As one product manager put it: “We’re JetBrains. We write parsers for breakfast”. But it’s busy work.

Nitra is designed to take away this busy work, allowing us to support more languages, quicker, and to focus on the higher levels in the stack of language tooling, such as analyses and quick fixes.

Nitra is a language for describing grammars, and the abstract syntax trees to represent them. It is also a build tool to compile the grammars into parsers, and generate the classes required to represent and navigate the AST. And it’s also a set of language services that use the grammar to provide automatic syntax highlighting, outlining and brace matching.

What’s more, Nitra’s grammars are designed to be extensible and reusable. A simple example would be the base building block grammars that ship with Nitra, such as Whitespaces and CStyleComments. Both clearly useful in many languages, and can be simply included and reused in a larger grammar.

A better example would be HTML, which allows for embedded “islands” of JavaScript and CSS. The HTML grammar could include the JavaScript and CSS grammars and allow full parsing of a typical HTML file. Take it further, and look at .net’s Razor web templating engine. This is a server-side HTML file that includes not only JavaScript and CSS, but also C#. Build a grammar that includes HTML, JS, CSS and C#, and you’ve got a parser and an AST that can handle Razor files.

Read Hadi’s Introduction to Nitra post to see an example grammar, and check out the wiki for documentation, especially for the syntax of the grammars.

Current Status

While we’re making Nitra Open Source today, this should not be considered an end-user preview. It’s a low-level language toolkit, and if you want to use it, expect to get your hands dirty. Here’s what it currently supports:

  1. Grammars to define lexer-less parsers and ASTs that can be extended at compile time or runtime, supporting composition of languages, or languages with extensible syntax, such as Nemerle
  2. A command line tool to compile grammars into an assembly containing the parser and the AST classes (grammars can also be compiled inside Visual Studio)
  3. A Windows application to load and test grammars
  4. A Visual Studio language service that uses Nitra to parse files and provide automatic syntax highlighting, outlining and brace matching. Current work is to increase the automatic support here to include go to declarations, find usages and rename refactorings

But this is still a work in progress. There are things that don’t yet work. The source reflects this in places, with incomplete grammars (for example, C++) and some references to internal test tools and file locations that aren’t included in the source code. This just reflects the status of the code as an active project, and will get cleaned up over time.

Building Nitra

Before you can use Nitra, you need to build it. Full instructions for building Nitra can be found here. Here’s an outline:

  1. Install Nemerle. Nitra is built by the Nemerle team, in Nemerle, so requires Nemerle in order to compile. You can download the latest version from the Nemerle site (there currently isn’t a VS2013 build that supports Nitra. One will be made available shortly. In the meantime, you need to use VS2012)
  2. Once installed, you need to open a command prompt in the Nitra folder, and run “BuildBoot.cmd”. This bootstraps the build process, building the boot versions of the assemblies required by the main product in order to build. you only need to do this once.
  3. Build the Nitra.sln solution in Visual Studio
  4. Set the environment “NitraPath” to point to the root of the source distribution. The Visual Studio integration uses this to locate the “NitraGlobalConfig.xml” file which tells it which grammars to load

Using Nitra

The solution contains several test applications you can use to give Nitra a go.

Firstly, there is Nitra.Visualizer, which is a Windows application that is pre-configured to load test files from the Nitra\Tests\Visualizer folder. It will use the grammars defined in the Nitra solution to load test files and compare the output with a known-good “gold” file. If the output is different, the test fails, and you can see the differences in the “Diff” tab. If the test succeeds, the file is displayed in the right hand pane, with syntax highlighting and outlining applied. You can also navigate the AST under the “Reflection” tab, and see how Nitra will output it in a pretty-printed version.

Nitra.Visualiser showing a C# file

There are several different tests and languages supported here, including C# and JSON, and a calculator DSL, too.

In fact, there’s also another sample app that uses the calculator DSL directly – Sample.Calc.App. This is a console application written in Nemerle that references the calculator grammar and parses input from the command line. It then pretty prints the expression you enter, outputs it with syntax highlighting, and evaluates it, using a Value() method that is defined as part of the grammar.

Calculator sample app

And of course, there’s the Visual Studio integration. Once this project builds, you’ll get a Nitra.VisualStudio.vsix file in the bin\Debug folder. Double clicking on this will install it into Visual Studio (you can also set up the debugging command line so you can just F5 the project). Don’t forget to set the “NitraPath” environment variable as mentioned above. The extension adds support for *.json files, and will parse *.csn files as C#, but most usefully, it will parse, syntax highlight and add outlining and brace matching for *.nitra grammar files.


There are other sample projects in the solution, and you can also run the test suite with the “run.cmd” batch file in the Nitra\Tests folder.

Why Open Source?

The plan has always been to release Nitra as Open Source. After all, it’s written in Nemerle, by the Nemerle team, and Nemerle is Open Source. However, we wanted to wait until we had a core platform that was ready to be opened up further. We feel that we have that now, so here we are.

And of course, we’re accepting contributions! If you’d like to get involved and help build the next generation of language tooling, take a look, raise issues, send pull requests.

Please let us know what you think!

Comments below can no longer be edited.

27 Responses to Nitra goes Open Source!

  1. Stu says:

    May 27, 2014

    This is very nice .. any chance of adding it as a package to ? Then it can be installed as easily as apt-get works in linux.

    • Matt Ellis says:

      May 27, 2014

      There isn’t really anything to install just yet – this isn’t an end user preview, more like an early adopter, developer preview. The executable front ends we have right now are either test harnesses, samples, or early works in progress (e.g. the Visual Studio extension). They’re not yet products. As such, it doesn’t make much sense to package up.

      Instead, it’s really a source only release, intended for developers to get access to the framework, take a look, try it out and hopefully give us feedback, or even better, contribute. As the project progresses, we’ll be looking at having something a little more end-user focussed.

  2. Matt Davey says:

    May 27, 2014

    As a long time fan of Nemerle it’s great to see this project come to fruition. Congratulations to everyone involved. I’d love to see the Nemerle language itself be the first to benefit from this new tooling & be productized as a first class citizen in the JetBrains .NET ecosystem.

    • Vlad says:

      May 27, 2014

      All of this will, but only after we implement the binding and typing subsystem. We have already started work on it, but the amount of work is big.

  3. Technical Hedgehog says:

    May 27, 2014

    Nice link Matt!

  4. Ariel says:

    May 27, 2014

    What’s the difference between Nitra and MPS?
    well, i can see that one’s an IDE and the other is not etc., but when should i use one over the other?

    • Suminda Dharmasena says:

      May 28, 2014

      This currently seams to be focused on .Net. (Hopefully JetBrains will make this more generic in the future.)

      Basic usage seems to be much more succinct in Nitra than MPS. (Hopefully MPS Team will start supporting Nitra some time down the line.)

      I am an interested bystander. Somebody from JetBrains or Nitra team may be able to give you more colour on this.

    • Vlad says:

      May 28, 2014

      Nitra working with text (MPS use graphical editor of AST). Nitra allows dynamically extend a language. And Nitra based on .Net.

      • Ariel says:

        May 28, 2014

        i see all you’ve said, and it’s true, but it looks more like implementation differences than a new idea.
        i think it’s weird that Jetbrains will own two products that’s are have so much in common.

  5. Gordon says:

    May 28, 2014

    Why is the “visualizer” called Nirta instead of Nitra?

    • Suminda Dharmasena says:

      May 28, 2014

      This might be a typo?

      • Matt Ellis says:

        May 28, 2014

        Yes, it’ll just be a typo, no special reason. Except, I can’t see where it’s referred to as “Nirta”. Could you point it out so we can fix, please?

  6. Articles for 2014-May-28 | Readings for a day says:

    May 28, 2014

    […] Nitra goes open source […]

  7. Raoul Duke says:

    May 28, 2014

    Another +1 on how I’m excited to see Nemerle not being dead.

    (I just wish Mono were better, from what I have heard/read. I am a big fan of F# and Nemerle or heck even C# is better than Java in terms of the syntax, but any time I try to use or go read up on or talk to real world users of Mono I never make any happy progress.)

  8. 6/1/2014 Developer News Roundup | The Puntastic Programmer says:

    June 1, 2014

    […] Nitra goes Open Source! (probably because you’ve never heard of Nitra before this) […]

  9. JetBrains open-sources Nitra « SDTimes says:

    July 9, 2014

    […] a lot of future work here at JetBrains,” wrote JetBrains technical evangelist Matt Ellis on the company’s blog. “Now that we feel we’ve reached a minimum viable feature set, as well as a good architectural […]

  10. Herding Code 194: Hadi Hariri on Kotlin, Nitra, and Developing In A Decade says:

    September 29, 2014

    […] Blog: Nitra goes open source […]

  11. JetBrains 2014: The Year in Review | JetBrains Company Blog says:

    February 11, 2015

    […] those plans come to fruition. In February, ReShaper C++ Early Access Program went live, in May Nitra was made open source, September brought CLion EAP, and in December Upsource, our repository browsing and code review […]

  12. Renee Cammarere says:

    April 14, 2015

    I downloaded Nitra and am very anxious to start working with it. However, when I ran BuildBoot.cmd, I got a series of error messages similar to this:
    Internal\ExtensibleRuleParseTreeConstructor.n(74,36,74,39): error : typing fail
    s on accessing member Add in the ? type [C:\NitraBase\Nitra\Boot1\Nitra.Runtime\Boot.Nitra.Runtime.npro

    • Renee Cammarere says:

      April 14, 2015

      Also, I get this error when building in VS . . . The referenced assembly “C:\NitraBase\Nitra\Nitra.Runtime\bin\Debug\Nitra.Runtime.dll” was not found. If this assembly is produced by another one of your projects, please make sure to build that project before building this one. Nitra.TestsLauncher

      • Matt Ellis says:

        April 15, 2015

        Hi Renee. The best thing to do is to raise an issue with the GitHub project:

        • Renee Cammarere says:

          April 15, 2015

          Yes, thanks. I realized yesterday that you had an issue site that I should have posted this comment to.

  13. ask says:

    May 5, 2015

    May I just say what a relief to find somebody that genuinely understands what
    they’re talking about on the web. You certainly understand how
    to bring a problem to light and make it important.
    More and more people should check this out
    and understand this side of your story. I was surprised that you are not more popular given that you surely have the

  14. Often these situations come up during times of financial hardship and because we believe you should have access to experienced legal counsel regardless of the situation says:

    September 21, 2015

    Thanks for sharing such a good opinion, article is pleasant, thats
    why i have read it fully

Discover more