Fleet Below Deck, Part VI – UI With Noria
This is a multipart series on building Fleet, a next-generation IDE by JetBrains.
- Part I – Architecture Overview
- Part II – Breaking Down the Editor
- Part III – State Management
- Part IV – Distributed Transactions
- Part V – The Story of Code Completion
- Part VI – UI With Noria
In Part V of this series, we discussed code completion, which is just one of Fleet’s services. Now it’s time to talk about something as big as our very own declarative UI framework for JVM, Noria. We built Fleet with Noria. Let’s look at the ideas behind Noria, the main concepts, and other exciting features.
Where it all starts: A Noria window
How is a UI built? First, we have a display with graphical capabilities that is able to present an application’s state and is also responsible for making our UI a GUI. We may also have one or more input devices, such as a keyboard, a mouse, or a touchpad, to deliver commands and control the application’s behavior. In these settings, an application is effectively an event loop responsible for reacting to events initiated by users and other computer system components (timers, a file system, a network, and others). Primarily, reactions are visible changes in what is displayed on the screen.
In addition to being an event loop, a GUI application usually has some kind of window, an area of the screen it is responsible for, and some drawing capabilities in that area. A window is normally provided by an underlying operating system or a window manager on top of it. An operating system may provide a graphics API or delegate drawing to a graphical framework.
Fleet is both a GUI application and a JVM application. It runs on all major operating systems, including Windows, macOS, and Linux. Fleet relies on the Java AWT/Swing framework to get a window from an operating system, but it doesn’t use the Java platform for managing its GUI components besides one JFrame and JPanel on top of it. Fleet doesn’t use the JVM’s screen drawing capabilities, either. Instead, it employs Skia, a native 2D graphics library that is available to JVM applications via the skiko-awt binding library, which JetBrains develops.
The following diagram presents a combination of the aforementioned frameworks and libraries, resulting in a screen area fully managed by our home-grown UI framework – a Noria window:
Everything you see in a Fleet window is a Noria component. Panels, tabs, buttons, tooltips, text editors, terminals, diff views, and docker views are managed by Noria and are constantly changing as a result of Noria’s event loop reactions.
You might ask, “Why Noria?” Why invent a new UI framework when we have so many brilliant UI frameworks out there begging to be applied for creating highly responsive GUI applications with a modern look and feel for all major desktop platforms at once? Well, that’s not entirely the case. Let’s refer to the history of those UI frameworks and see whether we had such options to choose from when we started developing the product that eventually became Fleet.
A brief history of UI approaches
Ideas and approaches to GUI frameworks (not GUI itself) can be traced back to the 70s, when a team of researchers at Xerox PARC, including Alan Kay, developed a graphical environment for the Smalltalk programming language. From the early days of UI frameworks, their developers had a clear focus on architectural matters. It wasn’t a simple task to organize an application over an event loop with many UI components on the screen, as well as factoring in unpredictable user actions and their corresponding UI reactions, and not end up having an application that would be a complete mess from an architectural perspective. One example of finding such an approach is detailed in some notes by Trygve M. H. Reenskaug, where he reflects on the inception of MVC, a notable Model-View-Controller architectural style, which was a set of ideas to provide a structure for GUI applications.
There was no clear way to apply an MVC style, which led to many attempts to reformulate it using other terms (for example, MVP – Model-View-Presenter). Every UI framework that claimed to support it provided some specific way to do that. Those ways didn’t have a lot in common besides several generic things, including the following:
- Relying on the object-oriented features of programming languages.
- Separating the business logic from the presentation (except that, unfortunately, presentation may require some logic, too).
- Observing changes via the Observer pattern (and experiencing related code readability issues, because it may be too hard to figure out what’s going on in the program from reading sources that heavily exploit this pattern).
There was also another line of development, an attempt to simplify an architecture by structuring it into a form, with a set of controls on that form, and with all the logic that connects the application state to components of that form. This idea, popularized by Borland Delphi and Visual Basic in the 90s, led to a generation of Button1Click developers who were happy to develop large applications without thinking about architecture. This approach was always heavily criticized by object-orientation-inclined researchers and practitioners, but, frankly speaking, sometimes it was too difficult to distinguish between the complete messes produced by either no-architecture or strict architectural guidelines.
Martin Fowler explored these 2 lines of development for UI frameworks in this brilliant excerpt from his (unfortunately, unfinished) book Further Patterns of Enterprise Application Architecture.
And then we’ve got PHP. Developing web applications in PHP following its inception in the middle of the 90s became an exceptionally pleasing experience, extremely easy to fire up and get straight to production. PHP popularized template processing, an approach when you mix logic (instructions of a programming language) and presentation (HTML tags) in one piece of code – a sordid crime according to MVC purists. No other programming language has ever received so much backlash from developers. Just google “PHP hate” to get an idea. The “Guards of Clean Code Heaven” still hate PHP for this (and maybe also for opening the gates to the industry to many new developers who were raised unaware of the only “right way” to develop software).
Well, let’s not judge programming languages but try to see exactly what this new idea was and where it would lead us. How we see data is intimately linked with how we manipulate it. With PHP, we no longer had to separate logic and presentation. After all, most of the logic is in the presentation anyway because it’s the users who manipulate the presentation, not some abstract data that can’t be seen or touched. PHP made it clear that we needed components – active entities with their own look, reactions to user actions, and other behavior. Components may be composed of other components, and they can effectively interact with neighboring components through some shared state. For example, a list of items in one component can deliver change signals to another component whose content depends on the selected items in the first component.
Ajax (Asynchronous JavaScript and XML) took this idea even further, with web components as the main building blocks of web applications. These web components combined backend and frontend logic while mixing them with the presentation. Good or bad, this idea brought us the current web and web development experience.
In the early 2010s, React introduced a brand new world of UI development. Frontend developers may argue about which is better, React or Angular, and which state manager should be used, but now they are used to the idea of active UI components. They even apply their web technologies to developing desktop applications (though the resulting experience doesn’t always provide perfect responsiveness, look, and feel).
One thing we should praise React for is that it made declarative programming and reactive programming the mainstream. These previously hidden techniques were rarely used in production programming languages, but with React they began to dominate in many areas of software development, including, most notably, UI frameworks.
By the end of the 2010s, we started to observe the same approaches to UI pervading UI frameworks for mobile systems, Apple’s SwiftUI and Google’s Jetpack Compose being the most prominent examples. We didn’t have to wait long to see the same trends in UI frameworks for desktop systems. In the early 2020s, we have the beautiful Compose for Desktop, based on Google’s Jetpack Compose and brought to you by JetBrains. When we started working on Fleet, however, Jetpack Compose didn’t exist, and thus Noria was born.
Incremental computations
Interestingly, Noria is not a UI framework at its core. Instead, it’s a platform for incremental computations. Imagine a large multi-part mathematical formula with interdependent components that sometimes have to be recalculated or trigger the recalculation of dependent components – not unlike a spreadsheet. Noria shines at computing and recomputing these sorts of formulas. The main principle here is to avoid any unnecessary computations and to recompute only the parts required to produce the final result.
What about the UI, then? Well, it’s also such a formula. It has a tree-like structure built of components and subcomponents. Those components may also have other dependencies besides being a part of another component. This makes a directed acyclic graph (DAG) of dependencies. A change of state in one of the components also triggers a change in another component that depends on it, directly or indirectly.
In general, incremental computations support skipping some calculations if we can determine that their expected results won’t change. In terms of UI, we avoid touching (and redrawing) components that don’t require any changes.
Suppose that we are about to make a move in this simple game of Tic-Tac-Toe:
| BEFORE | AFTER |
 |  |
We have a grid with cells. Which cells should be changed when we click (add an X) in the top-right corner? The cells in the first row will need to be redrawn as it becomes the winning row. None of the other cells changes, though, so we’d better refrain from redrawing them. To implement the required logic, we structure our components and computations as follows:
- A 2D grid contains the cells and offers a way to describe the layout with rows, columns, padding, etc.
- Each cell has information about its content (whether there’s an X or an O in it) and the possibility of being a part of a winning row, column, or diagonal, and it can be redrawn when the state of these Xs and Os is changed.
- Clicking on the top-right cell should change its content (and trigger redrawing), but it should also trigger checking whether the game is over and changing the corresponding settings of the winning cells.
- Changing the settings of the winning cells should trigger the redrawing of them with a cross-through yellow line.
Thus, reacting to a user action (clicking on a cell) leads to partial recomputation of the grid and the redrawing of some of its cells while keeping others intact. Noria provides us with everything we need to handle such behavior, including:
- A declarative Kotlin DSL to describe the layout of components as a tree structure along with tree-structure dependencies that enable event propagation.
- onClick-hooks for components that implement immediate reactions.
StateCells to express the out-of-tree-structure dependencies responsible for triggering recalculations of neighboring nodes.
Under the hood, Noria does not trigger recalculations when they are not needed. That’s standard behavior for any incremental computation platform. A Noria user writes a function describing the whole UI in a declarative way. This function is called every time something happens. But Noria knows precisely what was changed from the previous run and what wasn’t, and it decides which parts of the UI tree need recalculating.
Declaring UI components
Let’s look at some code examples and get a feeling of what it’s like to work with Noria.
The Tic-Tac-Toe component mentioned in the previous section can be described in Noria as follows:
val grid = Grid(3) { state { GridCell() } }
val gameState = state { GameState() }
clickable(onClick = { grid.checkEndOfGame(gameState) }) {
vbox {
grid.gridRows.forEach {
hbox {
it.forEach {
cell(it, gameState)
}
}
}
}
}
We have a grid with cells and a game state responsible for the next turn and end-of-game status. Note the 2 state functions in the first 2 lines. They are responsible for creating additional dependencies: Updating their content triggers the recalculation of components that read them.
Every cell is responsible for rendering its content and reacting to user clicks:
private fun UIContext.cell(gridCell: StateCell<GridCell>,
gameState: StateCell<GameState>, ...) {
val gs = gameState.read()
val cell = gridCell.read()
clickable(onClick = { ... }, propagate = Propagate.CONTINUE) {
decorate(backgroundColor = ...) {
layout {
render(Rect(Point.ZERO, Size(size, size))) {
...
}
}
}
}
}
Technically, cell is an extension function for the UIContext class, which implements the machinery behind Noria. The cell reads some state, thus expressing its dependence on that information. Noria uses this dependency while deciding whether it’s necessary to re-render this cell.
Noria components can be as simple as the following text label with a tooltip:
withTooltip("Tooltip text") {
uiText("Point on me")
}
They can also be quite sophisticated, like a text editor or other panes and windows that you regularly see in your Fleet instance.
Noria provides all the essential components for implementing desktop UIs and other obvious UI framework capabilities, including:
- Laying the components out.
- Constraining and setting boundaries.
- Rendering visible parts and providing support for scrolling.
- Defining focus traversals.
- Implementing overlays (such as a tooltip in the example above or error messages next to your erroneous code fragments).
Every time you run Fleet, Noria machinery is working hard to deliver the best UI experience possible.
Summary
Fleet is implemented with Noria, a home-grown UI framework for the JVM. Noria allows us to describe UIs in a modern declarative way using Kotlin features. It is based on the incremental computations core, responsible for expressing dependencies between UI components and minimizing needless re-renderings.
Noria is by no means revolutionary. It only takes modern design ideas, applies them to UI frameworks, and adds some Kotlin flavor to them. But what we’ve learned so far is that developing your own fundamental UI frameworks works and can actually be a lot of fun!
The Fleet Below Deck series is far from over. We still have a lot of details about Fleet internals to share. Stay tuned for more!
Subscribe to Fleet Blog updates
