Fleet
More Than a Code Editor
Fleet 后台探秘，第六部分 – UI 和 Noria
在本系列博文中，我们将以多个部分为您介绍构建 Fleet 这款由 JetBrains 打造的下一代 IDE。
在本系列的第五部分中，我们讨论了 Fleet 的一项服务 – 代码补全。 现在，该谈谈我们自己的 JVM 声明式 UI 框架 Noria 了。 我们使用 Noria 构建了 Fleet。 来看看 Noria 背后的想法、主要概念和其他精彩功能。
一切开始的地方：Noria 窗口
UI 是如何构建的？ 首先，我们有一台具备图形功能的显示器，它能够呈现应用程序的状态，也负责使我们的 UI 成为 GUI。 我们可能还有一个或多个输入设备，例如键盘、鼠标或触控板，用来传递命令和控制应用程序的行为。 在这些布置中，应用程序实际上是一个事件循环，负责回应用户和其他计算机系统组件（计时器、文件系统、网络等）发起的事件。 首先，回应是屏幕上显示的可见变化。
除了事件循环之外，GUI 应用程序通常还有某种窗口、其负责的屏幕区域以及该区域的一些绘图功能。 窗口通常由底层操作系统或它上面的窗口管理器提供。 操作系统可以提供图形 API 或将绘图委托给图形框架。
Fleet 既是 GUI 应用程序，也是 JVM 应用程序。 它可以在所有主流操作系统上运行，包括 Windows、macOS 和 Linux。 Fleet 依靠 Java AWT/Swing 框架从操作系统获取窗口，但除了它上面的一个 JFrame 和 JPanel 之外，它不使用 Java 平台管理其 GUI 组件。 Fleet 也不使用 JVM 的屏幕绘图功能。 它使用的是 Skia。这是一个原生 2D 图形库，可以通过 JetBrains 开发的 skiko-awt 绑定库供 JVM 应用程序使用。
下图呈现了上述框架和库的组合，得到完全由我们的自制 UI 框架管理的屏幕区域 – Noria 窗口：
Fleet 窗口中显示的一切都是 Noria 组件。 面板、选项卡、按钮、工具提示、文本编辑器、终端、差异视图和 docker 视图由 Noria 管理，并且根据 Noria 的事件循环回应而不断变化。
您可能会问：“为什么要选择 Noria？” 我们毕竟已经有大量出色 UI 框架，可以立即用于为所有主流桌面平台创建具有现代观感的快速 GUI 应用程序，为什么还要发明新的 UI 框架？ 事实并非完全如此。 这就要从这些 UI 框架的历史讲起，您可以看看在开发初期，Fleet 产品尚未成型时，我们是否有这么多选择。
UI 方式简史
GUI 框架（不是 GUI 本身）的想法和方式可以追溯到上世纪 70 年代，当时 Xerox PARC 的一组研究者，包括 Alan Kay，为 Smalltalk 编程语言开发了一个图形化环境。 从 UI 框架的早期开始，他们的开发者就明确关注架构问题。 如果屏幕上有许多 UI 组件，同时考虑到不可预测的用户操作及其 UI 回应，还要避免应用程序架构混乱，在事件循环上组织应用程序并非易事。 Trygve M. H. Reenskaug 在一些笔记中详细描述了找到这种方式的示例。他回顾了著名的 Model-View-Controller 架构样式 MVC 的起源，这是一系列为 GUI 应用程序提供结构的思想。
由于没有明确的方式来应用 MVC 样式，这让许多人尝试使用其他术语（例如，MVP – Model-View-Presenter）重新表述它。 每个声称支持它的 UI 框架都提供了一些特定方式。 这些方式没有太多共同点，只有几个通用的地方：
- 依赖编程语言的面向对象特性。
- 将业务逻辑与表示分离（不幸的是，表示可能也需要一些逻辑）。
- 通过 Observer 模式观察变化（并遇到相关代码可读性问题，因为阅读大量利用这种模式的源代码可能很难理清程序中的情况）。
另一条开发路线试图通过将架构构建为窗体来进行简化，在窗体上有一组控件，以及将应用程序状态连接到窗体组件的所有逻辑。 这个想法在上世纪 90 年代由 Borland Delphi 和 Visual Basic 推广开来，让一代 Button1Click 开发者可以开心开发大型应用程序而无需考虑架构。 这种方式一直受到倾向于面向对象的研究人员和从业者的严厉批评，但坦率地说，无架构和严格架构准则所产生的混乱有时很难区分。
Martin Fowler 在他的（可惜未完成）Further Patterns of Enterprise Application Architecture 一书的精彩摘录中探索了 UI 框架的这两条开发路线。
然后，是 PHP。 从上世纪 90 年代中期 PHP 诞生以来，使用 PHP 开发 Web 应用程序已成为一种非常愉快的体验，非常容易启动并直接投入生产。 PHP 普及了模板处理，这是一种将逻辑（编程语言指令）和表示（HTML 标记）混合在一段代码中的方式 – MVC 纯粹派眼中的肮脏罪行。 没有任何编程语言受到过开发者如此强烈的反对。 搜一下“PHP hate”您就懂了。 “清洁代码天堂的守卫”仍然因此讨厌 PHP（也许还因为它向许多新开发者打开了行业大门，这些人从来不知道开发软件仅有的“正确方式”）。
好吧，我们不评判编程语言，而是看看这个新思想到底是什么，以及它将把我们引向何方。 我们如何看待数据与我们如何处理数据密切相关。 有了 PHP，我们不再需要将逻辑和表示分开。 毕竟，大多数逻辑都在表示中，因为操作表示的是用户，而不是某些看不见摸不着的抽象数据。 PHP 明确表明我们需要组件 – 具有自己的外观、对用户操作的回应和其他行为的活动实体。 组件可能由其他组件组成，可以通过一些共享状态与相邻组件有效交互。 例如，一个组件中的一列条目可以将更改信号传递给另一个组件，后者的内容取决于第一个组件中的选定条目。
Ajax（异步 JavaScript 和 XML）在这个思想上更进一步，将 Web 组件作为 Web 应用程序的主要构建块。 这些 Web 组件结合了后端和前端逻辑，同时将它们与表示混合。 不论好坏，这个想法给我们带来了现在的 Web 和 Web 开发体验。
2010 年代初期，React 开创了一个全新的 UI 开发世界。 前端开发者可能会争论 React 和 Angular 哪个更好，以及应该使用哪个状态管理器，但现在他们已经习惯了活动 UI 组件的思想。 他们甚至将 Web 技术应用于桌面应用程序开发（尽管最终体验并不总能提供完美的响应、外观和感觉）。
我们应该称赞 React 的一点是，它使声明式编程和反应式编程成为主流。 这些以前隐藏的技术很少用于生产编程语言，但随着 React 的出现，它们开始在软件开发的许多领域占据主导地位，包括最引人注目的 UI 框架。
2010 年代末，我们开始观察到相同的 UI 方式在移动系统 UI 框架中普及，Apple 的 SwiftUI 和 Google 的 Jetpack Compose 是最突出的例子。 我们很快就看到同样的趋势出现在桌面系统的 UI 框架中。 在 2020 年代初期，我们有精美的 Compose for Desktop，它基于 Google 的 Jetpack Compose，由 JetBrains 推出。 然而，当我们开始开发 Fleet 时，Jetpack Compose 还不存在，于是 Noria 诞生了。
增量计算
有趣的是，Noria 的核心并不是 UI 框架。 它是一个增量计算的平台。 想象一个大型多部分数学公式，其中包含相互依赖的组件，有时必须重新计算或触发依赖组件的重新计算 – 这与电子表格没什么不同。 Noria 擅长计算和重新计算这类公式。 主要原则是避免一切多余计算，只重新计算生成最终结果所需的部分。
那么 UI 呢？ 它也是这样的公式。 它具有由组件和子组件构建的树状结构。 这些组件除了是另一个组件的一部分之外，还可能具有其他依赖项。 这就形成了依赖项的有向无环图 (DAG)。 一个组件的状态更改也会触发直接或间接依赖于它的另一个组件的更改。
一般来说，如果我们可以确定它们的预期结果不会改变，增量计算支持跳过一些计算。 在 UI 方面，我们避免触及（和重绘）不需要任何更改的组件。
假设我们要在这个简单的井字棋游戏中移动：
| 之前 | 之后 |
 |
 |
我们有一个带单元格的网格。 点击右上角（添加 X）时，哪些单元格应该被改变？ 第一行中的单元格需要重绘，成为获胜行。 不过，其他单元格都没有变化，所以最好不要重绘。 为了实现所需逻辑，我们按如下方式构建组件和计算：
- 一个 2D 网格，包含单元格，并提供了一种用行、列、填充等描述布局的方式。
- 每个单元格都有关于其内容的信息（无论其中有 X 还是 O），以及成为获胜行、列或对角线的一部分的可能性，并且当这些 X 和 O 的状态发生变化时可以重绘。
- 点击右上角的单元格应更改其内容（并触发重绘），但它也应触发检查游戏是否结束并更改获胜单元格的相应设置。
- 更改获胜单元格的设置应触发使用交叉黄线的重绘。
因此，回应用户操作（点击单元格）会导致网格的部分重新计算和部分单元格的重绘，同时保持其他单元格不变。 Noria 为我们提供了处理此类行为所需的一切，包括：
- 一种声明式 Kotlin DSL，将组件布局描述为树结构以及支持事件传播的树结构依赖项。
- onClick-hook，用于实现即时回应的组件。
StateCell，用于表示负责触发相邻节点重新计算的树结构外依赖项。
在后台，Noria 不会在没有必要时触发重新计算。 这是所有增量计算平台的标准行为。 Noria 用户以声明方式编写描述整个 UI 的函数。 每次有事情发生时都会调用这个函数。 但 Noria 准确知道与前一次运行相比什么发生了改变，什么没有改变，并决定 UI 树的哪些部分需要重新计算。
声明 UI 组件
我们来看一些代码示例，体验一下使用 Noria 的感觉。
上一部分提到的井字棋组件在 Noria 中可以这样描述：
val grid = Grid(3) { state { GridCell() } }
val gameState = state { GameState() }
clickable(onClick = { grid.checkEndOfGame(gameState) }) {
vbox {
grid.gridRows.forEach {
hbox {
it.forEach {
cell(it, gameState)
}
}
}
}
}
我们有一个包含单元格的网格和一个负责下一回合与游戏结束状态的游戏状态。 注意前两行中的两个 state 函数。 它们负责创建额外的依赖关系：更新它们的内容会触发读取它们的组件的重新计算。
每个单元格都负责呈现其内容和回应用户点击：
private fun UIContext.cell(gridCell: StateCell<GridCell>,
gameState: StateCell<GameState>, ...) {
val gs = gameState.read()
val cell = gridCell.read()
clickable(onClick = { ... }, propagate = Propagate.CONTINUE) {
decorate(backgroundColor = ...) {
layout {
render(Rect(Point.ZERO, Size(size, size))) {
...
}
}
}
}
}
从技术上讲，cell 是 UIContext 类的扩展函数，它实现了 Noria 背后的机制。 单元格读取一些状态，从而表达它对该信息的依赖。 Noria 在决定是否有必要重新呈现此单元格时使用这个依赖关系。
Noria 组件可以像以下带有工具提示的文本标签一样简单：
withTooltip("Tooltip text") {
uiText("Point on me")
}
它们也可以非常复杂，例如文本编辑器或 Fleet 实例中常见的其他窗格和窗口。
Noria 为实现桌面 UI 和其他明显的 UI 框架功能提供了所有必要组件，包括：
- 布置组件。
- 限制和设置边界。
- 呈现可见部分并提供对滚动的支持。
- 定义焦点遍历。
- 实现叠加（例如上例中的工具提示或错误代码段旁边的错误消息）。
每次运行 Fleet 时，Noria 机制都会全力提供最佳 UI 体验。
总结
Fleet 以用于 JVM 的自制 UI 框架 Noria 实现。 Noria 让我们可以使用 Kotlin 功能以现代声明方式描述 UI。 它基于增量计算核心，负责表达 UI 组件之间的依赖项，并最大限度地减少不必要的重复呈现。
Noria 绝不是革命性的。 它只是采用了现代设计思想，将其应用于 UI 框架，并添加了一些 Kotlin 风格。 但到目前为止，我们学到的是，开发自己的基础 UI 框架是可行的，而且实际上非常有趣！
《Fleet 后台探秘》系列远未结束。 我们还有许多 Fleet 的内部细节要分享。 敬请关注！
本博文英文原作者：
Subscribe to Fleet Blog updates
Discover more
Fleet Below Deck, Part VI – UI With Noria
This is a multipart series on building Fleet, a next-generation IDE by JetBrains.
- Part I – Architecture Overview
- Part II – Breaking Down the Editor
- Part III – State Management
- Part IV – Distributed Transactions
- Part V – The Story of Code Completion
- Part VI – UI With Noria
In Part V of this series, we discussed code completion, which is just one of Fleet’s services. Now it’s time to talk about something as big as our very own declarative UI framework for JVM, Noria. We built Fleet with Noria. Let’s look at the ideas behind Noria, the main concepts, and other exciting features.
Where it all starts: A Noria window
How is a UI built? First, we have a display with graphical capabilities that is able to present an application’s state and is also responsible for making our UI a GUI. We may also have one or more input devices, such as a keyboard, a mouse, or a touchpad, to deliver commands and control the application’s behavior. In these settings, an application is effectively an event loop responsible for reacting to events initiated by users and other computer system components (timers, a file system, a network, and others). Primarily, reactions are visible changes in what is displayed on the screen.
In addition to being an event loop, a GUI application usually has some kind of window, an area of the screen it is responsible for, and some drawing capabilities in that area. A window is normally provided by an underlying operating system or a window manager on top of it. An operating system may provide a graphics API or delegate drawing to a graphical framework.
Fleet is both a GUI application and a JVM application. It runs on all major operating systems, including Windows, macOS, and Linux. Fleet relies on the Java AWT/Swing framework to get a window from an operating system, but it doesn’t use the Java platform for managing its GUI components besides one JFrame and JPanel on top of it. Fleet doesn’t use the JVM’s screen drawing capabilities, either. Instead, it employs Skia, a native 2D graphics library that is available to JVM applications via the skiko-awt binding library, which JetBrains develops.
The following diagram presents a combination of the aforementioned frameworks and libraries, resulting in a screen area fully managed by our home-grown UI framework – a Noria window:
Everything you see in a Fleet window is a Noria component. Panels, tabs, buttons, tooltips, text editors, terminals, diff views, and docker views are managed by Noria and are constantly changing as a result of Noria’s event loop reactions.
You might ask, “Why Noria?” Why invent a new UI framework when we have so many brilliant UI frameworks out there begging to be applied for creating highly responsive GUI applications with a modern look and feel for all major desktop platforms at once? Well, that’s not entirely the case. Let’s refer to the history of those UI frameworks and see whether we had such options to choose from when we started developing the product that eventually became Fleet.
A brief history of UI approaches
Ideas and approaches to GUI frameworks (not GUI itself) can be traced back to the 70s, when a team of researchers at Xerox PARC, including Alan Kay, developed a graphical environment for the Smalltalk programming language. From the early days of UI frameworks, their developers had a clear focus on architectural matters. It wasn’t a simple task to organize an application over an event loop with many UI components on the screen, as well as factoring in unpredictable user actions and their corresponding UI reactions, and not end up having an application that would be a complete mess from an architectural perspective. One example of finding such an approach is detailed in some notes by Trygve M. H. Reenskaug, where he reflects on the inception of MVC, a notable Model-View-Controller architectural style, which was a set of ideas to provide a structure for GUI applications.
There was no clear way to apply an MVC style, which led to many attempts to reformulate it using other terms (for example, MVP – Model-View-Presenter). Every UI framework that claimed to support it provided some specific way to do that. Those ways didn’t have a lot in common besides several generic things, including the following:
- Relying on the object-oriented features of programming languages.
- Separating the business logic from the presentation (except that, unfortunately, presentation may require some logic, too).
- Observing changes via the Observer pattern (and experiencing related code readability issues, because it may be too hard to figure out what’s going on in the program from reading sources that heavily exploit this pattern).
There was also another line of development, an attempt to simplify an architecture by structuring it into a form, with a set of controls on that form, and with all the logic that connects the application state to components of that form. This idea, popularized by Borland Delphi and Visual Basic in the 90s, led to a generation of Button1Click developers who were happy to develop large applications without thinking about architecture. This approach was always heavily criticized by object-orientation-inclined researchers and practitioners, but, frankly speaking, sometimes it was too difficult to distinguish between the complete messes produced by either no-architecture or strict architectural guidelines.
Martin Fowler explored these 2 lines of development for UI frameworks in this brilliant excerpt from his (unfortunately, unfinished) book Further Patterns of Enterprise Application Architecture.
And then we’ve got PHP. Developing web applications in PHP following its inception in the middle of the 90s became an exceptionally pleasing experience, extremely easy to fire up and get straight to production. PHP popularized template processing, an approach when you mix logic (instructions of a programming language) and presentation (HTML tags) in one piece of code – a sordid crime according to MVC purists. No other programming language has ever received so much backlash from developers. Just google “PHP hate” to get an idea. The “Guards of Clean Code Heaven” still hate PHP for this (and maybe also for opening the gates to the industry to many new developers who were raised unaware of the only “right way” to develop software).
Well, let’s not judge programming languages but try to see exactly what this new idea was and where it would lead us. How we see data is intimately linked with how we manipulate it. With PHP, we no longer had to separate logic and presentation. After all, most of the logic is in the presentation anyway because it’s the users who manipulate the presentation, not some abstract data that can’t be seen or touched. PHP made it clear that we needed components – active entities with their own look, reactions to user actions, and other behavior. Components may be composed of other components, and they can effectively interact with neighboring components through some shared state. For example, a list of items in one component can deliver change signals to another component whose content depends on the selected items in the first component.
Ajax (Asynchronous JavaScript and XML) took this idea even further, with web components as the main building blocks of web applications. These web components combined backend and frontend logic while mixing them with the presentation. Good or bad, this idea brought us the current web and web development experience.
In the early 2010s, React introduced a brand new world of UI development. Frontend developers may argue about which is better, React or Angular, and which state manager should be used, but now they are used to the idea of active UI components. They even apply their web technologies to developing desktop applications (though the resulting experience doesn’t always provide perfect responsiveness, look, and feel).
One thing we should praise React for is that it made declarative programming and reactive programming the mainstream. These previously hidden techniques were rarely used in production programming languages, but with React they began to dominate in many areas of software development, including, most notably, UI frameworks.
By the end of the 2010s, we started to observe the same approaches to UI pervading UI frameworks for mobile systems, Apple’s SwiftUI and Google’s Jetpack Compose being the most prominent examples. We didn’t have to wait long to see the same trends in UI frameworks for desktop systems. In the early 2020s, we have the beautiful Compose for Desktop, based on Google’s Jetpack Compose and brought to you by JetBrains. When we started working on Fleet, however, Jetpack Compose didn’t exist, and thus Noria was born.
Incremental computations
Interestingly, Noria is not a UI framework at its core. Instead, it’s a platform for incremental computations. Imagine a large multi-part mathematical formula with interdependent components that sometimes have to be recalculated or trigger the recalculation of dependent components – not unlike a spreadsheet. Noria shines at computing and recomputing these sorts of formulas. The main principle here is to avoid any unnecessary computations and to recompute only the parts required to produce the final result.
What about the UI, then? Well, it’s also such a formula. It has a tree-like structure built of components and subcomponents. Those components may also have other dependencies besides being a part of another component. This makes a directed acyclic graph (DAG) of dependencies. A change of state in one of the components also triggers a change in another component that depends on it, directly or indirectly.
In general, incremental computations support skipping some calculations if we can determine that their expected results won’t change. In terms of UI, we avoid touching (and redrawing) components that don’t require any changes.
Suppose that we are about to make a move in this simple game of Tic-Tac-Toe:
| BEFORE | AFTER |
| |
We have a grid with cells. Which cells should be changed when we click (add an X) in the top-right corner? The cells in the first row will need to be redrawn as it becomes the winning row. None of the other cells changes, though, so we’d better refrain from redrawing them. To implement the required logic, we structure our components and computations as follows:
- A 2D grid contains the cells and offers a way to describe the layout with rows, columns, padding, etc.
- Each cell has information about its content (whether there’s an X or an O in it) and the possibility of being a part of a winning row, column, or diagonal, and it can be redrawn when the state of these Xs and Os is changed.
- Clicking on the top-right cell should change its content (and trigger redrawing), but it should also trigger checking whether the game is over and changing the corresponding settings of the winning cells.
- Changing the settings of the winning cells should trigger the redrawing of them with a cross-through yellow line.
Thus, reacting to a user action (clicking on a cell) leads to partial recomputation of the grid and the redrawing of some of its cells while keeping others intact. Noria provides us with everything we need to handle such behavior, including:
- A declarative Kotlin DSL to describe the layout of components as a tree structure along with tree-structure dependencies that enable event propagation.
- onClick-hooks for components that implement immediate reactions.
StateCells to express the out-of-tree-structure dependencies responsible for triggering recalculations of neighboring nodes.
Under the hood, Noria does not trigger recalculations when they are not needed. That’s standard behavior for any incremental computation platform. A Noria user writes a function describing the whole UI in a declarative way. This function is called every time something happens. But Noria knows precisely what was changed from the previous run and what wasn’t, and it decides which parts of the UI tree need recalculating.
Declaring UI components
Let’s look at some code examples and get a feeling of what it’s like to work with Noria.
The Tic-Tac-Toe component mentioned in the previous section can be described in Noria as follows:
val grid = Grid(3) { state { GridCell() } }
val gameState = state { GameState() }
clickable(onClick = { grid.checkEndOfGame(gameState) }) {
vbox {
grid.gridRows.forEach {
hbox {
it.forEach {
cell(it, gameState)
}
}
}
}
}
We have a grid with cells and a game state responsible for the next turn and end-of-game status. Note the 2 state functions in the first 2 lines. They are responsible for creating additional dependencies: Updating their content triggers the recalculation of components that read them.
Every cell is responsible for rendering its content and reacting to user clicks:
private fun UIContext.cell(gridCell: StateCell<GridCell>,
gameState: StateCell<GameState>, ...) {
val gs = gameState.read()
val cell = gridCell.read()
clickable(onClick = { ... }, propagate = Propagate.CONTINUE) {
decorate(backgroundColor = ...) {
layout {
render(Rect(Point.ZERO, Size(size, size))) {
...
}
}
}
}
}
Technically, cell is an extension function for the UIContext class, which implements the machinery behind Noria. The cell reads some state, thus expressing its dependence on that information. Noria uses this dependency while deciding whether it’s necessary to re-render this cell.
Noria components can be as simple as the following text label with a tooltip:
withTooltip("Tooltip text") {
uiText("Point on me")
}
They can also be quite sophisticated, like a text editor or other panes and windows that you regularly see in your Fleet instance.
Noria provides all the essential components for implementing desktop UIs and other obvious UI framework capabilities, including:
- Laying the components out.
- Constraining and setting boundaries.
- Rendering visible parts and providing support for scrolling.
- Defining focus traversals.
- Implementing overlays (such as a tooltip in the example above or error messages next to your erroneous code fragments).
Every time you run Fleet, Noria machinery is working hard to deliver the best UI experience possible.
Summary
Fleet is implemented with Noria, a home-grown UI framework for the JVM. Noria allows us to describe UIs in a modern declarative way using Kotlin features. It is based on the incremental computations core, responsible for expressing dependencies between UI components and minimizing needless re-renderings.
Noria is by no means revolutionary. It only takes modern design ideas, applies them to UI frameworks, and adds some Kotlin flavor to them. But what we’ve learned so far is that developing your own fundamental UI frameworks works and can actually be a lot of fun!
The Fleet Below Deck series is far from over. We still have a lot of details about Fleet internals to share. Stay tuned for more!
Subscribe to Fleet Blog updates
