Functional Highlighting for Functional Programming

Pavel Fatin

Statically typed programming languages, such as Java or Scala, are all about types. But not all types are created equal: there’s a difference between how types work in imperative programming and how they work in functional programming. In this article, we show why “imperative” highlighting doesn’t mix well will “functional” code, and present a new, “functional” solution.

The new highlighting is now enabled in the 2019.2 nightly builds of the Scala Plugin, so this update is definitely worth a detailed description. Your feedback is very welcome as always.

TLDR: Before. After. Enjoy!

1. Overexpression of expressions
2. A match for type mismatch
3. The code of code
4. Type ascription to type ascription
5. A diff makes a difference
6. The improvement of improvement

Overexpression of expressions

To illustrate the problem, let’s first consider the following Java code:

Don’t ponder the semantics too much; focus on the syntax. This code contains a type mismatch error: the actual type of the s2 expression (String) doesn’t conform to the expected type – the type of the add method parameter (Integer). As you can see, IntelliJ IDEA highlights such an error with a red squiggly underline. While the particular style depends on the color scheme, it’s of no particular importance – the main point here is that it’s the expression that is highlighted. This way of highlighting type-mismatch errors is the de-facto standard today. And yet it is a legacy of the past, when typical code resembled the snippet above. To IntelliJ IDEA, this mode came from Java and then extended to other supported languages, including Scala.

What’s so special about Java? Typical “good old” imperative Java code is a finely dispersed mix of statements and expressions, which naturally limits the scope of “type mismatch” highlighting. But that’s no longer the case – here’s a Java 8 version of the above code:

In “functional” code, expressions are no longer separated by statements. This doesn’t go well with the legacy highlighting, which is geared toward “imperative” code. Long underlines are hardly pretty, especially if they span across multiple lines. Outer errors mask inner errors. And if you’re in the process of writing such an expression, it would be constantly underlined solely for being incomplete.

All this poses a problem even in Java, let alone in Scala. Given that typical Scala code is more “functional” than Java code, the problem only gets worse. Because a block is an expression in Scala, arbitrary amounts of code, even imperative code, can be underlined “from tip to toe” if we apply the Java recipe directly:

This problem was encountered as soon as we started to implement type-aware highlighting in the Scala plugin. And so, another recipe was borrowed from Java: to highlight the closing brace:

However, this solution is imperfect, for many reasons. First, a single underlined } char is difficult to notice, especially in large blocks of code. Second, the semantics doesn’t match: in Java, this highlighting indicates that a non-void method is missing a return statement, which has nothing to do with the type mismatch (what’s more, you don’t even “return” values from a block in Scala):

The worst of all, though, is that such a solution is partial – it only addresses one special case without addressing the underlying problem. Not every expression is enclosed in a block:

The scheme could possibly be improved by highlighting the final expression instead of the closing brace. This would make the error easier to spot and would get the semantics right. But, again, this only has to do with blocks, and does nothing otherwise. What’s more, this would interfere with writing new code, as each subsequent expression in a block would be underlined.

A match for type mismatch

To find the solution, let’s look closely at the essence of “type mismatch”:

Does the problem lie in the expression? Sometimes you may need to fix the expression to fix the error. Other times, the problem is somewhere else (which is often the case in Scala because of type inference). As the name suggests, “type mismatch” is about a type, and about a mismatch (more precisely, about the actual type and its mismatch against the expected type). Insightful! But why then do we underline the expression? Well, because we need to underline something, and the type is not visible. Whereas the expected type may be present in the code, the actual type is (almost always) inferred, both in Scala and in Java.

But what if we can highlight a type, not an expression? After all, we already use inlay hints to indicate a missing implicit value instead of underlining the expression. We also use the hints to display implicit conversions – a noticeable improvement over the legacy mode. Thus, we can ascribe the actual type, and then highlight it as we see fit:

This way, we can shift the focus from the expression, which is not necessarily the source of the problem, to the actual cause. Even more importantly, that’s how we can limit the scope of highlighting, regardless of the expression length.

The code of code

Just like with the redesign of implicits, it’s crucial that inlay hints represent valid syntax. This lets people employ the same mental model they use to read code. When a notation is uniform and already familiar, the code is the UI, and even complex highlighting looks simple.

But to achieve that, we need cooperation from the language. For example, since Java syntax lacks named arguments, IntelliJ IDEA has to invent the extraneous notation:

Apparently, this cannot be a valid piece of code. Fortunately, Scala supports named arguments out of the box:

The same goes for type ascription. But is the chosen representation indeed a valid piece of code? Let’s ask the scalac:

This code is not valid in the sense that it doesn’t compile, but no wonder – there’s a type mismatch error to begin with. However, instead of complaining about the syntax, the compiler reports exactly the same error in exactly the same place! Thus, the inlay hint behaves as if it were present in the code “as is”.

Apart from the syntax, there’s the question of style. Originally, parameter name hints look different from code. This is somehow justified in Java, where hints don’t represent code. In Scala, however, hints represent valid code. What’s more, Scala hints behave like code: they provide error highlighting, code folding, tooltips, navigation, and brace matching (and we use the current code style for all of that). When we designed the implicit hints, we unanimously voted to display them “like code”, because this feels more natural. It makes sense to do the same with the type mismatch hints.

It also makes sense to display the existing type annotation hints in the same way, for uniformity. This is also more in line with the implicit hints. As a nice bonus, all the hints then support navigation, folding, tooltips, and brace matching:

The monospaced font is slightly wider, but this is compensated by the added dynamic folding and by showing type hints only when truly needed.

Interestingly, type mismatch hints are not just for type mismatch – they also provide editing assistance. Type ascription for expressions works in the same way as the type annotations for definitions (which often are just named expressions). Both kinds of hints complement the on-demand Type Info action.

Type ascription to type ascription

Showing type ascription as though it was present in the code is great and all, but what if it actually is present? Previously, we highlighted a typed expression as any other expression. That is, we underlined it:

But what are we supposed to do now? Should we ascribe a type twice? While Scala lets you do that – (expression: Actual): Actual, there’s a better way. If you think about it, a type ascription hint is just a blank canvas for error highlighting. If the canvas is already present, so much the better – we can do the highlighting right away:

When some expression lacks an ascribed type, the Scala compiler first has to infer it, and then check for a possible type mismatch. But when a type is already present, the compiler can skip the type inference, and get right to the type checking. That’s what we do now, and this perfectly matches the highlighting of inlay hints (interestingly, this also reduces the scope of type mismatch underlines even without using inlay hints).

So far so good, but what if an ascribed type is incompatible? For example:

Previously, we highlighted the expression as though the type ascription was in fact a type annotation. Should we now ascribe an actual type as (expression: Actual): Incompatible? Not really. There’s a difference between type annotation and type ascription. The former is added to declarations and definitions, and corresponds to explicit types in Java. The latter is added to the expression and corresponds to the cast operator in Java (but that supports only upcasts). In the case of type annotation, the type goes first and the expression follows. By contrast, with type ascription, the expression goes first and the type follows. So, type ascription always specifies the actual type; it’s just that the type might be incompatible. When an expression cannot be upcast to a given type, it’s the type that is the culprit:

That’s how we can reduce the scope of highlighting and keep the highlighting uniform (again, this would work even without using inlay hints).

A diff makes a difference

We’ve done a good job so far. But we need to go deeper narrower. Many Scala types comprise other types and those components are often the original source of a type mismatch. Previously, we compared types as indivisible units:

While technically this wasn’t wrong, it was hardly convenient, because you had to “parse” and “type check” the constituent types in your head to understand why exactly the types do not match. Now, we highlight only the truly “mismatching” parts of a type:

We can even show a fine-grained, vertically-aligned tooltip for a pairwise comparison:

It’s not just a text-based diff either – we take syntactic sugar, subtyping, variance, and other stuff into the account.

The improvement of improvement

That’s the big picture. There are also some special cases, for example, the handling of literal types or method invocations. Although the main work has been completed, the polishing is still in progress. Please use nightly builds for the the most recent updates, and report any bugs or suggestions to YouTrack.

On the whole, it seems that the new highlighting is a substantial improvement over the legacy, “imperative” scheme, and the more “functional” your code, the bigger the improvement. We hope this feature will prove useful for Scala, and we expect other IntelliJ IDEA languages to follow :)