IntelliJ IDEA 2017.2.2: Kotlin 1.1.4, Better Performance and More

We’re glad to announce the availability of IntelliJ IDEA 2017.2.2, that brings you numerous bugfixes and improvements, most notable of which are:

  • Major performance regression (KT-18921)
  • Crashes that occurred with the use of using custom fonts (JRE-426)
  • Wrong HiDPI scaling on multiple monitors for Windows 10 (JRE-427 and JRE-431)

Kotlin plugin has been updated to v1.1.4 and brings new refactorings, inspections, quick-fixes, and more.

Also, IntelliJ IDEA 2017.2.2 brings:

  • Support for CSS Modules with LESS and SASS
  • XQuery/XPath with Language Injection
  • Favorites in the Database tool window

You can install this update either via Toolbox App, or as a patch for IntelliJ IDEA 2017.2.1 (use Check for Updates), or otherwise just get an installer from our website.

Bug reports and your feedback, as always, are very welcome in our issue tracker.

The Drive to Develop

Posted in Releases | Tagged , | 1 Comment

Code Smells: Deeply Nested Code

Or: I wants all your data, give it to me… my precious….

Continuing the series of code smells and what do about them, in this post I examine some fairly innocent looking code that defies the obvious refactoring.  Although the code example itself is fairly trivial, it’s actually a symptom of a problem found again and again in this particular project: deep nesting of code. This can be for loops, if statements, even lambda expressions or inner classes, or combinations of all of the above.

Continue reading

Posted in Tips & Tricks | Tagged , | Leave a comment

Code Smells: Null

During my research on refactoring I’ve seen a number of patterns (smells) crop up again and again. None of these are particularly new, and there are plenty of books, blogs and videos that cover smells and how to deal with them, but I wanted to demonstrate some specific, non-trivial examples and, of course, how IntelliJ IDEA may (or may not) be able to help you.

The first problem I’ve being trying to counter is the use of nulls, particularly when it leads to null-checks scattered around the code.

I thought Java 8’s Optional should solve a lot of these problems. I assumed that a type that specifically states that it may be null, that is designed to let you say what to do in these occasions, is exactly the right solution.

However, things are never as simple as they seem, and I suspect that’s why there’s no magic “Optionalise my project” intention in IntelliJ IDEA.  Rather disappointingly, this is an area that needs me, the developer, to think hard about what should be done.

We have to come to terms with the fact that null means many things.  It can mean:

  1. Value was never initialised (whether accidentally or on purpose)
  2. Value is not valid
  3. Value is not needed
  4. No such value exists
  5. Something went terribly wrong and something that should be there is not
  6. …probably dozens of other things

You can assume some of these don’t apply to your code base if you’re a disciplined team with clear design goals – for example, having final fields that are always initialised in the constructor, using builders or factories to validate correct combinations before instantiation, not using nulls in the core of your application (i.e. pushing all checks to the edges) and so on.

Optional really solves only one of these cases, case 4. For example, you ask the database for a Customer with a given ID.  Previously, you might have used null to represent this, but that could be ambiguous – does it mean the customer wasn’t found? Or that there’s a customer with that ID, but has no values? Or was null returned because the connection to the database failed? By returning an empty Optional, you’ve removed the ambiguity – there’s no customer with that ID.

In a normal, mature, code base that’s been worked on by numerous people, is it possible to tell what all our nulls mean, and what to do about them?  We should be able to make some progress by taking very tiny bites.

The Case Study

I’m using the Morphia project as an example in this blog post, as I have in previous talks and articles about refactoring.  This is a great example project because it’s a) open source b) small enough to easily download and explore and c) mature enough to contain the sort of real life example code you probably have in your own applications.

The Smell: return null

I did a Find in Path for all the places where null is explicitly returned.  I have a theory that maybe we can change these methods into something that returns an Optional.

Find places that return null

Results of “Find in Path” for “return null”

Example 1: Converter.decode()

Given that lots of these *Converter classes seem to return a null value in the decode method, it seems reasonable that we might want to change the Converter superclass (an abstract class called TypeConverter) to return Optional here.  But a closer look at the code shows what’s actually happening: we see exactly the same pattern happening again and again – the method checks if a value passed in was null, and if so returns null:

The first question is can fromDBObject actually be null? The code is sufficiently complicated that it’s difficult to tell, but theoretically it seems plausible, since this is a value from the database.

A quick search shows that all instances of this method are actually only called from one central place, so instead of making this check in 21 different places, we can just do it in a single place and from there on assume fromDBObject can no longer be null.

Solution: @NotNull parameter

My original assumption that we can use Optional here turned out to be wrong.  Instead, I’m going to change the method signature of decode to declare that fromDBObject cannot be null – I’ve opted to use the JetBrains annotations to do this.

Then I move the null check (and subsequent null return) in the place where fromDBObject could actually be null, the single place that calls decode.

Add null check to call site

Moving the null check to the single call site where fromDBObject can be null

It makes the call site a bit more untidy, but constrains this logic to a single place instead of scattered through all implementations. Now, we can either document the fact that fromDBObject will not be null or rely on the annotation to do this for us, so we never have to perform this null check in the Converter implementations.

Next we can tidy up and remove all the duplicated code. Here IntelliJ IDEA can help us.  If we go back to the abstract method in TypeConverter that we annotated with the @NotNull parameter, we are given a warning:

Overridden parameters are not annotated

Warning that implementations don’t annotate this parameter

The quick fix suggested when we use Alt+Enter on this warning lets us apply this annotation to all the implementations of this method in one easy step, so let’s do that.

Quick fix to apply the annotation to all implementations

Use the quick fix to apply to all implementations

Our VCS log shows that all the Converter implementations have been updated

All Converters updated

This has added the @NotNull annotation to the decode method’s parameter, but we still haven’t tracked down and removed all the duplicate null-check code.  The good news is that now we’ve said fromDBObject cannot be null, we can use the inspections to find all of our redundant null checks and remove them.

One way to do this is to navigate into an implementation that we know has the check (in this case I chose StringConverter) to see what warning IntelliJ IDEA gives me.

Null check not needed as value is never null

Null check not needed as value is never null

This turns out to be triggered by the “Constant conditions & exceptions” inspection. We can run just this inspection over the code base to find other examples easily, using Ctrl+Shift+Alt+I and typing the inspection name.

Run inspection by name

Using Ctrl+Shift+Alt+I to run an inspection by name

This returns more results than just my Converters, but by grouping the results by directory I can see all the Converter implementations easily

Inspection results

I can use the preview window on the right to check which code has been flagged, and see what IntelliJ IDEA suggests doing with each problem.  After scanning through and making sure this is the null check I’m looking for, I select the converters directory and press the “Simplify boolean expression” button.

To sanity check the changes, I go into the VCS Local Changes window and use Show Diff (Ctrl+D) to check the changes that were applied to the 36 files there.

Using the diff view to sanity check changes

Using the diff view to sanity check changes

I can even use the diff view to make small edits if I want – in this case I use Ctrl+Y to remove the unnecessary empty lines 26 and 27 of my file (the file on the right). By using Alt+Right to check all the files that have been changed, I find an example where the null check wasn’t removed (this was a test converter and therefore was not in the converters directory), but I can easily use quick fixes even in the diff view to remove it.

This sanity check gave me peace of mind about the automatic changes IntelliJ IDEA made, and let me make some additional tweaks too. The last thing to do is run all the tests, and when they pass (hurrah!) now seems like a really good time to commit all these changes.

If you’re uncomfortable adding a new library to document that a parameter can’t be null and make the IDE warn you if you pass in null values, you can always remove the @NotNull just before committing – after all, it’s done its job now. In this case, you should at least update the Javadoc to state that the parameter is assumed to be not null.

Example 2: Converter.encode()

Earlier we saw that the decode method was only one of several that explicitly returned null values.  Now we’ve dealt with that, we can take a look at another example.

The encode method on the same TypeConverter class can also return null.  But unlike decode, there are valid times when the converter implementations explicitly return null:

Solution: Optional

This is a good place for Optional– we’re explicitly stating that there are some cases where the encode method does not return a value. Optional.empty() seems like a good representation of this fact.  So I change TypeConverter.encode() method to return an Optional instead.  It makes all the implementations a little more untidy, as things need to be wrapped in an Optional, but it is more explicit about the fact that this method sometimes does not return a value. I’d love to show you IntelliJ IDEA magic that does this for you (in some cases, Type Migration will work, but not in my example) but I made this change the hard way – I changed the return type of the super class to Optional and fixed all the places that broke. The good news is that once I’d changed the method signature to return an Optional, IntelliJ IDEA did offer a quick fix to wrap my return values in an Optional.

Wrap value in Optional

Wrap value in Optional

Similarly, null values can also be replaced

Return Optional.empty() instead of null

Return Optional.empty() instead of null

Now we’re returning an Optional, we have to work with this new type at the call site.  In my case, I actually got no compiler warnings as my return type was originally an Object, and obviously an Optional is also an Object (note: this is why we shouldn’t use raw Object types, because it basically removes the benefits of having a strongly typed language. This code would have been better with Generics). I just have to change the one place that calls the encode method to unwrap the Optional.  I’m going to do something that’s pretty nasty and use orElse(null), but since this is the same behaviour the original methods were doing anyway, and it’s restricted to one place in the code, I’m happy with this – I can flag it as tech debt and we can gradually deal with the issue rather than chasing it all around the code.

Dealing with the optional return

This preserves the original behaviour, but should be considered tech debt to be addressed later.

In tests that call the encode method I simply updated them to call encode(o).get() – usually this is unsafe, but a) the tests should not return an empty Optional and b) if they do, they’ll fail and that’s correct. In fact in my case this highlights that there are no tests for the case where no value is returned, so I should add new tests for empty Optional returns.

Changing your API to use Optional is often a non-trivial task, but in cases like this one it can really help clarify the intent – instead of returning nulls when there’s no valid value, it will return you an empty Optional and you can declare what to do – return an alternate value, throw an Exception, or perform some other operation. Beware though, this can get hairy.

Note: This example was OK, but there are other methods in this code that are perfect for returning Optional, but the upstream effect of applying this change was huge – the methods are called in multiple places and the return types used in all sorts of very complex methods, making it nearly impossible to track down the right place to test the Optional and unwrap it vs letting it propagate around the code. The lesson I have learnt is apply this if you can limit the impact to one or two call sites and deal with the Optional return type immediately.

Example 3: Mapper.getId()

Here’s another example of null returns:

This is a great example of where null means two different things. The first null means we were given a null input, therefore we give you a null output.  Seems fairly valid, if a little unhelpful. The second null means “an error happened and I couldn’t give you a value, so here, have null”.  Presumably a null ID is a valid response, given the first case, but it would probably be more helpful to the caller to know what the error was and deal with it accordingly.  Even if the Exception is such that the caller cannot deal with it, it’s pretty unfriendly to catch the Exception and return null, especially when the Exception isn’t even logged.  This is a really good way to hide genuine problems.  Exceptions should be handled where they’re found, or passed on in some useful way, they should not be swallowed unless you really really know it’s not an error.

So this null means “something unexpected happened and I don’t know what to do about it, so I’m going to give you null and hope that’s OK”, which is not at all clear to the caller.  These cases should definitely be avoided, and Optional is not going to be your solution. The solution is to implement much clearer error handing.

In summary

Null is a particularly difficult problem. The main problem with null is we don’t know what it means. It’s an absence of a value, but this could be for any number of reasons. Using null is not a useful way to flag what any of these reasons are, since by its very definition it has no value, no meaning.

Symptoms:

  • Null checks extensively used throughout the application code, without it being clear to you, the developer, what null really signifies or if the value can really be null at all.
  • Explicit return null

Possible Solutions:

  • Optional.  Not a solution to everywhere you find a null.  But if a method is deliberately returning a null value to mean “not found” or “I don’t have one of those”, this is a good candidate for returning an Optional.
  • @NotNull/@Nullable. One of the problems with complex code bases is understanding what are actually valid values. If you’re checking for null parameters, try putting @NotNull on the signature and seeing if null values are ever passed.  If nulls are never passed, you can remove the null check.  If you’re uncomfortable adding a dependency just to use the @NotNull annotation, you can apply the annotation temporarily, get IntelliJ IDEA to tell you if there are any problems, fix up if necessary and run all your tests to make sure everything works.  If it’s all good, you can remove the annotation, add a Javadoc comment (not as safe as the annotation, but still enormously useful for developers who call or maintain the method) and commit the updated code.
  • Exception handling. Returning null for an Exceptional case is very unfriendly, and unexpected from the caller’s point of view.  It can lead to NullPointerException further down the line, or at least more of the pervasive null checks (without a clear idea of why the value is null).  Throwing a descriptive Exception is much more valuable for those downstream.
  • Sometimes, it’s fine.  Field-level nulls (where it should be well understood what can be null and why) are examples where null may be perfectly acceptable.
Posted in Tips & Tricks | Tagged , , , , | 24 Comments

IntelliJ IDEA 2017.2.2 EAP is Out

Following the v2017.2.1 update released just a couple of days ago, we’re opening the EAP for IntelliJ IDEA 2017.2.2.

Among various fixes, the EAP build addresses the following issues:

  • A performance issue due to the Kotlin plugin regression (KT-18921)
  • The crash issue that occurred due to using custom fonts (JRE-426)
  • Wrong HiDPI scaling on multiple monitors for Windows 10 (JRE-427 and JRE-431)

The Kotlin plugin has been updated to v1.1.4-eap-54. The built-in JRE has been updated to u152b915.10.

The Drive to Develop

Posted in EAP Releases | Leave a comment

Java Annotated Monthly – August 2017

​Here in Europe we’re well into the vacation season. July and August are usually quiet, but at JetBrains we’ve been working hard on a number of 2017.2 releases, including IntelliJ IDEA and Upsource, so this month’s Java Annotated includes links to all the information about these releases. In addition, there’s loads of Java 9 news, and a good chunk of Design and Architecture this month. I was going to squeeze in a new section this month, but we’ve got so much content you’ll have to wait and see if I ever get space for it…

JavaAnnotatedMonthly@2x

Continue reading

Posted in News & Events | Tagged , | Leave a comment

IntelliJ IDEA 2017.2.1: Better Performance and Spring Run Dashboard

If you haven’t yet tried the newly-released IntelliJ IDEA 2017.2, now definitely is the time for that because we’ve just released a slightly better version of it.

Along with better performance and bugfixes, the update finally lets you use the Spring Boot feature that has been announced a while ago: Run Dashboard.

This dashboard lets you manage Run configurations and see their Spring Boot specific properties.

For the complete list of changes refer to the release notes.

If you’re using Toolbox App, you will be prompted to install this update. If you don’t (but you really should), check for the updates manually via Check for Updates menu action. Patches are available if you’re already running IntelliJ IDEA 2017.2, otherwise it’s a full upgrade.

As always, we appreciate your feedback. Please share it here in the comments. In case of issues, report it to our issue tracker.

The Drive to Develop!

Posted in New Features, Releases | Tagged , | 19 Comments

Loads of Small Improvements in IntelliJ IDEA 2017.2

Many of the features we’re going to talk about in this post have been mentioned elsewhere in this blog, but here we’re going to bring them all together and show how to use them.

Better Warnings

You’ll have seen before that IntelliJ IDEA can analyse your code and tell you if something isn’t needed, or perhaps doesn’t make sense.  IntelliJ IDEA 2017.2 has even better analysis in several areas.  The first we’re going to look at is Control Flow Analysis – IntelliJ IDEA can look at the logic in your code and tell you if certain cases just won’t occur.

For example, if a Collection or String is known to be empty, IntelliJ IDEA will tell you that certain types of operations will error or simply aren’t required, like iterating over an empty collection.

Warnings on empty collections

This applies to a number of methods on List, Map and Set, as well as String.

Other warnings on empty collections

There’s also improved analysis around nulls, particularly when passing Collections as parameters.  If, for example, you have a Collection that can contain nulls and you use it when calling a method that expects a Collection that does not contain nulls, you’ll get a warning.

Warnings on nulls

See the blog post on Smarter Flow Control Analysis for more details on the analysis of empty Collections/Strings and potential null problems.

Similarly, IntelliJ IDEA 2017.2 can also detect if a number is going to be out of range and warn you about this before your program errors at run time.

Warnings on unlikely number ranges

See the section on Smarter control flow analysis: number ranges in this blog post for more details.

The Reflection API has always had its risks, which is one of the reasons Java 9 modularity (JPMS) aims to provide better encapsulation. IntelliJ IDEA now has more help for working with reflection if you do need to use it, with coding assistance and added warnings.

Warnings on reflection methods

For more information see the Reflection API section in this blog post.

More Advanced Refactoring

As well as improved warnings that can spot bugs before running the code, IntelliJ IDEA 2017.2 also helps you to refactor existing code to simplify it or to make it more readable. For example, if you have a statement that contains a number of or clauses, this can be converted into a Set.contains statement – new values can then easily be added to the Set, rather than added to a long and difficult to read condition.

Replace with Set.contains

IntelliJ IDEA continues to add refactoring and suggestions to help you use the features in Java 8.  Now, if you extract a variable in a Stream operation, the IDE can suggest this is extracted as an additional map operation rather than adding an extra variable to the lambda expression.

Extract variable as map

Previously, you could extract functional parameters, letting you change a method to accept a lambda expression as a parameter.  Now you can also extract functional variables, so you reuse can some of these lambda expressions, or as a stepping stone to a larger refactoring, or maybe simply to create more readable code.

Extract functional variable

The last Java 8 refactoring to mention is the ability to reshape your code to use method references immediately. You may previously have used something like a combination of Extract Method and the “Lambda can be replaced with method reference” inspection in order to achieve this, now you can cut out a number of steps and extract to a method reference directly.

Refactor to method reference

Better Code Completion

Code completion has also been improved in IntelliJ IDEA 2017.2, smart completion can suggest much more than before. For example, if your code uses the classic builder pattern to create an instance (rather than calling new directly), smart completion will suggest using the builder when values of that type are required.

Suggested builder

When you select this option, the caret will automatically be placed into the correct position to set more values via the builder.

For more information see the Smart completion: builder classes section of this blog post.

Smart completion is also better at suggesting appropriate method chains in certain situations – if, for example, you frequently call a chain of methods this is now suggested on the first call to smart completion, whereas it was previously only suggested on the second call.

Chained method calls suggested

This feature relies on the project being indexed, and having been built by IntelliJ IDEA (not the Gradle compiler).

See the Smart Completion blog post for more information on this feature for chained method calls.

Notable Bug Fixes

Of course, releases aren’t just about new features, they’re also about improving the quality of the tool.  Here’s a summary of the most notable bug fixes that went into 2017.2.

  • The project window sometimes didn’t close on Windows IDEA-171010
  • The editor tabs limit didn’t work properly when set to 1 (2017.2.1) IDEA-173369
  • The project window didn’t properly restored in a maximised state on Windows IDEA-96168
  • When re-importing a Maven project, the IDE didn’t respect the language level IDEA-115977
  • The active editor/tool window tab was difficult to distinguish from a non-active IDEA-131683 & IDEA-103206
  • Per-monitor HiDPI scale-factor on Windows IDEA-164566
  • Automatically adjusted font size based on the HiDPI scale factor IDEA-151754
Posted in New Features | Tagged , , , | Leave a comment

Database Tools: What’s New in IntelliJ IDEA 2017.2

If you’re following our recent news, you already know that last week we released the new massive IntelliJ IDEA update, 2017.2. However, we didn’t mention its database tools much in the release announcement, so now it’s time to catch up on that, because there’s plenty of new features in that area.

Amazon Redshift and Microsoft Azure

These cloud database providers are now supported out of the box with dedicated drivers and their specific SQL syntaxes.

If you’re already using any of them via the standard SQL Server or PostgreSQL drivers, make sure that you migrate to the new dedicated drivers after upgrading to 2017.2.

Connecting to multiple PostgreSQL databases

You don’t need to create separate Data Sources for each of PostgreSQL databases you’re using anymore. Just select as many databases as needed when configuring a single PostgreSQL Data Source.

Evaluate expression

This feature, with which you could already be familiar if you’ve worked with Java code,, is now available for SQL expressions, too and allows you get a preview of a table, a column, or a query.

It can be invoked either via Alt+clicking an expression or selecting it and pressing Ctrl+Alt+F8 (Cmd+Alt+F8 for macOS). In case the expression you need to evaluate isn’t in the code, press Alt+F8, and you will get a dialog where you can enter it and see what it produces.

Transaction control

Auto-commit option in the Console has been replaced with its more advanced version, called Transaction Control, which is made available not only in the Console toolbar, but as the Data Source configuration option as well, and in addition to controlling the transaction mode (automatic or manual), also defines the transaction isolation level.

When this option is set to Manual, the toolbar will display the Commit and Rollback buttons.

Table DDL

The DDL tab is removed from the Table Editor, and now if you want to see DDL, call Edit Source by clicking the its icon in the Database tool window toolbar, or via the Ctrl+B (Cmd+B for macOS).

By the way, DataGrip 2017.2 release that will include all these features is just around the corner, so stay tuned for updates!

The IntelliJ IDEA and DataGrip teams,
The Drive to Develop!

Posted in New Features | Tagged , , , , , | 4 Comments

Support for JUnit 5 M4 in IntelliJ IDEA 2017.2

IntelliJ IDEA provides support for the upcoming JUnit 5 release. JUnit has pretty much been the standard for Java unit testing for years, and the latest version brings the library right up to date, making use of Java 8 features and bringing in ways of testing that are familiar if you’ve been using other frameworks like Spock.  IntelliJ IDEA 2017.2 adds support for some of these features, to make the experience of testing code as painless and quick as possible.

JUnit 5 supports parameterized tests. These tests let you use the same basic structure of a test with a number of different inputs. This reduces duplication in your test code and also lets you easily add new cases without having to create entire new tests.

IntelliJ IDEA helps you with parameterized tests in a number of different ways,  Firstly, it reports “suspicious” use of both @Test and @ParameterizedTest annotations – you only need one of these in order to declare the method is a test.

Suspicious Annotations

If you press Alt + Enter on the highlighted @Test annotation, IntelliJ IDEA suggests removing this unnecessary annotation.

Remove Test Annotation suggestion

If you’re using the @ParameterizedTest annotation, you need to supply a source for the parameters, otherwise there are no tests to run.  IntelliJ IDEA warns you if you use the @ParameterizedTest annotation without an associated source (annotations like @MethodSource, @ValueSource etc.)

No sources supplied

If you’re using a source annotation, but it doesn’t match the expectations set by the  test method, you’ll also get a warning about this.  For example, if you use a @ValueSource that defines a series of ints as the input, but your test method takes a String parameter, IntelliJ IDEA will warn you that these types don’t match

Incorrect parameter types

Using a @MethodSource lets you choose a method that returns a Stream of values to use for parameters, and IntelliJ IDEA offers suggestions of suitable methods

Code completion for MethodSource

If you select a method that does not provide the right values for the test method’s arguments, you’ll get a warning about this as well. Here, we’ve given @MethodSource a method that returns a Stream of Strings, but because our test takes more than one argument we should have chosen a method that returns a Stream of Arguments.

Incorrect method source type

You’ll notice as well that when you provide a method to @MethodValue, this method is not marked as unused because IntelliJ IDEA recognises it’s used in the annotation.

No unused method

In addition to this support for parameterized tests, you also get warnings for using the @Test annotation with @RepeatedTest.

Help for repeated tests

IntelliJ IDEA 2017.2 makes it simple to use all the features in JUnit 5, so that it’s easier and quicker to write effective tests.

Posted in New Features | Tagged , , | 2 Comments

Debugger: Filtering Arrays & Collections in IntelliJ IDEA 2017.2

As usual, the newest version of IntelliJ IDEA contains updates to help you debug applications.  Given that we are working more and more with large data sets, IntelliJ IDEA 2017.2 has added the ability to filter arrays and collections in our variables or watches.

Breakpoint during debugging

In this example, I have a variable allWords that’s a list of Strings. This is a large list, so when I’m debugging my application I may want to look for specific values, or types of values, in this list.  I can do this by right-clicking on the list and selecting “filter”.

Select "filter" on a list or array

Then I can enter the criteria to filter. As you’d expect, IntelliJ IDEA offers code completion and suggestions here.

Code completion on the filter field

The debug window will now show me the subset of values in the list that match this criteria.

List filtered by start of word

If I right click on the filter, I can edit the filter to change the original criteria or add further filters

The filter can be changed

And I can easily remove the filter using the clear button.

Press clear to remove the filter

This feature also works on other Collection types, and arrays.  Here, we’ve added a watch expression that creates an array of random ints

An int array in watches

We can filter these values too

A filtered list of ints

This ability to filter the values in large collections like lists and arrays make it easier for us, as we’re debugging an application, to see which values are available, and particularly if one or more of them match some criteria we’re looking for.

Posted in New Features | Tagged , | 3 Comments