Writing Tests with JUnit 5

Trisha Gee

In this tutorial we’re going to look at features of JUnit 5 that can make it easier for us to write effective and readable automated tests. All code in this tutorial can be found in this GitHub repository.

This blog post covers the same material as the video. This provides an easy way for people to skim the content quickly if they prefer reading to watching, and to give the reader/watcher code samples and links to additional information.

Setting up Gradle for JUnit 5

This tutorial uses Gradle, for information on how to add JUnit 5 via Maven take a look at our blog and video on Migrating to JUnit 5 from JUnit 4.

Given a Gradle build file, use ⌘N (macOS) or Alt+Insert (Windows/Linux) to add a new dependency. Typing "junit" in the artifact search box should give a list of possible dependencies.

Use Tab to jump into the dependencies list and use the down arrow until org.junit.jupiter:junit-jupiter is selected. Use the right arrow to open up the version options for this dependency, and choose version 5.6.2 (the most recent production version at the time of writing).

NOTE: if you try to search for a dependency and you don’t get the results you expect (either no results, or the versions seem out of date), make sure IntelliJ IDEA has an updated Maven Repository via the settings.

You should see an icon in the top right of the Gradle build file when it has been changed. You must load the Gradle changes if you want IntelliJ IDEA to apply them.

Click on the icon, or use ⇧⌘I, or Ctrl+Shift+O on Windows and Linux, to load the changes. Once the Gradle dependency changes have been loaded, we can see the junit-jupiter dependencies in the External Libraries section of our project window.

There’s one last step we need to do for Gradle in order to correctly use JUnit 5. We need to tell Gradle to use the JUnit Platform when running the tests, by adding useJUnitPlatform() to the test section. The final build.gradle file should look like this:

plugins {
    id 'java'
}

repositories {
    mavenCentral()
}

dependencies {
    compile 'org.junit.jupiter:junit-jupiter:5.6.2'
}

test {
    useJUnitPlatform()
}

Creating and Running a Test

Now the JUnit dependency is set up correctly, we can create our first JUnit 5 test. Create an ExampleTest using the shortcut to generate code (⌘N or Alt+Insert) in the project window.

Use the same shortcut again inside the class itself to get IntelliJ IDEA to generate a new valid test method for us.

If you’re familiar with JUnit 4, you’ll see the basic test method looks exactly the same, and we can use whichever format name we usually use for our tests. The only difference with JUnit 5 is that it uses the Test annotation from the jupiter package.

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

public class ExampleTest {
    @Test
    void shouldShowSimpleAssertion() {

    }
}

JUnit 5 has an Assertions class for all the common assertions we might want to make. We can use partial completion to find the assertion that we want, for example assertEquals.

Now we have our most basic test case:

@Test
void shouldShowSimpleAssertion() {
    Assertions.assertEquals(1, 1);
}

Run it to make sure everything works. You can run with:

• Run: ⌃R or Shift+F10
• Run Context Configuration: ⌃⇧R or Ctrl+Shift+F10 (Windows/Linux) with the caret inside this method to run just this single test method. If the caret is outside the method, this will run all the tests in the class.
• You can click the green arrow in the gutter of either the test method (to run just the test) or the class name (to run all tests in the class).

When the test runs, IntelliJ IDEA shows the result in the run tool window (⌘4 or Alt+4). If the details of the passing tests are hidden, we can show all the tests that passed by clicking on the tick in the top left.

Double clicking on the test method name takes us back to that method in the code.

One thing to note for JUnit 5 tests is that the test method doesn’t need to be public in order to work. IntelliJ IDEA will let you know if the class or method can have reduced visibility and still work. Use Alt+Enter to have the IDE remove public from the class declaration, and re-run the test to make sure it works as expected.

Change the test so that it should fail:

class ExampleTest {
    @Test
    void shouldShowSimpleAssertion() {
        Assertions.assertEquals(1, 2);
    }
}

When a test fails, IntelliJ IDEA shows the failing test in amber since the test failed an assertion, rather than causing an error (which would be shown in red). We can see the expected value and the actual value side by side, and this should give us an idea of what failed and how.

In our case the cause of the problem should be quite clear since we intentionally put the wrong number in as the "actual" argument.

Configuration: Parameter Hints

Note that IntelliJ IDEA’s parameter hints feature is really helpful for assertion methods. It’s not clear from the method signature which argument is the expected result and which is the actual result. IntelliJ IDEA shows the names of the method parameters as hints, so we can see at a glance which is which.

If we decide this is too much noise in the editor, we can turn off hints for a specific method using Alt+Enter on the hint and selecting "Do not show hints for current method". We can also configure the parameter hints from the IDE preferences, in Editor -> Inlay Hints -> Java -> Parameter hints. We can turn hints on or off and configure which types of methods show hints. We can also see the Exclude list, and remove items from the Exclude list if we decide we want to see hints for this method.

Configuration: Test Runner

We can configure how IntelliJ IDEA runs our unit tests if we’re using Gradle. By default IntelliJ IDEA uses Gradle to build and run the code and tests in Gradle projects. This ensures that when we run the application or tests in the IDE, it works the same way as it would in other environments like the command line or a continuous integration environment. It also ensures that any complex build or setup logic, or code generation, is done. However we might choose to use the IntelliJ IDEA runner to run our tests. In some circumstances this might be faster than using Gradle and provide a faster feedback loop.

Disabling or ignoring tests

Quite often we want to say we don’t want a test to be run. This is common with Test Driven Development as tests will, by definition, fail when we first write them. JUnit 5 supports this with a @Disabled annotation. We can add descriptive text to state why the test is not to be run.

@Test
@Disabled("Not implemented yet")
void shouldShowSimpleAssertion() {
    Assertions.assertEquals(1, 1);
}

NOTE: tests should usually only be disabled for a short period of time, until the code they are testing is working. If a test is disabled for a long time, perhaps because we don’t know why it doesn’t work or what its expected behaviour is, it’s not adding any value to the test suite. A test like this should be removed.

Like passing tests, IntelliJ IDEA usually hides the full list of disabled tests so we can focus on just the failures. Show all disabled tests by clicking on the grey disabled icon. Click on the test name to see the reason the test was disabled.

Helpful test names for display

JUnit 5 supports a @DisplayName for the test method, so we can add a helpful descriptive name for the test.

@Test
@DisplayName("Should demonstrate a simple assertion")
void shouldShowSimpleAssertion() {
    Assertions.assertEquals(1, 1);
}

When we run the test, it’s this DisplayName that shows in the run window:

Not only does this encourage us to be descriptive, since it’s a text string and not a method name, it supports special characters, which can help readability.

IDE Tip: Live Templates

If we have a standard template for new test methods that we’d like to follow, we could change the default test method template in IntelliJ IDEA, or we could write a Live Template which helps us to create new test methods that look exactly the way we want.

Let’s create a live template to generate a new test method with a DisplayName that is initially converted into a CamelCase and applied to the method name. This encourages us to use the DisplayName annotation to write readable test descriptions, and uses them to create valid method names so the method name is also helpful. The code our Live Template should generate will look something like this:

@Test
@DisplayName("Should check all items in the list")
void shouldCheckAllItemsInTheList() {

    fail("Not implemented");
}

It’s good practice to have generated tests automatically insert a fail into the generated method – any test should fail first even if we haven’t finished writing it yet

To create this live template, open the preferences and go to Editor -> Live Templates.

Using the "+" in the top right of the scroll pane, create a new live template group called "Test". With this group selected, using the "+" again to create a new live template.

In the live template details in the bottom of the screen:

• Give the template an abbreviation of "test"
• Give it a helpful description, like "JUnit 5 test method"

The key to live templates is creating the template text. This is quite a complex template, so the text is quite advanced:

@org.junit.jupiter.api.Test
@org.junit.jupiter.api.DisplayName("$TEST_NAME$")
void $METHOD_NAME$() {
    $END$
    $BODY$
}

NOTE: Use fully qualified names (package name plus class name) for the annotations so IntelliJ IDEA knows exactly which class you want. Tick "Shorten FQ names" to have IntelliJ IDEA automatically add the correct import and use only the class name in the annotation.

I like to tick:

• Reformat according to style
• Use static import if possible
• Shorten FQ names

on my live templates, then, when the code is inserted into the class file it usually follows the same standards as the rest of the application.

You need to define the scope the live template applies to, otherwise the IDE won’t know in which sorts of files and at which time it should suggest this template. Click the "define" link next to the "No applicable contexts" warning, and select Java -> Declaration. IntelliJ IDEA will now add this to the list of suggestions when we’re in a Java class file.

Notice the variables in the template. Some of these are built in to the IDE, for example $END is where the caret will end up when the live template finishes inserting all the code. Some are values you’re going to have to define. Let’s define those now. Click on the "Edit variables" button to bring up the variables window.

Set the following values for the variables:

• TEST_NAME:
  • Expression=<leave blank>
  • Default value="Test name"
• METHOD_NAME:
  • Expression=camelCase(TEST_NAME)
  • Default value="methodName"
• BODY:
  • Expression=<leave blank>
  • Default value="org.junit.jupiter.api.Assertions.fail(\"Not implemented\");"
  • Tick "Skip if defined"

Press OK on the variables window, and OK on the preferences window.

Check the live template in the editor. Make sure the caret is inside the Java test class, but outside of an existing test method. Type test and press tab. IntelliJ IDEA should generate a test method skeleton, and the caret should be in the value of the DisplayName annotation. Type a readable test description here, and you should see the text description is turned into a valid Java camelCase method name as well. Press Enter when you’ve finished the value for DisplayName, and the caret should move to select the method name in case you want to edit it. Pressing Enter again should place the caret above the fail call.

Multiple Assertions

As we already saw, JUnit 5 supports standard assertions that may be familiar if we’ve used other testing frameworks. In the real world, we often have to check more than one thing to prove something worked the way we expected. Take a list, for example. If we want to check every item in it is correct, we might write multiple assertions to check each value.

@Test
@DisplayName("Should check all items in the list")
void shouldCheckAllItemsInTheList() {
    List<Integer> numbers = List.of(2, 3, 5, 7);
    Assertions.assertEquals(2, numbers.get(0));
    Assertions.assertEquals(3, numbers.get(1));
    Assertions.assertEquals(5, numbers.get(2));
    Assertions.assertEquals(7, numbers.get(3));
}

This works, it will certainly pass if all the items in the list are as expected. The problem comes when one of the assertions fails. Change the first assertion so it fails:

List<Integer> numbers = List.of(2, 3, 5, 7);
Assertions.assertEquals(1, numbers.get(0));

The output shows that the test fails, and why that was.

What we don’t know though is whether the other assertions passed or failed, because JUnit won’t run the assertions after the first failure. You can see that if you change all the other assertions to fail:

@Test
@DisplayName("Should check all items in the list")
void shouldCheckAllItemsInTheList() {
    List<Integer> numbers = List.of(2, 3, 5, 7);
    Assertions.assertEquals(1, numbers.get(0));
    Assertions.assertEquals(1, numbers.get(1));
    Assertions.assertEquals(1, numbers.get(2));
    Assertions.assertEquals(1, numbers.get(3));
}

NOTE: you can use column selection mode or multiple carets to easily edit all the "expected" values at once.

Run the test to see once again that only the first assertion fails, we have no idea the others are also broken.

This could be a problem – we’d go back and fix the first assertion, re-run the test, have to fix the next one, re-run the test, and so-on. This is not the fast feedback we’re looking for.

JUnit 5 supports an assertAll assertion. This will check every assertion even if one of them fails. We do this by putting all of the assertions we want to group together into the assertAll call as a series of lambda expressions.

@Test
@DisplayName("Should check all items in the list")
void shouldCheckAllItemsInTheList() {
    List<Integer> numbers = List.of(2, 3, 5, 7);

    Assertions.assertAll(() -> assertEquals(1, numbers.get(0)),
                         () -> assertEquals(1, numbers.get(1)),
                         () -> assertEquals(1, numbers.get(2)),
                         () -> assertEquals(1, numbers.get(3)));
}

Let’s keep the test with values that should fail, so we can see what happens when we run a failing assertAll:

We can see that all the assertions failed – they were all run even though the first one failed. This makes it much easier for us to see the issues and fix them all in one pass, instead of having to repeatedly re-run the test.

Make the changes to fix the test:

@Test
@DisplayName("Should check all items in the list")
void shouldCheckAllItemsInTheList() {
    List<Integer> numbers = List.of(2, 3, 5, 7);

    Assertions.assertAll(() -> assertEquals(2, numbers.get(0)),
                         () -> assertEquals(3, numbers.get(1)),
                         () -> assertEquals(5, numbers.get(2)),
                         () -> assertEquals(7, numbers.get(3)));
}

Re-running the test should show everything works:

Assumptions

Now let’s look at assumptions in JUnit 5. Later versions of JUnit 4 supported assumptions, but those of us who are used to working with older tests might not have come across this concept before. We may want to write tests that only run given some set of circumstances are true – for example, if we’re using a particular type of storage, or we’re using a particular library version. This might be more applicable to system or integration tests than unit tests. In these cases we can set an assumption at the start of the test, and the test will only be run if the criteria for that assumption are met. Let’s write a test that should only be run if we’re using an API version that’s higher than ten.

@Test
@DisplayName("Should only run the test if some criteria are met")
void shouldOnlyRunTheTestIfSomeCriteriaAreMet() {
    Assumptions.assumeTrue(Fixture.apiVersion() > 10);
    assertEquals(1, 1);
}

When we run the test, we see that this test runs and passes as expected because the Fixture is returning an API version higher than 10 (for this tutorial, Fixture.apiVersion() returns 13).

Let’s flip the check in the assumption, so the test only runs if the API version is less than 10:

Assumptions.assumeTrue(Fixture.apiVersion() < 10);
assertEquals(1, 1);

Rerun the test – it should not go green. Since our API version is higher than ten, this check returns false, the assumption is not met, and the test is not run. It shows as a disabled or ignored test:

Data Driven Tests

Earlier we saw that we can use assertAll to group a number of assertions and make sure they’re all run. This is one way of performing multiple checks. There are other cases where we might want to do the same set of checks on different sets of data. For this, we can use parameterised tests. Parameterised tests are where we can pass data into the test as parameters, and with JUnit 5 there are a number of different ways to do this (see the documentation, it’s very good). We’re going to look at the simplest approach to show how it works.

Let’s use the @ValueSource annotation to give the test method a series of individual values to test.

@ParameterizedTest
@DisplayName("Should create shapes with different numbers of sides")
@ValueSource(ints = {3, 4, 5, 8, 14})
void shouldCreateShapesWithDifferentNumbersOfSides(int expectedNumberOfSides) {

}

JUnit 5 supports many different types of array input for this annotation, let’s use an array of hardcoded ints for this test. Each one of these values will be passed into the method individually, so the test method needs a single int parameter, expectedNumberOfSides, to pass the value in.

NOTE: IntelliJ IDEA can help us with parameterised tests in JUnit 5. It lets us know that if we’re using a ValueSource annotation, we shouldn’t be using the @Test annotation but ParameterizedTest instead. We can use Alt+Enter to get IntelliJ IDEA to change any @Test annotations to @ParameterizedTest.

Inside the test method, call the constructor of Shape, passing in the number of sides given to us, and check that the Shape can give us the correct number of sides.

@ParameterizedTest
@DisplayName("Should create shapes with different numbers of sides")
@ValueSource(ints = {3, 4, 5, 8, 14})
void shouldCreateShapesWithDifferentNumbersOfSides(int expectedNumberOfSides) {
    Shape shape = new Shape(expectedNumberOfSides);
    assertEquals(expectedNumberOfSides, shape.numberOfSides());
}

Run the test. In fact, the test runs more than once. The test is run for each one of the int values we put into the ValueSource annotation.

We can change the way these individual tests are shown in the results, by creating a custom name in the ParameterizedTest annotation. For this test, show the value of the number of sides the shape is being created with by using the first parameter (expectedNumberOfSides) as the test instance name:

@ParameterizedTest(name = "{0}")

When the test is run, we see the run window shows the number of sides used as the name for each test instance:

Checking Exceptions

Parameterized tests are very helpful for testing large sets of valid data, but they’re also really useful for checking lots of invalid input with the same assertions.

Create a new test to check invalid input. Set up a new ValueSource of ints, but this time the int values will all be invalid numbers of sides for a polygon. Assume that you need to check for too few sides, and assume the code doesn’t support creating Shapes with a very large number of sides:

@ParameterizedTest
@DisplayName("Should not create shapes with invalid numbers of sides")
@ValueSource(ints = {0, 1, 2, Integer.MAX_VALUE})
void shouldNotCreateShapesWithInvalidNumbersOfSides(int expectedNumberOfSides) {

}

At this point we should be asking ourselves: "what’s the expected behaviour when the input is invalid?". If we decide that the constructor should be throwing an exception when it is passed invalid values, we can check that with an assertThrows. We tell it which Exception we expect to be thrown, and we use a lambda expression to pass in the method that we expect to throw the exception.

@ParameterizedTest(name = "{0}")
@DisplayName("Should not create shapes with invalid numbers of sides")
@ValueSource(ints = {0, 1, 2, Integer.MAX_VALUE})
void shouldNotCreateShapesWithInvalidNumbersOfSides(int expectedNumberOfSides) {

    assertThrows(IllegalArgumentException.class,
                 () -> new Shape(expectedNumberOfSides));
}

Grouping tests with @Nested

In this final section we’re going to look at one of my favourite features of JUnit 5, nested tests. Nested tests allow us to group specific types of tests together inside a larger class. There are lots of reasons we might want to do this. For example, to group together tests with similar setup or tear down, but that are not so different from other tests in the class that they need to be in their own test file.

We’re going to use this feature to group together all the tests that require a Shape that’s already been set up.

Create an inner class, and add the Nested annotation. We can also add a DisplayName to this the same way we would to a test method.

class ExampleTest {
    @Nested
    @DisplayName("When a shape has been created")
    class WhenShapeExists {

    }
}

The nested class can contain fields, of course, and we can use these to store values that all the tests inside this inner class will need. Let’s create a simple Shape to use in these tests.

class ExampleTest {
    @Nested
    @DisplayName("When a shape has been created")
    class WhenShapeExists {
        private final Shape shape = new Shape(4);

    }
}

We can even create Nested classes inside our Nested class. This can be useful to do further grouping. We’re going to use it in this example to group together Happy Path tests, the tests that check everything works as expected under normal circumstances.

class ExampleTest {
    @Nested
    @DisplayName("When a shape has been created")
    class WhenShapeExists {
        private final Shape shape = new Shape(4);

        @Nested
        @DisplayName("Should allow")
        class ShouldAllow {

        }
    }
}

Now we can create our specific tests inside our nested classes. With nested classes we’ll probably want to define a naming convention that makes sense when the test results are printed, which we’ll see in a minute. Let’s make this first happy path test a simple check that shows the Shape returns the correct number of sides. We can then create another test which checks the correct description is returned for our shape.

@Nested
@DisplayName("Should allow")
class ShouldAllow {
    @Test
    @DisplayName("seeing the number of sides")
    void seeingTheNumberOfSides() {
        assertEquals(4, shape.numberOfSides());
    }

    @Test
    @DisplayName("seeing the description")
    void seeingTheDescription() {
        assertEquals("Square", shape.description());
    }
}

(Note that I’m just showing the inner-most class in this snippet, but it’s still part of the larger class)

Now let’s create a group for tests that show what behviour is not supported, or is not expected. Let’s say that in our example two Shapes with the same number of sides are not supposed to actually be the same shape. This is the listing for the whole class:

class ExampleTest {
    @Nested
    @DisplayName("When a shape has been created")
    class WhenShapeExists {
        private final Shape shape = new Shape(4);

        @Nested
        @DisplayName("Should allow")
        class ShouldAllow {
            @Test
            @DisplayName("seeing the number of sides")
            void seeingTheNumberOfSides() {
                assertEquals(4, shape.numberOfSides());
            }

            @Test
            @DisplayName("seeing the description")
            void seeingTheDescription() {
                assertEquals("Square", shape.description());
            }
        }

        @Nested
        @DisplayName("Should not")
        class ShouldNot {
            @Test
            @DisplayName("be equal to another shape with the same number of sides")
            void beEqualToAnotherShapeWithTheSameNumberOfSides() {
                assertNotEquals(new Shape(4), shape);
            }
        }
    }
}

If we run all the tests in the class (⌃R or Shift+F10), we can see our nested tests in the test results. We can see the grouping means the results of similar tests are all grouped together. We can also see how the display name can help us to understand the grouping of the tests.

IDE Tip: Code Folding

If all of these annotations are adding too much noise to the editor, we can always collapse them by pressing on the minus in the gutter, or by using the keyboard shortcut to fold code, ⌘. or Ctrl+. – where "." is the full stop or period on the keyboard. We can hover over the collapsed annotations to see them.

Conclusion

This tutorial has just scratched the surface of the features offered by JUnit 5. To find out more, go to the JUnit 5 documentation, it covers a huge host of topics, including showing the features we’ve seen in this video in more detail. It also covers the steps to take to migrate to JUnit 5 from JUnit 4, which was also covered in blog and video.

Top shortcuts

This blog post includes some shortcuts, but many more were demonstrated in the video and not all of them were mentioned here:

• ⌘[ or Ctrl+Alt+left arrow Navigate back – makes it easy to navigate between all the places you’ve been while writing code and running tests
• ⌘⇧T or Ctrl+Shift+T Navigate between test and test subject. This is explored in the Top 5 Navigation Tips (blog and video).
• ⌃⇧J or Ctrl+Shift+J Join Lines to create compiling code.
• ⇧⌘⏎ or Ctrl+Shift+Enter Complete Statement close off the brackets and statement, and it also formats the code too.
• ⌃⇧Space or Ctrl+Shift+Space Smart completion

See also:

• Migrating from JUnit 4 to JUnit 5 – blog, with embedded video
• Unit Testing and Coverage in IntelliJ IDEA (video)
• Editor Tips & Tricks (blog and video)
• Top 15 IntelliJ IDEA Shortcuts Blog (blog and video)

Download IntelliJ IDEA

Subscribe to IntelliJ IDEA Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!