One of the new features in IntelliJ IDEA 2016.2 is support for the new JUnit 5 testing framework. Almost all Java developers will have used JUnit at some point, so it’s pretty exciting to find the newest version has evolved with the times and provides a number of new features, some of which may be familiar to those who have used other frameworks.
IntelliJ IDEA supports the ability to actually run tests written for JUnit 5 – there’s no need to use the additional libraries (like the Gradle or Maven plugins for example), all you need is to include the JUnit 5 dependency. Here we’re using Gradle to include the dependency to the M2 version:
Once you’ve done this, you can start writing tests that use the new annotations:
Running these tests will give you familiar-looking results in the IntelliJ IDEA run window:
You’ll notice JUnit Jupiter is the test engine for the new JUnit tests, and that you can run both new and old tests.
At first glance, the new JUnit 5 tests are no different to JUnit 4 ones, other than the annotations being imported from a different package, and showing two types of runner on the results.
But JUnit 5 comes with some new features, such as the ability to add a readable name or description for the test, so you no longer have to rely on a method name, using @DisplayName:
You can group assertions so that all assertions are run, regardless of whether one (or more) fails, giving you better visibility over the true state of a failure:
And you can use the @Nested annotation, so you can write BDD-style tests or group the tests in a class by some dimension:
JUnit 5 also has a new way of handling expected Exceptions, by letting you assert specific Exceptions are thrown
And you can even get the thrown Exception to check for further details:
These are just some of the features available to use in JUnit 5, and IntelliJ’s support for the new framework is also evolving.
If you think JUnit 5 looks interesting, download IntelliJ IDEA 2016.2 and take a look at:
- The JUnit 5 User Guide, for reference and examples
- JUnit 5 – An Early Test Drive for a walkthrough of the new features
- The vJUG session on JUnit 5.
And, of course, you can check out more of the features available in IntelliJ IDEA 2016.2.
Getting the following error when trying to run test with only the junit-jupiter-api dependency above:
Exception in thread “main” java.lang.NoClassDefFoundError: junit/textui/ResultPrinter
Running IntelliJ IDEA 2016.2.1
Thanks for your feedback. This bug will be fixed in 2016.2.2. Learn more: https://youtrack.jetbrains.com/issue/IDEA-158667
A hacky work-around is to add JUnit 4 as a dependency as well:
dependencies {
testRuntime ‘junit:junit:4.12’
testCompile ‘org.junit.jupiter:junit-jupiter-api:5.0.0-M2’
testRuntime(“org.junit.jupiter:junit-jupiter-engine:5.0.0-M2”)
}
Is there a way to include/exclude tagged Tests?
You mean the tags described here? These can be used via Gradle, so you could run the tests in IntelliJ IDEA using the Gradle runner.
I’m also investigating if there is a way to provide these tags as arguments to the IntelliJ IDEA test runner, I’ll get back to you when I find out.
IntelliJ IDEA doesn’t provide explicit support for including/excluding tests with the @Tag annotation, but it is on the roadmap.
How is the support for running tagged tests looking right now. Is it already in the IDE?
FYI, it appears IntelliJ doesn’t correctly handle tests generated via a @TestFactory method: the “DynamicTests” run, but failures are not reported in the console; instead, the TestFactory method is listed as “Empty test suite.”
Sorry, more accurately: if the @TestFactory method ITSELF throws an exception, this is treated as an empty test suite, with no indication of the problem.
Heh, actually, that isn’t quite accurate, either. I’ll post a new comment with a correct description of the problem…
FYI, if a @TestFactory method returns a Stream, and traversing the Stream throws an exception, it appears IntelliJ doesn’t correctly handle this: the failure is not reported in the console; instead, the TestFactory method is listed as “Empty test suite.”
Thanks for reporting it. The problem has been fixed: https://youtrack.jetbrains.com/issue/IDEA-161850
I have a class with a bunch of @TestFactory methods. It runs smoothly in IntelliJ but IntelliJ don’t see @TestFactory as a test-class. (Warning: isn’t a test class).
I know it’s a minor issue but it confused me a bit.
IntelliJ IDEA 2016.2.5
Build #IC-162.2228.15, built on October 14, 2016
JRE: 1.8.0_112-b15 amd64
JVM: Java HotSpot(TM) 64-Bit Server VM by Oracle Corporation
The fix would be available in 2016.3, see https://youtrack.jetbrains.com/issue/IDEA-161850
It seems I can’t run single tests only… or @Nested classes. While the latter would be really nice, the former actually prevents me from switching to Jupiter: I often set a breakpoint in my code and then start only a single test.
I also have a Run Config using the pattern
.*Testto run only my unit- but not my integration tests (that end withIT). IDEA doesn’t find _any_ tests, when I run it with the Jupiter runner on the classpath. This pretty much forces me to stick to JUnit 4Please vote/watch for https://youtrack.jetbrains.com/issue/IDEA-164088
Anna
This was fixed by junit team but the fix is not released yet.
Anna
With 2016.3, if my Maven projects uses version M3, it has throws NoSuchMethod errors due to a conflict with IntelliJ M2 jars. If I switch the version to M2 the tests run fine.
org.junit.jupiter
junit-jupiter-api
5.0.0-M3
test
Is there a way to update 2016.3 to M3, and eventually the release version?
2016.3.1 with support for M3 will be out within a couple of hours.
When you try to run maven project with basic junit5 tests from here:
https://github.com/junit-team/junit5-samples/tree/master/junit5-maven-consumer
Inteij 2016.3.7743.44 says No tests were found :(((
With maven clean install it works fine, so it is clearly ide problem
For anyone having a similar problem: I could fix it by finding the specific JUnit milestone that is supported by my current IntelliJ version.
see: https://stackoverflow.com/questions/45040070/junit4-and-junit5-tests-not-running-in-the-same-project/45040161#45040161
Exception in thread “main” java.lang.NoSuchMethodError: org.junit.platform.commons.util.Preconditions.notNull([Ljava/lang/Object;Ljava/lang/String;)[Ljava/lang/Object;
at org.junit.platform.launcher.core.DefaultLauncher.registerTestExecutionListeners(DefaultLauncher.java:71)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:44)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Could you solve it? Have the same issue…
Same here, I replaced SNAPSHOT/M5+ dependencies with M4 in my build.gradle and it worked:
junitVintage : ‘4.12.0-M4’,
junitPlatform: ‘1.0.0-M4’,
junitJupiter : ‘5.0.0-M4’,
Adding this dep. seems to help out.
org.junit.platform
junit-platform-launcher
1.0.0-M3
Are there any plans to support for the Gradle Test Runner when using JUnit5?
I have IntelliJ 2016.3 and tried to run https://github.com/junit-team/junit5-samples/tree/r5.0.0-M3/junit5-maven-consumer in the IDE
IntelliJ test runner plugin said “No tests found”, it did not work.
Hi Ernesto. Try version 2016.3.1. It has been released yesterday and it does the trick
I am using the latest EAP and it doesn’t seem to recognize @BeforeEach or @BeforeAll. Should this be working?
A few questions. What EAP? How do you expect it to “recognize” them?
It looks like the Jupiter engine isn’t being called so methods with those annotations don’t get called before running tests. I am always using the latest EAP’s and it has never behaved like this blog where it calls out the Jupiter engine. I guess it still processes everything using the old Junit 4 facade.
Could you please share the command line used by the IDE for running tests. You can find it inside the Run tool window. This might help us better understand the problem.
Hi
It wans’t working for me either until i changed this:
import org.junit.Test;
to:
import org.junit.jupiter.api.Test;
Maybe this helps someone googling for it.
Thank you!! It did!
I am getting compilation warning
Warning:java: unknown enum constant org.junit.platform.commons.meta.API.Usage.Stable
reason: class file for org.junit.platform.commons.meta.API$Usage not found
using
IntelliJ IDEA 2017.1
Build #IC-171.3780.107, built on March 22, 2017
JRE: 1.8.0_112-release-736-b13 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 7 6.1
I am getting
Warning:java: unknown enum constant org.junit.platform.commons.meta.API.Usage.Stable
reason: class file for org.junit.platform.commons.meta.API$Usage not found
IntelliJ IDEA 2017.1
Build #IC-171.3780.107, built on March 22, 2017
JRE: 1.8.0_112-release-736-b13 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 7 6.1
The moment I read
MoodAnalyzerI said, did Trisha write this? Username checks outBtw, for those using the latest JUnit release as of today
5.0.0-M4, make sure to upgrade to your IDE to2017.1.2. https://blog.jetbrains.com/idea/2017/04/intellij-idea-2017-1-2-update-is-available/Support has just been added, if you don’t upgrade you’ll get the weird red “-” circular icon on your tests and an error in the event log that says “Failed to start: 0 passed, X not started”.
I sometime run JUnit5 test, and I can see idea print the exception:
Warning:java: unknown enumeration constant org.junit.platform.commons.meta.API.Usage.Stable reason: cannot find org.junit.platform.commons.meta.API$Usage class file( original： Warning:java: 未知的枚举常量 org.junit.platform.commons.meta.API.Usage.Stable 原因: 找不到org.junit.platform.commons.meta.API$Usage的类文件)
using
IntelliJ IDEA 2017.2
java version “1.8.0_73”
Java(TM) SE Runtime Environment (build 1.8.0_73-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.73-b02, mixed mode)
Windows 7
Currently I am testing to use JUnit 5 Test suites and run them with IntelliJ (they work when run with Gradle).
My test suite looks like:
@RunWith(JUnitPlatform.class)
@SelectClasses({LoggerExtensionTest.class, AnotherLoggerExtensionTest.class})
public class LoggerExtensionTestSuite {
}
And my IDEA is:
IntelliJ IDEA 2017.1.4
Build #IC-171.4694.23, built on June 6, 2017
JRE: 1.8.0_112-release-736-b21 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Mac OS X 10.12.5
But when I run the suite test I got:
Jun 26, 2017 12:22:10 PM org.junit.vintage.engine.discovery.DefensiveAllDefaultPossibilitiesBuilder$DefensiveAnnotatedBuilder buildRunner
WARNING: Ignoring test class using JUnitPlatform runner: LoggerExtensionTestSuite
Jun 26, 2017 12:22:10 PM org.junit.vintage.engine.discovery.DefensiveAllDefaultPossibilitiesBuilder$DefensiveAnnotatedBuilder buildRunner
WARNING: Ignoring test class using JUnitPlatform runner: LoggerExtensionTestSuite
i have read that there were some problems with that in previous versions, but now I think I have a pretty new class.
Any idea?
Please check out https://youtrack.jetbrains.com/issue/IDEA-160253
I’m trying to run maven project with basic junit5 tests from:
https://github.com/junit-team/junit5-samples/tree/master/junit5-maven-consumer
IntelliJ IDEA 2017.1.5
Build #IC-171.4694.70, built on July 4, 2017
JRE: 1.8.0_112-release-736-b21 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Linux 4.4.0-78-generic
And i get the follow error:
”
Exception in thread “main” java.lang.NoSuchMethodError: org.junit.platform.commons.util.ReflectionUtils.getDefaultClassLoader()Ljava/lang/ClassLoader;
at org.junit.platform.launcher.core.ServiceLoaderTestEngineRegistry.loadTestEngines(ServiceLoaderTestEngineRegistry.java:30)
at org.junit.platform.launcher.core.LauncherFactory.create(LauncherFactory.java:53)
at com.intellij.junit5.JUnit5IdeaTestRunner.createListeners(JUnit5IdeaTestRunner.java:39)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:49)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Process finished with exit code 1
Empty test suite.
”
With maven test in the command line it works fine
Same error for me:
IntelliJ IDEA 2017.1.5
Build #IU-171.4694.70, built on July 4, 2017
JRE: 1.8.0_112-release-736-b21 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Linux 3.19.0-32-generic
See here: http://junit.org/junit5/docs/snapshot/user-guide/#running-tests-ide-intellij-idea
I got this error because I was using the M5 release.
I’m trying to use @ParameterizedTest but it seems Intellij can’t find the annotation for some reason. I’ve chosen “Add Junit5 to the classpath” it seems that the only jupiter-api and opentest have been added. These don’t contain the @ParameterizedTest but that seems to be the case of all the jars in the //$APPLICATION_HOME_DIR$/plugins/junit/lib/ folder
Any ideas how to work around the problem? Did I miss something? It’s not really that important but it’s quite annoying.
Any help will be appreciated.
That’s a great question, I ran into a similar problem. You need to add junit-jupiter-params to your path: https://mvnrepository.com/artifact/org.junit.jupiter/junit-jupiter-params
Thanks a lot. That did the trick.
Hello,
is JUnit 5 running under IntelliJ 14?
No, cause IDEA 14 was released in 2014 and junit 5 project started in late 2015.
I’m trying to run maven project with basic junit5 tests from:
https://github.com/junit-team/junit5-samples.git
IntelliJ IDEA 2017.1
Build #IU-171.3780.170, built on March 22, 2017
JRE: 1.8.0_112-release-736-b21 amd64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Window10
And i get the follow error:
Exception in thread “main” java.lang.NoSuchMethodError: org.junit.platform.launcher.Launcher.execute(Lorg/junit/platform/launcher/LauncherDiscoveryRequest;)V
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:61)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:237)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
I update idea to 2017.2.6, it works well.
When running my junit5 tests from Intellij 2017.3 I see a lot of warnings like:
Nov 30, 2017 10:07:58 PM org.junit.platform.commons.util.ClasspathScanner logGenericFileProcessingException
WARNING: Failed to load java.lang.Class for path [/com/intellij/rt/execution/junit/FileComparisonFailure.class] during classpath scanning.
java.lang.NoClassDefFoundError: junit/framework/ComparisonFailure
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.junit.platform.commons.util.ReflectionUtils.loadClass(ReflectionUtils.java:460)
at org.junit.platform.commons.util.ClasspathScanner.processClassFileSafely(ClasspathScanner.java:141)
at org.junit.platform.commons.util.ClasspathScanner.lambda$findClassesForPath$3(ClasspathScanner.java:126)
at org.junit.platform.commons.util.ClassFileVisitor.visitFile(ClassFileVisitor.java:45)
at org.junit.platform.commons.util.ClassFileVisitor.visitFile(ClassFileVisitor.java:28)
at java.nio.file.Files.walkFileTree(Files.java:2670)
at java.nio.file.Files.walkFileTree(Files.java:2742)
at org.junit.platform.commons.util.ClasspathScanner.findClassesForPath(ClasspathScanner.java:126)
at org.junit.platform.commons.util.ClasspathScanner.findClassesForUri(ClasspathScanner.java:110)
at org.junit.platform.commons.util.ClasspathScanner.lambda$findClassesForUris$0(ClasspathScanner.java:99)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at org.junit.platform.commons.util.ClasspathScanner.findClassesForUris(ClasspathScanner.java:102)
at org.junit.platform.commons.util.ClasspathScanner.scanForClassesInPackage(ClasspathScanner.java:78)
at org.junit.platform.commons.util.ReflectionUtils.findAllClassesInPackage(ReflectionUtils.java:640)
at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.lambda$resolveSelectors$1(DiscoverySelectorResolver.java:55)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:54)
at org.junit.jupiter.engine.JupiterTestEngine.resolveDiscoveryRequest(JupiterTestEngine.java:68)
at org.junit.jupiter.engine.JupiterTestEngine.discover(JupiterTestEngine.java:61)
at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:130)
at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:117)
at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:82)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:48)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.ClassNotFoundException: junit.framework.ComparisonFailure
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
… 44 more
But the tests are green.
All these warnings are very annoying and I’m wondering if I mis-configured something.
If you are not using gradle how do you add junit as a dependency? It tried to add it in the dependencies tab of my project but no luck…
You can add libraries via IntelliJ IDEA if you’re not using Gradle or Maven to manage dependencies:
https://www.jetbrains.com/help/idea/library.html
IntelliJ IDEA Ultimate 2018.1.6
Warning for Junit5 tests: “JUnit test method contains no assertions”.
https://i.imgur.com/v7BL17Z.png
Fix: https://i.imgur.com/nRWwgOI.png
What the hell makes you write the configuration text in a image that I CAN’T copy the text from!!!!
Apologies, I agree that’s not useful at all.
Was it the dependency code that you wanted? That’s currently:
testImplementation (‘org.junit.jupiter:junit-jupiter-api:5.3.1’)
Just a small issue: the Show/Hide Ignored button in the Run tab doesn’t work when @Disabled is used together with @ParameterizedTest (the test is marked as ‘successful’ rather than ‘ignored’)