Easy way of running the same junit test over and over

Question

Like the title says  I m looking for some simple way to run JUnit 4 x tests several times in a row automatically using Eclipse   An example would be running the same test 10 times in a row and reporting back the result   We already have a complex way of doing this but I m looking for a simple way of doing it so that I can be sorta sure that the flaky test I ve been trying to fix stays fixed   An ideal solution would be an Eclipse plugin setting feature that I am unaware of

User · Answer

I build a module that allows do this kind of tests. But it is focused not only in repeat. But in guarantee that some piece of code is Thread safe.

https://github.com/anderson-marques/concurrent-testing

Maven dependency:

<dependency>
    <groupId>org.lite</groupId>
    <artifactId>concurrent-testing</artifactId>
    <version>1.0.0</version>
</dependency>

Example of use:

package org.lite.concurrent.testing;

import org.junit.Assert;
import org.junit.Rule;
import org.junit.Test;
import ConcurrentTest;
import ConcurrentTestsRule;

/**
 * Concurrent tests examples
 */
public class ExampleTest {

    /**
     * Create a new TestRule that will be applied to all tests
     */
    @Rule
    public ConcurrentTestsRule ct = ConcurrentTestsRule.silentTests();

    /**
     * Tests using 10 threads and make 20 requests. This means until 10 simultaneous requests.
     */
    @Test
    @ConcurrentTest(requests = 20, threads = 10)
    public void testConcurrentExecutionSuccess(){
        Assert.assertTrue(true);
    }

    /**
     * Tests using 10 threads and make 20 requests. This means until 10 simultaneous requests.
     */
    @Test
    @ConcurrentTest(requests = 200, threads = 10, timeoutMillis = 100)
    public void testConcurrentExecutionSuccessWaitOnly100Millissecond(){
    }

    @Test(expected = RuntimeException.class)
    @ConcurrentTest(requests = 3)
    public void testConcurrentExecutionFail(){
        throw new RuntimeException("Fail");
    }
}

This is a open source project. Feel free to improve.

User · Answer

You could run your JUnit test from a main method and repeat it so many times you need   package tests   import static org junit Assert     import org junit Test  import org junit runner Result   public class RepeatedTest         Test     public void test             fail  Not yet implemented               public static void main String args               boolean runForever   true           while  runForever                Result result   org junit runner JUnitCore runClasses RepeatedTest class                if  result getFailureCount    gt  0                    runForever   false                   Do something with the result object

User · Answer

I ve found that Spring s repeat annotation is useful for that kind of thing         Repeat value   10    Latest  Spring Framework 4 3 11 RELEASE API  doc    org springframework test annotation Unit Testing in Spring

User · Answer

In JUnit 5 it is much simpler by using  RepeatedTest   RepeatedTest is used to signal that the annotated method is a test template method that should be repeated a specified number of times        RepeatedTest 3     Runs N Times     public void test Prime            More info on www baeldung com

User · Answer

With JUnit 5 I was able to solve this using the  RepeatedTest annotation    RepeatedTest 10  public void testMyCode           your test code goes here     Note that  Test annotation shouldn t be used along with  RepeatedTest

User · Answer

There s an Intermittent annotation in the tempus-fugit library which works with JUnit 4 7 s  Rule to repeat a test several times or with  RunWith   For example    RunWith IntermittentTestRunner class  public class IntermittentTestRunnerTest       private static int testCounter   0       Test     Intermittent repition   99     public void annotatedTest           testCounter             After the test is run  with the IntermittentTestRunner in the  RunWith   testCounter would be equal to 99

User · Answer

Inspired by the following resources    blog post this solution commented version   Example  Create and use a  Repeat annotation as follows   public class MyTestClass         Rule     public RepeatRule repeatRule   new RepeatRule          Test      Repeat 10      public void testMyCode               your test code goes here           Repeat java  import static java lang annotation ElementType ANNOTATION TYPE  import static java lang annotation ElementType METHOD  import java lang annotation Retention  import java lang annotation RetentionPolicy  import java lang annotation Target    Retention  RetentionPolicy RUNTIME    Target   METHOD  ANNOTATION TYPE    public  interface Repeat       int value   default 1      RepeatRule java  import org junit rules TestRule  import org junit runner Description  import org junit runners model Statement   public class RepeatRule implements TestRule        private static class RepeatStatement extends Statement           private final Statement statement          private final int repeat               public RepeatStatement Statement statement  int repeat                this statement   statement              this repeat   repeat                      Override         public void evaluate   throws Throwable               for  int i   0  i  lt  repeat  i                      statement evaluate                                         Override     public Statement apply Statement statement  Description description            Statement result   statement          Repeat repeat   description getAnnotation Repeat class           if  repeat    null                int times   repeat value                result   new RepeatStatement statement  times                     return result            PowerMock  Using this solution with  RunWith PowerMockRunner class   requires updating to Powermock 1 6 5  which includes a patch

User · Answer

With IntelliJ  you can do this from the test configuration  Once you open this window  you can choose to run the test any number of times you want      when you run the test  intellij will execute all tests you have selected for the number of times you specified   Example running 624 tests 10 times

User · Answer

The easiest  as in least amount of new code required  way to do this is to run the test as a parametrized test  annotate with an  RunWith Parameterized class  and add a method to provide 10 empty parameters   That way the framework will run the test 10 times   This test would need to be the only test in the class  or better put all test methods should need to be run 10 times in the class   Here is an example    RunWith Parameterized class  public class RunTenTimes         Parameterized Parameters     public static Object     data             return new Object 10  0              public RunTenTimes                 Test     public void runsTenTimes             System out println  run              With the above  it is possible to even do it with a parameter-less constructor  but I m not sure if the framework authors intended that  or if that will break in the future   If you are implementing your own runner  then you could have the runner run the test 10 times  If you are using a third party runner  then with 4 7  you can use the new  Rule annotation and implement the MethodRule interface so that it takes the statement and executes it 10 times in a for loop  The current disadvantage of this approach is that  Before and  After get run only once  This will likely change in the next version of JUnit  the  Before will run after the  Rule   but regardless you will be acting on the same instance of the object  something that isn t true of the Parameterized runner   This assumes that whatever runner you are running the class with correctly recognizes the  Rule annotations  That is only the case if it is delegating to the JUnit runners   If you are running with a custom runner that does not recognize the  Rule annotation  then you are really stuck with having to write your own runner that delegates appropriately to that Runner and runs it 10 times   Note that there are other ways to potentially solve this  such as the Theories runner  but they all require a runner  Unfortunately JUnit does not currently support layers of runners  That is a runner that chains other runners

User · Answer

Anything wrong with    Test void itWorks            stuff     Test void itWorksRepeatably         for  int i   0  i  lt  10  i              itWorks              Unlike the case where you are testing each of an array of values  you don t particularly care which run failed    No need to do in configuration or annotation what you can do in code

User · Answer

This is essentially the answer that Yishai provided above  re-written in Kotlin     RunWith Parameterized  class  class MyTest        companion object            private const val numberOfTests   200           JvmStatic          Parameterized Parameters         fun data    Array lt Array lt Any  gt  gt    Array numberOfTests    arrayOfNulls lt Any  gt  0                Test     fun testSomething

User · Answer

This works much easier for me    public class RepeatTests extends TestCase        public static Test suite             TestSuite suite   new TestSuite RepeatTests class getName              for  int i   0  i  lt  10  i                            suite addTestSuite YourTest class                                   return suite

[java] Easy way of running the same junit test over and over?

Examples related to java

Examples related to eclipse

Examples related to junit