[testing] Order of execution of tests in TestNG

How to customize the order of execution of tests in TestNG?

For example:

public class Test1 {
  @Test
  public void test1() {
      System.out.println("test1");
  }

  @Test
  public void test2() {
      System.out.println("test2");
  }

  @Test
  public void test3() {
      System.out.println("test3");
  }
}

In the above suite, the order of execution of tests is arbitrary. For one execution the output may be:

test1
test3
test2

How do I execute the tests in the order in which they've been written?

There are ways of executing tests in a given order. Normally though, tests have to be repeatable and independent to guarantee it is testing only the desired functionality and is not dependent on side effects of code outside of what is being tested.

So, to answer your question, you'll need to provide more information such as WHY it is important to run tests in a specific order.

Use this:

public class TestNG
{
        @BeforeTest
        public void setUp() 
        {
                   /*--Initialize broowsers--*/

        }

        @Test(priority=0)
        public void Login() 
        {

        }

        @Test(priority=2)
        public void Logout() 
        {

        }

        @AfterTest
        public void tearDown() 
        {
                //--Close driver--//

        }

}

Usually TestNG provides number of annotations, We can use @BeforeSuite, @BeforeTest, @BeforeClass for initializing browsers/setup.

We can assign priority if you have written number of test cases in your script and want to execute as per assigned priority then use: @Test(priority=0) starting from 0,1,2,3....

Meanwhile we can group number of test cases and execute it by grouping. for that we will use @Test(Groups='Regression')

At the end like closing the browsers we can use @AfterTest, @AfterSuite, @AfterClass annotations.

Piggy backing off of user1927494's answer, In case you want to run a single test before all others, you can do this:

@Test()
public void testOrderDoesntMatter_1() {
}

@Test(priority=-1)
public void testToRunFirst() {
}

@Test()
public void testOrderDoesntMatter_2() {
}

use: preserve-order="true" enabled="true" that would run test cases in the manner in which you have written.

<suite name="Sanity" verbose="1" parallel="" thread-count="">   
<test name="Automation" preserve-order="true"  enabled="true">
        <listeners>
            <listener class-name="com.yourtest.testNgListner.RetryListener" />
        </listeners>
        <parameter name="BrowserName" value="chrome" />
        <classes>
            <class name="com.yourtest.Suites.InitilizeClass" />
            <class name="com.yourtest.Suites.SurveyTestCases" />
            <methods>
                <include name="valid_Login" />
                <include name="verifyManageSurveyPage" />
                <include name="verifySurveyDesignerPage" />
                <include name="cloneAndDeleteSurvey" />
                <include name="createAndDelete_Responses" />
                <include name="previewSurvey" />
                <include name="verifySurveyLink" />
                <include name="verifySurveyResponses" />
                <include name="verifySurveyReports" />
            </methods>
        </classes>
    </test>
</suite>

In case you happen to use additional stuff like dependsOnMethods, you may want to define the entire @Test flow in your testng.xml file. AFAIK, the order defined in your suite XML file (testng.xml) will override all other ordering strategies.

This will work.

@Test(priority=1)
public void Test1() {

}

@Test(priority=2)
public void Test2() {

}

@Test(priority=3)
public void Test3() {

}

priority encourages execution order but does not guarantee the previous priority level has completed. test3 could start before test2 completes. If a guarantee is needed, then declare a dependency.

Unlike the solutions which declare dependencies, tests which use priority will execute even if one test fails. This problem with dependencies can be worked around with @Test(...alwaysRun = true...) according to documentation.

@Test(dependsOnMethods="someBloodyMethod")

To address specific scenario in question:

@Test
public void Test1() {

}

@Test (dependsOnMethods={"Test1"})
public void Test2() {

}

@Test (dependsOnMethods={"Test2"})
public void Test3() {

}

If I understand your question correctly in that you want to run tests in a specified order, TestNG IMethodInterceptor can be used. Take a look at http://beust.com/weblog2/archives/000479.html on how to leverage them.

If you want run some preinitialization, take a look at IHookable http://testng.org/javadoc/org/testng/IHookable.html and associated thread http://groups.google.com/group/testng-users/browse_thread/thread/42596505990e8484/3923db2f127a9a9c?lnk=gst&q=IHookable#3923db2f127a9a9c

An answer with an important explanation:

There are two parameters of "TestNG" who are supposed to determine the order of execution the tests:

@Test(dependsOnGroups= "someGroup")

And:

@Test(dependsOnMethods= "someMethod")

In both cases these functions will depend on the method or group,

But the differences:

In this case:

@Test(dependsOnGroups= "someGroup")

The method will be dependent on the whole group, so it is not necessarily that immediately after the execution of the dependent function, this method will also be executed, but it may occur later in the run and even after other tests run.

It is important to note that in case and there is more than one use within the same set of tests in this parameter, this is a safe recipe for problems, because the dependent methods of the entire set of tests will run first and only then the methods that depend on them.

However, in this case:

@Test(dependsOnMethods= "someMethod")

Even if this parameter is used more than once within the same set of tests, the dependent method will still be executed after the dependent method is executed immediately.

Hope it's clearly and help.

In TestNG, you use dependsOnMethods and/or dependsOnGroups:

@Test(groups = "a")
public void f1() {}

@Test(groups = "a")
public void f2() {}

@Test(dependsOnGroups = "a")
public void g() {}

In this case, g() will only run after f1() and f2() have completed and succeeded.

You will find a lot of examples in the documentation: http://testng.org/doc/documentation-main.html#test-groups

The ordering of methods in the class file is unpredictable, so you need to either use dependencies or include your methods explicitly in XML.

By default, TestNG will run your tests in the order they are found in the XML file. If you want the classes and methods listed in this file to be run in an unpredictible order, set the preserve-order attribute to false

If you don't want to use the @Test(priority = ) option in TestNG, you can make use of the javaassist library and TestNG's IMethodInterceptor to prioritize the tests according to the order by which the test methods are defined in the test class. This is based on the solution provided here.

Add this listener to your test class:

package cs.jacob.listeners;

import java.util.Arrays;
import java.util.Comparator;
import java.util.List;

import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;
import javassist.NotFoundException;

import org.testng.IMethodInstance;
import org.testng.IMethodInterceptor;
import org.testng.ITestContext;

public class PriorityInterceptor implements IMethodInterceptor {
    public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {

    Comparator<IMethodInstance> comparator = new Comparator<IMethodInstance>() {
        private int getLineNo(IMethodInstance mi) {
        int result = 0;

        String methodName = mi.getMethod().getConstructorOrMethod().getMethod().getName();
        String className  = mi.getMethod().getConstructorOrMethod().getDeclaringClass().getCanonicalName();
        ClassPool pool    = ClassPool.getDefault();

        try {
            CtClass cc        = pool.get(className);
            CtMethod ctMethod = cc.getDeclaredMethod(methodName);
            result            = ctMethod.getMethodInfo().getLineNumber(0);
        } catch (NotFoundException e) {
            e.printStackTrace();
        }

        return result;
        }

        public int compare(IMethodInstance m1, IMethodInstance m2) {
        return getLineNo(m1) - getLineNo(m2);
        }
    };

    IMethodInstance[] array = methods.toArray(new IMethodInstance[methods.size()]);
    Arrays.sort(array, comparator);
    return Arrays.asList(array);
    }
}

This basically finds out the line numbers of the methods and sorts them by ascending order of their line number, i.e. the order by which they are defined in the class.

By using priority paramenter for @Test we can control the order of test execution.

By Specifying test methods to be executed in testNg.xml we can execute the test cases in desired order

<suite>_x000D_
<test name="selenium1">_x000D_
   <classes>_x000D_
   <class name="com.test.SeleniumTest" >_x000D_
       <methods><include name="methodB"></include>_x000D_
           <include name="methodA"></include>_x000D_
       </methods>    _x000D_
    </class>_x000D_
  </classes>_x000D_
 _x000D_
 </test>_x000D_
</suite>

I've faced the same issue, the possible reason is due to parallel execution of testng and the solution is to add Priority option or simply update preserve-order="true" in your testng.xml.

<test name="Firefox Test" preserve-order="true">

Tests like unit tests? What for? Tests HAVE to be independant, otherwise.... you can not run a test individually. If they are independent, why even interfere? Plus - what is an "order" if you run them in multiple threads on multiple cores?