Priority in TestNG

 

What is priority in TestNG?

The TestNG Priority is a mechanism used to force a specific execution order among test methods that belong to the same TestNG class. By default, TestNG does not guarantee any particular order for running test methods within a class; they might run in alphabetical order or another unpredictable sequence. Priority allows the developer to override this default behavior.

In TestNG, the priority attribute is used to define the order of execution of test methods in a class.

By default, TestNG executes test methods in alphabetical order of method names (not the order they appear in the code). If you want to control the execution order explicitly, you can assign a priority to each @Test method.

Key Points about priority:

• The lower the priority number, the earlier the method will run.

• priority = 0 will run before priority = 1, and so on.

• If no priority is specified, TestNG assigns it a default priority of 0.

• You can assign negative values as well.

• If two or more tests have the same priority, they run in alphabetical order.

Example Java Code for Priority in TestNG

Below is the test class having multiple test methods having attribute priority in @Test annotation.

Output is shown below of the execution of test methods depends on priority values.

import org.testng.annotations.Test;

public class PriorityExample {

    @Test(priority = 1)
    public void testC() {
        System.out.println("Test C - Priority 1");
    }

    @Test(priority = 0)
    public void testA() {
        System.out.println("Test A - Priority 0");
    }

    @Test(priority = 2)
    public void testB() {
        System.out.println("Test B - Priority 2");
    }

    @Test  // No priority mentioned, default is 0
    public void testD() {
        System.out.println("Test D - Default Priority (0)");
    }
}

Output:

Test A - Priority 0
Test D - Default Priority (0)
Test C - Priority 1
Test B - Priority 2

Important points:

• testA() and testD() both have priority 0. Since testA() comes alphabetically before testD(), it runs first.

• Priorities help in defining explicit flow of tests, especially in dependent or staged testing scenarios.

• In TestNG, when multiple test methods have the same priority, TestNG does not guarantee any specific execution order among them. It means that the methods with the same priority may be executed in any order, and this order can change across different runs.

• Lower priority value = executed earlier

• Same priority value = no guaranteed order among those methods

Example Code: Same Priority Test Methods

Below in test class, first three test methods have same priority that 1, last method has priority as 0.
Below is the output shown the execution flow of all test methods. Please have a look.

import org.testng.annotations.Test;

public class SamePriorityExample {

    @Test(priority = 1)
    public void testA() {
        System.out.println("Test A - Priority 1");
    }

    @Test(priority = 1)
    public void testB() {
        System.out.println("Test B - Priority 1");
    }

    @Test(priority = 1)
    public void testC() {
        System.out.println("Test C - Priority 1");
    }

    @Test(priority = 0)
    public void testD() {
        System.out.println("Test D - Priority 0");
    }
}

Output:

Test D - Priority 0
Test A - Priority 1
Test B - Priority 1
Test C - Priority 1

Another Run Might Give:

Test D - Priority 0
Test C - Priority 1
Test A - Priority 1
Test B - Priority 1

If multiple test methods share the same priority, TestNG may run them in any order. So, do not rely on order of execution if you assign the same priority to multiple test methods.

Suggested Posts:

1. Introduction to TestNG
2. Annotations in TestNG
3. TestNG Asserts
4. First Script in TestNG
5. Parameterization in TestNG

Parameterization in TestNG

  

What is Parameterization in TestNG?

The theory of Parameterization in TestNG is a mechanism for running the same test logic multiple times with different input values (parameters). This technique eliminates the need to duplicate test code when you want to verify a feature across various scenarios, datasets, or environments. Parameterization allows you to externalize the data used by the test methods.

Parameterization in TestNG allows us to run the same test with different sets of data. It helps in data-driven testing, where test data is separated from the test logic. TestNG provides multiple ways to do this, mainly:

• @Parameters annotation (via testng.xml)

• @DataProvider annotation (method-based)

1. Using @Parameters (from XML)

This approach uses parameters defined externally in the TestNG XML configuration file, suitable for configuration-based data.

(a) Goal: To pass environment-specific or global configuration values into a test without hardcoding them.

(b) Mechanism:

• Parameters are defined using the <parameter> tag within the testng.xml file (at the Suite, Test, or Class level).
• The test method uses the @Parameters annotation to declare the name of the configuration parameter it expects to receive.

(c) The Contract: TestNG reads the XML file, finds the defined parameter value, and passes it directly to the annotated test method.

(d) Benefit: This is ideal for parameters that don't change per execution but vary per deployment environment, such as a browser name (chrome, firefox), a base URL, or a database connection type. The test method runs only once, using the configured value.

Pass parameters directly from the testng.xml file to your test methods.

Steps for Parameterization in TestNG

• Define parameters in testng.xml

• Use @Parameters in the test method

• Use @BeforeMethod or @Test to access them

Java Code for parameterization in TestNG

Below is Test class with name loginTest() having two parameters user and pass. These parameters get values from @Parameters annotations variables: username and password. these variables username and password gets values from parameter tag in testng.xml. We can see below in testng.xml file

import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

public class ParameterExample {

    @Test
    @Parameters({"username", "password"})
    public void loginTest(String user, String pass) {
        System.out.println("Username: " + user);
        System.out.println("Password: " + pass);
    }
}

testng.xml:

It contains two parameters username and password having name and values as defined below

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="LoginTest">
    <parameter name="username" value="admin" />
    <parameter name="password" value="admin123" />
    <classes>
      <class name="ParameterExample" />
    </classes>
  </test>
</suite>

2. Using @DataProvider




The most flexible and common theoretical approach uses the @DataProvider annotation.

(a) Goal: To separate the test data from the test logic.

(b) Mechanism: A dedicated method is marked with the @DataProvider annotation. This method is theoretically responsible for generating and returning multiple sets of input data (e.g., a list of usernames and passwords, or a list of numbers to test a calculator function).

(c) The Contract: The main test method (marked with @Test) declares that it will accept parameters and specifies the name of the Data Provider it needs. TestNG acts as the mediator:

• It calls the Data Provider method first.
• For every set of data returned by the provider, TestNG instantiates and runs the @Test method once.

(d) Benefit: This creates Data-Driven Testing, which is crucial for efficiency and coverage, as one test method can cover hundreds of cases simply by supplying different data.

We can pass multiple sets of data to a test method from a @DataProvider method.

Below is the Java Code for @DataProvider in TestNG:

Below, there is a getData() which is annotated with @DataProvider annotation having name as 'loginData'. 

In test method that is loginTest, We have to include dataprovider method with dataProvider attribute with name of dataprovider method.

Like below we have, @Test(dataProvider = "loginData")

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class DataProviderExample {

    @Test(dataProvider = "loginData")
    public void loginTest(String username, String password) {
        System.out.println("Testing with Username: " + username + ", Password: " + password);
    }

    @DataProvider(name = "loginData")
    public Object[][] getData() {
        return new Object[][] {
            {"admin", "admin123"},
            {"user", "user123"},
            {"guest", "guest123"}
        };
    }
}

Suggested Posts:

1. Groups in TestNG
2. Annotations in TestNG
3. TestNG Asserts
4. First Script in TestNG
5. Read Excel Data by Data provider

TestNG groups

  

What are Testng Groups?

In Testng, groups are used to categorize your test methods logically so you can run:

• A specific set of tests (like "smoke", "regression", "sanity")

• Multiple tests together, even if they are in different classes

Groups allow modular test execution, making large test suites more manageable and efficient.

TestNG Groups revolves around the concept of categorization and selective execution of test methods. They provide a flexible mechanism to label, filter, and run subsets of your total test suite based on defined criteria, without altering the structure of your test classes.The theory of TestNG Groups revolves around the concept of categorization and selective execution of test methods. They provide a flexible mechanism to label, filter, and run subsets of your total test suite based on defined criteria, without altering the structure of your test classes.

1. Categorization and Tagging Theory

At its core, a TestNG Group acts as a label or tag that can be applied to individual test methods, or even entire classes.

• The Goal: To logically organize a large, heterogeneous set of tests into manageable categories.
• The Mechanism: A test method is simply assigned one or more group names (e.g., "Smoke," "Regression," "Database," "UI"). A single test can belong to multiple groups.
• Benefit: This creates a many-to-many relationship where tests are tagged based on their function, scope, or execution time.

2. Selective Execution Theory 

The primary utility of groups is to control which tests are executed during a specific test run. This control is managed through the testng.xml configuration file.

• Inclusion: You can theoretically instruct the TestNG runner to include specific groups. For example, by specifying the "Sanity" group, the runner will only execute methods and classes tagged with "Sanity," skipping all others. This is essential for quickly verifying critical functionality.
• Exclusion: You can also instruct the runner to exclude certain groups. For instance, you might run a full regression suite but explicitly exclude the "LongRunning" group to save time, or exclude the "Database" group if the database is temporarily unavailable.

3. Execution Scope and Filtering

Groups allow for granular filtering at different levels of the TestNG hierarchy.

• Method Filtering: The most common use is to tag and filter individual test methods.
• Class/Package Filtering: A group can be applied at the class or package level, meaning every test method inside that container automatically inherits that group tag. This simplifies organization when dealing with large modules.
• Pre-defined Execution: By combining inclusion and exclusion rules in the testng.xml, a single test class file can support multiple, pre-defined test runs (e.g., a "CI-Build" run that uses the "Smoke" group, and a "Nightly" run that uses the "Regression" group).

Why Use Groups?

• Run a subset of tests without running the entire suite
• Reuse the same test in multiple groupings
• Organize tests based on functionality, feature, or type (example: @smoke, @regression, @login)

Example: TestNG Groups using Java

Step 1: Create Java Test Class with Groups

Below in Test class, we have created Tests which are annotated with @Test and created groups with groups attribute.(ex: groups = {"smoke, "login"}

We can name multiple groups in groups attribute separated with comma. 

import org.testng.annotations.Test;

public class GroupTestExample {

    @Test(groups = {"smoke", "login"})
    public void loginTest() {
        System.out.println("Smoke/Login Test: Login functionality");
    }

    @Test(groups = {"regression"})
    public void profileTest() {
        System.out.println("Regression Test: Profile page validation");
    }

    @Test(groups = {"sanity", "checkout"})
    public void paymentTest() {
        System.out.println("Sanity/Checkout Test: Payment flow");
    }

    @Test
    public void generalTest() {
        System.out.println("General Test: No group assigned");
    }
}

Step 2: Define Groups in testng.xml




Below is the testng.xml file, here only smoke group is included as shown below so tests which is in smoke group will be executed. In above java class only one test is there which has smoke group, so only that test will execute.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="GroupSuite">
  <test name="SmokeGroupOnly">
    <groups>
      <run>
        <include name="smoke"/>
      </run>
    </groups>
    <classes>
      <class name="GroupTestExample"/>
    </classes>
  </test>
</suite>

Output

Smoke/Login Test: Login functionality

We can also run multiple groups:

<include name="smoke"/>
<include name="sanity"/>

We can also exclude specific groups:

<groups>
  <run>
    <exclude name="regression"/>
  </run>
</groups>

If a test case execution depends upon successful execution of group of test cases then we can also create dependencies on test cases by using 'dependsOnGroups' attribute in @Test annotation, as shown below in code snippet.

@Test(groups = "db")
public void connectToDB() {
    System.out.println("Connected to DB");
}

@Test(groups = "sanity", dependsOnGroups = "db")
public void fetchUserData() {
    System.out.println("Fetched user data after DB connection");
}

Suggested Posts:

1. Priority in TestNG
2. TestNG Annotations
3. TestNG Asserts
4. First Script in TestNG
5. Read Excel Data by Data provider

TestNG Suite

Creating a TestNG Test Suite allows you to group and run multiple test classes, groups, or methods together using an XML configuration file. This is very useful for organizing your tests logically and controlling test execution.

What is a TestNG Test Suite?

The TestNG Suite centers on its role as the highest organizational container for an entire test execution run. It is the control hub that defines the scope, configuration, and execution strategy for all the individual tests beneath it.

1. The Role as the Top Container 

The TestNG Suite is defined by the root element in the testng.xml configuration file.

• Logical Grouping: Theoretically, the Suite groups one or more Tests (which are themselves groupings of test classes). It allows you to run a complete, predefined set of tests as a single unit. For instance, an organization might define one Suite for "Daily Smoke Tests" and another for "Full Regression Testing."

• Single Execution Cycle: A Suite defines a single, complete execution cycle. All setup actions defined at the Suite level (@BeforeSuite) run once at the very beginning, and all cleanup actions (@AfterSuite) run once at the very end, regardless of how many individual tests or classes are executed within it.

2. Configuration and Parameter Control

The Suite provides the highest level of external configuration control.

• Global Parameters: The Suite level is where you define parameters that are meant to be shared across all the tests, classes, and methods within that Suite. This is ideal for global settings like a base URL, a shared database connection string, or a target environment (e.g., "production" or "staging").

• Execution Strategy: It dictates how the tests within it will be run. For example, the Suite defines whether the tests should run sequentially (one after the other) or in parallel (concurrently). This is crucial for managing test execution time and efficiency.

3. Reporting Scope

The Suite defines the boundary for the overall reporting mechanism.

• Consolidated Report: When a Suite completes its run, it generates a single, consolidated report detailing the status (pass, fail, skip) of every Test, every class, and every method executed under its definition. This provides a holistic view of the system's quality status from that specific run.

A Test Suite is defined in an XML file generally named as 'testng.xml'. It can:

• Include multiple test classes

• Run specific methods or groups

• Control execution order

• Pass parameters to tests

Below are the steps to create a TestNG Suite

Step 1: Create Your Test Classes

LoginTest.java

Below is the LoginTest class having two tests annotated with @Test and when Test get executed, it prints a statement on console.

import org.testng.annotations.Test;

public class LoginTest {

    @Test
    public void loginWithValidUser() {
        System.out.println("Login with valid user executed.");
    }

    @Test
    public void loginWithInvalidUser() {
        System.out.println("Login with invalid user executed.");
    }
}

HomePageTest.java

Below is the HomePageTest class having two tests annotated with @Test and when Test get executed, it prints a statement on console.

import org.testng.annotations.Test;

public class HomePageTest {

    @Test
    public void checkTitle() {
        System.out.println("Home page title check executed.");
    }

    @Test
    public void checkProfileButton() {
        System.out.println("Profile button check executed.");
    }
}

Step 2: Create the testng.xml Test Suite File

Below is Testng.xml file having name of bot Test classes in class tag. each class tag tag is in classes tag which is in test tag. All the tests are in suite tag.

Tags Hierarchy of below Testng.xml file: suite > test > classes > class

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="MyTestSuite">

    <test name="Login Tests">
        <classes>
            <class name="LoginTest"/>
        </classes>
    </test>

    <test name="Home Page Tests">
        <classes>
            <class name="HomePageTest"/>
        </classes>
    </test>

</suite>

Save testng.xml file in the root of your project (or src/test/resources/ for Maven).

Step 3: Run the Test Suite

In Eclipse or IntelliJ:

• Right-click on testng.xml
• Select Run As > TestNG Suite

Suggested Posts:

1. Priority in TestNG
2. TestNG Annotations
3. TestNG Asserts
4. Parameterization in TestNG
5. Read Excel Data by Data provider