Priority in TestNG

 

What is priority in TestNG?

The TestNG Priority is a mechanism used to force a specific execution order among test methods that belong to the same TestNG class. By default, TestNG does not guarantee any particular order for running test methods within a class; they might run in alphabetical order or another unpredictable sequence. Priority allows the developer to override this default behavior.

In TestNG, the priority attribute is used to define the order of execution of test methods in a class.

By default, TestNG executes test methods in alphabetical order of method names (not the order they appear in the code). If you want to control the execution order explicitly, you can assign a priority to each @Test method.

Key Points about priority:

• The lower the priority number, the earlier the method will run.

• priority = 0 will run before priority = 1, and so on.

• If no priority is specified, TestNG assigns it a default priority of 0.

• You can assign negative values as well.

• If two or more tests have the same priority, they run in alphabetical order.

Example Java Code for Priority in TestNG

Below is the test class having multiple test methods having attribute priority in @Test annotation.

Output is shown below of the execution of test methods depends on priority values.

import org.testng.annotations.Test;

public class PriorityExample {

    @Test(priority = 1)
    public void testC() {
        System.out.println("Test C - Priority 1");
    }

    @Test(priority = 0)
    public void testA() {
        System.out.println("Test A - Priority 0");
    }

    @Test(priority = 2)
    public void testB() {
        System.out.println("Test B - Priority 2");
    }

    @Test  // No priority mentioned, default is 0
    public void testD() {
        System.out.println("Test D - Default Priority (0)");
    }
}

Output:

Test A - Priority 0
Test D - Default Priority (0)
Test C - Priority 1
Test B - Priority 2

Important points:

• testA() and testD() both have priority 0. Since testA() comes alphabetically before testD(), it runs first.

• Priorities help in defining explicit flow of tests, especially in dependent or staged testing scenarios.

• In TestNG, when multiple test methods have the same priority, TestNG does not guarantee any specific execution order among them. It means that the methods with the same priority may be executed in any order, and this order can change across different runs.

• Lower priority value = executed earlier

• Same priority value = no guaranteed order among those methods

Example Code: Same Priority Test Methods

Below in test class, first three test methods have same priority that 1, last method has priority as 0.
Below is the output shown the execution flow of all test methods. Please have a look.

import org.testng.annotations.Test;

public class SamePriorityExample {

    @Test(priority = 1)
    public void testA() {
        System.out.println("Test A - Priority 1");
    }

    @Test(priority = 1)
    public void testB() {
        System.out.println("Test B - Priority 1");
    }

    @Test(priority = 1)
    public void testC() {
        System.out.println("Test C - Priority 1");
    }

    @Test(priority = 0)
    public void testD() {
        System.out.println("Test D - Priority 0");
    }
}

Output:

Test D - Priority 0
Test A - Priority 1
Test B - Priority 1
Test C - Priority 1

Another Run Might Give:

Test D - Priority 0
Test C - Priority 1
Test A - Priority 1
Test B - Priority 1

If multiple test methods share the same priority, TestNG may run them in any order. So, do not rely on order of execution if you assign the same priority to multiple test methods.

Suggested Posts:

1. Introduction to TestNG
2. Annotations in TestNG
3. TestNG Asserts
4. First Script in TestNG
5. Parameterization in TestNG

TestNG Dependent Tests

 

The concept of Dependent Tests in TestNG is a mechanism used to establish a sequential, conditional relationship between test methods. It ensures that a specific test method (the dependent) is executed only if one or more other test methods (the dependencies) have successfully completed their run.

The Core Concept: Conditional Execution

• Goal: To prevent subsequent tests from running when a critical prerequisite test has failed.
• Mechanism: You use a specific TestNG setting (an annotation parameter) on a test method to declare which other test methods it depends on.

What Are Dependent Tests in TestNG?

TestNG allows you to create test dependencies, meaning you can specify that:

• A test method depends on another method or group
• If the depended method fails, is skipped, or is not run, the dependent method will also be skipped

Types of Dependencies in TestNG

Type	Annotation Attribute	Description
Method Dependency	dependsOnMethods	Test method runs only if other method(s) pass
Group Dependency	dependsOnGroups	Test method runs only if group(s) pass

If a test case execution depends upon successful execution of other test cases, then in this condition we can use attribute 'dependsOnMethods' in @Test annotation.

Example Java Code for method dependency

Below is the Test class contains 4 test cases, 

• Here, login() depends upon successful execution of launchApp() test case. if launchApp will execute successfully then only login() will execute otherwise, it will be skipped.
• searchproduct(), depends upon successful execution of login() test case. if login() will execute successfully then only searchproduct() will execute otherwise, it will be skipped.
• logout(), depends upon successful execution of searchproduct() test case. if searchproduct() will execute successfully then only logout() will execute otherwise, it will be skipped.

import org.testng.annotations.Test;

public class DependentTestExample {

    @Test
    public void launchApp() {
        System.out.println("App launched successfully.");
    }

    @Test(dependsOnMethods = "launchApp")
    public void login() {
        System.out.println("Logged in successfully.");
    }

    @Test(dependsOnMethods = {"login"})
    public void searchProduct() {
        System.out.println("Product searched successfully.");
    }

    @Test(dependsOnMethods = {"searchProduct"})
    public void logout() {
        System.out.println("Logged out successfully.");
    }
}

Output

App launched successfully.
Logged in successfully.
Product searched successfully.
Logged out successfully.

Note: If a Test Fails then searchProduct() and logout() will be skipped.

Example Java Code for group dependency

Below there are two test cases comes in a group that is database. If all the test cases present in group database will be executed successfully then only generateReport() will execute.

import org.testng.annotations.Test;

public class GroupDependencyExample {

    @Test(groups = "database")
    public void connectToDB() {
        System.out.println("Database connected.");
    }

    @Test(groups = "database")
    public void fetchData() {
        System.out.println("Data fetched from DB.");
    }

    @Test(dependsOnGroups = "database")
    public void generateReport() {
        System.out.println("Report generated using DB data.");
    }
}

Optional Attributes

Attribute	Description
alwaysRun = true	Run test even if dependency fails
enabled = false	Disable the test
dependsOnMethods	Declare method dependencies
dependsOnGroups	Declare group dependencies

Example of attribute: alwaysRun




alwaysRun attribute always executes test cases. Below sendEmail() testcase has attribute alwaysRun, which means even if dependent test case fails then also this testcase sendEmail() will execute.

@Test(dependsOnMethods = "login", alwaysRun = true)
public void sendEmail() {
    System.out.println("Email sent (even if login fails).");
}

Suggested Posts:

1. Groups in TestNG
2. TestNG Annotations
3. TestNG Asserts
4. First Script in TestNG
5. Read Excel Data by Data provider

TestNG groups

  

What are Testng Groups?

In Testng, groups are used to categorize your test methods logically so you can run:

• A specific set of tests (like "smoke", "regression", "sanity")

• Multiple tests together, even if they are in different classes

Groups allow modular test execution, making large test suites more manageable and efficient.

TestNG Groups revolves around the concept of categorization and selective execution of test methods. They provide a flexible mechanism to label, filter, and run subsets of your total test suite based on defined criteria, without altering the structure of your test classes.The theory of TestNG Groups revolves around the concept of categorization and selective execution of test methods. They provide a flexible mechanism to label, filter, and run subsets of your total test suite based on defined criteria, without altering the structure of your test classes.

1. Categorization and Tagging Theory

At its core, a TestNG Group acts as a label or tag that can be applied to individual test methods, or even entire classes.

• The Goal: To logically organize a large, heterogeneous set of tests into manageable categories.
• The Mechanism: A test method is simply assigned one or more group names (e.g., "Smoke," "Regression," "Database," "UI"). A single test can belong to multiple groups.
• Benefit: This creates a many-to-many relationship where tests are tagged based on their function, scope, or execution time.

2. Selective Execution Theory 

The primary utility of groups is to control which tests are executed during a specific test run. This control is managed through the testng.xml configuration file.

• Inclusion: You can theoretically instruct the TestNG runner to include specific groups. For example, by specifying the "Sanity" group, the runner will only execute methods and classes tagged with "Sanity," skipping all others. This is essential for quickly verifying critical functionality.
• Exclusion: You can also instruct the runner to exclude certain groups. For instance, you might run a full regression suite but explicitly exclude the "LongRunning" group to save time, or exclude the "Database" group if the database is temporarily unavailable.

3. Execution Scope and Filtering

Groups allow for granular filtering at different levels of the TestNG hierarchy.

• Method Filtering: The most common use is to tag and filter individual test methods.
• Class/Package Filtering: A group can be applied at the class or package level, meaning every test method inside that container automatically inherits that group tag. This simplifies organization when dealing with large modules.
• Pre-defined Execution: By combining inclusion and exclusion rules in the testng.xml, a single test class file can support multiple, pre-defined test runs (e.g., a "CI-Build" run that uses the "Smoke" group, and a "Nightly" run that uses the "Regression" group).

Why Use Groups?

• Run a subset of tests without running the entire suite
• Reuse the same test in multiple groupings
• Organize tests based on functionality, feature, or type (example: @smoke, @regression, @login)

Example: TestNG Groups using Java

Step 1: Create Java Test Class with Groups

Below in Test class, we have created Tests which are annotated with @Test and created groups with groups attribute.(ex: groups = {"smoke, "login"}

We can name multiple groups in groups attribute separated with comma. 

import org.testng.annotations.Test;

public class GroupTestExample {

    @Test(groups = {"smoke", "login"})
    public void loginTest() {
        System.out.println("Smoke/Login Test: Login functionality");
    }

    @Test(groups = {"regression"})
    public void profileTest() {
        System.out.println("Regression Test: Profile page validation");
    }

    @Test(groups = {"sanity", "checkout"})
    public void paymentTest() {
        System.out.println("Sanity/Checkout Test: Payment flow");
    }

    @Test
    public void generalTest() {
        System.out.println("General Test: No group assigned");
    }
}

Step 2: Define Groups in testng.xml




Below is the testng.xml file, here only smoke group is included as shown below so tests which is in smoke group will be executed. In above java class only one test is there which has smoke group, so only that test will execute.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="GroupSuite">
  <test name="SmokeGroupOnly">
    <groups>
      <run>
        <include name="smoke"/>
      </run>
    </groups>
    <classes>
      <class name="GroupTestExample"/>
    </classes>
  </test>
</suite>

Output

Smoke/Login Test: Login functionality

We can also run multiple groups:

<include name="smoke"/>
<include name="sanity"/>

We can also exclude specific groups:

<groups>
  <run>
    <exclude name="regression"/>
  </run>
</groups>

If a test case execution depends upon successful execution of group of test cases then we can also create dependencies on test cases by using 'dependsOnGroups' attribute in @Test annotation, as shown below in code snippet.

@Test(groups = "db")
public void connectToDB() {
    System.out.println("Connected to DB");
}

@Test(groups = "sanity", dependsOnGroups = "db")
public void fetchUserData() {
    System.out.println("Fetched user data after DB connection");
}

Suggested Posts:

1. Priority in TestNG
2. TestNG Annotations
3. TestNG Asserts
4. First Script in TestNG
5. Read Excel Data by Data provider

TestNG Suite

Creating a TestNG Test Suite allows you to group and run multiple test classes, groups, or methods together using an XML configuration file. This is very useful for organizing your tests logically and controlling test execution.

What is a TestNG Test Suite?

The TestNG Suite centers on its role as the highest organizational container for an entire test execution run. It is the control hub that defines the scope, configuration, and execution strategy for all the individual tests beneath it.

1. The Role as the Top Container 

The TestNG Suite is defined by the root element in the testng.xml configuration file.

• Logical Grouping: Theoretically, the Suite groups one or more Tests (which are themselves groupings of test classes). It allows you to run a complete, predefined set of tests as a single unit. For instance, an organization might define one Suite for "Daily Smoke Tests" and another for "Full Regression Testing."

• Single Execution Cycle: A Suite defines a single, complete execution cycle. All setup actions defined at the Suite level (@BeforeSuite) run once at the very beginning, and all cleanup actions (@AfterSuite) run once at the very end, regardless of how many individual tests or classes are executed within it.

2. Configuration and Parameter Control

The Suite provides the highest level of external configuration control.

• Global Parameters: The Suite level is where you define parameters that are meant to be shared across all the tests, classes, and methods within that Suite. This is ideal for global settings like a base URL, a shared database connection string, or a target environment (e.g., "production" or "staging").

• Execution Strategy: It dictates how the tests within it will be run. For example, the Suite defines whether the tests should run sequentially (one after the other) or in parallel (concurrently). This is crucial for managing test execution time and efficiency.

3. Reporting Scope

The Suite defines the boundary for the overall reporting mechanism.

• Consolidated Report: When a Suite completes its run, it generates a single, consolidated report detailing the status (pass, fail, skip) of every Test, every class, and every method executed under its definition. This provides a holistic view of the system's quality status from that specific run.

A Test Suite is defined in an XML file generally named as 'testng.xml'. It can:

• Include multiple test classes

• Run specific methods or groups

• Control execution order

• Pass parameters to tests

Below are the steps to create a TestNG Suite

Step 1: Create Your Test Classes

LoginTest.java

Below is the LoginTest class having two tests annotated with @Test and when Test get executed, it prints a statement on console.

import org.testng.annotations.Test;

public class LoginTest {

    @Test
    public void loginWithValidUser() {
        System.out.println("Login with valid user executed.");
    }

    @Test
    public void loginWithInvalidUser() {
        System.out.println("Login with invalid user executed.");
    }
}

HomePageTest.java

Below is the HomePageTest class having two tests annotated with @Test and when Test get executed, it prints a statement on console.

import org.testng.annotations.Test;

public class HomePageTest {

    @Test
    public void checkTitle() {
        System.out.println("Home page title check executed.");
    }

    @Test
    public void checkProfileButton() {
        System.out.println("Profile button check executed.");
    }
}

Step 2: Create the testng.xml Test Suite File

Below is Testng.xml file having name of bot Test classes in class tag. each class tag tag is in classes tag which is in test tag. All the tests are in suite tag.

Tags Hierarchy of below Testng.xml file: suite > test > classes > class

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="MyTestSuite">

    <test name="Login Tests">
        <classes>
            <class name="LoginTest"/>
        </classes>
    </test>

    <test name="Home Page Tests">
        <classes>
            <class name="HomePageTest"/>
        </classes>
    </test>

</suite>

Save testng.xml file in the root of your project (or src/test/resources/ for Maven).

Step 3: Run the Test Suite

In Eclipse or IntelliJ:

• Right-click on testng.xml
• Select Run As > TestNG Suite

Suggested Posts:

1. Priority in TestNG
2. TestNG Annotations
3. TestNG Asserts
4. Parameterization in TestNG
5. Read Excel Data by Data provider