Tutorials123: TestNG Report Customization

What Are TestNG Reports?

TestNG Reports are the output summaries and detailed logs generated by the TestNG framework after a test suite execution. Their theoretical purpose is to provide a comprehensive, human-readable record of the quality status of the tested application and the efficiency of the test execution.

TestNG generally produces two types of reports:

1. HTML Reports

This is the most crucial report, generated as a set of static HTML files designed for easy viewing in a web browser.

(a) Holistic View: It offers a complete, consolidated view of the entire test run, encompassing all tests, classes, and methods executed within the defined Suite.

(b) Key Information: It summarizes the overall statistics:

Total Tests Run: The total number of test methods that were attempted.
Pass/Fail/Skip Count: The exact count of tests that succeeded, those that failed, and those that were skipped (often due to dependencies).
Execution Time: The total duration required to run the entire test suite.

Method Details: For failed tests, it displays the specific exception traceback (the error message and stack trace), explaining why the assertion failed.
Configuration Status: The status of all setup/teardown methods (@Before/After methods).

2. XML Reports

This report is generated in the JUnit XML format, making it ideal for integration with other development tools.

Integration with CI/CD: The primary theoretical function is to allow Continuous Integration (CI) tools (like Jenkins, GitLab CI, etc.) to understand the test results. CI tools cannot typically parse custom HTML.
Machine-Parseable Format: The XML structure follows a standardized schema, enabling other programs to reliably read, aggregate, and visualize the test data automatically.
Reporting Dashboard: This format allows TestNG results to be processed and displayed on centralized reporting dashboards, which is essential for automation in a professional environment.

TestNG Reports are automatically generated HTML/XML reports that give detailed feedback about:

Passed tests
Failed tests
Skipped tests
Execution time
Grouped tests
Exceptions and stack traces

These reports help QA teams and developers track the status of test executions and debug issues faster.

Types of TestNG Reports

By default, TestNG generates the following reports in the test-output folder after you run your tests:

Report File	Format	Description
index.html	HTML	Summary report of all suites
emailable-report.html	HTML	Email-friendly, basic summary report
testng-results.xml	XML	Machine-readable report for CI tools
testng-failed.xml	XML	Suite file with only failed tests
Default suite/Default test.html	HTML	Detailed class-level execution report

Where Are Reports Stored?

Reports are saved automatically in the `test-output` directory in your project after running your suite.

Example Java code of TestNG script to generate Reports

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestReportExample {

    @Test
    public void testPass() {
        System.out.println("This test will PASS");
        Assert.assertTrue(true);
    }

    @Test
    public void testFail() {
        System.out.println("This test will FAIL");
        Assert.assertTrue(false, "Intentionally failing the test");
    }

    @Test
    public void testSkip() {
        System.out.println("This test will be SKIPPED");
        throw new SkipException("Skipping this test intentionally");
    }
}

Create testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ReportSuite">
  <test name="ReportingTest">
    <classes>
      <class name="TestReportExample"/>
    </classes>
  </test>
</suite>

Run the Test

Right-click testng.xml in Eclipse/IntelliJ
Choose Run As > TestNG Suite
After execution, go to the test-output/ folder in your project
Open index.html or emailable-report.html in your browser

TestNG Reports

Types of TestNG Reports

Where Are Reports Stored?

Reports are saved automatically in the test-output directory in your project after running your suite.

Run the Test

Reports are saved automatically in the `test-output` directory in your project after running your suite.