TestNG Reporter Log

  

What is Reporter.log in TestNG?

Reporter.log() is a method provided by TestNG to log messages during test execution. These logs are then displayed in the TestNG report, particularly in emailable-report.html and index.html. This is useful for debugging and tracking what happens during a test.

Key purpose of Reporter Logs

1. Persistent Logging for Reports

The primary purpose of the Reporter Log is to provide persisting, method-specific log statements that can be viewed later in the test execution reports. Unlike temporary console output, these logs become a permanent part of the test result documentation.

2. Traceability and Debugging

Reporter logs significantly enhance traceability and debugging. By strategically placing Reporter.log() calls throughout a test method, a tester can record key actions, variable values, or intermediate statuses. If a test fails, these logs provide the necessary context and detailed step-by-step information to diagnose the failure without needing to rerun the test or step through the code manually.

3. Separation from Console Output

The Reporter Log offers a distinct advantage over using standard System.out.println():

• System.out.println() messages go to the console (standard output), which is often cleared or lost in large test suites and is not included in the default TestNG HTML reports.
• Reporter.log() messages are captured by the TestNG framework and stored, ensuring they appear specifically in the "Reporter Output" section of the index.html and emailable-report.html.

4. Dual Output Capability

While its main role is reporting, the Reporter class is flexible. It allows an optional boolean parameter in its log() method to direct the message to both the report and the standard console output simultaneously. This gives the user control over where the log message is visible.

5. Log Association

TestNG is designed to associate each log entry with the specific test method or configuration method that generated it. This organization makes the reports structured and easy to read, ensuring logs from one test don't clutter the output of another.

Key Features of Reporter.log():

Feature	Description
Output location	Logs are printed in TestNG's HTML reports (emailable-report.html, index.html) and console if configured.
Syntax	Reporter.log(String message)
Overloaded method	Reporter.log(String message, boolean logToStandardOutput) where second param lets you print to the console.
Use cases	Debugging, logging checkpoints, recording data inputs/outputs, etc.

Basic Syntax:

Reporter.log("Your log message here");

Example Java Code Using Reporter.log()




import org.testng.Assert;
import org.testng.Reporter;
import org.testng.annotations.Test;

public class ReporterLogExample {

    @Test
    public void testLogin() {
        Reporter.log("Opening browser", true);
        Reporter.log("Navigating to login page", true);
        
        String actualTitle = "Login - MyApp";
        String expectedTitle = "Login - MyApp";
        
        Reporter.log("Verifying page title", true);
        Assert.assertEquals(actualTitle, expectedTitle, "Page title doesn't match");

        Reporter.log("Login test passed", true);
    }

    @Test
    public void testSearch() {
        Reporter.log("Starting search test", true);

        String keyword = "TestNG";
        Reporter.log("Searching for keyword: " + keyword, true);
        
        // Dummy output
        boolean searchSuccess = true;

        Reporter.log("Search operation completed", true);
        Assert.assertTrue(searchSuccess, "Search failed");

        Reporter.log("Search test passed", true);
    }
}

Code Explanation:

In the class, two test methods are there, in each method: Reporter.log("message", true) logs message to:

• TestNG report

• Console output

• true as the second argument ensures it prints to the console during execution.
  If omitted or set to false, the message will only appear in the report.

Output Location:

After running the test, you’ll find logs in:

Console output when true is passed that TestNG Report Files:

• test-output/emailable-report.html

• test-output/index.html

At a glance:

• Reporter.log() is a handy tool for tracking and debugging in TestNG tests.
• It helps you monitor steps, values, and flow in reports.
• Use the second parameter as true to see logs on the console during execution.

Suggested Posts:

1. Read Excel Data by Data provider
2. Annotations in TestNG
3. TestNG Asserts
4. Priority in TestNG
5. Parameterization in TestNG

TestNG Reports

  

What Are TestNG Reports?

TestNG Reports are the output summaries and detailed logs generated by the TestNG framework after a test suite execution. Their theoretical purpose is to provide a comprehensive, human-readable record of the quality status of the tested application and the efficiency of the test execution.

TestNG Reports are the output summaries and detailed logs generated by the TestNG framework after a test suite execution. Their theoretical purpose is to provide a comprehensive, human-readable record of the quality status of the tested application and the efficiency of the test execution.

TestNG generally produces two types of reports:

1. HTML Reports

This is the most crucial report, generated as a set of static HTML files designed for easy viewing in a web browser.

(a) Holistic View: It offers a complete, consolidated view of the entire test run, encompassing all tests, classes, and methods executed within the defined Suite.

(b) Key Information: It summarizes the overall statistics:

• Total Tests Run: The total number of test methods that were attempted.
• Pass/Fail/Skip Count: The exact count of tests that succeeded, those that failed, and those that were skipped (often due to dependencies).
• Execution Time: The total duration required to run the entire test suite.

(c) Detailed Breakdown: It provides drill-down links to view:

• Method Details: For failed tests, it displays the specific exception traceback (the error message and stack trace), explaining why the assertion failed.
• Configuration Status: The status of all setup/teardown methods (@Before/After methods).

2. XML Reports 

This report is generated in the JUnit XML format, making it ideal for integration with other development tools.

• Integration with CI/CD: The primary theoretical function is to allow Continuous Integration (CI) tools (like Jenkins, GitLab CI, etc.) to understand the test results. CI tools cannot typically parse custom HTML.
• Machine-Parseable Format: The XML structure follows a standardized schema, enabling other programs to reliably read, aggregate, and visualize the test data automatically.
• Reporting Dashboard: This format allows TestNG results to be processed and displayed on centralized reporting dashboards, which is essential for automation in a professional environment.

TestNG Reports are automatically generated HTML/XML reports that give detailed feedback about:

• Passed tests

• Failed tests

• Skipped tests

• Execution time

• Grouped tests

• Exceptions and stack traces

These reports help QA teams and developers track the status of test executions and debug issues faster.

Types of TestNG Reports

By default, TestNG generates the following reports in the test-output folder after you run your tests:

Report File	Format	Description
index.html	HTML	Summary report of all suites
emailable-report.html	HTML	Email-friendly, basic summary report
testng-results.xml	XML	Machine-readable report for CI tools
testng-failed.xml	XML	Suite file with only failed tests
Default suite/Default test.html	HTML	Detailed class-level execution report

Where Are Reports Stored?

Reports are saved automatically in the test-output directory in your project after running your suite.

Example Java code of TestNG script to generate Reports

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestReportExample {

    @Test
    public void testPass() {
        System.out.println("This test will PASS");
        Assert.assertTrue(true);
    }

    @Test
    public void testFail() {
        System.out.println("This test will FAIL");
        Assert.assertTrue(false, "Intentionally failing the test");
    }

    @Test
    public void testSkip() {
        System.out.println("This test will be SKIPPED");
        throw new SkipException("Skipping this test intentionally");
    }
}

 

Create testng.xml




<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="ReportSuite">
  <test name="ReportingTest">
    <classes>
      <class name="TestReportExample"/>
    </classes>
  </test>
</suite>

Run the Test

• Right-click testng.xml in Eclipse/IntelliJ
• Choose Run As > TestNG Suite
• After execution, go to the test-output/ folder in your project
• Open index.html or emailable-report.html in your browser

Suggested Posts:

1. Groups in TestNG
2. Parameterization in TestNG
3. TestNG Asserts
4. First Script in TestNG
5. Read Excel Data by Data provider