TestNG custom listeners: ITestListener


When running TestNG tests, one could want to perform some common actions – after each test has finished successfully, after each failed test, after each skipped test, or after all the tests have finished running, no matter their result. To apply such a common behavior to a group of tests, a custom listener can be created, that implements TestNG’s ITestListener interface. There are two types of methods to implement: a set of them are related to a single test’s run (onTestStart, onTestSuccess, onTestFailure, onTestSkipped, onTestFailedButWithinSuccessPercentage), the other to the whole suite’s run (onStart, onFinish). These methods, depending on their type, have a parameter passed to them: result, which is of type ITestResult – for the test run, and context, which is of type ITestContext, for the suite run. These types offer different information, for example: ITestResult can tell you when a test began to run, when it finished running, what status the test had (whether failed, passed), whereas ITestContext can tell you the list of all the passed tests, all the failed ones, and so on.

For a better understanding of what these two types can offer, the following references should be read:   http://testng.org/javadocs/org/testng/ITestListener.html and http://testng.org/javadoc/org/testng/ITestContext.html.

Implementing the custom listener

The custom listener that will be used must implement all the ITestListener interface’s methods, as depicted below:

public class SimpleTestListener implements ITestListener{

@Override
public void onTestStart(ITestResult result) {
}

@Override
public void onTestSuccess(ITestResult result) {
}

@Override
public void onTestFailure(ITestResult result) {
}

@Override
public void onTestSkipped(ITestResult result) {
}

@Override
public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
}

@Override
public void onStart(ITestContext context) {
}

@Override
public void onFinish(ITestContext context) {
}

 As a simple example, one would like for a test to print the name of it’s method, the time when it started running (in milliseconds), and the result of the run.  In this case, the listener’s implemented methods would look something like:

@Override
public void onTestStart(ITestResult result) {
        System.out.println("Test has started running:"  + result.getMethod().getMethodName() + " at:" + result.getStartMillis());
}

@Override
public void onTestSuccess(ITestResult result) {
   System.out.println("Result was: success");
}

@Override
public void onTestFailure(ITestResult result) {
   System.out.println("Result was: failure");
}

@Override
public void onTestSkipped(ITestResult result) {
   System.out.println("Test was skipped!");
}

Also, the list of failed and list of passed tests should be displayed. In this case, the listener’s implemented methods would look something like:

@Override 
public void onFinish(ITestContext context) { 
System.out.println("Passed tests: " + context.getPassedTests());                      System.out.println("Failed tests:" + context.getFailedTests()); 
}
Using the listener

Now, after the listener has been implemented, it needs to be declared so that the tests know how to use it. There are two ways of doing that: the first one is by adding a @Listeners annotation to the tests that need to behave as the listener says. The disadvantage here is that, if there are many test classes that need to use the listener, each test class must introduce the new annotation. This might be a more cumbersome job. The advantage is that if you want to run this class from your IDE (not from the command line), by using the annotation, the listener will be applied to this class. Such a test would look something like (note that the @Listeners annotation must be placed above a @Test annotation):

@Listeners(CustomTestListener.class)
@Test
public void someTest() {
...
}

However, if tests are run from an .xml configuration file (as described in https://iamalittletester.wordpress.com/2014/03/01/running-testng-tests/), a ‘listeners’ section can be added to this file,  in the ‘suite’ section, right above the ‘test’ section, so that all the tests that are run by using this file will by default use the new listener.  Note that if you take this approach, when running any test from the IDE directly (with ‘Run as TestNG test’), the listener will not be applied. It applies only when running tests by specifying this configuration file. The listeners section would look like:

<listeners>
<listener class-name="path.To.The.Listener.CustomTestListener" />
</listeners>
Advertisements

8 thoughts on “TestNG custom listeners: ITestListener”

  1. Great article!
    Can I use ITestListener to rename test methods (which run with dataprovider) at runtime, and if possible, will the method name be overriden and displayed with it’s new name at testng report?

    Thanks!
    Noam.

    Like

  2. Good Article.
    I have one doubt though: I have one scenario where Assertions were failing. I handled it with try n catch block so that next code can get executed. Now, the problem is : Test Result is showing Pass but since, assertion was failing itshould show Fail. I used ITestListener but still it is showing Pass. Can you please guide me?
    Thanks

    Like

    1. So i understand that what you are doing is something like:
      try {
      some kind of assert here }
      catch (Assertion Exception) {
      do nothing }.
      What you want is that at the same time, when an assertion fails, you want the test to fail, but also to run all the code after the assertion.
      The solution for this is simple: remove the “try-catch”. What it does is that it catches your exception, but it will never throw another one, therefore the test will be considered as passed, and this is normal. Any exception that you “catch” would be considered as a desired behavior. The test cannot fail by doing this.
      Therefore, after removing the “try – catch” block, replace your regular assertions with SoftAssertions (as per this post: https://iamalittletester.wordpress.com/2016/04/27/softassert-dont-make-your-test-fail-on-the-first-assertion-failure/). SoftAsserts will pick up any assertion failures that you received, but will not stop the test from running any subsequent code. The last step of the test will be, as per the documentation, the code: softAssert.assertAll(); This line of code will fail the test if any assertions were failing, and will also display any assertions that failed, so that you know where the issues occurred.
      Hope this helps.

      Like

  3. Thanks for giving me reply.
    I tried the solution you mentioned but it didnt work.
    Soft Assert is failing the TC as Fail but is not executing the rest of the code. It is same as earlier scenario, when, i had used ITestListener Class. Please guide me how to handle this scenario.
    Code :->
    package testcases;
    import static org.testng.Assert.assertEquals;
    import static org.testng.Assert.assertFalse;
    import static org.testng.Assert.assertTrue;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.firefox.FirefoxDriver;
    import org.testng.Assert;
    import org.testng.annotations.Listeners;
    import org.testng.annotations.Test;
    import org.testng.asserts.Assertion;
    import org.testng.asserts.SoftAssert;
    //@Listeners(utilities.TestNGListerner.class)
    public class Validations {
    private SoftAssert softAssert = new SoftAssert();
    @Test
    public void testApp(){
    System.setProperty(“webdriver.gecko.driver”, “D://Rajiv//Selenium//geckodriver.exe”);
    FirefoxDriver driver = new FirefoxDriver();
    driver.get(“http://www.gmail.com&#8221;);
    String expectedTitle=”XGmail”;
    String actualTitle=driver.getTitle();
    System.out.println(“**************A”);
    /*Issue 1= The Code was executing till the line where it was giving error and program was getting stopped. We wanted it continue execution,
    * nex line of code in our program.
    * Resolution=To handle it, we used “try{} catch (Throwable e) {}”.
    *
    * Issue 2= The Code is showing error but the Result is showing passed, we want the test result to be reported as ‘Failure’
    * to handle that, we do it and use of the interface ‘ITestListener’. But, it was also not handling it. Then i tried “softAssert” Class, but, it is not executing rest of the code after encountering exception.
    */
    softAssert.assertEquals(actualTitle, expectedTitle);
    softAssert.assertFalse(false);
    softAssert.assertTrue(true);
    softAssert.assertAll();
    System.out.println(“**********B”); // not printing this
    driver.quit(); // not quiting browser
    }
    }

    Like

    1. Move the softAssert.assertAll(); code so that it is the last line in the test method. Right after quitting the browser.

      Like

  4. Thanks a lot. It worked.
    Perfect. 10 on 10, On Target!!
    Exactly this is what i was looking for.
    Your blog is excellent and very informative.
    Planning to read other stuff also in there now.

    Liked by 1 person

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s