Running TestNG tests

Running TestNG tests can be done in two ways: either directly from the IDE (by selecting the desired tests and choosing to ‘Run TestNG tests’) or from the command line. The latter option is very useful when trying to run only a selection of all the tests, that might spread across different classes or packages, or to run tests that belong to certain groups. To do this, you need to create some .xml files that will select the tests that need to run and/or exclude tests not to run, add a configuration in the Maven profile created for running the tests, and run the proper Maven command.

Creating the testng.xml files

Creating the .xml files for running tests is quite easy, and it has a few sections that need to be filled in:

• The first section simply declares a name for the test suite to run. It is not really used anywhere, it is just for convention.

 <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="SuiteName" verbose="1" >  
    <test name="TestName" >

• If tests to be run are selected based on the groups that they are assigned (@TestNG annotation group attributes, that is), the following section will declare how to handle these groups. There are two ways of selecting tests based on their groups: either to include specific groups, or exclude them. In the section below this one, the location in which the tests to be run is specified. If the <include> section is declared for selecting groups, all the tests in the specified location, having the included group, will be run. Any test outside of the declared location, or any tests in the declared location that do not have the declared group attached to them, will not be run. Note that a test can have many groups attached to it. If one of those groups is the one declared in the <include> section, the test will be run. On the other hand, if the <exclude> section is declared (without an <include> section), from the location specified in the section below, only those tests that do not have the specified group assigned to them will be run. If both an <include> and <exclude> tags are declared, the situation is more interesting: all the tests that have the ‘included’ group will run; all the tests that have the ‘excluded’ group will not be run; EXCEPT – if a test has a group that is considered included, and also a group that is considered excluded, the test will be run, since the <include> tag takes precedence. Declaring this section is done as follows:

<groups>
  <run>
     <include name = "includedGroupName" />
     <exclude name = "excludedGroupName" />
  </run>
</groups>

• Declaring the location from which to select tests to be run can be done on a package, class or method level. To select a package, the following syntax is to be used:

<packages>
    <package name = "packageName" />
</packages>

• To select a class, the following syntax is to be used:

<classes>
    <class name = "className" />
</classes>

• To select only some methods, the syntax is:

 <classes>
    <class name = "className" />
       <methods>
          <include name = "methodName" />
       </methods>
    </class>
</classes>

Examples

Select all the tests in a package (including those in sub-packages):

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="First suite" verbose="1" > <test name = "allTestsInAPackage" > <packages> <package name = "whole.path.to.package.* /> </packages> </test> </suite>

Select all the tests in a class:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="Second suite" verbose="1" > <test name = "allTestsInAClass" > <classes> <class name="whole.path.to.package.className /> </classes> </test> </suite>

Select a set of test methods from a class:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="Third suite" verbose="1" > <test name = "aFewTestsFromAClass" > <classes> <class name="whole.path.to.package.className > <methods> <include name = "firstMethod" /> <include name = "secondMethod" /> <include name = "thirdMethod" /> </methods> </class> </classes> </test> </suite>

Select all tests in a package that have the declared group attached:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="Fourth suite" verbose="1" > <test name = "includedGroupsInAPackage" > <groups> <run> <include name = "includedGroup" /> </run> </groups> <packages> <package name = "whole.path.to.package.* /> </packages> </test> </suite>

Select all tests in a package that do not have the declared group attached:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" > <suite name="Fifth suite" verbose="1" > <test name = "excludedGroupsInAPackage" > <groups> <run> <exclude name = "excludedGroup" /> </run> </groups> <packages> <package name = "whole.path.to.package.* /> </packages> </test> </suite>

 Declaring the .xml file location in the Maven profile

After creating a Maven profile for running tests (as per https://imalittletester.com/2014/02/22/create-the-maven-profile-for-running-tests/), the path from the project for finding the .xml files needs to be declared. For this, in the <configuration> section of the test profile, the following information must be declared:

<configuration>
   <suiteXmlFiles>                             <suiteXmlFile>src/test/resources/testNGFilesFolder/${testNgFileName}.xml</suiteXmlFile>
   </suiteXmlFiles>                         
 </configuration>

Here, the information inside the ‘suiteXmlFile’ tag must be the location to the .xml file used for selecting the tests to be run. They should all be placed together in the same folder, in a location like /src/test/resources. The file name is declared as a variable. It will be passed to the test executor from the command line.

Running the tests

After configuring the Maven profile, running a test will be done with the following command:

 mvn -PprofileName test -DtestNgFileName=firstTestSuite

Here, the parameter that follows ‘-D’ is the one declared in the <suiteXmlFile> section of the Maven profile. What comes after the ‘=’ is the name of the file that holds the test configurations, found in the location declared in the <suiteXmlFile> section of the Maven profile (so if the command from the command line says that the file name is ‘firstTestSuite’, this file, with the .xml extension, needs to exist in the location declared in the Maven profile, in this case: src/test/resources/testNGFilesFolder/firstTestSuite.xml).