Running Tests with Maven

Starting the Server Automatically
Executing Tests in the Integration Test Phase
Downloading WebDrivers Automatically

A Maven build is divided into different lifecycle phases. Below are the relevant ones:

compile Compiles the code;
test Runs unit tests which don’t require a packaged project;
pre-integration-test Makes preparations for integration tests;
integration-test Executes integration tests; and
post-integration-test Cleans up after integration tests.

TestBench tests fit into the integration-test phase. The pre-integration-test phase is the place to start a server and deploy the package. The post-integration-test phase is where you would stop the server.

Tip	If you name your tests `Test`, they are run automatically in the `test` phase. If you instead name your TestBench tests `IT`, they’re run automatically in the `integration-test` phase.

Note	Never execute TestBench tests in the `test` phase. They can’t be run without a packaged or deployed project.

Starting the Server Automatically

For applications without external dependencies, it’s often handy to start a test as part of the build. As an example, if you’re using Jetty to run the project, you can use the jetty-maven-plugin to start the server in the pre-integration-test and stop it in the post-integration-test phase, as follows:

<plugin>
    <groupId>org.eclipse.jetty</groupId>
    <artifactId>jetty-maven-plugin</artifactId>
    <version>9.2.3.v20140905</version>
    <configuration>
        <stopPort>9966</stopPort>
        <stopKey>something-goes-here</stopKey>
    </configuration>
    <executions>
        <execution>
            <id>start-jetty</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start</goal>
            </goals>
        </execution>
        <execution>
            <id>stop-jetty</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>stop</goal>
            </goals>
        </execution>
    </executions>
</plugin>

The stopPort and stopKey are Jetty specific parameters which must be given so that Jetty is able to stop the correct server instance. A fully working example of running Jetty as part of the build can be found in https://github.com/vaadin/testbench-demo/blob/master/pom.xml.

If you’re using Spring Boot, you can use the spring-boot-maven-plugin to achieve the same thing. See the Bakery starter for Spring for an example.

If you’re using JavaEE, you can start TomEE, WildFly or a Liberty server in a similar way. See the Bakery starter for JavaEE for an example — this is available at this time only for Vaadin 8.

Executing Tests in the Integration Test Phase

In Maven, unit tests are executed by the maven-surefire-plugin, automatically included in all projects. Integration tests are executed by the maven-failsafe-plugin instead, which needs to be included manually in the project as the following:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.19.1</version>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <trimStackTrace>false</trimStackTrace>
    </configuration>
</plugin>

The <executions> part is needed to execute the plugin during the integration-test phase. The <configuration> part is optional, but by including it you get the full stack trace when an error occurs. This typically makes it easier to determine what went wrong in a test. Running failed tests multiple times is also possible using the rerunFailingTestsCount property.

Downloading WebDrivers Automatically

The driver-binary-downloader-maven-plugin plugin downloads WebDrivers for a given browser and platform and makes them available for the TestBench tests. By downloading these as part of the build, you don’t need to do any setup on the machine where you are running the tests.

The plugin can be enabled as follows:

<plugin>
    <groupId>com.lazerycode.selenium</groupId>
    <artifactId>driver-binary-downloader-maven-plugin</artifactId>
    <version>1.0.17</version>
    <configuration>
        <downloadedZipFileDirectory>${project.basedir}/webdriver/zips</downloadedZipFileDirectory>
        <rootStandaloneServerDirectory>${project.basedir}/webdriver</rootStandaloneServerDirectory>
        <customRepositoryMap>${project.basedir}/webdrivers.xml</customRepositoryMap>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>selenium</goal>
            </goals>
        </execution>
    </executions>
</plugin>

This downloads the WebDrivers defined in webdrivers.xml (i.e., a repository map) in the project root during the test-compile phase, before the integration tests start. The downloaded WebDrivers are placed in the webdriver/zips folder in the project and unpacked to the webdriver folder. The file webdrivers.xml defines which version of the various WebDrivers to download. An example can be found at https://github.com/vaadin/testbench-demo/blob/master/webdrivers.xml.

Tip	The https://github.com/Ardesco/selenium-standalone-server-plugin repository map is kept quite up-to-date.

In addition to downloading the WebDrivers, the location of the unpacked drivers must be passed to the maven-failsafe-plugin so that the TestBench tests can find them during execution. This can be done by defining system properties in the <configuration> section of the maven-failsafe-plugin like so:

<configuration>
    <trimStackTrace>false</trimStackTrace>
    <systemPropertyVariables>
        <webdriver.chrome.driver>${webdriver.chrome.driver}</webdriver.chrome.driver>
        <!-- Similarly for other browsers -->
    </systemPropertyVariables>
</configuration>

2516DA74-34F6-4247-AAD3-44584BF5DBF3

Documentation