Vaadin 8 TestBench Tutorial

Introduction
Prerequisites
Setup
Adding TestBench to Your Project
Creating the First TestBench Test
Build Real Tests
Summary

Introduction

This tutorial shows how to add tests to an existing project, step-by-step. As the project to add tests to, we will use the "Address book tutorial" (Github).

In this tutorial, you will learn how to:

Set the browser driver and open the tested web page.
Add TestBench to an existing project.
Create a TestBench test case.
Identify and select different components using the new ElementQuery API.
Interact with different components using the relevant Element API.

The complete source code for the resulting project is available at https://github.com/vaadin/testbench-tutorial.

Prerequisites

To follow along with this tutorial exactly as it is written, you need to have the command line Git and Maven tools as well as a JDK installed. You may also follow along using your favorite IDE, but this tutorial does not contain instructions on how to use Maven or Git in your IDE.

Setup

To begin, we need to clone the Vaadin tutorial project from Github. Using command line git, you can clone the project by using the following command:

git clone https://github.com/vaadin/tutorial.git

After the repository is cloned on your local machine, make sure that you can run and view it in your browser. You can do that by using the run goal of the jetty plugin:

cd tutorial
mvn jetty:run

The above command will start a Jetty server and run the application on link: http://localhost:8080/. Go ahead, point your browser to that address and play around with the application for a bit.

When you have finished investigating the Address Book application, you can shut down the Jetty server by pressing Ctrl+C in the terminal where you started it.

Adding TestBench to Your Project

To start using TestBench, you need to add it as a dependency to your project and install a license key. These topics are covered in "Maven Dependency" and "Installing Vaadin TestBench License".

Creating the First TestBench Test

In the Maven world, all test classes and resources live in the src/test directory, so create that directory. Continue by creating a java directory under that so that you end up with a src/test/java directory structure.

Create a new file called AddressBookIT.java in the src/test/java directory.

First, make the class extend TestBenchTestCase:

import com.vaadin.testbench.TestBenchTestCase;
...

public class AddressBookIT extends TestBenchTestCase {
   ...
}

Extending this base class will provide you with all the Vaadin TestBench API that makes it easy for you to write UI-level tests for Vaadin applications.

We use Firefox driver in the code examples. Make sure you have Firefox installed. You also need to install Geckodriver, see see "Installing Browser Drivers".

We are now ready to make the empty JUnit test above use Vaadin TestBench. Let us start by opening a browser. This can be done in a method annotated with @Before method, which is run before each of the tests are run. We should also close the browser in the @After method. This way we have a clean browser for each test that is run.

import org.junit.After;
import org.junit.Before;
import org.openqa.selenium.firefox.FirefoxDriver;
...

public class AddressBookIT extends TestBenchTestCase {
   @Before
   public void setUp() throws Exception {
       setDriver(new FirefoxDriver());
   }

   @Test
   public void testFails() {
       Assert.fail("This is supposed to fail");
   }
   @After
   public void tearDown() throws Exception {
       getDriver().quit();
   }
   ...
}

TestBench also supports testing applications through other browsers, such as Chrome, Internet Explorer, or even in the headless (doesn’t show any UI) PhantomJS browser by instantiating ChromeDriver, InternetExplorerDriver, or PhantomJSDriver, respectively. See "Creating and Closing a Web Driver" on how to run on the different browsers.

Note

Before TestBench version 5.0.0, Firefox did not require a web driver setup. Since TestBench version 5.0.0, Geckodriver is used to run tests on Firefox. Geckodriver requires a separate installation. You can run your tests using other browsers. All supported browsers require a special driver and some additional setup, see "Installing Browser Drivers".

Note

Geckodriver used for the latest Firefox versions (48 and newer) does not support actions API. In practice, meaning that you can not simulate context click and double click in tests. It also means that some parts of the Element API just will not work It is recommended you use Firefox 45 ESR to execute tests on Firefox, before this is fixed. See "Known Issues" for details.

Run the test with the following command and observe that it opens up a Firefox browser and immediately closes it again.

mvn verify

Navigating to a Web Page

Let us change the failing test into something more useful. First, we will make it navigate to the URL where our application is running. Note that the server needs to be running for any test to work. Use the mvn jetty:run command to run it like we did above. Rename the testFails() method to testAddressBook() and tell the driver to navigate to http://localhost:8080/. We will also make the test verify that the application is started by verifying that a table is visible:

import com.vaadin.testbench.elements.GridElement;
...

public class AddressBookIT extends TestBenchTestCase {
   ...

   @Test
   public void testAddressBook() {
       getDriver().get("http://localhost:8080/");
       Assert.assertTrue($(GridElement.class).exists());
   }
}

Above, we looked for a Vaadin component on the web page by using an element query, that is, the $() method. The argument to this method is an element class corresponding to the element we are searching for. After obtaining a basic element query, we can specify additional rules such as .caption(“My Caption”) or .at(index), etc. See the "TestBench chapter" and especially "Querying Elements" for details.

We are now ready to run the test. Make sure that the application server is up and running. If you have not started it yet, issue the mvn jetty:run command to start it now and wait for the “Started Jetty server” message to appear in the terminal. Once Jetty is running, we can run our test in a different terminal – or even using the run as JUnit command in your favorite IDE. It is just a plain JUnit test, after all.

The test should open up a browser, load the application, successfully verify that a table is visible, close the browser, and passe the test. Awesome! Now stop the jetty server by pressing ctrl+C in the terminal.

Build Real Tests

Now we are ready to build real test cases for the Address Book application.

Your First Test - Data Validation

The first test will click the first row in the table and verify that a form containing the same information is shown.

First, create a new test in the AddressBookIT class by copying the testAddressBook() method and naming it testFormShowsCorrectData(). The test should do the following:

The contacts are shown using a Vaadin Grid component, so the test should find it using ElementQuery. You can use the Vaadin debug window to get an ElementQuery for the Grid component by pointing and clicking as follows:
1. Open the application by navigating to http://localhost:8080/?debug in your favorite browser. Note the ?debug parameter in the URL. Using it causes the debug window to open in the lower right corner of the window
2. In the debug window (see Vaadin Debug Screenshot)
  1. Click on the TestBench icon (1)
  2. Click on the “targeting” icon (2)
  3. Click on the Grid component (3)
  4. An ElementQuery code line will be printed in the debug window.
Vaadin Debug Screenshot
Store the first name and last name values shown in the first row of the table for later comparison
Click on the first row
Assert that the values in the first name and last name fields are the same as in the table

The resulting test looks like this:

import com.vaadin.testbench.elements.TextFieldElement;
...

@Test
public void testFormShowsCorrectData() {
    getDriver().get("http://localhost:8080/");

    // 1. Find the grid
    GridElement grid = $(GridElement.class).first();

    // 2. Store the first name and last name values shown
    // in the first row of the grid for later comparison
    String firstName = grid.getCell(0, 0).getText();
    String lastName = grid.getCell(0, 1).getText();

    // 3. Click on the first row
    grid.getCell(0, 0).click();

    // 4. Assert that the values in the first name and
    // last name fields are the same as in the grid
    Assert.assertEquals(firstName, $(FormLayoutElement.class).
        $(TextFieldElement.class).first().getValue());
    Assert.assertEquals(lastName, $(FormLayoutElement.class).
        $(TextFieldElement.class).get(1).getValue());
    }
}

Run the test and verify that the build is successful by issuing the mvn verify command.

Data Entry

In this second test, we will add a new person, search for this person, and verify that the same data that was entered is shown in the form.

Create a new test in the same class in the same way as earlier when we created our first test. Name this one testEnterNew(). The steps of this test are the following:

Click on the New button
Type “Tyler” in the first name field
Type “Durden” in the last name field
Save this new contact
Click on some other row, for example, row 7
Verify that the same name is not in the fields after selecting some other row
Click on the first row again
Verify that the first row and the form contains the new contact

The test should now look as follows:

import com.vaadin.testbench.elements.ButtonElement;
...

@Test
public void testEnterNew() {
    getDriver().get("http://localhost:8080/");

    // 1. Click the "New contact" button
    $(ButtonElement.class).caption("Add new customer").first().click();

    // 2. Enter "Tyler" into the first name field
    $(FormLayoutElement.class).$(TextFieldElement.class).
        first().setValue("Tyler");

    // 3. Enter "Durden" into the last name field
    $(FormLayoutElement.class).$(TextFieldElement.class).
        get(1).setValue("Durden");

    // 4. Save the new contact by clicking "Save" button
    $(ButtonElement.class).caption("Save").first().click();

    // 5. Click on some other row
    GridElement table = $(GridElement.class).first();
    table.getCell(6, 0).click();

    // 6. Assert that the entered name is not in the text
    // fields any longer
    Assert.assertNotEquals("Tyler", $(FormLayoutElement.class).
        $(TextFieldElement.class).first().getValue());
    Assert.assertNotEquals("Durden", $(FormLayoutElement.class).
        $(TextFieldElement.class).get(1).getValue());

    // 7. Click on the first row
    table.getCell(0,0).click();

    // 8. Verify that the first row and form
    // contain "Tyler Durden"
    Assert.assertEquals("Tyler",table.getCell(0, 0).getText());
    Assert.assertEquals("Durden",table.getCell(0, 1).getText());
    Assert.assertEquals("Tyler", $(FormLayoutElement.class).
        $(TextFieldElement.class).first().getValue());
    Assert.assertEquals("Durden", $(FormLayoutElement.class).
        $(TextFieldElement.class).get(1).getValue());
}

Again, run the test and verify that the build is successful by issuing the mvn verify command. Not bad!

Start the Server Automatically

You might feel that it is tedious to have to start and stop the jetty server. The good news is that Maven can do this for us automatically. The 'IT' in the test class name AddressBookIT, marks it as an Integration Test using the conventions of the Failsafe plugin. This results in the test being run in the integration-test phase of the Maven build. Next up: adding the Failsafe plugin declaration to the pom.xml file. Open pom.xml and add the following declaration of the Failsafe plugin after the <plugins> start tag directly inside of the <build></build> tags:

<plugin>
 <artifactId>maven-failsafe-plugin</artifactId>
 <version>2.17</version>
 <executions>
   <execution>
     <goals>
       <goal>integration-test</goal>
       <goal>verify</goal>
     </goals>
   </execution>
 </executions>
</plugin>

Next, find the jetty plugin section in the same file and replace it with the following configuration:

<plugin>
   <groupId>org.eclipse.jetty</groupId>
   <artifactId>jetty-maven-plugin</artifactId>
   <version>9.2.3.v20140905</version>
   <executions>
       <!-- start and stop jetty (running our app) when
            running integration tests -->
       <execution>
           <id>start-jetty</id>
           <phase>pre-integration-test</phase>
           <goals>
               <goal>start</goal>
           </goals>
           <configuration>
               <scanIntervalSeconds>0</scanIntervalSeconds>
               <stopKey>STOP</stopKey>
               <stopPort>8866</stopPort>
           </configuration>
       </execution>
       <execution>
           <id>stop-jetty</id>
           <phase>post-integration-test</phase>
           <goals>
               <goal>stop</goal>
           </goals>
           <configuration>
               <stopKey>STOP</stopKey>
               <stopPort>8866</stopPort>
           </configuration>
       </execution>
   </executions>
</plugin>

Now running all integration tests is as easy as typing:

mvn clean verify

Try it and see that it compiles and packages everything, starts the Jetty server, runs our integration test, and stops the Jetty server. The build is still successful.

Summary

Congratulations! Hopefully you now have a good idea of how to set up a project for tests with Vaadin TestBench and how to write tests. Now you can continue with more advanced topics such as:

Docs