Advanced Testing Concepts

Waiting for Vaadin
Waiting Until a Condition is Met
Scrolling
Profiling Test Execution Time

Waiting for Vaadin

Traditional web pages load a page that is immediately rendered by the browser. In such applications, you can test the page elements immediately after the page is loaded. In Vaadin and other SPAs (Single Page Applications), rendering is done by JavaScript code asynchronously, so you need to wait until the server has given its response to an AJAX request and the JavaScript code finishes rendering the UI. A major advantage of using TestBench compared to other testing solutions is that TestBench knows when something is still being rendered on the page and automatically waits for that rendering to finish before moving on with the test.

In most cases, this is not something you need to take into account as waiting is automatically enabled. It might be necessary to disable it in some cases though and you can do that by calling disableWaitForVaadin() in the TestBenchCommands interface. You can call it in a test case as follows:

testBench(driver).disableWaitForVaadin();

When disabled, you can wait for the rendering to finish by calling waitForVaadin() explicitly.

testBench(driver).waitForVaadin();

You can re-enable the waiting with enableWaitForVaadin() in the same interface.

Waiting Until a Condition is Met

In addition to waiting for Vaadin, it is also possible to wait until a condition is met. This could, for example, be used to wait until an element is visible on the web page.

waitUntil(ExpectedConditions.presenceOfElementLocated(By.id("first")));

The above waits until the specified element is present or times out after waiting for 10 seconds by default.

waitUntil(condition, timeout) allows the timeout duration to be controlled.

Scrolling

To be able to interact with an element, it needs to be visible on the screen. This limitation is set so that test which are run using a web driver shall simulate a normal user as closely as possible. TestBench handles this automatically by ensuring an element is in view before an interaction is triggered. In some cases you might want to disable this behavior and can then use TestBenchCommands.setAutoScrollIntoView(false).

Profiling Test Execution Time

It is not just that it works, but also how long it takes. Profiling test execution times consistently is not trivial, as a test environment can have different kinds of latency and interference. For example in a distributed setup, timings taken on the test server would include the latencies between the test server, the grid hub, a grid node running the browser, and the web server running the application. In such a setup, you could also expect interference between multiple test nodes, which all might make requests to a shared application server and possibly also share virtual machine resources.

Furthermore, in Vaadin applications, there are two sides which need to be profiled: the server-side, on which the application logic is executed, and the client-side, where it is rendered in the browser. Vaadin TestBench includes methods for measuring execution time both on the server-side and the client-side.

The TestBenchCommands interface offers the following methods for profiling test execution time:

totalTimeSpentServicingRequests(): Returns the total time (in milliseconds) spent servicing requests in the application on the server-side. The timer starts when you first navigate to the application and hence start a new session. The time passes only when servicing requests for the particular session.

Notice that if you are also interested in the client-side performance for the last request, you must call the timeSpentRenderingLastRequest() before calling this method. This is due to the fact that this method makes an extra server request, which will cause an empty response to be rendered.
timeSpentServicingLastRequest(): Returns the time (in milliseconds) spent servicing the last request in the application on the server-side. Notice that not all user interaction through the WebDriver cause server requests.

As with the total above, if you are also interested in the client-side performance for the last request, you must call the timeSpentRenderingLastRequest() before calling this method.
totalTimeSpentRendering(): Returns the total time (in milliseconds) spent rendering the user interface of the application on the client-side, that is, in the browser. This time only passes when the browser is rendering after interacting with it through the WebDriver.
timeSpentRenderingLastRequest(): Returns the time (in milliseconds) spent rendering user interface of the application after the last server request. Notice that not all user interaction through the WebDriver cause server requests.

If you also call the timeSpentServicingLastRequest() or totalTimeSpentServicingRequests(), you should do so before calling this method. The methods cause a server request, which will zero the rendering time measured by this method.

The following example is given in the VerifyExecutionTimeITCase.java file in the TestBench demo.

@Test
public void verifyServerExecutionTime() throws Exception {
    // Get start time on the server-side
    long currentSessionTime = testBench(getDriver())
            .totalTimeSpentServicingRequests();

    // Interact with the application
    calculateOnePlusTwo();

    // Calculate the passed processing time on the serve-side
    long timeSpentByServerForSimpleCalculation =
            testBench().totalTimeSpentServicingRequests() -
            currentSessionTime;

    // Report the timing
    System.out.println("Calculating 1+2 took about "
            + timeSpentByServerForSimpleCalculation
            + "ms in servlets service method.");

    // Fail if the processing time was critically long
    if (timeSpentByServerForSimpleCalculation > 30) {
        fail("Simple calculation shouldn't take " +
             timeSpentByServerForSimpleCalculation + "ms!");
    }

    // Do the same with rendering time
    long totalTimeSpentRendering =
            testBench().totalTimeSpentRendering();
    System.out.println("Rendering UI took "
            + totalTimeSpentRendering + "ms");
    if (totalTimeSpentRendering > 400) {
        fail("Rendering UI shouldn't take "
               + totalTimeSpentRendering + "ms!");
    }

    // A normal assertion on the UI state
    assertEquals("3.0",
        $(TextFieldElement.class).first()
        .getValue());
}

0BDE39CA-8E69-4E6D-BB3A-15D5604E60E9

Docs