Have you heard of “flaky tests”?

There are a lot of articles, blog posts, podcasts, conference talks about what are “Flaky tests” and how to avoid them.

Some of the ideas proposed are:

Automatically rerun failed tests a couple of times, and hope they pass
Automatically retry certain operations in the test (ex: retry click / checking for visibility of elements, etc.) and hope the test can proceed

Unfortunately, I do not agree with the above ideas, and I would term these as anti-patterns for fixing flaky / intermittent tests.

We need to understand the reasons for flaky / intermittent tests. Some of these reasons could be because of issues like:

timing issues (i.e. page loading taking time)
network issues
browser-timing issues (different for different browsers / devices)
data related (dynamic, changing, validity, etc.)
poor locator strategy (ex: weird & hard-wired xpaths / locators)
environment issue
actual issue in the product-under-test

In this session, with the help of demos, we will look at the following techniques you can use to reduce / eliminate the flakiness of your test execution:

Reduce number of UI tests
Use Visual Assertions instead of Functional Assertions
Remove external dependencies via Intelligent Virtualization

Demo will be done using the following and sample code will be shared with the participants

Specmatic
teswiz
Applitools Eyes