UX Testing Automation with ML

"UX testing cannot be automated" — a statement that was true until recently. Now, a bunch of tasks in UX testing and user experience validation can be automated using machine learning. In this talk, we will find out what tasks neural networks can already perform, as well as tools and tech under the hood. Of course, taking into consideration current limitations.

Who is this talk for: testing teams and individual UI testers, UX researchers and developers, as well as product managers and designers.

Technologies under the hood of the talk: machine/computer vision (CV, recognition and classification of UI elements), vector representation (understanding and structuring the user interface), WebDriver + adb (browser/device control methods), Deep Visual Models (simulation of human attention (UX)), web scarping/web parsing.

The solution presented in the talk will demonstrate the use of machine learning technologies to automate some UX testing cases and identify growth points in the user experience of the product.

P.S. The talk is based on my own coursework and diploma work at BMSTU and HSE.