AI and Testing: Tips from the trenches

AI is changing our world. It is being employed in all types of applications and increasingly impacts the way we build and test software. However, although machine learning has been around for decades, very little real-world experience is publicly available on how to test AI based systems.

In this talk, Adam will share practical AI testing tips from our years of experience building an AI-powered service at Applitools. We will cover common approaches to implementing AI: from hand-coded algorithms to classic machine learning and deep learning, and review their applicability to solving different types of problems and the challenges involved. Speaker will share practical tips for testing AI-based features including how to determine if accuracy is a sufficient quality measure for a statistical model, how to avoid degrading the overall quality of a system when improving the accuracy of a specific model, why and how to "canary test" your candidate models in production, and many others. Lastly, we'll explore areas where you can easily utilize AI to reduce your automated test maintenance overhead.