Android functional testing: Scaling the mountain

It's relatively easy to develop a small suite of Android functional tests, but as a project develops they can become harder to scale and slow.

Five years ago, Badoo was already an established and successful app with about 300 multi-platform functional tests to help ensure consistency between Android, iOS, and Mobile Web. The tests were relatively easy to write and run, but engineers found those tests to be a nuisance: they usually ran slowly overnight on the master branch, giving very late feedback.

Badoo team had to improve, to give engineers quick feedback for their particular changes before they merged to master: to do that, they needed to scale Android infrastructure. They had to move slowly, building confidence in the value of the tests, integrating with other systems, and acquiring more hardware to deploy on.

This talk covers a number of the things team tried with particular reference to the Android infrastructure, and problems Tim worked around while working towards the solution they have now, where the team can run 1500 tests in 30-40 minutes, more than 100,000 tests per day, providing support for multiple versions and configurations of Android where needed.