Problems and solutions when testing a heavy enterprise

A case of test automation and successfully dealing with the accompanying automation problems: test speed, test stability, and organizational problems. This talk will be useful both for the audience, which is just starting to automate the testing process and for quite experienced specialists, as well as those who are experimenting with new approaches.

Testing a very large project is never easy. Several factors contribute to this:

• Specificity: tests in a digital bank are complex. People are working with complex and often long business cases, for example, filling out a payment order;
• Scope: there are many tests, there are also many changes on which you need to run tests;
• Analysis: the great number of reports and test results makes it difficult to analyze problematic tests;
• Structure: there are also many teams writing tests and they are scattered across different divisions of the bank.

This caused a great number of problems:

• long and unstable work of tests;
• late detection of defects (after they have entered the release branch code);
• repairing some tests broke others, running all tests on pull request checks was impossible.

Solving these problems took several years and led to the creation and implementation of several tools and technologies. For example, a testing framework appeared in Sber with its mechanism for finding elements and executing the logic of checks on the browser side. There is also a portal with detailed statistics on test results, a harvester — a tool for analyzing the impact of changes in the code on tests, etc. These tools helped significantly, but revealed some organizational and quality problems (speed and stability). But Sber also successfully coped with them. How? Sergey will tell in this talk.