CI/CD automation. Managing a horde of Jenkins jobs

Nowadays when we are talking about automated test execution we usually consider that is done in the context of Continuous Integration (CI). Usually, QA engineers concentrate on preparing the tests and the test framework, but the CI tools are considered as something where you have to simply plug your tests. Often CI tools are fully owned by DevOps engineers or developers. However, understanding and skills of work with CI allow QA engineers to significantly improve approaches to test automation.

Evolution Engineering is using Jenkins — it is a popular open source CI system where lots of functionality developed through plugins. There used to be a lot of manually configured Jenkins builds for regression tests and their support, and the extension was inconvenient, ineffective. Vyacheslav's company implemented Jenkins automation, which made it possible to fully customize it through code. This is also allowed to speed up the creation of new jobs and improve their maintenance. This introduction also fostered the development of CI/CD expertise among QA engineers at Evolution Engineering.

Now Evolution Engineering is actively moving towards Continuous Delivery (CD) and the old approach with E2E regression tests no longer suits them. One of the most difficult places with great automation needs was the frontend, which is stored in a mono repository, on which about 200 engineers are working. To improve and speed up testing, the team focused on checking specific code changes and integrating automation with the code review process in GitLab. Jenkins has a Gitlab Plugin to provide basic integration, and GitLab also provides a rich API. The team has connected various Jenkins builds (tests and not only) to Merge Requests: they are launched on different events and make sure that the code has been tested well enough. Comments in the Merge Request are used both as an interface for displaying all information on checks and for starting and restarting builds. The result is an ecosystem of automation around the capabilities of Gitlab and Jenkins, in which Evolution Engineering is constantly expanding and improving, and this allows us to release new versions of the game frontend into products daily.