Kafka: Testing and operation in a limited resource environment

Targeted audience: developers, QA, ops in systems with Kafka.

During this session, you will hear about practices used for Kafka. You will hear interesting cases from testing and operating Kafka and applications that interact with it.

Kontur is one of the main suppliers of business solutions for tax reporting, electronic document management, bookkeeping, OFD, EGAIS and other B2B and B2G solutions. The number of services, applications, transactions are huge and for all of them you need to collect telemetrics — logs, metrics, traces, then transport and distribute them to the appropriate applications for alerting, storage and analysis.

Sergey and his team are developing transport for services telemetrics and applications with Kafka. The system pumps 300 thousand messages per second on a small cluster and this range continues to grow.

Speaker wants to tell how to survive in the conditions of a shortage of servers, what problems and interesting cases happened, what tools and tricks Kontur use to test Kafka and applications that write or read from Kafka, for monitoring cluster status, etc.