Using SQLancer to test ClickHouse and other database systems

SQLancer is a highly-popular and widely-used tool, which can automatically generate test cases to find bugs in Database Management Systems (DBMSs). While crash bugs in DBMS can be detected via fuzzers such as AFL or SQLsmith, SQLancer aims to detect correctness bugs, which cause a DBMS to produce an incorrect result set for a given query.

In this talk, speakers will give an overview of the testing techniques that they designed to automatically validate the result sets returned for a query. Manuel and Ilya used SQLancer to find and report over 450 unique, previously unknown bugs in widely-used DBMSs such as SQLite, MySQL, and PostgreSQL, and ClickHouse. Furthermore, speakers will give a hands-on tutorial to demonstrate how SQLancer can be extended to support a new DBMS and how the tool can be effectively used to find bugs. We will also discuss interesting examples and a case study on how to use SQLancer to test ClickHouse.