Automatically Finding Performance Issues in Compilers and Databases

While compilers are expected to produce efficient machine code, database systems (DBMSs) are expected to efficiently execute queries. Both kinds of systems can be affected by performance issues, or missed optimizations. Finding such issues is challenging due to the so-called test oracle problem. It is unclear how performance issues could be rigorously defined, even more so how to automatically detect them, since compilers and DBMSs base their optimizations on various heuristics, while balancing optimization time and expected speedups.

This talk will present two novel approaches on detecting performance issues in two domains—compilers, and DBMSs. Both approaches are based on non-obvious, domain-specific insights and revealed many issues in the systems that they targeted. The first approach’s core insight is that dead code elimination (DCE) in compilers is an optimization sink – DCE can be applied only if other optimizations could successfully simplify the code beforehand. Our approach detects missed optimizations by using differential testing to check whether optimization markers inserted into the programs can be eliminated by one compiler, but not another. The second approach is based on the insight that cardinality estimation in DBMSs is the most important component for optimization. We proposed a metamorphic testing approach that checks a monotonicity property: a cardinality estimator should estimate that a more restrictive query should fetch fewer rows than a less restrictive one. Any violations of this indicate a potential performance issue.

Manuel Rigger is an Assistant Professor leading the TEST Lab, which is part of the PL/SE group at the School of Computing, National University of Singapore. He is working on software reliability, data-centric systems, and programming language implementation. Prior to joining NUS, Manuel was a postdoc at the Advanced Software Technologies Lab at ETH Zurich, mentored by Zhendong Su. He completed his PhD at Johannes Kepler University Linz, mentored by Hanspeter Mössenböck, where he worked on the safe and efficient execution of unsafe languages (project Sulong).