Mining Oracles for Fully Automated Test Generation

Generating tests automatically promises to get exhaustive coverage even of large-scale systems. But current techniques for automatic test generation cannot produce oracles – the crucial checks whether a program behaves as expected. In this talk, I present recent research that automatically mines patterns of normal behavior from a multitude of instances – for instance, by comparing web site behavior across multiple browsers, or comparing app behavior against similar apps from app stores. These techniques easily scale to industrial settings, and companies like Google or our Testfabrik startup are rapidly adopting them for full-fledged testing systems.

Andreas Zeller is a full professor for Software Engineering at Saarland University in Saarbrücken, Germany, since 2001. His research concerns the analysis of large software systems and their development process. In 2010, Zeller was inducted as Fellow of the ACM for his contributions to automated debugging and mining software archives. In 2011, he received an ERC Advanced Grant, Europe’s highest and most prestigious individual research grant, for work on specification mining and test case generation.