Double Session: "Shadow of a Doubt: Testing for Divergences Between Software Versions" and "Analysing the Program Analyser"

In this seminar, there will be two talks given in a conference-like setting.

Shadow of a Doubt: Testing for Divergences Between Software Versions

While developers are aware of the importance of comprehensively testing patches, the large effort involved in coming up with relevant test cases means that such testing rarely happens in practice. Furthermore, even when test cases are written to cover the patch, they often exercise the same behaviour in the old and the new version of the code.

In this paper, we present a symbolic execution-based technique that is designed to generate test inputs that cover the new program behaviours introduced by a patch. The technique works by executing both the old and the new version in the same symbolic execution instance, with the old version shadowing the new one. During this combined shadow execution, whenever a branch point is reached where the old and the new version diverge, we generate a test case exercising the divergence and comprehensively test the new behaviours of the new version.

We evaluate our technique on the Coreutils patches from the CorREBench suite of regression bugs, and show that it is able to generate test inputs that exercise newly added behaviours and expose some of the regression bugs.

Tomasz Kuchta is a final year PhD candidate in the Department of Computing, Imperial College London, working under the supervision of Dr Cristian Cadar in the Software Reliability Group. His research interests span across the areas of software engineering, software reliability and systems. In his PhD work, he explores applications of symbolic execution technique to the problems of program input recovery and testing software patches. Prior to joining Imperial, he has been working in the telecommunications industry as a software engineer. Tomasz holds an MSc degree from Cracow University of Technology.

Analysing the Program Analyser

The reliability of program analysis tools is clearly important if such tools are to play a serious role in improving the quality and integrity of software systems, and the confidence which users place in such systems. Yet our experience is that, currently, little attention is paid to analysing the correctness of program analysers themselves, beyond regression testing. I will present our vision that, by 2025, the use of more rigorous analyses to check the reliability of program analysers will be commonplace. Inspired by recent advances in compiler testing, I will set out initial steps towards this vision, building upon techniques such as cross-checking, program transformation and program generation.

Joint work with Cristian Cadar.

Alastair Donaldson is a Senior Lecturer and EPSRC Early Career Fellow in the Department of Computing, Imperial College London, where he leads the Multicore Programming Group. He has published more than 70 peer-reviewed papers in formal verification, multicore programming and software testing. Before joining Imperial, Alastair was a Visiting Researcher at Microsoft Research Redmond, an EPSRC Postdoctoral Research Fellow at the University of Oxford and a Research Engineer at Codeplay Software Ltd. He holds a PhD from the University of Glasgow.