A symbolic execution-based technique for patch testing. The old and the new version are run simultaneously in order to discover the introduced behavioural divergences.
While developers are aware of the importance of comprehensively testing patches, the large effort involved in coming up with relevant test cases means that such testing rarely happens in practice. Furthermore, even when test cases are written to cover the patch, they often exercise the same behaviour in the old and the new version of the code.
In this project, we present a symbolic execution technique that is designed to generate test inputs that cover the new program behaviours introduced by a patch. The technique works by executing both the old and the new version in the same symbolic execution instance, with the old version shadowing the new one. During this combined shadow execution, whenever a branch point is reached where the old and the new versions diverge, we generate a test case exercising the divergence and comprehensively test the new behaviours of the new version.
We evaluate our technique on the Coreutils patches from the CoREBench suite of regression bugs, and show that it is able to generate test inputs that exercise newly added behaviours and expose some of the regression bugs.
Shadow is available in a binary form (along with the necessary infrastructure) as a downloadable virtual machine. Please contact Tomasz Kuchta for more details.
This research project is generously sponsored by the UK EPSRC through the grant EP/J00636X/1 and by Microsoft Research through its PhD Scholarship Programme.
Shadow Symbolic Execution for Testing Software Patches
ACM Transactions on Software Engineering and Methodology (TOSEM 2018)
Shadow of a Doubt: Testing for Divergences Between Software Versions
International Conference on Software Engineering (ICSE 2016)
Shadow Symbolic Execution for Better Testing of Evolving Software
International Conference on Software Engineering, New Ideas and Emerging Results (ICSE NIER 2014)