Closer to the Edge: Testing Compilers More Thoroughly by Being Less Conservative About Undefined Behaviour

Abstract

Randomised compiler techniques require a means of generating programs that are free from undefined behaviour (UB) in order to reliably reveal miscompilation bugs. Existing progarm generators such as Csmith heavily restrict the form of generated programs in order to achieve UB-freedom. We hypothesis that the idiomatic nature of such program limits the test coverage they can offer. Our idea is to generate less restricted programs that are still UB-freeā€”programs that get closer to the edge of UB, but that do not quitecross the edge. We present preliminary support for our idea via aprototype tool,CsmithEdge, which uses simple dynamic analysis to determine where Csmith has been too conservative in its use of safe math wrappers that guarantee UB-freedom for arithmetic operations. By eliminating redundant wrappers,CsmithEdge was able to discover two new miscompilation bugs in GCC that couldnot be found via intensive testing using regular Csmith, and toachieve substantial differences in code coverage on GCC comparedwith regular Csmith.