Abstract

This paper presents a new lightweight technique for automatically generating high coverage test suites for Haskell library code. Our approach combines four main features to increase test coverage: (1) automatically inferring the constructors and functions needed to generate test data; (2) using needed narrowing to take advantage of Haskell’s lazy evaluation semantics; (3) inspecting elements inside returned data structures through the use of case statements, and (4) elementsciently handling polymorphism by lazily instantiating all possible instances.

We have implemented this technique in Irulan, a fully automatic tool for systematic black-box unit testing of Haskell library code. We have designed Irulan to generate high coverage test suites and detect common programming errors in the process. We have applied Irulan to over 50 programs from the spectral and real suites of the nofib benchmark and show that it can ectively generate high-coverage test suites—exhibiting 70.83% coverage for spectral and 59.78% coverage for real—and find errors in these programs.

Our techniques are general enough to be useful for several other types of testing, and we also discuss our experience of using Irulan for property and regression testing.