This paper presents a new lightweight technique for automatically
generating high coverage test suites for Haskell library code. Our
approach combines four main features to increase test coverage: (1)
automatically inferring the constructors and functions needed to
generate test data; (2) using needed narrowing to take advantage of
Haskell’s lazy evaluation semantics; (3) inspecting elements inside
returned data structures through the use of
case statements, and (4)
elementsciently handling polymorphism by lazily instantiating all
We have implemented this technique in Irulan, a fully automatic tool for systematic black-box unit testing of Haskell library code. We have designed Irulan to generate high coverage test suites and detect common programming errors in the process. We have applied Irulan to over 50 programs from the spectral and real suites of the nofib benchmark and show that it can ectively generate high-coverage test suites—-exhibiting 70.83% coverage for spectral and 59.78% coverage for real—-and find errors in these programs.
Our techniques are general enough to be useful for several other types of testing, and we also discuss our experience of using Irulan for property and regression testing.