Skeleton program enumeration (SPE) is
a novel framework for systematic, rigorous compiler
testing. We view a program as a syntactic structure P (syntactic
skeleton) parameterized by a collection of identifiers V
(variable names). Given a syntactic skeleton P and a set of variables V,
the goal of SPE is to enumerate a set of programs P exhibiting all possible variable usage patterns within P. We can then use
the enumerated programs in P to stress-test compilers.
insights behind SPE are: (1) Programs with different variable
usage patterns exhibit diverse control- and data-dependence, and
help exploit different compiler optimizations; (2) most real
compiler bugs were revealed by small tests this "small-scope"
observation opens up SPE for practical compiler validation; and
(3) SPE is exhaustive w.r.t. a given syntactic skeleton and
variable set, offering a level of guarantee absent from all
existing compiler testing techniques.
We continue our team efforts working with the open-source community on
various compiler projects. The following table gives an overview
of the bugs found by SPE. This list is not exhaustive as it does not include some of
the GCC/LLVM bugs reported by Chengnian and Zhendong (link1,
link4) using SPE.