Mutation testing aims to evaluate the fault detection capability of a test suite. This evaluation substitutes faults with mutants by transforming program code to be defective. Evidences of the relationship between the detection rates of mutants and real faults have supported the use of mutants. It has also been known that the test suite size was a significant factor affecting the relationship. Our study revealed that the selection of the mutated code was another factor affecting the relationship. We generated mutants by transforming the code modified to fix defects, while the modified code was located at three granularity levels. The experiments conducted on the defects4j dataset demonstrated that the granularity level caused a significant difference in the relationship; the detection rate of mutants was more strongly correlated with and more indicative of the fault detection capability at a fine level than at a coarse level. Moreover, the influence of the test suite size was different at each granularity level. These findings implied a strong correlation between the detection rates of mutants and real faults, independently of test suite size, when the error-prone code was located precisely.
|Number of pages
|International Journal of Software Engineering and Knowledge Engineering
|Published - 2020 Aug 1
Bibliographical noteFunding Information:
Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2019-0-00099, Formal Speci¯cation of Smart Contract).
This study was partly supported by Samsung Electronics Co. Ltd, the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (2020R1C1C1014611), and Institute of Information & communications
© 2020 World Scientific Publishing Company.
- Mutation testing
- irrelevant mutants
- real faults
ASJC Scopus subject areas
- Computer Networks and Communications
- Computer Graphics and Computer-Aided Design
- Artificial Intelligence