An experimental study on combining automated and stochastic test data generation

Abstract

Test data plays an important role in improving the quality and e ectiveness of test cases and automatic generation of meaningful test data saves a lot of human e ort. There are several automatic test data generation approaches; stochastic test data generation is one of them. To investigate the bene ts and challenges of using a stochastic test data generation technique, this study presents JuliaTest, an automatic test data generation tool integrating stochastic test data generation framework with the unit testing framework JUnit5. Using JuliaTest two empirical studies were conducted on open-source projects to compare different automatic test generation techniques and to investigate the behavior of JuliaTest in di erent settings (e.g., varied number of data generated, di erent generators used). Performed experiments of limited scope showed promising results indicating test data generated by stochastic technique is able to provide better mutation coverage and detect more faults when compared to other existing alternatives. These experiments aims at being used as a baseline for future work with broader scope and setup. Moreover, the study discusses di erent design choices made during the implementation of the tool. Finally, a number of future research concepts are discussed to open the door for researchers interested in this eld.