Automated analysis of machine-generated test-results - An exploration of the efficiency gains from automated testing

Abstract

Testing and measuring has become increasingly prevalent and important in industrial settings as products and processes move towards greater complexity. To be able to more fully judge quality and progress in a developing system, companies strive to test widely and test often. How, then, can these results be best taken advantage of? Many organisations favour using automated frameworks generating data based on either random permutations of configurations or constant running of pre-defined tests. However, it becomes difficult and infeasible for humans to manually trawl through the amounts of data produced when running such automated tests. Clearly, a tool or system to assist is required. This study will examine the potential improvements to efficiency made with the introduction of a bespoke test analysis system to one such company.