Visualization of Test Failure Data to Support Fault Localisation in Distributed Embedded Systems within the Automotive Industry

Abstract

In this thesis we present the design, development and evaluation of a software tool with the purpose of assisting in the localisation of root causes of test case failures in distributed embedded systems, specifically vehicle systems controlled by a network of electronic control units (ECUs). Fault localisation is especially hard in such systems due to its distributed nature, and often organisations rely on the knowledge of in-house specialists for detecting and rectifying the underlying root cause of test case failures. The study took place in-situ at the Research and Development division of Volvo Car Corporation, a large automotive manufacturer. Researchers had access to a vast number of test execution logs from large-scale software integration testing under a continuous integration process. The main objectives of the research were to develop and evaluate a data visualisation tool to support root-cause identification of failures in order to foster a continuous feedback loop in the fault localisation process. Our contributions encourage the improvement of testing quality and supporting the development and adoption of test case writing guidelines and test failure debugging procedures. The research concludes that the use of data visualisation techniques can considerably boost the failure debugging procedures by presenting data in a clear and concise manner and making use of test harnesses to directly assist in reducing possible causes of failures. Additionally it encourages a systematic and continuous analysis of the current state of testing by aggregating, categorising and displaying large amounts of historical data in a concise manner that allows stakeholders to identify patterns and trends in test results.