Testing in microservice systems: A repository mining study on open-source systems using contract testing

Abstract

Context:There is a trend in the software industry to migrate systems from a monolithic to a microservice architecture (MSA) due to the gains in agility and scalability. An MSA-based system consists of a suite of small microservices which can be developed, tested, and deployed independently. Problem: MSA puts challenges on software testing due to the complexity caused by the integration of many autonomous microservices into one system. New best practices with testing strategies in terms of test types and test proportions (pyramid shape) emerged, but are only studied in isolated cases. Objective: The aim of this study is to explore microservice testing in a real-life context and to compare it to best practices. Approach: Repository mining is used to identify open-source MSA-based systems. Consumer-driven contract testing (CDC), a method recommended by MSA practitioners, was selected as inclusion and system identification criteria. Four systems with overall 22 microservices were analyzed. Results: The recommended five test types were identified, but not in all systems. The bigger a microservice, the better its test strategy aligned to the pyramid shape, but the test proportions also dependent on the microservice’s distinct function in the system. Using CDC tests seems to minimize the number of system tests.