Achieving Cost-Effective Testing for Serverless Based Applications

Abstract

Testing serverless applications poses unique challenges compared to traditional applications, such as the abstraction of its inner workings and how the pay-per-usage model means tests incur monetary costs. With the goal of studying practitioner’s approaches on this topic of cost-efficiency, we conducted five interviews and distributed a survey which received six responses. The questions were around testing practices, cost factors, and evaluation of cost-efficiency. After an inductive thematic analysis, the results revealed that the importance of unit tests increases as the development of serverless applications progresses. Participants emphasized the significance of monitoring metrics, particularly response time, to ensure optimal performance and validity. To mitigate costs, they employed strategies such as the incorporation of observability tools, granularity, wherein they reduced the size and scope of functions, and utilized local emulation for smaller tasks. Additionally, the choice of serverless provider emerged as a topic of discussion, as participants faced challenges in smoothly transitioning between vendors due to the varying advantages offered by each provider.

These results provide insight into the challenges of achieving cost-effective testing for serverless-based applications — though we believe the topic should receive further attention in research due to how new this field is.