Statistical measures for evaluation of methods for syndromic surveillance

Abstract

Introduction

In syndromic surveillance there is a need for continual observation of one or more time series, with the goal of detecting an important change in the underlying process as soon as possible after it has occurred. Statistical methods are necessary to separate important changes from stochastic variation. The statistical methods suitable for this differ from the standard hypothesis testing methods. Also the measures for evaluation differ.

Objectives An overview of statistical optimality issues and statistical measures for evaluation of prospective surveillance will be given. Timeliness and the control of false alarms are important issues.

Methods Surveillance methods. Some commonly used methods for surveillance are examined. Optimality criteria. Most of the commonly used methods are optimal in some respect. Different criteria of optimality are used in different subcultures of statistical surveillance. The shortcomings of some criteria of optimality are demonstrated. One criterion discussed is based on the average run length, ARL. This is the most commonly used optimality criterion. Another criterion is based on a utility function. From the perspective of optimal decisions costs are given to the different errors which can be made and a utility function is maximized. A third criterion discussed is that of 2 minimax. Evaluation measures. Several measures for false alarms, detection ability and predictive value will be discussed and illustrated.

Conclusions Evaluation of methods for syndromic surveillance in practice is very important. It is necessary to know the basic properties of a system before it is implemented. This involves many important aspects. The use of relevant statistical measures is one of them.