Polling accuracy of vote intentions in Sweden using different weighting and sampling strategies

Abstract

Over and over again, vote intention polls have been reported to fail their forecasts of votes such as the UK “Brexit” referendum and the US presidential election of 2016. The trustworthiness of opinion polls are called into question and this thesis aims to provide detailed knowledge of some of the circumstances that lead to inaccurate measurements of voting intention. In past research, the various determinants of accuracy are usually only considered separately. Using an innovative method of creating all possible weight covariate combinations—which results in a dataset with over 98,000 bias adjustments of vote intention measurements in 21 probability and nonprobability samples collected in Sweden—a more holistic analytic approach is possible and the effects of accuracy determinants may be estimated simultaneously. Adjusting for demographic variables such as gender, age and education are found to be relatively ineffective, while employing psychographic variables, such as vote recall and political interest is more fruitful. The choice of weighting technique, cell-weighting, raking or propensity score adjustment, matters little for the resulting accuracy. Probability samples produce more consistent and higher measurement accuracy, although making a distinction between different levels of quality in nonprobability samples reveal significant variation within the nonprobability category. The application of weights should be done with care, since there is a risk that the weights introduce more bias than they remove. For survey research, these results suggest that there is a need to find unorthodox adjustment covariates similar to that of political interest to get more accurate measurements.