Abstract 

An index for assessing selection bias in non-probability samples: A validation study and a real-world application. 

Sabine Zinn & Angelina Hammon
Sozio-ökonomisches Panel am Deutschen Institut für Wirtschaftsforschung, Berlin

Fast online surveys without sampling frames are becoming increasingly important in survey research. Their recruitment methods result in non-probability samples. Since the mechanism of data generation is always unknown in such samples, the problem of non-ignorability arises making generalization of calculated statistics to the population of interest highly questionable.  

In this talk, I will present sensitivity analyses as a valuable tool to deal with non-ignorability. They capture the impact of different sample selection mechanisms on target statistics. Recently, Andridge et al. (2019) propose an index to quantify potential (non-ignorable) selection bias in proportions that combines the effects of different selection mechanisms. I present a validation study in which we examined the performance of the index. Overall, the index shows good functionality under realistic assumptions. Applying the index to current data on political attitudes collected using river sampling shows the danger of simply generalizing from non-probability samples to the population level.