Assessing the effect of data collection mode on measurement

Abstract

We review the methods typically used to assess the effects of mode on measurement and data comparability and then discuss some of the challenges, including 1) the need to avoid confounding effects, 2) the sensitivity of conclusions to methods of analysing experimental mode comparison data, 3) the difficulty of assessing whether measurement differences matter in practice, and 4) the assessment of which mode provides better measurement. We illustrate the challenges and implications of mixed modes research for survey design with examples from experiments conducted in the context of the European Social Survey (ESS). The paper concludes with implications for mixed modes research.