Estimating reliability in cross-national longitudinal survey measures

Summary

Reliability in survey measurement is a crucial aspect of data quality. Yet surprisingly little attention has been given to assessing reliability for commonly used survey measures or evaluating how reliability may vary across different survey contexts. This study leverages the Comparative Panel File (CPF), a dataset compiled from long-running household panel surveys in seven countries—Australia, Germany, Russia, South Korea, Switzerland, the United Kingdom, and the United States—to estimate the reliability of 14 survey indicators from 2001 to 2020 using quasi-simplex models. We find that reliability is high and consistent across countries for factual items but substantially lower and more variable for self-assessment items (e.g., satisfaction with life, satisfaction with work, self-rated health). Additionally, we find that variations in question wording across the panels (e.g., asking about health currently versus in general) led to different reliabilities. We discuss the implications these findings have for measurement comparability in cross-national research.