Assessment of Cross-National Differences using Survey Data: A Cautionary Note

Abstract

Sociologists, political scientists and economists often assess cross-national differences in outcomes using individual-level survey data from multiple countries. Examples of such data include the ECHP, ESS, EU-SILC, EVS, ISSP, and SHARE. We review regression-based approaches to analysing such data and point out problems in the assessment of country-level factors that appear not to be widely appreciated. With a large number of observations in each country but a small number of countries, analysts can reliably estimate relationships within each country, but not country-related parameters of regression models. Application of 'multilevel' ('hierarchical') modelling approaches is no panacea. Cross-national comparative research may benefit from combining a statistical approach to within-country differences with a greater reliance on discursive or graphical approaches to analysing cross-country differences.