Without an indication of its precision, a point estimate is pointless
When doing quantitative research on the basis of a sample, the estimation of the precision and reliability of survey estimates is an important basic quality requirement. Unfortunately, many applied researchers struggle with estimating correctly the sampling variance and its related precision indicators (standard errors, confidence intervals, p-values, significance tests). More in particular, researchers often ignore the sample design when estimating the sampling variance, resulting potentially in strongly biased estimates of statistical precision.
On this page, you can find some tools that can help with estimating correct standard errors and confidence intervals, with an application to the EU Survey on Income and Living Conditions (EU-SILC), the principal data source for cross-national research on poverty and inequality in Europe. The tools are described in more detail in the following article and CSB working paper:
- Goedemé, T. (2013). ‘How much Confidence can we have in EU-SILC? Complex Sample Designs and the Standard Error of the Europe 2020 Poverty Indicators‘ in Social Indicators Research, 110(1): 89-110, doi:10.1007/s11205-011-9918-2.
- Zardo Trindade, L. and Goedemé, T. (2016) Notes on updating the EU-SILC UDB sample design variables 2012-2014, CSB Working Paper 16/02, Antwerp: Herman Deleeck Centre for Social Policy, University of Antwerp.
Most statistical software packages can handle in a rather generic way complex sample designs. However, it is necessary to inform the programme about sample design settings, by indicating so-called ‘sample design variables’. In EU-SILC, these variables are not readily available. Therefore, I have developed with Lorena Zardo Trindade several tools to create in your EU-SILC microdata file sample design variables that can be easily used:
- Download Stata do-files for (1) preparing the sample design variables for the EU-SILC cross-sectional data (2005-2011); and (2) reconstructing the Europe 2020 poverty reduction indicator (as bonus material)
- Download Stata do-files for preparing the sample design variables for the EU-SILC cross-sectional data (2012-2014)
- Download CSV-files (11MB) which can be merged with the Eurostat EU-SILC User database (UDB) (using db020 (country) and db030 (household ID). The CSV files can be merged with the following versions of the (cross-sectional) Eurostat UDB: 2005 – version 5; 2006 – version 4; 2007 – version 6; 2008 – version 7; 2009 – version 7; 2010 – version 6; 2011 – version 5; 2012 – version 4; 2013 – version 3; 2014 – version 1.
The do-files reconstruct as precisely as possible the original sample design variables. Before merging the various EU-SILC data files (D, H, R and P), the do-files have to be run on the D-file of EU-SILC. The procedures embedded in the do-file have been validated for EU-SILC 2008 during a research stay at Eurostat. The previously mentioned CSB working paper explains with an example how the sample design variables should be used for correctly estimating standard errors and confidence intervals. Please acknowledge our work by citing both the article and the paper. If you encounter any trouble with the do-file or csv-files, please do not hesitate to contact me.
More information on variance estimation for EU-SILC can be found in:
- Goedemé, T. (2010), The standard error of estimates based on EU-SILC. An exploration through the Europe 2020 poverty indicators, CSB Working Paper Series, WP 10/09, Antwerp, Herman Deleeck Centre for Social Policy, University of Antwerp.
- Guio, A.-C., Goedemé, T. (2011), ‘Stratégie Europe 2020 : quelles implications pour la mesure de la pauvreté et de l’exclusion en Belgique?’ in Reflets et perspectives de la vie économique, 50(4): 31-44, http://dx.doi.org/doi:10.3917/rpve.504.0031.
- Goedemé, T. (2013), The EU-SILC sample design variables: critical review and recommendations, CSB Working Paper Series, WP 13/02, Antwerp: Herman Deleeck Centre for Social Policy.
- Osier, G., Berger, Y., and Goedemé, T. (2013), Standard error estimation for the EU–SILC indicators of poverty and social exclusion, Eurostat Methodologies and Working papers, Luxembourg: Publications Office of the European Union, 54p.
- Goedemé, T. (2013), Measuring change with the Belgian Survey on Income and Living Conditions (SILC): taking account of the sampling variance, Report prepared for the Federal Public Service Social Security, Antwerp: Herman Deleeck Centre for Social Policy, University of Antwerp.
- Goedemé, T., Van den Bosch, K., Salanauskaite, L. & Verbist, G. (2013), ‘Testing the statistical significance of microsimulation results: A plea‘ in International Journal of Microsimulation, 6(3): 50-77.
- Berger, Y., Osier, G. and Goedemé, T. (2017) ‘Standard error estimation and related sampling issues’ in Atkinson, A.B., Guio, A.-C. and Marlier, E. (eds.) Monitoring social inclusion in Europe, Luxembourg: Publications Office of the European Union, p. 465-478. http://dx.doi.org/doi:10.2785/60152.
- Goedemé, T., Zardo Trindade, L. and Vandenbroucke, F. (2017), A pan-European perspective on low-income dynamics in the EU, CSB Working Paper 17/03, Antwerp: Herman Deleeck Centre for Social Policy – University of Antwerp.
- Decerf, B., Van den Bosch, K. and Goedemé, T. (2017), A new measure of income poverty for Europe, CORE Discussion Paper 2017/08, Louvain-La-Neuve: CORE.
- Goedemé Tim and Collado Diego (2016), The EU convergence machine at work. To the benefit of the EU’s poorest citizens?, Journal of Common Market Studies, 54(5), 1142–1158, doi:10.1111/jcms.12382.
- Decancq Koen, Goedemé Tim, Van den Bosch Karel, Vanhille Josefine (2014) The evolution of poverty in the European Union: concepts, measurement and data, in Cantillon, Bea & Vandenbroucke, Frank (eds.) Reconciling work and poverty reduction : how successful are European welfare states? Oxford: Oxford University Press, p. 60-93, also published as ImPRovE Working Paper 13/01.
- Goedemé Tim, Collado Diego, Meeusen Leen (2014), Mountains on the move: recent trends in national and EU-wide income dynamics in old and new EU member states, ImPRovE working paper 14/05, Antwerp, University of Antwerp, Herman Deleeck Centre for Social Policy, 34 p.