1 ANNEX Quality criteria and statistical concepts, Regulation (EU) 2019/2180

In forcesince 16 Dec 2019

The quality report should contain quality-related data and metadata in line with the following quality criteria and statistical concepts.

If a particular statistical concept is not relevant to a statistical operation, that concept should remain in the quality report, accompanied by the words ‘Not applicable’.

Individual or organisational contact points for data or metadata, including contact details.

Description of the data disseminated, which can be displayed to users as tables, graphs or maps.

Description of the main characteristics of the dataset.

Where applicable, the list of classifications and breakdowns used in data, and any deviations from European statistical standards or international standards.

Description of the main themes covered by the dataset.

The list of all the variables that deviate from the standard definition, mentioning the national concepts used and any differences between national concepts and the respective data collections.

Description of the observation units.

Description of the target statistical population or populations to which the dataset refers, i.e. the population about which information is to be collected.

Information about any sub-populations not covered by the data collection (e.g. homeless people or people living in institutions), including description of any such population and its best quantitative estimate.

Description of the geographical area to which the statistical phenomenon measured relates: the geographical area covered and a list of any regions that are excluded.

The periods or points in time to which the observation refers.

Operations performed on data to derive new information in accordance with a given set of rules.

Description of the source of the raw statistical data (e.g. interviews, administrative data, any other sources). If administrative registers are used, they should be clearly described (source, primary purpose, possible shortcomings, etc.)

Description of the methods used to obtain or create the sampling frame.

Description of the following aspects:

Type of sampling design (stratified, multi-stage, clustered, one stage, two stages)

Stratification and sub-stratification criteria

Sample size

Information on frequency at which a dataset is collected.

Description of the methods used to gather data (CAPI, CAWI, CATI, etc.). The national questionnaire used for data collection should be attached, along with its translation into English.

Description of procedures used for checking and validating the source and output data, including explanation of how the results of these validations are monitored and used.

Description of the data compilation process (e.g. data editing, imputation, weighting, adjustment for non-response, calibration, model used, etc.) Each step of weighting should be described separately: calculation of design weights; non-response adjustment (how the design weight is corrected, taking account of differences in response rates); calibration (the level and variables used in the adjustment, method applied); calculation of final weights.

Systems and frameworks in place within an organisation to manage the quality of statistical products and processes.

Description of the quality assurance framework and/or quality management system (e.g. EFQM, ISO 9000) used in the organisation.

Description of the overall quality of statistical outputs, summarising the main strengths and any quality deficiencies in the standard quality criteria: relevance, accuracy, reliability, timeliness, punctuality, comparability and coherence. Any trade-offs between quality aspects and any quality improvements foreseen can be mentioned.

Information (if available) on (any new) needs of the users in relation to data collected.

Information (if available) on level of satisfaction of data users in relation to data collected and made available.

Description of any non-compliance in terms of variables that are not transmitted.

Summary of the various components of an assessment of accuracy, linked to a given dataset or domain:

Description of the main sources of random and systematic errors in the statistical outputs, with a summary assessment of all errors, focusing particularly on the impact on key estimates.

If relevant, data revision aspects.

Description of the methodology for calculating precision estimates.

Precision measures of estimates in accordance with the technical specifications of the individual datasets.

The standard errors at national and, where required, regional level (NUTS 2) for the main indicators as mentioned in Annex II to Regulation (EU) 2019/1700.

Description of divergence between the frame population and the target population.

Frequency and timing of frame updates.

Errors due to discrepancies between the sampling frame and the target population and sub-populations (over-coverage, under-coverage, misclassifications).

Description of errors that occur during data collection, causing recorded values of variables to differ from true values.

Description of efforts made in questionnaire design and testing (including addressing errors arising from multi-mode and/or multi-source data collection).

Description of interviewer training.

Proxy interview rates.

Description of:

Available characteristics of non-respondents.

Unit and item non-response rates.

Substitution rates.

Gross sample size (initial sample size), number of eligible units and net sample size, including substitution units (achieved sample size)

Description of any error in processing and its impact on final data collection results, arising from faulty implementation of correctly planned implementation methods.

Description of quality checks and the data editing process.

Description of imputation procedures.

Imputation rates.

Where applicable: description of error arising from domain-specific models needed to define the target of estimation.

Description of statistical techniques used to remove seasonal effects that influence a data series.

Description of policy aimed at ensuring the transparency of disseminated data, with preliminary data being revised after compilation.

Information on data revision practice.

Information on:

Date of the dissemination of national results.

Number of days between the end of fieldwork and the first fully validated delivery of data to the Commission (Eurostat).

Date of the first full delivery of data to the Commission (Eurostat). If the delivery of data does not meet the deadline laid down in Regulation (EU) 2019/1700, a reason for the delay should be given.

Description of how the requirements established in the specific domain have been fulfilled, including, where relevant, the impact of any deviations from the questionnaire and definitions.

Description of any problems of comparability between different regions of the country.

Information on the length of comparable time series, including the years when any series breaks occurred, and the reasons for them.

Comparison with external sources for all relevant variables, where the Member States concerned consider such external data to be sufficiently reliable.

If applicable.

If applicable.

Information on any lack of coherence in the output of the statistical process.

Information on:

Dissemination formats.

Documentation on methodology and quality.

Burden on respondents and, if available, cost associated with the collection and production of the statistical product. The average duration of household interviews should be stated. If possible and relevant, the duration of household interview should be reported by mode of data collection.

Information on ownership of data, indicating the extent to which their unauthorised disclosure could be prejudicial or harmful to the interests of the source or other relevant parties.

Confidentiality policy - description of any provisions in addition to European legislation that are relevant to the statistical confidentiality applied to the data.

Confidentiality – data treatment: general description of the rules applied to treating microdata and macrodata (including tabular data) with regard to statistical confidentiality.

Supplementary descriptive text that can be included in the quality report.