Practical information about use of PIAAC data

PIAAC data gives valuable information on how information processing skills differ between the adult populations in different countries and within different groups of the population. In addition, the survey also contains a lot of other variables making it possible to see skills in relation to a range of other variables, thereby providing possibilities for using the data in policy-based analyses. Yet, the nature of the data makes them complex to use and to interpret. Below, you can find more information about how to use and interpret PIAAC data.

How do I interpret and use results from the PIAAC survey?

PIAAC cycle 2 measures adult skills in literacy, numeracy and adaptive problem solving. According to the OECD, the skills in PIAAC cycle 2 are defined as follows:

“Literacy is accessing, understanding, evaluating and reflecting on written texts in order to achieve one’s goals, to develop one’s knowledge and potential and to participate in society.”

“Numeracy is accessing, using and reasoning critically with mathematical content, information and ideas represented in multiple ways in order to engage in and manage the mathematical demands of a range of situations in adult life.”

“Adaptive problem solving involves the capacity to achieve one’s goals in a dynamic situation, in which a method for solution is not immediately available. It requires engaging in cognitive and metacognitive processes to define the problem, search for information, and apply a solution in a variety of information environments and contexts.”

These are key-information-processing skills that are important for accessing, analyzing information and that are of high importance for functioning in today’s working life. To measure skills, a cognitive assessment is undertaken. To understand the nature of the data, find more information about the survey design of PIAAC cycle 2 on the websites of the OECD.

In PIAAC cycle 1, skills were measured in literacy, numeracy and in problem-solving in technology-rich environments. In PIAAC cycle 2, the latter is replaced with adaptive problem solving. In the technical report for PIAAC and in Reader’s companion you can read more about the survey design in PIAAC1 and find more information about the definition of these skills.

PIAAC data is complex and must be interpreted carefully. In each of the three skills domains, data is reported in terms of proficiency scores. These proficiency scores are estimated using Item Response Theory. Yet, respondents do not answer all items and therefore, there is uncertainty connected to the test results. To account for this uncertainty, a set of so-called plausible values represents the distribution of a person’s proficiency.

The higher the score achieved by a respondent, the higher the proficiency of the particular skill and the more complex items the respondent has managed to solve. Based on the proficiency scores, different proficiency levels are derived. These proficiency levels give information on the distribution of different groups with regard to their skill levels. For literacy and numeracy, there are 6 proficiency levels, while for problem solving in technology rich environments, there are 4 different levels. To calculate proficiency levels and to use the skill variables in analyses, special analysis tools are needed.

Which tools can I use to analyse the data?

PIAAC-data use complex psychometric designs. Thus, the nature of the data makes it necessary to use tools specifically designed to analyse the data, using advanced procedures to estimate results. Tools to analyse PIAAC data have been developed for several programming languages:

• A macro for SAS-users
• The STATA Repest macro and the STATA command piaactools
• The International Database (IDB) Analyzer (SPSS)
• R-package intsvy
• The International Data Explorer (IDE) is accessible via the webpages of the OECD. With this tool, you can create tables and charts to explore adult skills by country and in relation to different background variables, read more here.

You can find more information and a documentation of these tools at the webpages of the OECD.

How can I use the results for developing policy measures?

PIAAC data give information about key adult skills for information processing; literacy, numeracy and problem-solving. PIAAC data can be used to gain more information about how such skills differ between the adult populations in the different participating countries. For example, which of the participating countries has the highest share of adults performing at the highest level in literacy? Moreover, PIAAC data make it possible to explore how such skills vary between different groups within the countries’ populations, as PIAAC contains a lot of different background variables. Hence, the results of the survey make it possible to gain more information about the extent of low skills and to identify the low-skilled groups within the population. For example, PIAAC data make it possible to explore whether skills vary according to age, gender, immigration background, educational background, employment status or health.

Moreover, the data give important insight to factors that influence the development of skills. In addition to demographic background variables, PIAAC contains information about highest educational attainment, participation in education and training, participation in the labour market, as well as on the use of different skills in daily life and at work. Hence, data can also be used to gain more insight into the education and training systems of different countries and how people transition from education to work.

You can find more information about the survey’s content to explore possibilities of analysing the data in the documentation provided on the webpages of the OECD and in the background questionnaires for the different countries, see here.