8.4 Learning progress | Covid-19 Lessons Learned

Covid by the Numbers Report

Download report 9.8 MB

8.4. Learning progress | Te ahunga ako

It is difficult to quantify the impact of the pandemic on learning progress. We can observe snapshots of achievement of cohorts of students across the world through international studies such as those led by the OECD (section 8.4.1).

In New Zealand, we cannot use National Certificate of Educational Achievement qualifications to assess the impact of the pandemic on student learning, as there were changes to assessment standards during 2020, 2021 and 2022 so that students were not disadvantaged in later life by the qualifications they attained under pandemic learning conditions.⁴⁵

However, the Ministry of Education was able to examine student learning for students in Years 4–10 (section 8.4.2). We have also commissioned new research on this topic (section 8.4.3).

8.4.1. International comparisons of 15-year-olds | Ngā whakataurite ā-ao mō ngā 15-tau

New Zealand participates in the OECD-led Programme for International Student Assessment (PISA), which tests the competency of 15-year-olds' application of reading, mathematics and science to real-life problems.⁴⁶

The strength of international studies such as PISA is that they use consistent methods that enable comparison across countries and over time. However, they are conducted at 3–5-year intervals and are therefore better suited to understanding the evolution of the education system rather than sudden shocks to the system.

That said, the results from the 2022 PISA survey allow comparison of the proficiency of 15-year-olds with results of previous surveys and results from other countries.

New Zealand has been experiencing a long-term decline in mathematics proficiency, as measured by PISA. This decline continued in 2022 (Figure 119). Of the comparator countries shown in the Figure, all except Australia and Japan experienced a statistically significant decline in PISA scores between 2018 and 2022.

Figure 119: Programme for International Student Assessment, proficiency in mathematics

PISA average proficiency in mathematics, New Zealand nd comparator countries, 2003–2022

Source: Organisation for Economic Co-operation and Development, PISA 2022 Database https://www.oecd.org/en/data/datasets/pisa-2022-database.html, Table 1.B.5.4
Notes: Decline in PISA scores between 2018 and 2022 are statistically significant in all countries except Australia and Japan.

Similar declines occurred in New Zealand students' reading and science proficiency (Figure 120). The pattern, however, was more mixed in our comparator countries.

Figure 120: Programme for International Student Assessment, proficiency in reading and science

PISA average proficiency in reading and science, New Zealand and comparator countries, 2000–2022 (reading) and 2006–2022 (science)

a) Reading

b) Science

Source: Organisation for Economic Co-operation and Development, PISA 2022 Database https://www.oecd.org/en/data/datasets/pisa-2022-database.html, Tables 1.B.5.5 and 6
Notes: Declines in reading performance between 2018 and 2022 were statistically significant in Hong Kong, Sweden and the United Kingdom, as was the increase in Japan's score. Increases in science performance were statistically significant in Italy and Japan.

Research published in 2025 using PISA data⁴⁷suggests that learning loss in mathematics from school closures due to the pandemic increased with the length of the closure (Figure 121). The learning loss was roughly equal to seven months of learning across the countries in the PISA dataset. This was measured by the deviation of mathematics scores in the 2022 PISA study from the trend since 2000. The researchers measured the length of school closures four different ways, all of which gave similar estimates of learning loss.

Figure 121: Programme for International Student Assessment, estimated learning loss in mathematics, by length of school closure

Deviation of mathematics scores in the 2022 PISA study from the trend since 2000, four measures

Source: Maciej Jacubowski, Tomasz Gajderowicz and Harry Anthony Patrinos, ‘COVID-19, school closures, and student learning outcomes. New global evidence from PISA’, npj Science of Learning. 10,5 (2025), pp 1–7
Note: -20 corresponds with one year of learning loss.

8.4.2. Measuring the impact of the pandemic on learning in Years 4–10 in New Zealand using e-asTTle data | Te ine i te pānga o te mate urutā
ki te ako i ngā Tau 4–10 i Aotearoa mā te raraunga e-asTTle

New Zealand teachers use assessment tools as a part of teaching practice, including the e-asTTle tool.⁴⁸ The Ministry of Education used anonymised results from that tool to understand the patterns of learning progress in reading, writing and maths for students in Years 4–10 (that is those aged approximately 8–15). This data covers hundreds of thousands of students, but it is not a representative sample of the student population as not all teachers or schools use e-asTTle.

The Ministry's initial analysis⁴⁹ found that reading and maths learning progress for most student groups in 2020 was essentially unchanged compared with previous years. There was some evidence that progress in writing was slower. This amounted to an equivalent loss of five weeks of learning. This is in the context of declining writing progress in most year groups since 2013.

The Ministry of Education looked at whether these results varied by demographic factors, but found no significant change in pre-existing patterns of learning progress by ethnicity, by socioeconomic background,⁵⁰ or in Auckland relative to the rest of New Zealand.⁵¹

The Ministry subsequently updated its analysis.⁵² The updated analysis suggests that any learning losses in 2020 were reversed in subsequent years (2021 and 2022). See Figure 122 for the combined results. The changes experienced during the COVID-19 pandemic were not unprecedented in previous years. For example, Year 9 and 10 writing scores have varied quite considerably from year to year since 2013.

Figure 122: Yearly progress in reading, writing and mathematics, Years 5–10 in New Zealand
As measured by e-asTTle, 2012–2022

(a) Reading

(b) Writing

(c) Mathematics

Source: Andrew Webber, "Student learning during COVID-19: Literacy and Maths in Years 4–10 updated to 2022', (Wellington: Ministry of Education, 2022).
Notes: Average yearly progress calculated by average difference in e-asTTle scores.

8.4.3. Measuring the impact of the pandemic on learning in Years 4–10 in New Zealand using Progressive Achievement Test data | Te ine i te pānga o te mate urutā ki te ako i ngā Tau 4–10 i Aotearoa mā te raraunga Progressive Achievement Test

Another tool used by teachers to understand students' achievement levels and learning progress is the Progressive Achievement Test (PAT) tool. PATs are standardised, multiple-choice assessments developed by the New Zealand Council for Educational Research (NZCER). PATs can be administered to students in Year 4 through Year 10 (roughly ages 8–15).

We asked the NZCER to analyse data from schools that used the PAT to measure students' mathematics and reading comprehension over the period of the pandemic. A comparison of the average PAT scores during and after the pandemic with scores from 2019 allows us to investigate the impact of the pandemic on learning. Comparing schools in Auckland with the rest of the country helps us understand how the extended 2021 Auckland lockdown affected learning.

Figure 123 shows the difference in average mathematics learning over 2020, 2021 and 2022, relative to that during 2019.⁵³ We provide error bars to distinguish between changes that are statistically significant, and those that might be due to random variation.⁵⁴ In the following text, we focus on observations that are statistically significant.

The average mathematics learning score (the orange line) fell in 2020, fell even further in 2021, then rose in 2022 to slightly above their 2019 levels (Figure 123). The drop between 2019 and 2021 represents about one month of learning.⁵⁵ On average, students leaving at the end of 2021 experienced the 2021 learning loss in mathematics but did not experience the 2022 recovery.

Mathematics learning scores in Auckland (the green dotted line) are normally slightly above those for the rest of New Zealand (the blue dashed line). This difference is not statistically significant in 2019 but is in 2020. However, in the year when schools in Auckland were affected by the extended lockdown, their students' mathematics scores dropped below those of the rest of New Zealand. This difference was statistically significant, even when accounting for differences in populations.⁵⁶ By 2022, learning scores at Auckland schools had risen, and those scores were statistically indistinguishable from non-Auckland schools.

Figure 123: Mathematics learning, Years 4–10 in New Zealand, Auckland and non-Auckland

Difference between average mathematics learning scores in 2020, 2021 and 2022 compared to the national average in 2019, as measured by Progressive Achievement Test

Figure 124 shows the equivalent results for reading comprehension. These show a different pattern to the mathematics results. National reading comprehension scores dropped in 2020 and 2021. Both drops were statistically significant. Average reading comprehension stayed below 2019 levels in both 2022 and 2023. On average, students leaving at the end of 2020, 2021, 2022 and 2023 suffered a reading comprehension learning loss relative to those leaving at the end of 2019.

Reading comprehension scores in Auckland are statistically indistinguishable from those in the rest of New Zealand, except from 2021. Auckland students did experience a statistically significant fall in reading comprehension scores in 2021, when they shifted from slightly above the rest of New Zealand to slightly below. This fall coincided with the extended Auckland lockdown, like the pattern we saw with mathematics scores.

Figure 124: Reading comprehension learning, Years 4–10 in New Zealand, Auckland and non-Auckland

Difference between average reading comprehension learning scores 2020–2023 compared to the national average in 2019, as measured by Progressive Achievement Test

Figure 125 shows how learning scores for mathematics and reading comprehension differed according to socioeconomic status, as measured by the Ministry of Education's Equity Index.⁵⁷ The Equity Index is an estimate of the extent to which the students attending a school face socioeconomic barriers to achievement in education. Schools are divided into three bands – schools where their students face 'fewer' socioeconomic barriers to achievement, 'moderate' socioeconomic barriers to achievement, or 'more' socioeconomic barriers to achievement.⁵⁸ The following analysis is of students, where each student is assigned the Equity Index of their school.

There is no simple relationship between socioeconomic barriers and learning loss. Auckland students from all socioeconomic backgrounds experienced falls in mathematics scores in 2021, when Auckland was in extended lockdown unlike the rest of the country (panel (a) in Figure 125). This drop in scores for all groups was statistically significant relative to the national average in 2019. Mathematics scores for all groups recovered in 2022, such that they were almost identical to what they were prior to the pandemic.

Conversely, there was relatively little change in schools outside of Auckland over the entire period. Students in schools that face 'more' socioeconomic barriers saw almost no change in their mathematics scores. These are the students with the lowest overall mathematics achievement, but they did not drop further. Students with fewer or moderate barriers, saw a small decline in 2020 and 2021, recovering in 2022.

The most disadvantaged students (those in the 'more' barriers group) experienced a greater drop in mathematics score in Auckland than they did outside of Auckland. Similarly, the least disadvantaged students (those in the 'fewer' barriers group) in Auckland also experienced larger drops than their compatriots in the rest of New Zealand. Both differences were statistically significant.

The pattern for reading comprehension (panel (b) in Figure 125) is different, and less clear. Students in Auckland schools with more socioeconomic barriers had lower reading comprehension scores than those with moderate or fewer socioeconomic barriers in 2019, before the COVID-19 pandemic began.59This pattern remained (indeed the differences declined slightly) over the period of the pandemic. While there were declines in all groups in 2021, these were too small to be statistically significant.

Conversely, the students with more socioeconomic barriers outside of Auckland improved their reading comprehension relative to those with moderate of fewer barriers prior to COVID-19. It is the students with moderate socioeconomic barriers who experience the largest fall in reading comprehension in schools outside of Auckland. Between 2019 and 2021, reading comprehension in the top and bottom socioeconomic groups (the fewer and more barrier groups) remained unchanged, and only began to fall slightly in 2021.

Figure 125: Learning by socioeconomic group, Auckland and non-Auckland

Difference between average learning scores and the 2019 national average, by Equity Index of student's school, for (a) mathematics 2020–2022 and (b) reading comprehension 2020–2023

(a) Mathematics

(b) Reading comprehension

Source: NZCER, COVID-19 and student achievement in Aotearoa New Zealand (Wellington: 2025)
Notes:
1. Learning scores for a given year relate to tests taken at the beginning of the following year.
2. Error bars denote 95% confidence intervals.
3. Fewer/moderate/more = students at schools whose students face fewer/moderate/more socioeconomic barriers to achievement.

Figure 126 shows patterns of learning across students of different ethnicities. Mathematics learning scores moved broadly together within Auckland and the rest of New Zealand. Mathematics scores for North Asian students outside of Auckland increased over the whole period, as did those for South Asian students, from a lower base. While patterns outside of Auckland largely remained flat or increasing, those within Auckland show a clear dip in 2021. The difference in scores between Māori ākonga in Auckland and those outside Auckland was around 1.16 units in 2021, and remained 0.73 units in 2022 (both differences between Auckland and Non-Auckland scores for Māori were statistically significant). It was a similar story for Pacific students in 2021, with students in Auckland achieving scores in mathematics 0.7 units lower than those in the rest of New Zealand. However, this difference had disappeared for Pasifika by 2022.

Reading comprehension scores for Māori ākonga declined in both Auckland and the rest of New Zealand. The reading comprehension scores of Māori in Auckland remained below those of outside of Auckland over the entire period. This difference was statistically significant in all years, excepting 2023 when the rise in Auckland narrowed the difference. The reading comprehension of Māori ākonga outside of Auckland was more like students of other ethnicities. Once more, there was a significant difference in Non-Auckland North Asian students and those of other ethnicities over the entire period. This difference expanded as the reading comprehension scores of North Asian students increased for much of the period.

Figure 126: Learning by ethnicity, Auckland and non-Auckland

Difference between average learning scores and the 2019 national average, by ethnicity, for (a) mathematics 2020–2022 and (b) reading comprehension 2020–2023

(a) Mathematics

(b) Reading comprehension

⁴⁶ See New Zealand Qualifications Authority, 'COVID-19 related changes to NCEA and University Entrance', last updated 12 October 2023, https://www2.nzqa.govt.nz/ncea/about-ncea/covid-19-related-changes-to-ncea-and-ue for further details.

⁴⁶ New Zealand also participates in two studies led by the International Association for the Evaluation of Educational Achievement: The Progress in International Reading Literacy Study (PIRLS) study of reading (comprehension) achievement in 9/10-year-olds, and the Trends in International Mathematics and Science Study (TIMSS) assessment of the mathematics and science knowledge of year 5 and 9 students.

⁴⁷ Maciej Jacubowski, Tomasz Gajderowicz and Harry Anthony Patrinos, 'COVID-19, school closures, and student learning outcomes. New global evidence from PISA', *npj Science of Learning*, Volume 10, Issue 5 (2025), pp. 1–7.

⁴⁸ For more on e-asTTle, see Ministry of Education, 'Achievement and progress in mathematics, reading and writing in primary schooling: Analysis of e-asTTle assessment data, 2011 to 2016', (Wellington: April 2018), https://www.educationcounts.govt.nz/__data/assets/pdf_file/0019/185023/20171213-Achievement-and-Progress-in-mathematics-reading-and-writing.pdf

⁴⁹ Andrew Webber, 'Student learning during COVID-19: Literacy and maths in years 4–10', *he Whakaaro: Education Insights*, (Wellington: Ministry of Education, 2021)

⁵⁰ As measured using the old decile system.

⁵¹ Note that there is not enough data to examine the progress of students in Māori medium and settings such as specialist schools, alternative education providers, teen parent units, or activity centres.

⁵² Andrew Webber, 'Student learning during COVID-19: Literacy and Maths in Years 4–10 updated to 2022', (Wellington: Ministry of Education, 2022).

⁵³ The mathematics test was changed in 2023, so there is no time-series consistency between that year and the previous years.

⁵⁴ 'Statistically significant' in the text means that the difference is statistically significant at the 95% level. That is, there is only a 5% chance that this difference could have occurred by chance. For the national data, if the error bars above and below a 2020, 2021 or 2022 data point (what is called the confidence interval) do not include zero, we can say the difference is statistically significant. When comparing the Auckland with the non-Auckland data, if the error bars for a datapoint are non-overlapping, the difference is statistically significant.

⁵⁵ One year's progress tends to be between 6–7 units in upper primary years, and about 3–5 units in later years.

⁵⁶ NZCER also used a statistical model to account for differences in populations across the gender, socioeconomic background and ethnicity of students (as well as their year of study) between Auckland and non-Auckland schools. They found that once they accounted for these differences, scores in Auckland schools fell by 0.73 scale points in 2021.

⁵⁷ For more information on the Equity Index, see Ministry of Education, He Whakaaro: Accounting for educational disadvantage, (Wellington: Ministry of Education, 2019), https://www.educationcounts.govt.nz/publications/schooling/he-whakaaro-accounting-for-educational-disadvantage

⁵⁸ While the bands contain approximately the same number of schools, those schools with more socioeconomic barriers to achievement tend to be smaller, which means that the three bands contain unequal numbers of students.

⁵⁹ Although the difference from the moderate group is not statistically significant, because of the larger variation within that group (in other words, the 95% confidence intervals are wider than those for students in the fewer group).