All that gets lost in our conversation around Pisa results.

This piece by Tom Pearce was originally published in The Post and The Press.

Every three years the OECD carries out reading, maths and science tests on 15-year-olds around the world, known as the Programme for International Student Assessment (Pisa). The 2022 results were released on Monday, kicking off the triennial round of panic about educational failure.

A lot of useful information comes out of international large-scale assessments (ILSAs) like Pisa. The 2022 results highlighted the pandemic, socio-economic differences, gender differences, distractions and attendance as significant issues to address in New Zealand’s education system, among other things.

Some education researchers will be eagerly digging into the data. For education researchers like myself however, how the data from these assessments is used is just as interesting as the data itself.

This time, as usual, teachers are being bashed, politicians are justifying whatever education reforms they were already planning, and most of the focus is on where New Zealand sits compared to other countries.

This reveals the increasingly political function of ILSAs. They form a powerful regime of truth around what counts as educational quality, especially within the discourse of a globally competitive knowledge economy. A discourse that has come to dominate our educational tradition.

Governments and policymakers are unable to ignore ILSAs. Domestic education policy around the world is increasingly shaped through reference to whichever countries are scoring highly, and Aotearoa is no exception. But we should question how valuable the rankings produced by ILSAs are as drivers of educational improvement.

For one thing, policy-makers understand neither the content of these assessments nor the impact of their decisions taken as a result. A raft of factors influence student performance in tests, well beyond particular policy settings or pedagogical approaches.

Poverty and inequality are huge factors. Unfortunately the response from our new Government, as well as others like free-market think-tank The New Zealand Initiative, is focused primarily on what they think teachers need to be doing better. There is no talk of addressing the barriers that poverty creates in education; a glaring omission from people claiming we need to use the“best scientific evidence”. Perhaps because seriously tackling poverty might mean compromising much of the new Government’s political platform.

Motivation is another key factor in ILSA scores. All tests are at least partly a measure of how much students actually care about doing well. Students with low motivation are much more likely to skip questions, guess, or not read closely.

Research suggests that our students are not as motivated to do well in Pisa as those in high ranking countries, possibly because there is nothing at stake for them in these tests.

Other research raises concern over transplanting educational approaches from one country to another. Teaching and learning are cultural and social activities as much as they are technical practices, so what works well in one context may not in another.

Politicians also cherry-pick educational policy and practice that aligns with their political narrative, further removing them from the context in which they work.

With National focused on curriculum reform we can expect “basics” to be lifted from places like top-ranked Singapore, but not policies like their massive investment in quality teaching through high teacher salaries. High pay and high status for teachers are unifying features of top-ranked Pisa countries.

Most importantly though, ILSA results do not paint a complete picture of what good education looks like.

Good education must produce the critical, empathetic, engaged young people needed to meet the challenges of the next 80 years. Challenges that already include catastrophic climate change, politics and media rife with disinformation, and a digital landscape shaped by generative AI and driven by algorithms that do more to divide us than bring us together.

This means that in their time at school, students need to develop competencies like self-regulation, relating well to others, thinking critically, and practising responsible digital citizenship; all of which defy test-based assessment.

And so we must be wary of the tail wagging the dog. ILSAs are one valuable source of data on student achievement, but we mustn’t let them limit the scope of educational improvement. Our education system needs to equip students with far more than what is measured by tests like Pisa.