Andreas Schleicher, a statistician and researcher in the field of education, is the Division Head and coordinator of the OECD Programme for International Student Assessment (PISA) and the OECD Indicators of Education Systems programme. He recently made a visit to Michaela Community School, which you can read about in a blog entitled “where working hard and being kind” are part of the curriculum.
Michaela is a flagship school for current government policy and divides opinion along political lines. It is hard to envisage Schleicher’s intervention as anything other than a positive endorsement of current government policy, which advocates teacher-directed instruction against oft-described progressive approaches such as enquiry-based instruction.
In Ofsted terms, Michaela is a successful school, but also controversial. Headteacher, Katherine Birbalsingh, sprang to notoriety by delivering an impassioned speech at the Conservative Party conference in 2010. This blog, therefore, is about politicised discourse and PISA data.
What PISA actually says
In its 2015 report, Results Policies and practices for successful schools volume II, PISA attempted to identify effective teaching of science in schools. Online commentators have criticised PISA suggesting it has downplayed findings suggesting teacher-directed instruction was more effective than enquiry-based instruction, but included enquiry-based instruction as part of its definition of effective teaching.
Schleicher’s blog appears to accept this criticism suggesting that:
One of the most debated findings from PISA has been that teacher-directed instructional practices tend to better predict student achievement than student-oriented learning.
Schleicher goes on to address criticisms that the research approaches PISA adopts tends to favour teacher-directed instruction:
(…) others suggest that teacher-directed instruction only prepares well for tests predicated on recall and memorisation. But that’s not what PISA is about; to do well in PISA, students have to be able to extrapolate from what they know, think across the boundaries of subject-matter disciplines, apply their knowledge creatively in novel situations and demonstrate effective learning strategies.
It sounds convincing, but the reality is somewhat different. PISA simply asked science students how often certain activities happened in the classroom. Apparently, PISA considers students more reliable than teachers because teacher’s tend overstate the practices they perceive as currently fashionable:
While students may not always recall exactly what happens in their science lessons, students’ reports are often more reliable than teachers’ reports, as teachers will often overstate how much they expose their students to activities that are positively viewed by others (Hodson, 1993).
How does PISA define teacher-directed and enquiry-based instruction?
It is not easy to see from the report how PISA manages the data, but teacher-directed instruction appears to be defined by four variables (See Figure 1) whilst enquiry-based instruction by nine.
Figure 1 teacher-directed instruction
Teacher-directed instruction looks to be a solid if somewhat uninspiring teaching approach whilst more controversial enquiry-based approaches such as “spend time in the laboratory” or “design their own experiments” barely happens in OECD schools according to PISA data (See Figure 2).
Figure 2 enquiry-based instruction
The question in my mind, when looking at the data, is whether PISA is simply differentiating between good and bad teaching rather than nuanced teaching approaches? Further scrutiny of the report suggests so, but there are also other explanations for the data.
PISA found teachers used enquiry-based instruction in socio-economically disadvantaged schools concluding that “this correlational evidence should be interpreted with caution” because:
(…) teachers may be using hands-on activities to make science more attractive to disengaged students.
Indeed, teachers may use enquiry-based approaches in circumstances where students are less likely to be striving for test success or where behaviour is an issue. You can’t correlate social deprivation with bad behaviour but you can consider whether social deprivation is a proxy for underlying variables not identified by insufficiently nuanced research.
Do teachers use enquiry-based approaches in the UK?
Contrary to the opinion of some, the UK is below the OECD average for enquiry-based approaches (see Figure 3). The more controversial activities identified as enquiry-based learning is not prevalent in OECD countries, and even less so in the UK.
Figure 3 enquiry-based instruction in the OECD
Like much educational research, PISA attempts to define complex social practices by slicing and dicing data barely reflected by the categories then used to define those social practices. I think better methodological approaches to data would result in better research findings.
Nonetheless, PISA data has the potential to contribute much to educational debate. The research finding that teachers in most countries could be adapting their teaching method to reflect the needs of their learners is useful. I wonder what is happening in those countries that do not?
Ironically, Schleicher finishes the blog by saying “principal Birbalsingh listened to the research findings from PISA with great interest, but she founded her school on something simpler: common sense. Makes me wonder what motivated Schleicher to state such a thing? Can we infer anything about the upcoming PISA rankings due on the 3rd of December?
The point is, if PISA conducted better research and more accurately reflected its own research findings headteachers like Katherine Birbalsingh could rely on it more and less on “common sense”.