Andreas Schleicher, a statistician and researcher in the field of education, is the Division Head and coordinator of the OECD Programme for International Student Assessment (PISA) and the OECD Indicators of Education Systems programme. He recently made a visit to Michaela Community School, which you can read about in a blog entitled “where working hard and being kind” are part of the curriculum.
Michaela is a flagship school for current government policy and divides opinion along political lines. It is hard to envisage Schleicher’s intervention as anything other than a positive endorsement of current government policy, which advocates teacher-directed instruction against oft-described progressive approaches such as enquiry-based instruction.
In Ofsted terms, Michaela is a successful school, but also controversial. Headteacher, Katherine Birbalsingh, sprang to notoriety by delivering an impassioned speech at the Conservative Party conference in 2010. This blog, therefore, is about politicised discourse and PISA data.
What PISA actually says
In its 2015 report, Results Policies and practices for successful schools volume II, PISA attempted to identify effective teaching of science in schools. Online commentators have criticised the fact that PISA’s own findings suggested teacher-directed instruction was more effective but included enquiry-based instruction as part of its definition of effective teaching.
Schleicher’s blog appears to accept this criticism suggesting that:
One of the most debated findings from PISA has been that teacher-directed instructional practices tend to better predict student achievement than student-oriented learning.
Schleicher goes on to address criticism of teacher-directed instruction:
(…) others suggest that teacher-directed instruction only prepares well for tests predicated on recall and memorisation. But that’s not what PISA is about; to do well in PISA, students have to be able to extrapolate from what they know, think across the boundaries of subject-matter disciplines, apply their knowledge creatively in novel situations and demonstrate effective learning strategies.
It sounds convincing but the reality is somewhat different. Students may have to extrapolate from what they know, think across boundaries, apply knowledge creatively, but in the study cited above PISA simply asked science students how often certain activities happened in the classroom.
Apparently, PISA considers students more reliable than teachers:
While students may not always recall exactly what happens in their science lessons, students’ reports are often more reliable than teachers’ reports, as teachers will often overstate how much they expose their students to activities that are positively viewed by others (Hodson, 1993).
PISA then grouped the activities into four categories (1) teacher-directed instruction, (2) perceived feedback, (3) adaptive instruction and (4) enquiry-based instruction.
How does PISA define teacher-directed and enquiry-based instruction?
It is not easy to see from the report how PISA manages the data, but teacher-directed instruction appears to be defined by four variables (See Figure 1) whilst enquiry-based instruction by nine.
Figure 1 teacher-directed instruction
The activities seem to be categorised somewhat haphazardly, even the most ardent instructivist would presumably offer teacher explanations of different phenomena and give students the opportunity to explain their ideas.
Teacher-directed instruction looks to be a solid if somewhat uninspiring teaching approach whilst more controversial enquiry-based approaches such as “spend time in the laboratory” or “design their own experiments” barely happens in OECD schools according to PISA data (See Figure 2).
Figure 2 enquiry-based instruction
The question in my mind, when looking at the data, is whether PISA is simply differentiating between good and bad teaching rather than nuanced teaching approaches? Further scrutiny of the report suggests so, but there are also other explanations for the data.
PISA found teachers used enquiry-based instruction in socio-economically disadvantaged schools concluding that “this correlational evidence should be interpreted with caution” because:
(…) teachers may be using hands-on activities to make science more attractive to disengaged students.
Indeed, teachers may use enquiry-based approaches in circumstances where students are less likely to be striving for test success or where behaviour is an issue. You can’t correlate social deprivation with bad behaviour but you can consider whether social deprivation is a proxy for underlying variables not identified by insufficiently nuanced research.
Just a thought, but how about asking teachers whether they adopt enquiry-based approaches in classrooms with less academic learners? Regardless, evidence should always be treated with caution; sadly, too few reflect the limitations of studies in public discourse.
Do teachers use enquiry-based approaches in the UK?
Contrary to the opinion of some, the UK is below the OECD average for enquiry-based approaches (see Figure 3). The more controversial activities identified as enquiry-based learning is not prevalent in OECD countries, and even less so in the UK.
Figure 3 enquiry-based instruction in the OECD
Like much educational research, PISA attempts to define complex social practices by slicing and dicing data barely reflected by the categories then used to define those social practices. I think better methodological approaches to data would result in better research findings. As it is I wonder whether PISA research says much about teaching approaches, whether, teacher-directed or enquiry-based. Consequently, where research findings suggest caution researchers should heed those warnings.
Nonetheless, PISA data has the potential to contribute much to educational debate. The research finding that teachers in most countries could be adapting their teaching method to reflect the needs of their learners is useful. I wonder what is happening in those countries that do not?
Ironically, Schleicher finishes the blog by saying “principal Birbalsingh listened to the research findings from PISA with great interest, but she founded her school on something simpler: common sense”. Makes me wonder what motivated Schleicher to state such a thing: Is he dancing to a political tune? Can we infer anything about the upcoming PISA rankings due on the 3rd of December?
It is true that common sense is better than politicised research. The point is if PISA more accurately reflected its own research in public discourse headteachers like Katherine Birbalsingh could rely more on it and less on “common sense”; an odd phrase for an influential statistician who seeks to offer reliable evidence to educational policymakers.