The matrix revisited: where is the evidence in the Sutton report?

matrix revisitedIn my last BLOG I offered the view that neo-traditionalist pedagogy offers a diminished view of teaching and learning.

The recently released Sutton report has been presented as being in support of traditionalist pedagogic approaches. I’m not sure that it consciously does but like most aspects of  the social world it reflects fashionable discourse. The cited authors say much about the perspective of the report.

Actually unpicking the essence of the report is not easy. It is presented as one thing by the Sutton Trust but seems to be another when the body of the report is further scrutinised. It seems to be both a review of the available research, a list of what is good and bad about teaching from a causal perspective and finally, I suspect, it’s actual purpose, which is an attempt at developing a framework for evidence based practice.

It would actually make the basis of a useful research primer for teachers, or perhaps an on-going bulletin to ensure that the profession is continually updated with the latest research information. What is isn’t, or doesn’t seem to me to be, is any kind of  guide to research. Of course, I have no doubt the authors would contextualize, and indeed do, that claim anyway.  Unfortunately it’s purpose is to make headlines and grab attention, that’s the nature of think-tank’s or in the case of the Sutton Trust, “do-tank” And therein lies the problem.

Some of the headline statements seem to be underpinned by relatively little research evidence. For example:

(Pedagogical) Content knowledge

A number of studies have found a relationship between measures of a teacher’s knowledge of the content they are teaching and the gains made by their students. It seems intuitively obvious that ‘Teachers cannot help children learn things they themselves do not understand’ (Ball, 1991, p5).

However, the search for a relationship between characteristics such as academic qualifications or general ability and student performance has been rather disappointing: correlations are typically very small or non-existent (Rockoff et al, 2011).

The view that learners can’t be taught, that which the teacher doesn’t know, seems to be the underpinning statement. The rest of the text offers a more nuanced view of teacher knowledge. It seems to be that if you know nothing or relatively little about a subject you can’t easily teach it. Other than that content knowledge is not such a big issue.

Others passages have even less evidence. Discovery learning is written off with a simple reference to Kerschner et al:

Allow learners to discover key ideas for themselves

Enthusiasm for ‘discovery learning’ is not supported by research evidence, which broadly favours direct instruction (Kirschner et al, 2006). Although learners do need to build new understanding on what they already know, if teachers want them to learn new ideas, knowledge or methods they need to teach them directly.

And finally no evidence at all. This passage contains one of DT Willingham’s enigmatic quotes, but it’s hardly relevant:

Ensure learners are always active, rather than listening passively, if you want them to remember

This claim is commonly presented in the form of a ‘learning pyramid’ which shows precise percentages of material that will be retained when different levels of activity are employed. These percentages have no empirical basis and are pure fiction. Memory is the residue of thought (Willingham, 2008), so if you want students to remember something you have to get them to think about it. This might be achieved by being ‘active’ or ‘passive’.

I suppose much will depend upon the definitions of active and passive. There is an awful lot of theory and educational thinking disposed of in one paragraph. You would assume that being actively engaged, whether in an activity or sat listening to an engaging speaker is better than passively doing one or the other. The terms derive, I assume from on-going polemics about traditionalism or progressivism and some of the context is lost in translation.

Interestingly the one model that the report claims to have some efficacy, Creemers and Kyriakides’ Dynamic Model (see: appendix 1) actually includes references to; group work, problem solving, modelling and hints at self efficacy and inquiry / discovery learning. It’s complexity is such that you do wonder how it was tested empirically (the report makes the claim but provides no references).

A huge body of research in the educational effectiveness tradition has focused on the characteristics of schools and teachers that are associated with high learning gains. Much of the evidence is correlational, cross-sectional and lacking a strong theoretical foundation (Scheerens et al, 2001).

However, the Dynamic Model (Creemers & Kyriakides, 2006, 2011) is empirically grounded, well enough specified to be testable and has indeed been subjected to considerable testing and verification.

Anecdotally speaking, as a guide to classroom practice, it’s not bad at all. My guess is that this model reflects how many in the profession actually teach but it bares little relation to what is described in the executive summary.

The problem is that each aspect of this model is a research field in itself. Every activity can be done well or poorly and that will impact upon learning. For example; at what point does feedback become “spoon feeding”.  Pedagogical approaches are not intrinsically good most are good dependent upon context.

There is a purposeful project here but any claims to being evidence-based is a little far fetched. Writing off an entire field with one reference to a heavily criticized report does not in my view constitute an evidence based approach. I suppose that it is the price you pay when you attempt to find causal relations for complex social situations.

In the end you are left wondering where the evidence actually is,  and pondering whether this is really aimed at policy makers rather than educators. It seems to play to a narrative that is popular at the moment and has a very political feel about it. Its essential incoherence stems from the need of someone, or other, to lose all context and purpose by deriving a bullet point list of what is good or bad about teaching and learning. Avoiding that kind of nonsense is surely the purpose of evidence. Then again evidential practice doesn’t lend itself to “do tanks” and newspaper headlines.

The history of education suggests that even well researched initiatives can be poorly understood and implemented in practice particularly if they are traduced into sound bites and executive summaries. If there is one thing to be learned from the Sutton report it is that you cannot shortcut pedagogy or pedagogical knowledge into a series of bullet points. It doesn’t work, the evidence is overwhelming.

So my, humble or maybe not, advice would be to maintain the project but ditch the sound bites albeit given the politicised nature of education that is possibly easier said than done.

Appendix 1

The dynamic model of educational effectiveness (Creemers & Kyriakides, 2006)

(1) Orientation

(a) Providing the objectives for which a specific task/lesson/series of lessons take(s) place

(b) Challenging students to identify the reason why an activity is taking place in the lesson. (2) Structuring (a) Beginning with overviews and/or review of objectives (b) Outlining the content to be covered and signalling transitions between lesson parts (c) Drawing attention to and reviewing main ideas.

(3) Questioning

(a) Raising different types of questions (i.e., process and product) at appropriate difficulty level

(b) Giving time for students to respond

(c) Dealing with student responses

(4) Teaching modelling .

(a) Encouraging students to use problem-solving strategies presented by the teacher or other classmates

(b) Inviting students to develop strategies

(c) Promoting the idea of modelling

(5) Application

(a) Using seatwork or small-group tasks in order to provide needed practice and application opportunities

(b) Using application tasks as starting points for the next step of teaching and learning.

(6) The classroom as a learning environment

(a) Establishing on-task behaviour through the interactions they promote (i.e., teacher–student and student–student interactions)

(b) Dealing with classroom disorder and student competition through establishing rules, persuading students to respect them and using the rules.

(7) Management of time

(a) Organizing the classroom environment

(b) Maximizing engagement rates.

(8) Assessment

(a) Using appropriate techniques to collect data on student knowledge and skills

(b) Analysing data in order to identify student needs and report the results to students and parents.

(c) Teachers evaluating their own practices.


3 thoughts on “The matrix revisited: where is the evidence in the Sutton report?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s