An instrumentalist education system that cannot measure its own progress is not fit for purpose

The education system likes to think it can measure things that are notoriously hard to measure. Increasingly pragmatic instrumentalism is being held culpable  (more of that in a future BLOG). However it appears that the education system is struggling to measure its own progress. You wonder therefore what, if anything, it can measure with any certainty? It’s not just me, others are wondering too.

In the beginning there was…..

VAK learning styles. It’s purpose was not pedagogical but as a means to evidence personalisation and differentiation. Most now agree that VAK learning styles cannot identify the personal cognitive preferences of an individual learner based on; visual, auditory and kinesthetic characteristics.

Teacher’s throughout the land conducted tests, identified the personalised learning style of the individual learner and wrote up the results. Some might even have got a good grade in an OFSTED observation because of it. Then there was light and we all saw through VAK learning styles.

Yesterday’s issue of personalisation, a policy tool of New Labour,  has been replaced by the new policy mantras of the coalition. Ideas abound: closing the gap, mind growth, visual learning etc.

Of course there is a counter narrative; Professor Rob Coe’s assertion that education hasn’t measurably improved. David Didau also has a point about memory retention. What do learners learn? Is there much point learning stuff you won’t remember or don’t really understand simply to scrape a “C” at GCSE? Individual learners benefit but what about society?

Is the culture of compliance, rote learning and constant exams the kind of environment that will deliver the innovative thinking required in our social institutions and in 21st century business? It may deliver measurable outcomes but little else.

And you will know the truth, and the truth will set you free

So what is the truth about learning? The truth is Nearly 40 per cent of schools leavers do not achieve a grade “C” at GCSE English:

Almost one in five are functionally illiterate when they leave school and this has been the case for some time. A recent article in TES wrote about the problems of functional illiteracy and measuring; reading and writing skills:

The Sheffield report – The levels of attainment in literacy and numeracy of 13- to 19-year-olds in England, 1948-2009 – says the latest evidence shows that 22 per cent of 16- to 19-year-olds are functionally innumerate. Professor Greg Brooks, one of the study’s authors, said this had remained at around the same level for at least 20 years.

The latest evidence on reading shows 17 per cent of 16- to 19-year-olds are functionally illiterate and Professor Brooks said this had also been the case for at least two decades. “People at this level can handle only simple tests and straightforward questions on them where no distracting information is adjacent or nearby,” his report says. “Making inferences and understanding forms of indirect meaning, eg allusion and irony, are likely to be difficult or impossible. This is less than the functional literacy needed to partake fully in employment, family life and citizenship and to enjoy reading for its own sake.”

The Sheffield study found that average reading scores for 13- to 19-year- olds had improved between 1948 and 1960 and remained “remarkably constant” from 1960 to 1988. Data showed a “gentle rise” between 1997 and 2004, followed by a further plateau. The researchers found no evidence on achievement in writing before 1979 and no significant change between 1979 and 1988. Examination scripts suggested this static picture continued until 2004.

Sheffield’s figures are contradicted by Skills for Life 2011 (SfL). This research put the functional illiteracy level of 16-18 year-olds at 14% not 17%. But SfL found functional innumeracy was worse, 27% not 22%. These figures in turn are contradicted by GCSE results.

We look not to the things that are seen but to the things that are unseen

As the TES article states; these figures in turn are contradicted by GCSE results. All roads lead to the thorny issue of grade inflation, which makes it difficult to understand what is happening in schools. A brief look at grade inflation shows it has been dramatic. In fact in one period from 1988 to 1994 the A – C success rate improved by ten per cent; in 5 years. It’s not possible, is it?

The grade A* grade (2012) introduced in 1994 now has similar success rates to the old A grade in 1988:

It is possible to ascertain some relational information based on one year but beyond that it’s impossible to make any credible judgement on the data. It’s almost completely meaningless. How do you measure progress with data that few now have faith in?

OFSTED

What does the sector watchdog make of this situation? Well OFSTED is currently more credible than it has been in the past; for most of the last 10 years it’s been frankly talking complete gibberish. It’s a highly political organisation reliant on data that is highly politicised. It’s own data as well as the data derived from GCSE’s. OFSTED’s problem being that its inspectors simply haven’t seemed to know the difference between good and bad teaching

Even if you are generous it may award grades in one year but has no real way of knowing how those grades equate over time. The base assessments change as do the frameworks as does the data. This is just one of a number of recent critical reports:

 Certainly, heads and teachers seem unconvinced that inspection, in its present form, is an
effective way of securing school improvement.  One study found two-thirds of heads felt inspection did not lead to improvement, whilst another for Ofsted suggested only 35% of schools felt the benefits of inspection outweighed the bad effects (Thomas). Both were in the early days of Ofsted but a survey in 2008 found only 5% of teachers thought inspection had positively influenced their teaching ‘to a great extent’, whilst another 40% felt it had made no difference at all (ofsted.gov.uk/node/2315).

In 2004, however, a study commissioned by Ofsted had claimed that inspection did lead to widespread improvement. It is difficult to determine Ofsted’s role in securing school improvement as there is so little agreement as to whether schools have improved at all in the past 20 years and, if they have, by how much.

Debates about standards continue despite Ofsted’s own evidence pointing to schools having improved. Nor have changes to the Ofsted process – ‘the fiddling at the margins’, as one senior ex-HMI described it – clarified its contribution to school improvement.

In the 20 years since Ofsted’s foundation there have been 10 new inspection frameworks. The impact of other educational reforms and wider social factors must also be considered. Kenneth Clarke had said that informing parents better about schools would be a key aim of Ofsted and an early survey found most parents thought inspection reports helpful and accessible (Tabberer).

In 2004, 85% of parents surveyed by Ofsted said they would read the inspection report if choosing a new school (Ofsted [2]), although other evidence suggests many parents place more weight in choosing a school on local reputation than any published material (Maras, Moon and Bradshaw).

Some 20 years after its foundation, Ofsted’s role in the English education system remains
both significant and highly controversial

Conclusion

Rob Coe’s research is food for thought . Martin Robinson’s BLOG on pragmatic instrumentalism and grit is also thought provoking. Pragmatic instrumentalism is based on a practical “what works” approach.

The problem is nothing seems to work in education data simply isn’t joined up. It’s particularly concerning when schools use cramming, testing and excessive use of feedback to check learning. Research has shown that knowledge is not retained blaming exam cramming and the lack of conceptual development, You do wonder what, if any, learning is going on.

Statistics that don’t measure anything worth measuring; gurus and personalities making claims based on dubious interpretations of research and personality head teachers that brag about making teachers quit their jobs is only justifiable if you can know for sure that progress is being made.

The data has to join up and learning has to be purposeful. Whatever pedagogical polemic individual educationalists hang onto I think we can all agree that an instrumentalist education system that cannot measure its own progress is not fit for purpose.