On critical realism in education – part one: the neo-traditionalist fallacy, meta analysis and other epistemic sleights of hand

This eduBLOG was inspired by a “twittersation” with @imagineinquiry last week about critical realism.

I aim to discuss critical realism over two blogs. The first is to show that neo-traditionalism and its underpinning ideologies can never work in education.

The second  blog will deal with “fixing” progressivism, for want of a better word, and to show how an ontological position such as critical realism can help to better understand knowledge in educational practice and the role it plays in constructing reality.


The underlying epistemic perspectives of neo-traditionalism and progressivism are: positivism and constructivism respectively. It is quite possible that either or both would reject the notion of a label even whilst accusing the other of being one. Either way both are epistemological perspectives that struggle to offer an ontological explanation of knowledge. In this blog I will look at neo-traditionalism, its underpinning positivist philosophy and aim to show that neo-traditionalism offers a theory of how we acquire knowledge (epistemology) but cannot offer an explanation of knowledge itself (ontology).

Moore (2007) describes positivism as such:

This view (associated with logical positivist theories of knowledge and science) sees the social as representing, actually or potentially, the distortion of knowledge through historically received metaphysics or superstition, arbitrary traditional authority, bias and sectional interest. Knowledge is secure only when founded upon what can be empirically demonstrated to be in fact the case.

Although this kind of positivism ceased to have any serious standing in epistemology and the philosophy of science many decades ago (it was dealt a fatal blow by Karl Popper in the 1930s), its spectre still haunts certain areas of contemporary thinking in the humanities and social sciences (see Moore & Muller, 1999).

I would argue that neo-traditionalism has adopted, and adapted, the position of logical positivism. Their view is that although science is the best way to acquire knowledge; science cannot always provide the answers. Neo-traditionalists therefore argue that in the absence of a scientific perspective there is an arbitrary body of knowledge described as “the best that has been thought and said”, which will have to do instead.

In other words neo-traditionalists replace ontology with the view that knowledge; just is. It is one of a number of sleights of hands used by neo-traditionalists to circumvent ontology.

Critical realism: structure and agency

According to critical realists the fundamental ontological questions in education can be viewed through the prism of structure and agency:

The ontological assumptions in the social sciences pertain to the structure–agency problem, i.e. the problem of defining social reality in terms of structures, agents or some form of structure–agency interplay. Bhaskar (1998) criticizes structuralist positions for determinism, and individualist positions for failing to account for the social context, with the consequence being that for him the task is to link structure and agency.

To do this he turns to his natural scientific ontology to argue that social reality is a stratified open system. It is stratified because social structures are held to be emergent properties that exist in interaction with agents who are conditioned but not determined by structures.

Critical realists view reality as an open stratified system. In the absence of any type of constructivist thought in neo-traditionalism it is hard to see that neo-traditionalists see the mind and the world as anything other than two closed and separate systems. The purpose of the mind is to come to know the facts of the world external to it.

Neo-traditionalism and the structure-agency question

A neo-traditionalist education therefore is designed to teach students the knowledge that helps them represent the facts of the world. In this view the student is rendered impotent. Knowledge of the world outside is transmitted to the student by the master and the education system measures how well the student has assimilated that knowledge.

The problem for neo-traditionalists is that without an ontological position there is no way of accounting for knowledge other than the previously mentioned concept of the “best that has been thought and said”:

This brings us to the classic problem with the subject–object dualism that underpinned the Vienna Circle positivism (and the rationalist tradition too in epistemology). If one started by separating the mind of the subject from the world of objects (including the corporeal form of the subject), then one had to reunite them by arguing that ideas in the subject’s mind must correspond to the objects outside, otherwise scepticism sets in and we cannot know anything.

With empiricism it is maintained that the ideas the mind received through sense-data inputs are ideational copies of the objects outside the mind. However,as we cannot step outside the ideas in the mind to compare them to the objects we hope they correspond to and act as ideational copies of, it is an act of metaphysical speculation to maintain that ideas of sensation are ideational copies of material objects.

Critical realists argue that this is an epistemic fallacy. The world, as it is, is not the same as how we come to know it; not least because how we come to know the world conditions how we think about it.

For Bhaskar the problem with modern philosophy, which is exemplified by the positivist tradition, is that it is based on epistemology not ontology. This is a problem for Bhaskar (1997) because it commits the epistemic fallacy of translating ontological questions about what reality is into epistemological questions about how we gain knowledge of reality. Or, to put it another way, reality is cut to fit a model of how the mind gets ideas of reality and doing this misconstrues reality.

Neo-traditionalists cannot ask the question of what knowledge is, or what function it performs in the social world because neo-traditionalism sees education as the transmission of knowledge from one mind to another (master to student). Nor can neo-traditionalism develop new knowledge because the scientific method cannot account for knowledge it simply generates random causal events.

Meta analysis: the ontological glue of positivism

Quantitative educational research underpins neo-traditionalist ideologies however quantitative educational research has developed a sleight of hand to extricate itself from the epistemic fallacy described above. Influential educational thinkers such as John Hattie, Rob Coe and even, on occasion, Dylan Wiliam take random, context-less quantitative research and socially construct a purpose and context in education using meta analysis.

In the case of Professor Rob Coe (CEM Centre, Durham University) and the EEF toolkit pedagogic interventions, such as feedback, are said to offer so many months worth of learning improvement. As discursive tools they are useful but the question remains as whether they are factual.

For Baskhar the problem lies in the methodologies used to generate the data:

Thus with induction one observes relations of cause and effect and with the hypothetico-deductive (H-D) method one tests a theory’s predictions by observing the fixed effects of underlying causal laws. In this case we end up with what Bhaskar (1997) calls a ‘closed systems ontology’, meaning an ontology that defines natural reality in terms of fixed empirical regularities that are closed to the possibility of change.

As knowledge is taken to come from observation, scientific method has to be in accord with this approach to knowledge, which means basing scientific method on observing fixed empirical patterns and assuming that nature is a closed system. Bhaskar (1997, 1998) argues that while one may create an artificial closed system in a laboratory, the world outside the laboratory is actually a stratified open system.

The consequence of using a closed system methodology in an open system is that there is no theoretical explanation of the system. Neo-traditionalists solution to the problem of numerous random causal studies with no ontological coherence is to invent a mathematical framework external to the those studies. Without the “meta glue”, of the mathematical framework, positivist research is simply generating lots of unrelated causal events with no means of extrapolating anything meaningful. Invariably each “event” would fail Popper’s test of falsifiability.

If we take one pedagogic intervention mentioned in the framework, feedback as an example. A research project could generate a single condition in which feedback works. Research could probably also generate conditions where it does not work by placing the emphasis on different variables; teacher expertise springs to mind or student motivation. It is not a given that feedback is even a good thing. One piece of feedback could generate numerous outcomes but without a theory or an ontological position the consequence would be numerous unrelated studies that generate data to no real effect.

The only way around this problem is to collate studies and use a constructed variable such as “effect size” that masks the problems with the individual studies; in particular their un-relatedness. Meta analysis is not so much a means to cohere lots of random studies but rather a way of avoiding ontological problems.


In this first blog I want to show how neo-traditonalism cannot offer a view of knowledge because it is based upon epistemological assumptions. In order to circumvent this problem neo-traditionalism has adopted a concept of the “best that has been thought and said” that is fraught with problems not least what constitutes the “best that has been thought and said” and why? As well as ignoring the history of power relations and the exclusion of voices that have not traditionally shared power; the working classes, women and minorities.

In order to generate new pedagogical knowledge neo-traditionalism has adopted a view based upon the empirical sciences. The problem with the positivist epistemological perspective is that it generates random causal events and  has no metaphysical explanation for knowledge.The sleight of hand that neo-traditionalists use to dig themselves out of the epistemological hole are socially constructed concepts such as “meta-analysis” and “effect-size”..

In the next blog I want to show that the epistemic assumptions in relativistic progressivism also have no explanation for knowledge and aim to use critically realist ideas (particularly those of Margaret Archer) to offer an ontologically underpinned critically progressive constructionism in educational practice.


Thanks to @imagineinquiry for the inspiration and the headaches!

8 thoughts on “On critical realism in education – part one: the neo-traditionalist fallacy, meta analysis and other epistemic sleights of hand

  1. This is interesting but I’m finding it hard to understand the discourse as I am unfamiliar with many of the ideas.
    “Nor can neo-traditionalism develop new knowledge because the scientific method cannot account for knowledge it simply generates random causal events.” If this is the case, surely all knowledge becomes problematic? I am not privileging the scientific method here, but it is often an example of careful, rigorous investigation. If all that uncovers is random causal events…?

    1. Perhaps in a long, quickly written, blog the language does get careless but only careless not meaningless.

      All knowledge is problematic. The greatest minds on the planet have been debating knowledge for years. The gold standard of scientific research is uncovering replicable and to some extend random causal events. Science doesn’t uncover the whole it uncovers the causal events and theorises on the rest.

      That is why there is little need for ontology. If something is replicable then the nature of it (ontology) is, as it is, as described by science. The theory is robust because to some extent the nature of what is being studied is robust.

      Secondary is the strong corelational research. The cause is not known but the effect is observable and reliable. Think of Humes billiard ball. The cause is not observable (or it wasn’t) but the effect is replicable. In this case the theorising is greater but equally reliable.

      More problematic are fields like Psychology where neither the cause nor the effect is known but there is observable output. You have experienced it yourself when arguments from science start to encroach on the social world. At this level the theory starts to take over from anything solid and can almost seem like speculation.

      Finally in the social world; there is no cause, effect or necessarily observable output. It is an open complex system and difficult to study. Of course so is much natural science hence the need for complexity theory etc but the problems start to emulate those in the social world.

      Remember I’m not criticising the scientific method per se rather suggesting that once you stray away from replicable events and strong correlational relationships the need for ontology becomes greater because what you are describing is not the same, necessarily, as what the method you are using is uncovering.

      The greater the need for theory the more language and human cognition start to construct reality and the claims to science are less reliable. That is what the post modernists were largely critisicing.

      Hope that makes sense.

  2. You seem to suggest that scientific observations and theories generate ontologies almost automatically, and that these become generally accepted without much debate. Just because observations are replicable doesn’t mean that there is only one possible ontology to explain them — it’s often a matter of extended debate. Often, of course, it can become a consensus, and generally I think scientists favour ontologically abstemious frameworks.

    What puzzles me is what comes first in your view: the ontology or the epistemology? In real ( as opposed to Popperian) science, it’s a confused and messy business — sometimes it’s one, sometimes it’s the other. There has to be a range of possible ontologies for any epistemology so there must be an issue of choice…? How can you be certain that the ontology chosen is the correct one (and how is that judged, anyway).

    Certain knowledge is problematic, of course, but to me it seems to become less problematic the more than one accepts that all knowledge is actually tentative.

    1. Remember in this blog I am addressing the discourse of edu-twitter not science itself so yes I am suggesting that scientific explanations are accepted without ontological explanations. I am not suggesting that serious scientists have that view or that it is the view of natural scientists that concern themselves with philosophy but I think its true nonetheless that scientific explanations are accepted based on methodology and not their theoretical or philosophical underpinnings.

      The scientific method itself is limited to the nature of knowledge it can generate. As you seem to suggest on twitter both of us would agree that mixed method studies are better perhaps suited to studying complexity particularly social complexity.

      With regard to ontology generally speaking the ontological perspective has to match the epistemic perspective. A radical constructivist would hardly likely to choose an objectivist ontology.Similarly a natural scientist less likely to choose pragmatism if they have a philosophical position at all.

      So it’s not a case of which comes first but that one matches the other. I actually really rate Popper I like his three world philosophy and his attempt to resolve Platonic epistemic positions with the falsifiable concept.

      I think ontologies and epistemologies can be mixed and matched depending on the problem but clearly some are incompatible.

      Finally yes a fallibilist position taken on knowledge is less problematic but now your are adopting a philosophical position less common in the discourse of scientism or in this case neo-traditionalism. The nub of the blog is a critique of epistemic fallacies such as “the best that has been thought and said” and mathematical ontologies such as meta analysis.

      As I say again I’m critiquing a discourse rather than science as a “thing in itself”.

  3. To my mind, the scientific method isn’t unique: it’s simply the most organised form of investigation that humans have developed. That said, it isn’t as uniform and consistent as some people like to pretend. My comment on Popper wasn’t meant to denigrate his contribution but simply to highlight that real science is a lot messier than his falsification model suggests. I think Kuhn is perhaps closer to the mark, but again real science doesn’t always follow his model either.

    Your picture of ontology and epistemology: is it analagous to the strands of a double-helix in the sense that a development in one leads to a development in the other?

    I do accept that there can be “other ways of knowing” in addition or as an alternative to scientific investiagtion e.g. traditional craft knowledge or life experience; but it seems to me that these “mixed methods” are still part of the class of pragmatic, empirical methods for the most part.

    Your point that you are critiquing a discourse rather than a method is well taken. Meta-analysis can be critiqued on many levels, and it’s certainly true that many neo-trad bloggers drink too deep of the scientism well at times. One of the conversations that shook up my neo-trad leanings a ittle bit was discussing Engelmann’s criticism of a “progressive” method of teaching primes using counters with a very experienced Maths colleague: my colleague admitted that, yes, the “progressive” method might indeed generate false inferences as Engelmann suggested; his counter was that any method generates false inferences and that sometimes it is a good to develop false inferences so that they can be challenged…

    Hmm, this whole education thing is more difficult than I thought

    1. The problem with the scientific method in the social world is that you cannot stand outside the social world. As you are investigating something in the social world you are creating discourse, in other words changing it. That is why how we come to know things (epistemology) is not adequate in the social world because we change the nature of them (ontology) as we interact with the social world.

      Agree about Kuhn. Both he and Popper influenced post modernism.

      When you use the double helix model, I agree, but I presume you mean the object of study not ontology itself. I find Hegel’s dialectic quite a useful analytical tool to help think about it. Social realism sees knowledge as external to the knower. Like scientific realism but not the scientific method the object is to understand the conditions of knowledge construction. The more we come to know about the thing (epistemology) the more we learn about its nature (ontology).

      With regard to mixed methods I agree most of the methods aim to generate empirical data but they are not the scientific method, which is a methodology in itself. Perhaps you include them in an umbrella description of scientific method, which would explain some of our differences.

      I’m not actually sure we have that many differences really. The problem is we all engage with polemical discourse, which is purposeful if there is a genuine desire to engage but not if there is simply a desire to win arguments.

  4. Me? Engage in polemical discourse as a cheap point scoring exercise? How very dare you, sir!

    But seriously, I would agree that nomenclature seems to often be more of a sticking point rather than anything substantive. I’m going to do more reading on postmodernism as its an area of thought that I’ve previously written off based on secondhand assessments. Many thanks for a very interesting discussion.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s