researchED London 2015 and knowing when to trust the experts

In 19th century North America salesmen travelled across the continent promising wonder cures for all manners of ailments.  However, once the 'miracle-cure' was purchased and the salesman was onto his next town, his or her customers soon found they had been hoodwinked and the miracle-cure was nothing of the kind.  Snake-oil was one of the most commonly sold miracles cures and the term 'snake-oil' is still used today to describe so-called miracle-cures.  Indeed, in 21st century England, there are still the more than occasional sighting in schools of snake-oil salesmen, this time in the form of consultants, peddling the latest intervention which will transform pupil learning.  And despite Tom Bennett and Helen Galdin O'Shea's best efforts, one or two may have sneaked into researchED London 2015.  I really hope they haven't - but just in case they have - I've written this post.

So what are we to do?  Fortunately, Daniel Willingham  in his 2012 book,  When Can You Trust the Experts: How to tell good science from bad in education,  provides us with guidance in how to work-out  whether to believe what a so-called expert (consultant, ex-headteacher, educational researcher) says about a subject just because of his or her authority.  In particular, Willingham identifies a number of ways you can go wrong when you trust an authority.  But first, let's look at the structure of the argument as to whether claim is supported by 'scientific' method.

What happens when we believe an authority

Adapting Willingham (p170) -   say you - the researchED delegate  - don't understand the science behind an idea or claim made by an educational researcher/expert/researchEd speaker.  But you believe that the educational researcher/expert does.  You know that the educational researcher is saying the research evidence supports the idea or claim being made.  So you - the researchED delegate - without understanding the underpinning research, trust that the evidence supports the claim.  In other words, as the educational researcher/researchED speaker is an expert on the evidence, when the researchers says something about the evidence, you - the researchED delegate - are more likely to believe it.

However, there are a number of situations when this argument can go wrong - again adapted from Willingham.

• What we take to be signs of authority - acadamic pre-eminence, experience, twitter profile or educational columnist - turn out not to be very reliable, and the person is not, in fact, scientifically knowledgeable
• We might arrive at a false belief because we misundertand the position taken the academic, tweeter, consultant or columnist
• The 'authority' might be knowledgeable, but be in error takes up position on a topic outside his or her topic of expertise
• Two equally credible authorities might disagree on an issue, leaving it unclear what to do and how to proceed (adapted from Willingham p 178)

The School Research Lead's dilemma

This can be best summarised by F Scott Fitzgerald who said The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function.  In this case the two opposing ideas are : I do not possess the expertise of an educational researcher, so I need to rely on those who do.  On the other hand, just because someone would appear to be a credible expert there are plenty of reasons not to believe such experts and authorities.  So what are we to do?  Unfortunately, and there's no other way round it, you the researchED delegate need to evaluate the strength of the evidence yourselves. This does not mean you have to become an instant expert in either qualitative or quantitative methods, but it does mean you have to have a list of questions which help unpack and understand educational research.  And in the case of researchED 2015 - make sure any speaker you listen to references a paper/book/article - where you can really unpick and analyse their argument.  If they haven't got a paper etc - then it's probably best to move onto someone who has.


Reference
Willingham, D. (2012) When Can You Trust The Experts: How to tell good science from bad in education, Jossey-Bass, San Francisco.