So what are we to do? Fortunately, Daniel Willingham in his 2012 book, When Can You Trust the Experts: How to tell good science from bad in education, provides us with guidance in how to work-out whether to believe what a so-called expert (consultant, ex-headteacher, educational researcher) says about a subject just because of his or her authority. In particular, Willingham identifies a number of ways you can go wrong when you trust an authority. But first, let's look at the structure of the argument as to whether claim is supported by 'scientific' method.
What happens when we believe an authority
Adapting Willingham (p170) - say you - the researchED delegate - don't understand the science behind an idea or claim made by an educational researcher/expert/researchEd speaker. But you believe that the educational researcher/expert does. You know that the educational researcher is saying the research evidence supports the idea or claim being made. So you - the researchED delegate - without understanding the underpinning research, trust that the evidence supports the claim. In other words, as the educational researcher/researchED speaker is an expert on the evidence, when the researchers says something about the evidence, you - the researchED delegate - are more likely to believe it.
However, there are a number of situations when this argument can go wrong - again adapted from Willingham.
- What we take to be signs of authority - acadamic pre-eminence, experience, twitter profile or educational columnist - turn out not to be very reliable, and the person is not, in fact, scientifically knowledgeable
- We might arrive at a false belief because we misundertand the position taken the academic, tweeter, consultant or columnist
- The 'authority' might be knowledgeable, but be in error takes up position on a topic outside his or her topic of expertise
- Two equally credible authorities might disagree on an issue, leaving it unclear what to do and how to proceed (adapted from Willingham p 178)
This can be best summarised by F Scott Fitzgerald who said The test of a first-rate intelligence is the ability to hold two opposed ideas in mind at the same time and still retain the ability to function. In this case the two opposing ideas are : I do not possess the expertise of an educational researcher, so I need to rely on those who do. On the other hand, just because someone would appear to be a credible expert there are plenty of reasons not to believe such experts and authorities. So what are we to do? Unfortunately, and there's no other way round it, you the researchED delegate need to evaluate the strength of the evidence yourselves. This does not mean you have to become an instant expert in either qualitative or quantitative methods, but it does mean you have to have a list of questions which help unpack and understand educational research. And in the case of researchED 2015 - make sure any speaker you listen to references a paper/book/article - where you can really unpick and analyse their argument. If they haven't got a paper etc - then it's probably best to move onto someone who has.
Willingham, D. (2012) When Can You Trust The Experts: How to tell good science from bad in education, Jossey-Bass, San Francisco.