The School Research Lead - The Evidence Says Its A Good Idea - Should You Do It?

So you've read the research papers and separated the wheat from the chaff.  You've identified the research papers where it's just old ideas recycled with new terminology. Research papers where there are real doubts about the claims that are being made. Research papers where there just might be onto something but the evidence is incomplete and lacking.  This leaves research papers with changes which might just work, but you are not quite sure whether to adopt it or not.  Drawing once again upon the Daniel Willingham's 2012 book - When Can You Trust the Experts? How to tell good science from bad education - we will look at what can be done to help make a good and wise decision on whether to proceed with the change.  In doing so, we will consider the following:

• the factors to take into account of even if there is good evidence that the change is scientifically sound.
• a check-list to be completed before adopting a change 
• Willingham on education and evidence-based medicine.

Factors to take into account even if there is good evidence that the change is scientifically sound

Having read the research papers, you think you've come across an intervention/change for which there is sound evidence base and which you think might be worth adopting.  However, just because the evidence-base appears to be sound, that in itself is not a good enough reason to adopt the intervention/change.  There are a range of other factors which need to be taken into account before deciding whether or not to proceed.  Willingham identifies four factors which are worthy of consideration.

• Implementing a Change likely incurs a cost in times, energy and other resources.  Even if you believe that promised benefits will accrue you must weight them against the anticipated costs.
• Any Change you adopt will brings opportunity costs
• A Change may work as described but may have negative side effects
• Could the Change directly impact upon others -

A check-list to be completed before adopting a change

Willingham, drawing upon the work of Atul Gawande,  has developed a 10 point check-list which is to be completed before adopting a change.

1. The thing I'm hoping to change is ....
2. The way I can see that things change (in other words, what I'm going to measure) is ...
3. I've measured it before I start the Change , and the level it ....
4. I'm also going to measure ....... It probably won't be held by the Change, but you never know.
5. The Change could have some negative effects.  I'm most suspicious that it might influence ..... To be confident about whether or not it does, I'm going to measure ......
6. Here's how often I plan to collect the measurements, and the circumstances under which I'll do so: ......
7. My plan to keep these data organised is ....
8. The date by which I expect to see some benefit of the Change is ....
9. The size of the benefit I expect to see on that date ....
10. If I don't observe the expected benefit on that date, my plan is to ...... (Willingham, p 217)

At first glance, this may appear to be a daunting check-list.  On the other hand, as Woody Allen is often misquoted as saying -Eighty percent of success is showing up,  so by just asking these questions will get you most of the way towards success.

Willingham on education and evidence-based medicine

In writing a blog-post or any other publication it's quite easy to 'cherry-pick' and only quote 'authorities' when they agree with you.   In a number of blog posts I have taken the stance that education has much to learn from evidence-based medicine so it's important to note Willingham's reservations about making undue comparisons between education and medicine.  First, Willingham argues that in medicine there is a single goal shared by both doctors and patients i.e good health, whereas in education there are a far greater and diverse range of goals, aims and objectives.  Second, the deterministic model of science is relevant to medicine, well-evidence treatments for patients will probably work, whereas in education well-evidenced interventions may possibly work.  Willingham goes onto argue that architecture serves as a better comparison than medicine.  Architects have to manage multiple competing needs - be it functionality, form, the environment.  Furthermore, their decisions are informed by the 'science' of building, though not determined, in that they set the overall conditions for any design decisions.  As such, educational research should set the boundary decisions by which teachers make decisions about teaching and learning.   My own views on this matter, for what it is worth, is that whilst accepting the limitations in applying a strict interpretation of evidence-based medicine within education, there is a still a huge amount to be learnt from doctors, nurses and other health practitioners about how to make best use of the available evidence. For me, applying the processes associated with evidence-based medicine are likely to lead to you being a more  effective evidence-informed educational practitioner.

Some final words on this series of posts

This is the last of five posts which have been inspired by When Can You Trust The Experts: How to tell good science from bad in education.  As Daniel Willingham himself argues - the 4 steps outlined - will not make you into an expert in any particular area, as the 4 steps themselves are themselves a heuristic, a short-cut or work-around to help you get round that lack of expertise.  However, just because gaining expertise is a difficult and time consuming process, does not mean that you cannot be informed about the changes and interventions that have a reasonable chance of working for your students, and identifying those changes which are just BS and are not worthy of you and your pupils.