The School Research Lead - Do we need to be vaccinated against the sharing of good practice virus?

The start of the academic year will see an outbreak of CPD and the sharing of good practice virus. Colleagues will spend hours,  if not days, being exposed to the self-espoused good practice of colleagues.  Unfortunately, like many viruses, the sharing of good practice 'virus' may do a great deal of harm, unless colleagues are vaccinated against infection from the well-meaning but unsubstantiated testimonials of colleagues.  Fortunately, a 'vaccine' has been developed by Daniel Willingham in his 2012 book - When Can You Trust the Experts : How to tell good science from bad in education.  Like all vaccines it can only provide the host with some protection against infection, as it will depend upon how the host responds to the vaccine.  However before we examine in more detail Willingham's vaccine, let's quickly remind ourselves of an activity we often see in a CPD and sharing of good practice day.

Sharing Good Practice - A Typical Activity

Some form of 'good-practice speed-dating' will no doubt form part of many schools start of term CPD events.  Colleagues will be asked to bring an example of what they perceive to be good practice and will be asked to explain how it works - often in less than two minutes -  to another colleague, with this process being repeated may be up to 20 - 30 times in a session. Unfortunately, there are a number of problems with this approach.  First, it gives insufficient time for colleagues to engage in an extended conversation about the 'good practice.  Willingham argues that such conversations are essential to ensure that the 'sharer of practice' fully understands his or her own practice and which is an important component of persuading others of the merit of a proposal.  However, the real problem with the speed-dating approach or others based upon testimonials is that it provides insufficient information to evaluate the practice being shared.  As Willingham states: You need not just stories of success from people who adopted the changes but also stories of failure.  You also need stories of success and failure from people who didn't adopt the change. (p 195)   The problem with testimonials in the sharing of good practice is illustrated in Figure 1

Figure 1 Information required to fairly evaluate a Change (Willingham, 2012 p 196)

	Things improved	Things didn’t improve
Adopted Change
Didn’t adopt Change (or adopted a placebo)

As such, Willingham argues that statements from individual teachers about his or her 'good-practice' only provides information from the top-left hand quadrant.  If we want to know whether some intervention or strategy has the potential to be good-practice than we will need information to allow to complete all four quadrants.

So what are we to do?

First it is essential for School Research Leads and CPD Co-ordinators to be aware of the design elements of effective professional development.  Fortunately, the Teacher Development Trust has recently provided us with a number of evidence-informed design elements, and about which I have written about previously.  Second, it is important that teachers talk about their practice in a 'disciplined' manner to determine whether espoused good practice is worth further discussion and possibly wider use within the school. Willingham suggests that when examining the potential of an educational change, strategy, or intervention teachers undertake a four-stage process.

1. Strip it - get rid of the fluff surrounding the idea/change/strategy/intervention and get to the heart of the actual claim.  What specific intervention, strategy or actions should the teacher adopt and what outcomes, say learning or achievement, are being promised.
2. Trace it - where did the idea come from.  Is the idea supported by a leading educational authority, as ironically in education this can be a weak indicator of validity or reliability 
3. Analyse it - what are you being asked to believe.  What is the evidence to support the claims being made?  How does this evidence relate to your own experience as a teacher?
4. Should I do it - is it something we already do.  Is it an old idea wrapped in new language? Has it failed previously in other settings with other students.  What are the opportunity costs 

'Flipping' Willingham to aid the sharing of practice 

 Willingham's four steps have been designed to help teachers discriminate between good and bad research evidence. Each of the four steps can be amended to help teachers structure the way they share practice with others.   In other words,  Willingham's four steps can be 'flipped' around for use by the teacher who is sharing the practice.  As such, if you intend to share practice with others the following elements needs to be covered.

• By doing X with my pupils there appears to be a Y percent chance that Z will happen
• Where the idea came from - is it supported by research evidence - if so, what are the criticisms of this research 
• State the 'class-based' evidence - what was measured, how was it measured, in comparison to what, how many pupils, how much did it help
• What were the opportunity costs associated with the change?  What negative-side effects were experienced? Who else was impacted upon?

Flipping the process around - with the steps being used by the sharer of practice has two main benefits.  It provides a structure for reflection by the teacher sharing the practice and which should facilitate more robust self-evaluation of practice before that practice is shared.   Second, it provides the 'audience' with a framework which allows them to 'test' through conversation how well the 'sharer'  understands his or her espoused good practice, which will in turn impact upon whether the sharees adopt the change.

In conclusion

As education can be a fashion-led profession and with fashions often based on so-called good practice it is essential that teachers can engage in the structured and disciplined evaluation of evidence.   Over the last 12 months as I have blogged about a evidence-informed practice,  attended events researchED events and followed twitter discussions, the need for teachers to be research literate is continually raised.  Unfortunately, there is very little discussion about what research literacy looks like in practice and how teachers should go about it.  With that in mind, if you are in the process of setting up a 'journal club' or are seeking a book to be read over the year, then Willingham's When Can You Trust the Experts : How to tell good science from bad in education is essential reading.  Future posts will help time-pressed School Research Lead by exploring Willingham's work in more detail, and expand upon each of the the four steps he recommends which will help you make significant progress to becoming a research literate and evidence-informed teacher.

Reference
Willingham, D. (2012) When Can You Trust The Experts: How to tell good science from bad in education, Jossey-Bass, San Francisco.