The School Research Lead and Stripping and Flipping - Essential skills for the evidence-informed practitioner

For teachers stripping and flipping may not sound like a way of helping them better do their jobs.  Indeed, it may even sound like the road to instant dismissal.  But seriously, stripping and flipping are two essential skills in becoming a research-literate and evidence-informed teacher.  Daniel Willingham in his 2012 book - When Can You Trust The Experts? How to tell good science from bad in education describes how stripping away verbiage and getting to the heart of actual claim being made, is the first step in a 'heuristic' process of evaluating educational interventions, programmes and research.  The rest of this post will now look at stripping and flipping in more detail. But first, lets define a few terms.

The Change refers to a new curriculum or teaching strategy or software package or school restructuring plan - generically anything than someone is urging you to try as a way to better educate kids.

The Persuader refer(s) to any personal who is urging you to try the Change, whether it's a teacher, administrator, salesperson, or the President of the United States (Willingham, p136) 

The Audience refers to individual teachers, department, schools, LEAs who are being asked to adopt the Change

Stripping 

In undertaking an evaluation of a Change Willingham states that it is essential to to be clear on the following:

1. precisely what Change is being suggested
2. precisely what outcome is being promised as a result of the Change
3. the probability that the promised outcome will actually happen if you undertake the Change (Willingham, p136)

For example : If an 'inadequate' school is converted to an academy, there is a 50 percent that the school will double the percentage of pupils gaining 5 GCSEs (inc English and Maths) at grade C or better

Flipping

Flipping is best explained by an example.  Let's suppose that a school claims that 65 percent of pupils leave school at 16 with a least a GCSE grade C in both English and Maths, then 35 percent of pupils leave school without at least a GCSE grade C in both English and Maths.  Now, all other things being equal, due to a framing effects a school with a 65 percent success rate is likely to be rated as better than a school with a 35 percent failure rate.

Things to watch out while flipping and stripping

Of course, there is far more to the stripping and flipping process than can be captured in a couple of examples.  Things to watch of for include"

• Emotional claims about the need or benefit of the Change
• Claims that the Persuader is like you
• False Analogies
• Postive and negative framing
• Old stuff which we have seen before
• Vague stuff which is difficult to discern either the output or outcomes
• Over-reach - claiming the Change will help all teachers and all pupils

Accordingly, there are a number of sub-steps to the Strip It/Flip It process and which are summarised in Table 1

Table 1 : Summary of the Strip It and Flip It Process

Suggested Action	Why You’re Doing This
Strip to the form “If I do X, then there is a Y percent chance of Z happening.”	To get rid of emotional appeals, peripheral cues, and proffered analogies that may influence your belief.  The scientific methods is suppose to be evidence-based and uninfluenced by these factors.
Consider whether the outcome (Z) has an inverse; if so, restate the stripped version of the claim using the inverse	To be sure that you appreciate all consequences of the action – for example, that an 85 percent pass rate implies a 15 percent failure rate.  We are subject to framing effects; we think something is better if the positive aspects are emphasised.
Consider the outcome if you fail to take action X.	To ensure that the promised outcomes if you do X seems much better than if you don’t do X.  When there is a problem; it’s tempting to lunge toward any action because it makes you feel that you are taking some action rather than standing idle.
Consider the outcome if you fail to take action, this time using the inverse of Z as the outcomes	To ensure that doing something versus doing nothing looks just as appealing when you think about good outcomes as when you think about bad outcomes.  People are generally less willing to take risks to increase their gains – they would rather have a sure thing.  But they don’t want a sure thing for losses – they will take a risk to try to minimise them.
Evaluate whether the stripped promise is something you already know	To be sure that what’s being sold to you is something you can’t do yourself.  Technical talk – especially neuroscientific talk – can make old ideas seem cutting-edge.
Evaluate whether the change is clear; ‘clear’ means that you feel confident that you know how the Change will affect students’ minds	To ensure the Change is implemented as intended.  Changes that sound good can go awry if they are not implemented in the classroom as intended or if students don’t do what you’re hoping they will do.
Evaluate whether the outcome (Z) is clear; ‘clear’ means that there is a reasonably objective measure of whatever outcomes you expect, how big the change in the outcome will be, and when will it happen.	To be sure you will able to tell whether or not the promised outcome is happening
Check the outcome against the list of frequently claimed but unlikely-to-work promises	To be sure that claims are not made that are unfeasible, from a cognitive perspective – for example An improvement in all cognitive processes An improvement in a specific cognitive process (for example, critical thinking) irrespective of material An improvement for all students who struggle with a complex skill such as reading

Willingham goes onto argue that while considering the merits of a change it is necessary to write down your thoughts, and in doing so, this will lead so an increase in both the quantity and quality of reflection.  However, even if you have gone through the check-list and considered each of the items and have decided that the proposed change may be worth pursuing, there are still three further steps in the evaluative process, each of which will be the subject of a forthcoming post.

Implications for the School Research Leads

It is clear that School Research Leads are under constant time-pressure, with many colleagues being given only a small amount of time to undertake the role and using this check-list will take both time and effort.  However, given the increased and relentless resource pressures on schools over the next 5 years, anything that helps make sure time, money and effort are invested in the right change initiatives is worthy of consideration and use.   Furthermore, the role of the School Research Lead is still in its infancy, and if the role is to be continued to be supported and developed it is important that the role delivers discernible benefits.  The chances of delivering these benefits will be increased if  research-informed Changes are subject to rigorous pre-evaluation prior to implementation.  And in doing so, yo are more likely to deliver to changes which benefit your pupils, which is, at the end of day, what it is all about.

Reference
Willingham, D. (2012) When Can You Trust The Experts: How to tell good science from bad in education, Jossey-Bass, San Francisco.