It seemed a good idea at the time BUT we really should have known better!

This academic year will have seen the introduction of  a wide range innovations in schools and colleges, many of which will be showing the first fruits of success.  On the other hand, there will be innovations which are not working and have little prospect of success and were introduced because the originator(s) had fallen in love with the idea.   These unfortunate failures highlight one of the major challenges of evidence based leadership and management which is to develop the processes which reduce the errors generated by cognitive and information processing limits which make decision-makers prone to biases and which subsequently lead to negative outcomes.

In this post I will be drawing upon the work of Kahneman, Lavallo and Sibony (2011) in how to find dangerous biases before they lead to poor-decision-making.  Developing the skills to to appraise and critically judge the trustworthiness and relevance of multiple sources of evidence is a critical element of evidence based practice.   Kahneman et al identify a number of specific biases, questions and actions which could be used to improve the rigor of decision-making.  These have been summarised and adapted and  in the following table.

Avoiding Biases and Making Better Decisions - A Checklist - Summarised and adapted from Kahneman, Lavallo and Sibony (2011)
 

Preliminary questions
Check/Confirm for		Action
Self-interested biases	Is there any reason to suspect that the team of individuals making the recommendation are making errors motivated by self-interest?	Review the proposal with care
Affect heuristic	Has the team fallen in love with its’ proposals?	Apply the check-list
Groupthink	Were there dissenting opinions, were these opinions fully explored?	Discretely obtain dissenting views
Challenge questions
Saliency bias	Could the diagnosis be overly influenced by an analogy to a memorable success?	Are there other analogies? How similar are this and other analogies to the current situation?
Confirmation bias	Are credible alternatives included with the recommendation?	Request additional options be provided
Availability bias	If this decision was to be made again in a year’s time, what information would you want and can you get more of it now?	Develop checklists of available information for different types decisions
Anchoring bias	Do you know where the numbers came from – are there unsubstantiated numbers – have they been extrapolated from historical data?	Check the figures against other models, are there alternative benchmarks which can be used for analysis.
Halo effect	Is the team assuming that a person, organisation or innovation  which is successful in one area will be just as successful in another?	Eliminate false inferences- seek alternative examples
Ask about the proposal
Overconfidence, planning fallacy, optimistic biases, competition neglect	Is the base case overly optimistic?	Have outside views been taken into account?
Check for disaster neglect	Is the worst case bad enough?	Conduct a pre-mortem to work out what could go wrong
Check for loss aversions	Is the recommending team overly cautious?	Realign incentives to share responsibility for the risk or remove the risk.

How could this check-list be used to improve decision-making within educational settings?

• Ensuring the check-list is applied before the action is taken which commits the school or college to the action being proposed.
• Ensuring the decision-check-list are applied by a member or members of staff who are both sufficiently senior within the school/college, whilst at the same time is not part of the group making the recommendation.  Separation from recommenders and decision-makes is desirable and which has implications for governance and leadership.
• Ensuring the check-list is used in whole and not in parts and is not 'cherry-picked' to legitimate a decision.

If colleagues are able to adopt the above process then it is likely to increase the chances of success in the use of evidence based practice.   As we approach the end of the first half-term  of the academic year I wonder how many schools, colleges, teachers, lecturers and most importantly students could have avoided the negative impacts of poorly thought out decisions, if the above checklist had been applied early on as part of the decision-making process.

References

Kahneman, D., Lovallo, D and Sibony, O.  (2011) Before you make that big decision ... Harvard Business Review, June 2011