Dangerous Half Truths and Total Nonsense - Evidence Based Practice and two avoidable mistakes

A few weeks ago I wrote about how difficult it is to learn from others using casual benchmarking and the implications this had for leadership teams who followed the Skills Commissioners advice to visit other colleges.  In this post I continue to draw upon the work of Jeffrey Pfeffer and Robert I Sutton and their 2006 book Hard Facts Dangerous Half-Truths and Total Nonsense : Profiting from evidence-based management  and some of the practices which are confused with evidence based management. In addition to casual benchmarking Pfeffer and Sutton identify two other practices - ,  doing what seems to have worked in the past and following deeply held yet unexamined ideologies -  which can lead to both to poor decisions and detrimental outcomes for college stakeholders i.e. staff, students, parents, employers and the community.   These two practices are now explored in more detail.

Doing what seems to have worked in the past
     As colleagues move one from role to another, be it within a school or college the temptation is use that experience and apply it to a new setting.  Pfeffer and Sutton argue that problems arise when the new situation is different from the past and when what we have learned worked in the past, may have actually have been wrong.  There is a huge temptation to import management practices and ideas from one setting to another without sufficient thought about context, either old or new.   Pfeffer and Sutton identify three simple questions that can assist in avoiding negative outcomes arising from inappropriately repeating past strategies and innovations.

• Is the practice that I am trying to import into a new setting, say lesson planning, peer tutoring, e-learning, pastoral arrangements or information and guidance arrangements directly linked with previous success.   Was the success which is now being attempted to be replicated despite of the innovation which was adopted rather than because of it?
• Is the new - department, school, college or other setting - so similar to past situations that what worked in the past will work in the new setting?  For example, getting the balance right between delegation and control may be different between an outstanding school/college and one that is on a journey of improvement.
• Why was past practice  say graded lesson observations - effective, what was the mechanism or theory which explains the impact of your previous actions.  If you don't know what was the theory or mechanism that underpinned past success then it will be difficult to know what will work this time.  (Adapted from Pfeffer and Sutton p9)

Following deeply held yet unexamined ideologies 
     The third basis for flawed decisions-making is deeply held beliefs or biases which may cause school and college leaders to adopt particular practices, even though there maybe little or no evidence to support the introduction of the proposed practice or innovation. Pfeffer and Sutton suggest a final three questions to address this issue.

• Is my preference for a particularly leadership style or practice because it fits in with my belief about people -  for example, do I support the introduction of PRP for teachers because of my belief of the motivational benefits of higher salaries and payment for results?
• Do I require the same level of evidence or supporting data, if the issue at hand is something that I am already convinced what the answer is?
• Am I  allowing my preferences to reduce my willingness to obtain and examine data and evidence which may be relevant to the decisions.  Do I tend to read the educational bloggers/tweeters who agree with me or do I search out views that are in contrast to my own.  Am I willing to look at research which challenges my preconceptions for example Burgess et al (2013) and the small but positive effect of league tables.(Adapted from Pfeffer and Sutton, p 12)

To conclude :

     If an evidence based approach is taken seriously it could potentially change the way in which every school/college leader thinks about educational leadership and management, and subsequently change the way the behave as school/college leader.  As Pfeffer and Sutton state:
     First and foremost, (evidence based management) it is way of seeing the world and thinking about the craft of management.  Evidence-based management proceeds from the premise that using better, deeper logic and employing facts to the extent possible permits leaders to do their jobs better. Evidence based management is based on the belief that facing the hard facts about what works and what doesn't, understanding dangerous half-truths that constitute as much conventional wisdom about management, and rejecting the total nonsense that too often passes for sound advice will help organizations perform better (p13)

References
Burgess, S., Wilson, D., and Worth, J. (2013) A natural experiment in school accountability the impact of school performance information on pupil progress, Journal of Public Economics, vol 106, pp 57-67

Pfeffer, J and Sutton, R., (2006) Hard Facts Dangerous Half-Truths and Total Nonsense : Profiting from evidence-based management. Harvard Business Review Press.