Evidence-informed practice : Some common misconceptions - Part Two

In my recent blog post for BERA I identified three of the most common misconceptions about evidence-based/informed practice.  In this post, adapting the work of Barends et al (2014) for a school context, I will highlight a further five misconceptions about evidence-informed practice  which get in the way of effective evidence-informed practice within schools.  In doing so, I hope this will contribute to: better outcomes for pupils; improved opportunities for teacher professional learning; and, the better use of increasingly scare financial resources. So let's now turn to these additional misconceptions.

Misconception 4: Evidence-based practice is all about numbers and statistics.

Evidence-based practice involves seeking out and using the best available evidence from multiple sources. It is not exclusively about numbers and quantitative data, randomized control trials, although many practice decisions involve figures of some sort. You do not need to become a statistician to undertake evidence-based practice, but it does help to have an understanding of basic statistical concepts that are useful to evaluate critically some types of evidence. The principles behind such concepts as sample size, statistical versus practical significance, confidence intervals and effect sizes, can be understood without any mathematics.

Misconception 5: School Leaders need to make decisions quickly and don’t have time for evidence-based practice.

Sometimes evidence-based practice is about taking a moment to reflect on how well the evidence you have can be trusted. More often it is about preparing yourself (and your school ) to make key decisions well – by identifying the best available evidence you need, preferably before you need it. Some management decisions do need to be taken quickly, but even split-second decisions require trustworthy evidence. Making a good, fast decision about when to evacuate a school or how to deal with a critical incidents up-to-date knowledge of emergency procedures and reliable measures providing trustworthy evidence. When important decisions need to be made quickly, an evidence-based practitioner anticipates the kinds of evidence that quality decisions require. The need to make an immediate decision is generally the exception rather than the rule. The vast majority of school management decisions are made over much longer time periods – sometimes weeks or even months – and often require the consideration of legal, financial, strategic, logistical or other school issues, which all takes time. This provides plenty of opportunities to collect and critically evaluate evidence about the nature of the problem and, if there is a problem, the decision most likely to produce the desired outcome. For evidence-based practice, time is not normally a deal breaker.

Misconception 6: Each school is unique, so the usefulness of scientific evidence is limited.

One objection practitioners have to using research evidence is the belief that their school or classroom is unique, suggesting that research findings will simply not apply. Although it is true that schools or classrooms do differ, they also tend to face very similar issues, sometimes repeatedly, and often respond to them in similar ways. Peter Drucker, a seminal management thinker, was perhaps the first to assert that most management issues are ‘repetitions of familiar problems cloaked in the guise of uniqueness’ . The truth of the matter is that it is commonplace for schools to have myths and stories about their own uniqueness.   In reality they tend to be neither exactly alike nor unique, but somewhere in between. Evidence-based practitioners need to be flexible enough to take any such similar-yet-different qualities into account. A thoughtful school leader, for instance, might use financial incentives for business development staff but reward knowledge workers (teachers  with opportunities for development or personally interesting projects, knowing that financial incentives tend to lower performance for knowledge workers (teachers  while increasing the performance of less-skilled workers  .

Misconception 7: If you do not have high-quality evidence, you cannot do anything.

Sometimes there is very little or no quality evidence available. This may be the case with a new management practice or the implementation of new technologies. In some areas the school’s context changes rapidly, which can limit the relevance and applicability of scientific and experiential evidence derived in a context different than that of today. In those cases, the evidence-based school-leader has no other option but to work with the limited evidence at hand and supplement it through learning by doing. This means pilot testing and treating any course of action as a prototype: systematically assess the outcome of the decisions we take through a process of constant experimentation, punctuated by critical reflection and evaluation about which things work and which things do not.

Misconception 8: Good-quality evidence gives you the answer to the problem.

Evidence is not an answer. It does not speak for itself. To make sense of evidence, we need an understanding of the context and a critical mindset. You might take a test and find out you scored 10 points, but if you don’t know the average or total possible score it’s hard to determine whether you did well or not. You may also want to know what doing well on the test actually means. Does it indicate or predict anything important to you and in your context? And why? Your score in the test is meaningless without this additional information. At the same time, evidence is never conclusive. It does not prove things, which means that no piece of evidence can be viewed as a universal or timeless truth. In most cases evidence comes with a large degree of uncertainty. Evidence-based school leaders therefore make decisions not based on conclusive, solid, up-to-date information, but on probabilities, indications and tentative conclusions. Evidence does not tell you what to decide, but it does help you to make a better-informed decision.

Some final words

Let's be clear, the jury is still out on the effectiveness of evidence-informed practice within schools,  although there are reported benefits to practitioners in engaging in evidence-informed practice (Supovitz, 2015).  Fortunately,  there are a number of EEF funded studies, for example – The Rise Project: Evidence – Informed School Improvement; Research into Practice : Evidence Informed CPD; Research Learning Communities; and Evidence for the Frontline – which are considering the impact of evidence-informed practices on teacher development and pupil outcomes, and these projects will begin to report from 2016 onwards.   Nevertheless, we are still waiting for the research evidence on school-based evidence-informed practice, and until that evidence becomes available it is best to proceed with caution and to learn by doing.

References

Barends, E., Rousseau, D. M., & Briner, R. B. (2014). Evidence-Based Management, The Basic Principles. In Search of Evidence, Center for Evidence-Based Management
Supovitz, J. (2015) Teacher Data Use for Improving Teaching and Learning, in Brown C. (ed) 

---