School Research Lead - From evidence to implementation

During this Thursday's #UKEdReschat there was a lively discussion hosted by @StuartKime which focussed on the implementation of research and the ingredients associated by with successful implementation

However, for me this cycle of implementation misses out a fundamental step - how to bring together all  the evidence and then make a decision about how to proceed. For as (Alonso-Coello et al., 2016) 

Often the process that decision-makers used, the criteria that they consider and the evidence that they used to reach their judgments are unclear.  They may omit important criteria, give undue weight to some criteria, or not use the best available evidence.  Systematic and transparent systems for decision-making can help to ensure that all important criteria are considered and that the best available research evidence informs decisions.  P1

Adopting some form of 'evidence' framework can have a number of benefits for school leaders. (Alonso-Coello et al., 2016) subsequently identify a number of such benefits, which I have adapted for the use in a school-setting.

• You and your fellow decision-makers, will have an improved understanding of the advantages and disadvantages of the various actions being proposed
• Helping ensure that you include all important criteria in the decision-making process
• Providing you with a concise summary of the all best available evidence – be it research evidence, school data, stakeholder views and practitioner expertise
• Helping colleagues will be in a better position to understand the decisions by senior leadership teams and the evidence supporting those decisions

As mentioned in a previous post  evidence-based school leaders make explicit the criteria they use to make a decision.  In the context of your school, these criteria may well change depending on what domain and sub-domain of school leadership and management you are concerned with (Neeleman, 2017).  The criteria for making decisions about teaching and learning – may well be different to the criteria you apply to making financial decisions.  In addition, you may want to take into account whether criteria are adjusted for different parts of the organisation.  In the context of your school, the criteria being applied at say level of the Board of MAT Trust, may well be different to how the criteria are applied at Key Stage 1 in a primary school.    Using (Alonso-Coello et al., 2016) as a starting point, let’s look at some of the criteria that could be applied to decision-making (see Figure 1)

 Figure 1 Evidence to Decision Template

Element	Criteria
Priority of the problem	Is the issue an important problem for which a remedy is sought and that can be locally implemented?
Benefits	How substantial are the desirable anticipated effects?
Costs	How substantial are the undesirable anticipated effects?
Certainty of the evidence	How robust and secure are the different sources  - research, practitioner expertise, stakeholder views and school data - of evidence?
Balance	Does the balance of the desirable and undesirable effects favour the intervention or the comparator?
Resource use	How large are the resource requirements – attention, time, money, professional learning? Does the balance of costs and benefits favour the intervention or the comparator?
Equity	What impact does the decision have on educational equity?  Will it help close gaps in attainment? Are there important ethical issues which need to be taken into account?
Acceptability	Are there key stakeholders – teachers, parents, trustees, who would not accept the distribution of the benefits, harms and costs? Would the intervention adversely affect the autonomy of teacher, department, school or MAT?
Feasible	Are there important barriers that are likely to limit the feasibility of implementing the intervention (option) or require consideration when implementing it? Is the intervention or strategy sustainable?
Additional comments and recommendation

Furthermore, application of the above framework needs to be seen in the context of the strength or otherwise of the evidence - be it research, practitioner expertise, school data or stakeholder views.  However, that is another discussion which I will explore in a future post.

And finally

Having reflected on the EEF school improvement cycle - it seems to me insufficient attention is being paid to how you turn evidence into a decision and the processes necessary to support evidence-based decision-making - and this represents a fundamental flaw in the five stage process put forward.  Evidence-based decision making is so much more sophisticated than a simple model of priorities, external research, implementation, evaluation and mobilisation - and involves critical appraisal of multiple sources of evidence, aggregation of that evidence, and subsequent integration of the evidence into the decision making process.


References

ALONSO-COELLO, P., OXMAN, A. D., MOBERG, J., BRIGNARDELLO-PETERSEN, R., AKL, E. A., DAVOLI, M., TREWEEK, S., MUSTAFA, R. A., VANDVIK, P. O., MEERPOHL, J., GUYATT, G. H. & SCHÜNEMANN, H. J. 2016. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ, 353.

NEELEMAN, A.-M. 2017. Grasping the scope of school autonomy: a classification scheme for school policy practice  Belmas. Stratford on Avon, England