The underpinning thesis
The main 'thesis' underpinning Brown et al's presentation was a call to integrate data based decision-making (DBDM) and research informed teaching practice (RITP) into a comprehensive professional learning based approach that is designed to enhance teaching quality, and a result increased student achievement. As such, the model is attempting to use the strengths of each approach to offset the weaknesses of the other. Brown et al summarise the offsetting benefits of each approach as follows:
- RITP is not based on a real need in the field vs DBDM starts with the vision and goals of a specific school, focusing on a context specific problem.
- DBDM data can inform educators about problems in their school, but what causes this problem vs RITP educators can draw upon a variety of effective approaches to school improvement
- RITP is the ‘one size does not fit all’ vs DBDM based on data, schools develop a context specific solution
- DBDM is that data can be used to pinpoint possible causes of a problem, but educators may still not know the best available course for school improvement vs RITP picking a promising solution, based on an existing evidence base (slide 23).
An eight stage cycle of evidence-based inquiry
Brown et al then go onto integrate DBDM and RITP into the following eight stage cycle.
On initial reflection this model provides an incredibly useful way of thinking about the relationship between data and research evidence within the inquiry cycle, with with the local data setting the scene for the necessary research evidence. Indeed, one of the things I have been thinking about lately is where 'local data-collection' fits within the models of evidence-based practice and evidence-based medicine which I have previously described and advocated. And this model, clearly places local data prior to seeking the research evidence.
However, for me, the model two limitations to be taken into account: first, the determination of causes coming before the collection of data; second, the potential of for increasing the risk of confirmation bias. So lets now examine each of these limitations in more detail.
Causes before data or vice versa
The eight stage model presented places the stage of determining possible causes of the problem before collecting data about causes, which to me seems to be out of sequence and in reverse to order to what they should be . Indeed, there are other models of data led inquiry where data collection comes before diagnosis - and this illustrated in the four stage model of inquiry popularised by Roger Fisher and William Ury in the their classic book Getting to Yes
In this model the data stage focuses on identifying what's wrong, what are the current symptoms, and then moves to the identifying of possible causes in the diagnosis phase, with the next two phases being direction setting and action planning. For me, and I know when you use these cycles you move back and forth between the stages - it seems far more sensible to place the data-collection before the diagnosis phase. In addition, and I thank Rob Briner for this observation, the proposed process seems to separate use of the of research evidence from the diagnosis phase. Yet, why would you not use research evidence to help you identify possible causes of the symptoms being experienced.
The potential for cognitive bias
By placing both vision and goal setting next to determining causes and the drawing of conclusions prior to the search for the solution in the research evidence, this may lead to confirmation bias i.e the tendency to selectively search or interpret information in a way that confirms your perceptions or hypotheses. So determining the vision and goals may lead to certain problems to be highlighted as they are consistent with the already established, whereas other problems or data inconsistent with the 'vision' may be ignored. Furthermore, given that conclusions are drawn before searching for research evidence - this may lead to research evidence being sought which is consistent with the conclusions that have been already been drawn.
A possible alternative
Barends, Rousseau and Briner (2014) provide an extremely useful definition of evidence-based practice, which identifies a six-stage process in making decisions which are based upon evidence. In this model the search for and acquiring of evidence happens at stage two, with research evidence, organisational data and stakeholder views being accessed. This leads to the subsequent appraisal and aggregation of the evidence resulting in the incorporation of the evidence into the decision-making and action planning process.
However, this process does not fully incorporate all the elements of included in the proposed eight-stage cycle of evidence-informed inquiry i.e vision and goal setting. This 'omission' can be rectified if so required, by the inclusion of another 'A' representing : Ambition, aims and goal setting and which relates to the vision, mission and values of the school, and which gives us a 7 stage cycle of evidence-based inquiry.
A seven stage cycle of evidence-informed inquiry
Some final words
Brown et al make a powerful case for combining DBDM and RITP and have developed a potentially useful initial synthesis of the two processes. However, the proposed model has some inherent limitations, be it diagnosis taking place before data collection and increased possibilities of confirmation bias taking place. Finally, an alternative model - the 7 A's of evidence-informed inquiry - is put forward as way to think about the process of linking DBDM and RITP.