The school research lead and the half-life of facts - why everything we know has an expiration date

One of the main reasons why school leaders need to be constantly engaged in evidence-based practice, is that the evidence we might use to support decisions can become ‘rusty’ (Hargreaves and Fullan, 2012).   Evidence that we have relied on in the past may no longer be current or applicable, indeed advice based on the evidence may have just been plain wrong.  Indeed, (Arbesman, 2012) argues that just like radioactive material, ‘facts’ have a half-life and decay over time.  Arbesman states:

Facts, in the aggregate, have half-lives: We can measure the amount of times for half of a subject’s knowledge to be overturned.  There is science that explores the rates at which new facts are created, new technologies are develop, and even how facts spread. (p3)

To illustrate this point, Arbesman cites the work of (Poynard et al., 2002) who demonstrate that in a particular field of medicine – cirrhosis and hepatitis -  that the half-life of facts in this field was approximately forty-five years.  In other words, half of what we thought we knew about cirrhosis and hepatitis was within forty-five years been shown to be incorrect. 

Now in the context of social science – within which much educational research finds a home – the half-life of facts is generally shorter than forty-five years.  – in part because of some difficulties associated with undertaking social science and trying to distinguish the signal from the noise (Silver, 2012).  In other words, the conduct of social science is much messier than the physical sciences, making it much more difficult in establishing ‘incontrovertible facts’.

So what is the importance of being aware of the half-life of facts.  Arbesman argues that being aware of how facts change, develop and spread is important in that facts – however defined, help us make sense of the world we are facing.  If we are aware of how facts change and develop over time, it will allow us to be more critical of the evidence on which we base decisions, as it will help us identify the both limitations of that evidence and what actions we can take to offset such limitations.   

However, in the context of being an evidence-based school leader, being aware of the half-life of facts has a number of important implications.  First, the need for for school leaders to be constantly searching for the best available evidence, recognising what is the best available evidence is constantly changing.  Second, evidence-based school leaders may have to act on the basis of that much of what research evidence suggests works, may in the long-run turn out to incorrect.   Indeed, there may-be parallels between educational research and the old saying about advertising expenditure, fifty percent works, fifty percent doesn’t, trouble is we don’t know which is which. Consequently, evidence-based school leaders need to proceed with a large-degree of humility.  Third, evidence-based school leaders are going to need to be able to adapt to a world of changing facts, and what was certain at the beginning of their working life, may be less so towards the end of your career.   What was once ‘cutting-edge’ may now be just a jagged old ‘rusty-edge.’   Finally, and probably most importantly, evidence-based school leaders will need to contribute to developing an evidence-based school culture, where all staff are constantly challenging what they think, and in doing so provide conditions for effective and stimulating professional learning for teachers, at all stages  of their career.

  

References

ARBESMAN, S. 2012. The half-life of facts: Why everything we know has an expiration date, Penguin.

HARGREAVES, A. & FULLAN, M. 2012. Professional capital: Transforming teaching in every school, Teachers College Press.

POYNARD, T., MUNTEANU, M., RATZIU, V., BENHAMOU, Y., DI MARTINO, V., TAIEB, J. & OPOLON, P. 2002. Truth survival in clinical research: an evidence-based requiem? Annals of internal medicine, 136, 888-895.

SILVER, N. 2012. The signal and the noise: the art and science of prediction, Penguin UK.

The school research lead and writing a critical synopsis

Given your responsibilities as either a research lead or senior leader within a school, you will be mindful of how to make the most of the relatively scare amount of time that you have available for engaging with research evidence.  (Wallace and Wray, 2016) suggests five questions that should act as a your starting point whenever you are reading and summarising research evidence and which will  contribute to capabilities to be an evidence-based practitioner. In addition, these five questions can form the basis for producing a written summary of research evidence, which has the benefit that it can easily be shared with colleagues, and may aid their own evidence-based practice.  It also has the benefit of you the school-leader, modelling evidence-based practice, and which is an essential components o the leadership strategies necessary to lead the research and evidence-based school. (Brown, 2015)

So what are Wallace and Wray’s five critical questions when reading research.

• Why am I reading this?
• What are the authors trying to achieve in writing this?
• What are the authors claiming that is relevant to my work?
• How convincing are these claims, and why?
• In conclusions, what use can I make of this? 

Again, using a template provided by (Wallace and Wray, 2016) a critical synopsis of a text is best illustrated by the development of a worked example. 

Author, date, tiles, publication details:  GODFREY, D. 2016. Leadership of schools as research-led organisations in the English educational environment Cultivating a research-engaged school culture. Educational Management Administration & Leadership, 44, 301-321

Code : 1 return for detailed analysis; (2) an important general text; (3) of minor importance; (4) Not relevant

Why am I reading this?  I am trying to find out more about the conditions necessary for the long-term development and sustainability for the evidence-based school.

What is the author trying to achieve in writing this? The author is trying to explain the conditions for growth and expansion of research-engaged schools.  In doing so, the author is trying to make a contribution to theory-development.  At the same time, the author’s article is informed by his own research  into the development of a school’s research culture.

What is the author is claiming that is relevant to my work? The author provides a biological analogy, for understanding the development of the research-engaged school movement.  As such, nourishing factors such a systemic connectedness, leadership for knowledge creations, teaching as a research-informed practice, and the school as a learning organisation need to be taken into account.  To make the most of the research-informed school requires a long-term and sustainable school improvement strategy.

How convincing are these claims, and why? Within the conceptual framework used the argument is convincing. However, a particularly narrow view is taken of ‘evidence-based practice’ and the term ‘research-informed practice’ is used instead. As such the nature of ‘evidence-based practice’ is misrepresented, which undermines the quality of the article as a whole.   In addition, with the use of a biological analogy, it is surprising that there is no reference to both the decline and ‘death’ of a culture of research culture.

In conclusion, what use can I make of this? The paper is useful as an overview of the development of research-informed/engaged schools.It also has a number of suggestions for strategies for what a research-engaged school could look like, and how a school’s culture might change over time.

Code 1/2 (1) return for detailed analysis; (2) an important general text;

Nevertheless, each one of the five critical questions is be supported by a number of sub-questions, which will allow you to unpick underlying detail of the main question.  (Wallace and Wray, 2016) have provided us with a useful list of questions, which I have amended to ensure a specific focus on the needs to the evidence-based practitioner

1. Why am I reading this?

• Is this reading going to help me gain a better understanding of background or foreground questions?  
• Am I still trying to formulate an answerable question?
• Is the reading to assist with making an evidence-based decision or for the purpose of academic study?
• Is this reading about helping me develop my skills evidence-based practitioner?

2.What are the authors trying to achieve in writing this?

The authors may have a range of differing purposes including:

• Suggestions to changes in practice
• Report on their own research activities
• Synthesise and evaluate the work of others.
• Contribute to the development of theory
• Criticising both policy and practice 

3. What are the authors claiming that is relevant to my work?

• What is the text actually about and what do the authors say about it?  
• How does text the relate to your own interests and problems of practice?
• Is the text directly linked to your problem of practice?
• Is the text indirectly related to your problem of practice?
• Does the text provide an alternative perspective on the problem of practice?

4. How convincing are these claims, and why?

• Are there any underpinning assumptions, which have not been made explicit?
• Are the conclusions of the study warranted?
• Is the evidence used to support an illogical conclusion?
• Does the evidence suggest other conclusions?

5. In conclusion, what use can I make of this?

• Does this research contribute to your understanding of your problem of practice?
• Is the research relevant to your context?
• Is there any evidence to suggest that any ‘intervention’ might work in your school?
• Does the research influence any decisions you need to make?
• Is this research worth sharing with colleagues?
• Should I undertake further reading?
• Should I seek further sources of ‘counter’ evidence, particularly if the research is consistent with own values and experiences? 
• How does this research relate to the other three sources of evidence used by an evidence-based practitioner.

Now, this is only a very brief introduction into the process of becoming a critical reader of research evidence.  Further guidance on how to develop your skills can be found in (Wallace and Wray, 2016) where they explain further the skills necessary to both develop an in-depth analysis of a text and to engage in critical writing.  Further guidance can also be found in (Booth et al., 2016) which examines the craft of research and how to develop your argument.

References

BOOTH, W. C., COLOB, G., G, WILLIAMS, J. M., J., B. & FITZGERALD, W. T. 2016. The Craft of Research (Fourth Edition), Chicago, The University Of Chicago Press.

BROWN, C. (ed.) 2015. Leading the use of research & evidence in schools: IOE Press.

WALLACE, M. & WRAY, A. 2016. Critical reading and writing for postgraduates, Sage.