Can nurse education help you become a better evidence based educational practitioner?

A fundamental skill in becoming an evidence-based educational practitioner is the ability to formulate questions in a manner which allows them to be answerable.  Too often questions are phrased about in a way which is relatively unhelpful in trying to generate evidence based answers.  In this post I intend to adapt the work of Stillwell, Melnyk, & Williamson (2010)  in the field of nurse education in order to help teachers, educators and leaders go about the task of devising well formulated questions which contribute to better decision-making.

Stillwell states that one way of thinking about formulating questions is to distinguish between background and foreground questions.  A background question involves asking a broad question, which can be answered by accessing basic information - for example – what are the potential benefits of e-learning? Whereas a foreground question is more specific and relevant to a particular pedagogic or managerial issue, for example, for level 3 students does 24/7 access to iPads compared to the classroom use of Chromebooks improve the  timely completion of coursework?

Asking better foreground questions using the PICOT format

In evidence-based medicine well formulated  questions are often structured using the PICOT approach. PICOT is an acronym for the components of a clinical question and are as follows:

P — Patient(Pupil/student) or Problem. How would you describe the group of patients or problem?

I — Intervention. What are you planning to do with your patients?

C — Comparison. What is the alternative to the intervention (e.g.different intervention)?

O — Outcomes. What are the effects of the intervention?

T  - Time.  What time does it take for intervention to achieve the outcomes.

(Note in much of the literature PICOT is abbreviated to PICO, however I believe the longer version is more useful to educators). Now let's try and see if we can use this technique using to ask better questions to inform our educational practice.

Worked examples*

• For general further education students aged 16 who failed to achieve at least at a grade C in GCSE English  (P) and subsequently retake GCSE English  (I) at the end of the academic (T) year, how well do they achieve (O) compared to students who have been prepared and entered  for iGCSE English (C). 

• For students requiring students requiring additional learning support (P)  how does the provision of 1 to 1 support (I) compared with group support (C) affect retention rates (O) in the first term (T).

• For level 3 BTEC Extended Diploma students who do not have a grade C or better in GCSE Mathematics on entry (P) and subsequently retake GCSE Mathematics (I), how do their two year (T) success  rates (O) on their level 3 BTEC Extended Diploma programme compare to students who chose not to re-sit GCSE Mathematics (C).

(* I'm sure readers can come up with their own examples which are more precise, as these are some of my first attempts at producing PICOTS - in that sense, this blog is personal CPD made public)

What are the practical implications for using PICOTs?

1. For teachers it provides a structured framework to think through the challenges which they are facing in their own classrooms.
2. It provides a structure for teachers to provide senior leaders within their school/college with constructive challenge when discussing substantive evidence based issues
3. The introduction of PICOT type questions provide an opportunity for capacity and capability building across the school/college as a whole, and provides a mechanism to facilitate the creation of a school/college culture which values evidence-based practice and enquiry.

Reference
Stillwell, S. B., Melnyk, B. M., & Williamson, K. M. 2010. Asking the Clinical Question : A key step in evidence based practice. American Journal of Nursing, 210(3).

So you want to be an evidence based educational leader

Last week I wrote about the need for a synthesis between the research/evidence informed education movement and the discipline of evidence based management.  In this post I will be using Barends, Rousseau, & Briner's (2014) pamphlet on the basic principles of evidence-based management in order to explore in more detail the implications of such an approach for evidence based educational leadership.
Barends et al define evidence based practice as the making of decisions through the conscientious, explicit and judicious use of the best available evidence from multiple sources by:

• Asking: translating a practical issue or problem into an answerable question
• Acquiring: systematically searching for and retrieving the evidence
• Appraising: critically judging the trustworthiness and relevance of the evidence
• Aggregating: weighing and pulling together the evidence
• Applying: incorporating the evidence into the decision-making process
• Assessing: evaluating the outcome of the decision taken

to increase the likelihood of a favourable outcome (p2)*.

In undertaking this task information and evidence is sought from four sources

1. Scientific evidence Findings from published scientific research.
2. Organisational evidence Data, facts and figures gathered from the organisation.
3. Experiential evidence The professional experience and judgment of practitioners.
4. Stakeholder evidence The values and concerns of people who may be affected by the decision.

Already it should be self-evident that there are a number of challenges, though not insurmountable, to the practice of evidence-based educational leadership.  Three challenges seem immediately apparent:

1. The need to develop a greater understanding that being an evidence based educational leader requires more than reading John Hattie or looking at the latest Education Endowment Foundation reports.  It is about thinking fundamentally about how you go about your practice as an educational leader  and ensuring it draws upon multiple and possibly conflicting sources of evidence. 
2. Evidence based educational leadership requires the building of capacity, both individually and organisationally, to create the conditions to make better decisions.  Given the current budgetary pressures faced by the vast majority of schools and colleges, creating the time and the space for this capacity building may be a challenge in itself.  On the other hand, these challenging conditions makes the need for better decision ever more important.
3. Given the overwhelming importance of OfSTED inspections to what extent does this lead to internal school/college evidence being produced and presented in ways which make evidence-based educational leadership more difficult. In other words is the school/college based evidence, presented and interpreted in such a way to support a 'positive' narrative  - spin- rather than aid in a deep understanding of the issues at hand. 

So, if we accept  these challenges, where is the best place to start for an aspiring evidence based educational leader and manager?    Well there is no better place to start than to begin by developing the mind-set to ask critical questions, followed by the skill to ask those questions in way that they become answerable.

My next post will look to  borrow a technique devised in evidence based medicine to help us develop answerable evidence based questions.

Reference
Barends, E., Rousseau, D. M., & Briner, R. B. 2014. Evidence-Based Management : The Basic Principles. Centre for Evidence Based Management (Ed.). Amsterdam.

 *This definition is partly adapted from the Sicily statement of evidence-based practice: Dawes, M., Summerskill, W., Glasziou, P., Cartabellotta, A., Martin, J., Hopayian, K., Porzsolt, F., Burls, A., Osborne, J. (2005). Sicily statement on evidence-based practice. BMC  Medical Education, Vol. 5 (1)

researchED14 and evidence based educational leadership and management - is there a missing synthesis?

At the weekend I had the privilege of attending researchED14 and taking part in some of the most inspirational CPD that I have experienced in my thirty years in the education profession.   The experience reminded me of the famous Jean Luc Goddard saying that a story should have a beginning, middle and an end but not necessarily in that order.  For many colleagues this was their first experience of the teacher as researcher movement, whereas for others, it was another rich experience in a long developmental journey as a research-led practitioner. 

So going back to the beginning - researchED14 prompted me to revisit the seminal contribution to evidence based medicine of Sackett, Rosenberg, Gray, Haynes, & Richardson (1996) and consider the implications for education, and particularly educational leadership and management.

Sackett at al define evidence-based medicine as  ... the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. (p71)  They go onto describe what good doctors do, which I have amended for teachers and educators. Good teachers use both individual teaching expertise and the best available external evidence, and neither alone is enough. Without teaching expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual pupil or school. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of pupils. (Amended from Sackett p71)

However, Sackett et al go on to argue what evidence-based medicine is NOT and using that as a starting point I have sought to make comparisons with the research-led/evidence based education movement.

Evidence based medicine is NOT(Sackett et al)	Evidence based education is NOT
Old-hat and is something everyone is doing.	Something all teachers are doing.
Conducted only by researchers in ‘ivory-towers.	Just conducted in university education departments but also takes place in school.
Cook-book or recipe led practice.	Attempting to replace teaching craft and skill with a check-list of pre-determined approaches.
Restricted to randomized trials and meta analyses.	Just about  EEF sponsored randomized trials  and requires individual teachers to reflect on how to improve their practice in their own particular school or college.
A method for reducing the cost of health-care.	About reducing the cost of education by making all teachers more effective but will require significant investment in teacher CPD in order to increase basic research literacy.

However, for me there is something missing in that research informed/ evidence based education needs to be more than about pedagogy, policy and practice, it also needs to be about educational leadership and management.  Most leadership and management decisions made by head-teachers and heads of department are not based on the best available evidence.  Often they are either based solely on prior experience or possibly the latest bestselling 'airport' business guru .   However, both are potentially hugely flawed in that personal experience is subject to a number of systematic errors and business best-sellers are often little more than anecdotes or when they do refer to evidence are neither reliable or valid (see Phil Rosenzweig's book The Halo Effect .... and the eight other business delusions that deceive managers  and my post of 29 April 2014)

So what does this mean for the research informed education movement.  First, the reach of this movement needs to extend to educational leadership and management practice, encouraging headteachers to ask the question ' what is the best available evidence' before making leadership and management decisions, and avoiding over-reliance on personal experience and judgement.  Second, the research informed education movement needs to  develop a synthesis with the field of evidence based management and draw upon the work of scholars such as Eric Barends, Denise Rousseau, Rob Briner and David Denyer.  

This potential of this work is enormous in that it will contribute to better informed decisions leading to better educational experiences for students and improved working lives for teachers.

Reference

Sackett, D. L., Rosenberg, W., Gray, J., Haynes, R. B., & Richardson, W. S. 1996. Evidence based medicine: what it is and what it isn't. BMJ, 312(7023): 71-72.