School Research Leads and Ambitiously seeking to improve practice through the use of evidence

Last week' s post, adapting the work of Barends, Rousseau and Briner (2014),  provided a definition of evidence-informed practice for teachers and schools.  This post, using the first of the 9 A's outlined in the definition - Ambitiously : seeking to improve practice through the use of evidence - will detail both what ambitiously means, and the steps which can be taken to help realise that ambition.  

So what does -  Ambitiously: seeking to improve practice through the use of evidence  mean in practice?  Well, there are clearly has similarities with Hargreaves and Fullan's (2012)  guidance for teachers on how they can develop their professional capital.    In the first of their 10 action points, Hargreaves and Fullan state teachers need to  - Become a true pro.  Hargreaves and and Fullan state :

Teachers are dedicated.  They care about their subjects and their students. They put in endless extra hours.  It is a lot.  But it is  not enough .  Not if you want to teach like a pro.  Teaching like a pro means preparing yourself properly : putting in years of study and practice until you reach your 10,000 hours of of highly accomplished performance and then honing your skills even more as you help develop the next generation of teachers …..Teaching like a pro means connecting with the latest research evidence, inquiring into your own practice - with other colleagues and other schools, down the street and across the word - to find new ideas, get advice, and sift what work from what doesn't. (p155)

In other words, your ambition is to become a tru pro in the use of evidence-informed practice to bring about favourable outcomes for those affected by your actions.  To achieve this there are four principles: first, aim to be expert in the use of evidence; second, where evidence is referred to - check-it-out. third, just because you think you know what the evidence is, check-it-out; four, you are responsible for you own professional learning.  

Aim to be expert in the use of evidence  

In my post of 16 February, 2015 I described the Dreyfus model of human learning, which identifies five levels within the human learning process: novice; advanced beginner; competent; proficient; and, expert.  Brown and Rogers (2014) - in a study designed to facilitate the use of evidence-informed practice - combine the Dreyfus model  with the work of Flyvberg (2001) and Hall and Hord (2001) to create a typology of the descriptors of expertise in the use of evidence. 

Self-assess your self against these descriptors and identify your current level of expertise in the use of research evidence in your day-to-day practice.  Make a plan on how you an move from your current level of expertise to the next stage, for example, how do you move from being a competent to a proficient researcher.  Identify a small-step that you can take right now, to help you make that change.  But remember, expertise in the use of research-evidence is just one part of being an expert evidence-informed practitioner; you will also need to develop your expertise in the use of the other three (stakeholder views, school data and professional expertise) sources of evidence. 

Whenever evidence is referred to - check-it-out 

Just because an expert or guru cites evidence you still need to check-it out. Hargreaves and Fullan cites the 10,000 hour rule for becoming expert or proficient, and  which has gained its prominence through the work of Malcolm Gladwell and his 2008 book Outliers.   Gladwell draws attention to  the work of Anders Ericsson and his colleagues who argue that even the most gifted performers required 10,000 hours of deliberate practice   Nevertheless, research by Hambrick et al (2014) reported  that only a third of differences in performance, in elite music and chess, can be explained by accumulated difference in deliberate practice. Let's be clear, I am not arguing that becoming an expert evidence-informed practitioner does not take time and deliberate practice. Instead, I am arguing that you need to check the evidence, especially when it is referred to be experts or gurus.  If in doubt - CHECK-IT-OUT. 

Just because you heard someone say something at a conference does not mean his or her view has not changed. 

Even when you think you know what the evidence is - CHECK-IT-OUT.  At researchED's 2014 conference I saw David Weston, of the Teacher Development Trust,  give a fantastic talk on how a teacher's expertise tended to develop rapidly in the first five years of teaching experience, and then quickly flattens out, with teachers of twenty years of experience being no more effective than those teachers with five years experience.   As such, I was going to use this summary of research to justify a claim that teachers should engage in evidence-informed practice as a career-long strategy for professional learning. NevertheIess,  I  thought I'd better check the evidence, and thanks to David himself, - I've been pointed in the direction of research evidence suggesting the relationship between years of experience and teacher expertise is more complicated than first thought.  For example, Kraft and Papay's (2014) analysis shows that teacher effectiveness is linked to professional environments, with teachers in more supportive environments improving their effectiveness more over time than teachers working in less supportive contexts.  

It's down to you  

Finally - acknowledge that getting better at using evidence is a career-long activity and is not something confined to the first few years of professional development.  No one else is responsible for your professional learning, your head-teacher or head of department may be able to help with time and resources, but they are not responsible for your learning.  Becoming an expert evidence-informed practitioner is important and should, wherever appropriate, be prioritised over the non-important and yet somehow urgent  activities that so-often consume our daily working-lives. 

References

Barends, E., Rousseau, D. M., & Briner, R. B. 2014. Evidence-Based Management : The Basic Principles. Centre for Evidence Based Management (Ed.). Amsterdam.
 *This definition is partly adapted from the Sicily statement of evidence-based practice: Dawes, M., Summerskill, W., Glasziou, P., Cartabellotta, A., Martin, J., Hopayian, K., Porzsolt, F., Burls, A., Osborne, J. (2005). Sicily statement on evidence-based practice. BMC  Medical Education, Vol. 5 (1)

Hargreaves, A. & Fullan, M. (2012) Professional Capital : Transforming teaching in every school, Routledge, Abingdon. 

The School Research Lead - A definition of evidence-informed practice for teachers and schools

In writing previous posts I have often wondered what is meant by evidence-informed practice.  Now I have thought about it some more and I wish I hadn't. Having now devised a definition of evidence-informed practice, I feel that I have to share it with the 'blogosphere' and 'twitterverse,' both of which are unforgiving environments for the half-formed idea.  Nevertheless, I'm going to send the definition out into the 'wilds' to see what happens.  So here goes:

Drawing upon the work of Barends, Rousseau and Briner (2014),*  and which I have adapted for use by teachers and schools, I would like to offer the following definition of evidence-informed practice

A definition of evidence-informed practice

Evidence-informed practice involves the making of decisions within schools through the conscientious, explicit, judicious and skilful use of four sources of information:  

• educator's (at all levels of the school) expertise and judgment  
• open and transparent evidence from both within the school and other sources, which is subject to rigorous evaluation 
• a critical analysis and appraisal of the best available scholarly research - both theory and evidence
• the values and preferences of  pupils, educators, parents, employers and other stakeholders who might be affected by the decision.   

And by 

• Ambitiously : seeking to improve practice through the use of evidence 
• Acknowledging : and challenging both cognitive and value biases through actively seeking non-confirmatory evidence and views 
• Asking : translating a practical issue or problem into an answerable question 
• Acquiring : systematically searching and retrieving both internal and evidence 
• Appraising : critically judging the trust-worthiness and relevance of the evidence 
• Aggregating : weighing and pulling together the evidence 
• Applying : the evidence into the decision-making process 
• Assessing : evaluating the outcome of the decision-taken and learning from that process to improve future evidence-informed practice 
• Acting : collegiality and collaboratively to build trusting yet mutually challenging relationships both within and between schools 

to increase the likelihood of favourable outcomes for pupils, educators, the school and the educational system.

So what do you think?

Note - 
I'd like to thank those colleagues who have helped with developing this definition, you know who you are.

Reference

Barends, E., Rousseau, D. M., & Briner, R. B. 2014. Evidence-Based Management : The Basic Principles. Centre for Evidence Based Management (Ed.). Amsterdam.
 *This definition is partly adapted from the Sicily statement of evidence-based practice: Dawes, M., Summerskill, W., Glasziou, P., Cartabellotta, A., Martin, J., Hopayian, K., Porzsolt, F., Burls, A., Osborne, J. (2005). Sicily statement on evidence-based practice. BMC  Medical Education, Vol. 5 (1)

We need to talk about researchEDthics - School Research Leads and Ethical Research and Evidence-Based School Cultures

This post was first published in March 2015

At the recent researchED Research Leads conference held in Cambridge, the 'ethics' of how school-based and led research should be conducted emerged as a key topic for discussion.   In this post I will explore some of the ethical issues associated with the role of the School Research Lead and school-based and led research. In doing so, I hope the post will generate, both discussion and debate amongst the burgeoning number of School Research Leads about the ethical challenges of conducting both school-based research and evidence-informed practice. This attempt to promote this much-need discussion is framed by the following questions.

• What do we me mean by research? 
• What are the implications for school-based research of conducting research within an ethical framework? 
• What is the difference between research and evidence-informed practice? 
• What are the ethical considerations which need to be considered when undertaking evidence-informed practice? 

What is meant by research?

This section draws upon the work of the National Research Ethics Service (NRES) and the National Patient Safety Agency guidelines: Defining Research. As such, research will be defined as the process of creating new generalisable knowledge, and which could include both the generation and testing of hypotheses. The NRES publication goes to state that research may involve some, it not all, of the following activities (amended for use in schools):

• defined aims and objectives 
• the testing or development of hypotheses 
• comparing interventions 
• collecting quantitative and qualitative data over and above data generally collected within the school - though could include normally collected data 
• significant change to teaching approaches/support strategies - which are in addition to what is provided for pupils 
• allocation of pupils to intervention groups 
• the use of a sampling framework to decide who is within the study 
• any form of intervention involving some form of randomnisation. 

What are the implications for school-based research of conducting research activities within an ethical framework?

• There would appear to be a lack of understanding of the difference between research, quality improvement and evidence-based practice.
• Much of the 'so-called' research or evidence-informed practice being undertaken in schools, could be classified as quality improvement
• However, ethical considerations are just as relevant to quality improvement as to research

 However, it is unrealistic expect all changes to teacher's practice to be subject to ethical approval, and I now turn to the difference between research and evidence-informed practice.

What is the difference between research and evidence-informed practice?

In the context of nurse education, Carnwell (2001) identified the following differences between research and evidence-based practice. These differences are illustrated in Table 1, and have been amended for a school context.

Table 1 - The differences between research and evidence-based/informed practice

Amended from Carnwell (2001, p58)

As such, the main difference between research and evidence based practice is, evidence-based/informed practice is concerned with the USE of the best current available evidence to inform practice, taking into account the needs and wishes of the learner/pupils. Whereas research involves an attempt at the PRODUCTION/CREATION of new generalisable knowledge, which can be applied in a range of contexts.

What are the ethical issues associated with evidence-informed practice?

Accepting the distinction between research and evidence-based/informed practice, a number of ethical issues need to be addressed whilst engaging in evidence-informed practice. Again, I will turn to the evidence-based medicine movement for initial insight and the four principles which inform medical-ethics. In the following paragraphs the four principles of medical ethics are 'mapped' against a school setting.

Justice

The ‘justice’ principle requires the burdens and benefits of new teaching approaches or innovations to be distributed equally among all pupils or staff within the school. The ‘justice’ principle requires evidence-informed practice to uphold existing legislation, and to be fair to all pupils and staff.

Beneficence

This principle requires the action or intervention being proposed is being done with the intent of doing good for the pupil, member of staff or school.

Non-maleficence

This principle requires an action or intervention not to do harm to the pupil, member of staff involved or other stakeholders. Educators are required to take a wider perspective and consider whether actions may have unintended negative consequences, and which are subsequently experienced by others.

Autonomy 

This principle requires the pupil or member of staff have autonomy of thought and action when involved in decision-making. With pupils, and given the nature of teaching and learning it is difficult to imagine a situation with young children, where it is possible to envisage fully-informed consent in all circumstances. However, when projects involve other colleagues, this seeking their consent is both, principled and practical.

What are the implications of these principles for evidence-informed practice?

The implications of the above principles are .

1. For teachers to ethically engage in evidence-informed practice they must be explicitly aware of an appropriate ethical framework. 
2. Teachers to give active consideration to the application of that ethical framework prior to any changes in practice arising from an investigation of the best current available evidence. 
3. Ethical considerations to be explicitly noted in some form; be it a note on a lesson plan; referenced to within a scheme work; or recorded in some way, be it a reflective diary or log. 
4. Any processes for the recording of ethical issues need to be both, proportional and fit for purpose. 

To conclude:

In this post I have argued that the leadership and management of ethical issues is an integral part of the school research leads role. Second, if schools are to engage in research - as defined as seeking to create new and generalisable knowledge, including the testing or development of hypotheses - a clearly defined ethical approval process must be applied within the school. Third, evidence-informed practice involves making use of the best-available current evidence, and which distinguishes it from the prior definition of research. Fourth, teachers engaged in evidence-informed practice must be aware of ethical issues, and engage in a process of evidenced self-regulation. Fifth, whilst the enthusiasm for teachers to engage with research should welcomed and developed, it is essential colleagues use appropriate processes - this being a case of in order to do the right thing it is also essential to do things right. Sixth, whilst conceptual and operational clarity is being developed in relation to ethics, research and evidence-informed practice, teachers should focus on developing their skills as critical consumers of research. Finally, I hope this post is helpful in generating debates with schools about the ethical implications of conducting both research and evidence-informed practice.

Resources

Carnwell, R. (2001). Essential differences between research and evidence-based practice. Nurse Researcher, 8(2), 55-68.

https://www.bera.ac.uk/researchers-resources/publications/ethical-guidelines-for-educational-research-2011

http://www.lancashirecare.nhs.uk/media/Publications/R_and_D/Guidance/NRES_leaflet_Defining_Research.pdf

The School Research Lead - Supporting teachers to improve not prove

At the recent Research Leads Conference held in Cambridge, there was a discussion about Stokes's (1997) notion of Pasteur's Quadrant. In particular, there was agreement that the role of the school research lead was to encourage colleagues to engage in Yes/Yes research activity, i.e. use-inspired basic research, and the creation of new and generalisable knowledge. In this post I argue this stance is both philosophically and practically mistaken, but rather the focus should be on No/No activity, which primarily develops individual colleague's skills as researchers. Furthermore, the Yes/Yes stance could have significant negative unintended consequences for the desired development of evidence-informed cultures within schools. First, let's examine Stokes's model of scientific research.

Stokes's Quadrant Model of Scientific Research

At September 2014's researchED conference, Dylan Wiliam gave a fascinating explanation of why education will never be a research-based profession. In this presentation Wiliam made reference to Stokes's (1997) notion of Pasteur's Quadrant. Stoke's developed his quadrant model of scientific research as an attempt to end, in his view the unhelpful separation between basic and applied research, which he argued made it more difficult to think about the relationship between scientific discovery and improvement. In order, to challenge this dichotomy 'pure' and 'applied' research Stokes developed a 2 x 2 table by which to classify research, and is illustrated in Table 1.

The upper left-hand cell is contains research which is driven purely by understanding for its' intrinsic values, for example, Bohr's work on the atomic structure. The upper right hand cell captures research which is undertaken to for both understand and use, the search for understanding is driven by the need to solve some very specific problems, for example, Pasteur's work on microbiology and pastuerisation. The bottom right hand cell focuses on research which has very specific applied goals, without necessarily seeking to gain engagement and understanding of the basic phenomena which are being 'exploited'. Edison's work in the commercial development telecommunications and electrical lighting sits neatly into this cell. (all examples are cited by Stokes).

However, Stokes argues that this does not mean that the bottom left-hand cell is empty. This cell may involve exploration of particular phenomena where there is no attempt to understand the issue being studied or even apply the findings. Examples, of this can be found where data is being classified and presented, without any subsequent discussion about the implications for action of such data. As such, this type of research is often driven by the intrinsic curiosity of the researcher. Furthermore, this cell is not just limited to 'curiosity,' research activity in this quadrant may involve the researcher developing their own skills as a researcher, rather than seeking to create new knowledge and innovations.

Implications for School Research Leads and Teacher Researchers

For School Research Leads and Teachers there are three main implications of Stoke's model of scientific research.

1. If you accept Flyvberg's argument that social sciences should give up on 'physics-envy', then focussing the work of the school research lead on supporting the development of Yes/Yes activity is probably misguided. It is misguided in a far as: one, it is not achievable; two, the vast majority of colleagues do not have the appropriate skills set for such activity; three, nor do they have the time.

2. If the focus is on applied research, then the tools and techniques of evidence-based practice should form an essential component of the School Research Lead. At the core of evidence-based practice is the idea that decisions are informed, not determined, by the best available evidence. The focus of evidence-based practice activity is on bringing about improvements in decision-making, which subsequently provides for the benefits of pupils and colleagues.  Evidence-based practice is not about the creation of 'new' generalisable knowledge, which is applicable outside of the current-setting.  Furthermore,  there is an emphasis on accessing evidence from a number of sources, not just 'scientific' academic research. School generated evidence is essential, alongside professional judgement and the views of stakeholders.

3. Teachers should focus on developing their skills as evidence-informed practitioners, be it by critiquing research or engaging in evidence-based practice, with the School Research Lead being one source of support. Indeed, to go back to the beginning of this blog and Dylan Wiliam's presentation, the challenge is for teachers throughout their careers to improve practice through the process of 'disciplined inquiry'.

In many ways the core message of this blog is as follows: do simple things well; ask good questions; make the most of the current best available evidence; develop your skills incrementally; apply what you find out to improve your practice; and, evaluate what you do. In doing so, you will end-up having a far more interesting and enjoyable career. More importantly, it will make positive differences to the lives of pupils and colleagues. Remember, you are teachers wishing to improve, rather than researchers seeking to prove.

References

Flyvberg. B. (2001). Making Social Science Matter : Why social inquiry fails and how it can succeed again, Cambridge University Press, Cambridge/

Stokes, D.E. (1997). Pasteur's Quadrant : Basic Science and Technological Innovation, Brookings Institution Press, Washington D.C.