The role of research evidence in educational improvement

If you are a teacher, school research lead or headteacher interested in the role of research evidence in  educational improvement, then this post is for you.  The recent White Paper places great store on the role of evidence in improving teaching and learning and this post, drawing upon the work of Hargreaves and Stone-Johnson (2009) will: first,  dentify some of the complexities associated with the relationship between research/evidence-based practice and educational improvement; second, consider the implications for evidence-based change of the different conceptions of the practice of teaching; third, explore the implications for you in your role - teacher, school research lead or head-teacher as you seek to bring about evidence-based educational improvement.

Complexities in evidence-based practice in educational improvement

Hargreaves and Stone Johnson argue that the debate about the use of research evidence* in educational improvement is particularly complex for two reasons.  First, the extent to which stronger and better evidence might advance the quality of teaching and its impact depends upon the nature of the evidence itself - on how it is defined, used, interpreted, and made legitimate.  Second, the relevance and impact of evidence-based practice in teaching also depends on what conceptions of teaching and learning are being employed, how evidences applies to each of them, and how these conceptions shape the conditions under which evidence itself will be used or misused. (p90)

Brown (2015) goes onto expand upon these areas of controversy and debate:

• the epistemological differences between academic researchers and policy-makers in terms of what counts as evidence, the quality of evidence and what evidence can or can't tell us;
• whether the evidence-informed movement serves to work agains the practitioners' professional judgement;
• issues in relations to how formal academic knowledge and professional or tacit knowledge might be effectively combined;
• differentials in power that can affect or limit interactions between teachers or policy-makers and research/ers;
• controversies in relation to to some of the methods commonly associated with enhancing evidence uses;
• how the capacity to engage with academic research might be enhanced;
• issues such as the inaccessibility of research to teachers and policy-makers, both in terms of where it is published and the language that is typically used in such publications. (adapted from p1)

In addition, Hargreaves and Fullan (2012) also provide a number of reasons why the case for the use of research-based evidence in decision-making can be overstated:

• evidence-based decisions can be tainted with self-interest;
• cast-iron evidence can get rusty later on;
• evidence-based principles are used very selectively;
• evidence isn't always self-evident;
• evidence on what to changes isn't the same as how to change;
• positive initiatives based on evidence in one area can inflict collateral damage;
• people can cook the data; 
• evidence-based teaching is only somewhat like evidence-based medicine;
• evidence comes from experience as well as research. (adapted from p47)

The Practices of Teaching 

Hargreaves and Stone Johnson state : Teaching - like medicine, law, or dentistry - is a professional practice.  It consists of systematically organised and thoughtfully as we as ethically grounded activities among a community of professional that is dedicated to the service of others.  

The practice of teaching is complex, It is not mechanical or predictable. Nor does it follow simple rules. Indeed teaching is an assemblage of many practices, each affecting, drawing on, or intersecting with the others.  Each of these practice relates to evidence in its own distinctive way. (p91)

Table 1 seeks to summarise - as identified by Harris and Stone-Johnson -  the relationship between the different practices of teaching and the role of research evidence in evidence-informed change.

Practice of teaching	Description	Role of evidence in evidence-informed change
Technical	Teaching involves the mastery and employment of a technical skill	Evidence could show how differing teacher behaviours effect on student outcomes, though perspective underplays the complexity and challenge of teachers’ work.  Approach needs to reflect deep understanding of evidence-based medicine and practice and role of practitioner expertise and patient values.
Intellectual	Teaching involves increasingly complex work that is highly cognitive and intellectual	Evidence provides a source for improving student learning through enhanced teacher learning about effects of their teaching; strengths and needs of their students; and alternative strategies that have externally validated record of success.
Experiential	Teaching understandings of their problems deeper than offered by theorist, teachers can provide common-sense insight	Evidence provides legitimate but imperfect basis for professional judgment and knowledge.  Practical experience is as important as research-driven knowledge. Validity of teacher knowledge depends upon the conditions in which it is produced as well as the processes by which it is validated.  Teachers need to become adaptive experts who actively seek to check existing practices and have a disposition towards career-long professional lean
Emotional	Teaching is an embedded practice that produces emotional alteration in the stream of experience – giving emotional culmination to thoughts, feelings and actions (amended from Denzin)	Evidence-based changes need to include emotional goals and processes of learning (empathy, resilience, self-esteem) as well as emotional conditions for learning (safety and security).  Evidence-based education must go through process which strengthens the relationships of the groups and communities that produce it.
Moral and Ethical	Teaching is never amoral – it always involved ethical and moral practices, either in a good or bade Teachers promote and produce virtues such as justice, fairness, respect and responsibility	Requires judgment about how evidence is produced, used and interpreted.  Colleagues to hold others to account within professional learning communities for the integrity of their practice and the evidence supporting their work.
Political	Teaching always in some measures involves a relationship of power	Given selective use of evidence by government essential that teachers have the professional capacity to review, critique, make informed decisions, and adapt the evidence accordingly.
Situated	Teaching varies in what is taught, who is taught, and how learning is assessed	For evidence-based improvement to be effective contextual contingencies need to be embraced, with realistic timelines for change
Cultural	As teaching practices become ingrained and accepted – they form part of the professional culture of teaching ie the attitudes, beliefs, values and the patterns of relationship between teachers	Evidence-based educational initiatives require significant investment in the culture building process. They can also push collaborative teachers cultures to focus more persistently on the student learning needs, especially when this might create professional discomfort for teachers themselves.  A systemmatic connection between culture and evidence-based inquiry in caring and trusting relationships of ethical integrity is at the heart of the one of the most powerful principles of implementing evidence-based improvement: professional learning communities (Summarised and amended from Hargreaves and Stone-Johnson p91 - p103)

What are the implications of for your role as a leader of research evidence-informed change?

First, given the various of practices of teaching it is important to reflect on which of practices most closely mirrors your own view of teaching - and reflect on how that view will impact upon on your approach to developing research/evidence-informed practice within your school.  In particular, it is essential that you challenge your own 'practice of teaching' and its relationship with evidence and acknowledge the inherent weaknesses of your approach

Second, given that within your school there will be a diverse range of views on the practice of teaching, it may be necessary to do the following.  Identify which model or models colleagues are using to describe their practice and then adapt your use of 'evidence' to one which is consistent to that paradigm.  In doing so, this may provide you with the opportunity to engage in on-going dialogue rather than engage in the 'dialogue of the deaf.' If you do this, you will keep the conversation going and hopefully each of you will learn from one another.

Third, leading evidence-based or evidence-informed change is clearly a complex and challenging space and is will require a significant personal investment of time, no little skill, and large dollops of patience.  As such, given the challenges and complexities of evidence-based/informed change it would be wrong to expect evidence-based/informed practices to provide 'wonder-cures' for a school's 'ailments.'  (If it did, I'd like to see the evidence).

And finally*

This post has focused on the use of research evidence in evidence-based change, so remember evidence-based practitioners draw upon four sources of evidence - research, school data, stakeholder views and practitioner expertise. Research evidence is not the only evidential fruit.

References

Brown, C (2015) Evidence-Informed Policy and Practice in Education : A sociological grounding.  Bloomsbury. London

Hargreaves, A., & Stone-Johnson, C. (2009). Evidence-informed change and the practice of teaching. The role of research in educational improvement, 89-109.

Hargreaves, A and Fullan, M. (2012) Professional Capital: Transforming, teaching in every school. Routledge, Abingdon

Fads, faddism and the evidence-based practitioner

If you are an evidence-based practitioner, school research lead or headteacher and interested in evidence-based educational change, then this post is for you.  This post, drawing upon the work of Robert Slavin, will consider explore the fashionable nature of educational reform and the various pendulum like stages - both upswing and downswing -  which educational innovations invariably undergo.  Finally, we will consider the actions necessary to ensure your own practice is based on substance rather than the whims of fashion.

The pendulum of educational innovation

Everyone involved in schools and colleges  can no doubt draw up their own list of 'fads' and a few are listed:

• Brain-based teaching
• Character education
• Emotional intelligence
• Learning styles
• Multiple intelligences
• Self-esteem movement
• Portfolio-based assessment

Indeed, every period has its own fads - for example - Growth Mindsets, Flipped Learning, Bring Your Own Device, Grit and dare I say it Multi-Academy Trusts.  Indeed in this week's TES Dan Williams - @furtheredagogy - identifies a number of current teaching practices which appear not to have much in the way of research evidence to support claims for their effectiveness.  These include

• Teaching generic skills over subject knowledge
• Teaching in blocks
• Individualised/personalised learning
• Student control over learning
• Raising aspiration

Now to help us understand the processes associated with the life-cycles of educational innovations it is worth turning to the work of Slavin (1989).  Slavin  argues that educational innovation is famous for its cycle of early enthusiasm, widespread dissemination, subsequent disappointment, and eventual decline - the classic swing of the pendulum. (p752).  Slavin then goes onto describe the 12 phases of the swing of the pendulum which educational innovations inevitably move through.  

The upswing

• The programme is proposed
• The programme is piloted
• The programme is introduced into innovative schools or MATs
• The programme becomes a hot-topic amongst staff-developers
• The programme expands rapidly
• Controlled evaluations begin

The downswing

• Innovative schools move to other programmes
• Complaints surface in professional publications
• Preliminary evaluations are disappointing
• Developer claims that disappointing results are due to poor implementation.
• Interest in the programme flags
• Controlled evaluation studies are published - invariably with disappointing results

Implications for you and your role

In seeking to improve your effectiveness as an evidence-based practitioner - be it as a teacher, school research lead, head of department or senior leader,  there would appear to be a number of implication of Slavin's analysis.

1. You may want to take a moment to reflect on innovations recently introduced into your school, and where you would place the innovation on the pendulum.  In doing so, I'm not asking to judge where the innovation might be within your own school's 'pendulum of change' but rather where the innovation could be located within the system.   Let's take Lesson Study as an example.  It could be argued that Lesson Study is at the mid-point of swing of the pendulum - there has been rapid expansion in the use of Lesson Study - even though we don't know whether it works and benefits pupil outcomes -  and the EEF evaluations have just begun and we are now waiting for the results.
2. Having located where an innovation is located on the swing of the pendulum, it is probably worth reflecting on the decision-making process which led to the innovation being adopted.  Were colleagues swept up in a colleague's enthusiasm post a CPD event.  Does the school have a culture of having to be seen to be innovative and adopting the latest new programme or intervention?  Was it because the school has a specific problem and needed to be seen to taking action to address the issues
3. For any new innovations or programmes you are thinking of adopting - consideration should be given as to their current position on the pendulum, as this will given you some insight into the evidence available to support the decision where to adopt the innovation.  Slavin argues that before a programme or intervention is adopted a critical analysis of available evaluations should be undertaken where the following questions are asked:  Has the group using the program been compared to a comparable control group? Did the post-test assess objectives that were being pursued equally by the experimental and control group? Was the programme evaluated under realistic conditions and over realistic time periods?
4. For any long-standing educational practices which may have been adopted within your school or college, it may be worth taking a moment to reflect on what research evidence is currently available to support claims for the practices effectiveness.  For as Hargreaves and Fullan (2012) evidence can often become stale and no longer current

And a few final words

Given the significant challenges in trying to establish 'what works, for whom, to what extent, for how long, and why' there is an obligation to try and participate in high quality evaluations of programmes and innovations.  If everyone waits for the evidence to be produced then there will be no evidence to evaluate.  Indeed, by participating in high quality evaluations this should increase the school's ability to discriminate between fashionable programmes and interventions with real substance.

References

Hargreaves, A and Fullan, M. (2012) Professional Capital: Transforming, teaching in every school. Routledge, Abingdon

Slavin, R.E., 1989. PET and the pendulum: Faddism in education and how to stop it. Phi Delta Kappan, 70(10), pp.752-58.

Educational Excellence Everywhere and a curriculum for the development of the evidence-based practitioner

If you are a school-research lead, evidence-based teacher or head-teacher interested in the implications of the recent White Paper for creating an evidence-informed teaching professions, then this post is for you.  In this post I  will begin by summarising how the government is intending to support the creation of an evidence-informed teaching profession. I will then go onto to outline a curriculum to help develop trainee teachers, experienced teachers and headteachers develop their knowledge and skills as evidence-based practitioners.

Educational Excellence Everywhere and Evidence-Informed Teaching

The White Paper makes a number of specific references* to the development of evidence-informed teaching profession.  However, the main elements can be summarised as:

• ensuring ITT and subsequent CPD is evidence-based; 
• increasing teachers' access to high quality evidence; 
• increasing the volume of high quality evidence available which can directly impact teachers' practice in the classroom.

However, there is little or no description of the skills or knowledge required by teachers to become evidence-based practitioners.  My own view is that providing teachers with access to research evidence is a necessary, though not sufficient condition for the development of teachers as evidence-based practitioners.  For teachers to make the most of this access to research evidence, they also need to have developed a range of skills that allow them to incorporate research evidence - along with with others sources of evidence and critical judgement to make decisions which will hopefully benefit their pupils and colleagues.

A curriculum for the evidence-based teacher?

In developing a curriculum for the development of the evidence-based teachers a useful place to start is  Dawes, Martin, et al (2005) and the "Sicily statement on evidence-based practice." which provides a 'clear statement of what Evidence-Based Practice (EBP) means, a description of the skills required to practise in an evidence-based manner and a curriculum that outlines the minimum requirements for training health professionals in EBP." So using the 'Sicily statement' as a starting point the following is an amended version of the educational outcomes for health professionals proposed in the 'Sicily statement' and adapted for teachers and headteachers.   It could also be used as a basis for a curriculum for trainee teachers on programmes of ITT.  The proposed curriculum is summarised in Table 1

Table 1 A curriculum for evidence-based practitioners - based on the Sicily Statement

Translation of uncertainty into an answerable question	The teacher/headteacher identifies knowledge gaps during the course of practice and asks foreground questions to fill these gaps, The teacher/headteacher should ask focused questions that lead to effective search and appraisal strategies
Search for and retrieval of evidence	The teacher/headteacher can design and conduct a search strategy to answer questions. The strategy should be effective and comprehensive: likely to retrieve all relevant evidence - research evidence and school data. The teacher/headteacher understands the strengths and weaknesses of the different sources of evidence.
Critical appraisal of evidence for validity and practical importance	The teacher/headteacher can appraise the validity of a study. The appraisal will include: the suitability of the type of study to the type of question asked, the design of the study and sources of bias, the reliability of outcome measures chosen, and the suitability and robustness of the analysis employed. The teacher/headteachers can appraise the importance of the outcomes and translate them into practical and meaningful summary statistics.
Application of appraised evidence to practice	The teacher/headteacher can assess the relevance of the appraised evidence to the need that prompted the question. The teacher headteacher can explore the stakeholder  - pupils, parents, staff and the community - values  and the acceptability of the answer.
Evaluation of performance	The teacher/headteacher asks focussed questions, searches sources of evidence, appraises or uses pre-appraised evidence and applies these in practice. The teacher/headteacher reflects on how well these activities are performed

To provide some further clarity as to the practical implications of this curriculum of evidence-based practice, it is probably worth taking a moment to say what evidence based practice is NOT.  So let's be absolutely clear about evidence-based practice is NOT.

• Evidence-based practice is NOT the same as research-based practice - in other words, evidence-based practice is not about looking up Hattie's effect size tables and seeking to implement those teaching strategies with the highest effect size 

• Evidence-based practice is NOT about teachers undertaking research BUT it is about teachers making the use of the best available current evidence to make better decisions

• Evidence-based practice is NOT about teachers losing their autonomy to make pedagogical decisions about what is best for their pupils BUT it is about teachers using their professional expertise alongside other sources of evidence to make conscientious and judicious decisions which hopefully will benefit their pupils

To conclude
There is currently much interest in evidence-based practice.  However, it be wrong to assume that by merely giving teachers and headteachers access to high quality evidence this will turn them into effective evidence-based practitioners.  To be an effective evidence-based practitioner requires the development of a range of skills knowledge - particularly the ability to develop and articulate well formulated questions.  To help with this task much useful guidance can be gained from the health professions in developing a clear curriculum for  new teachers, experienced teachers and headteachers - as effective evidence-based practitioners.  

*Educational Excellence Everywhere and Evidence-Informed Practice

• ... we start from the basis that the country’s best school leaders know what works, and that good, enthusiastic leaders should be able to use their creativity, innovation, professional expertise and up-to-date evidence to drive up standards.

• Strengthen university and school-led training, increasing the rigour of ITT content with a greater focus on subject knowledge and evidence-based practice

• Replace Qualified Teacher Status(QTS) with a stronger, more challenging accreditation based on a teacher’s effectiveness in the classroom, as judged by great schools. This new accreditation will raise the quality and status of the teaching profession, better recognising advanced subject knowledge and pedagogy that is rooted in up-to-date evidence

• Support teachers  to develop their skills through evidence-based continuing professional development and introduce a new Standard for Teachers’ Professional Development to help schools improve the quality of CPD 

• We will support the development of a high status, world-leading teaching profession, by supporting the establishment of an independent College of Teaching, a new professional body along the lines of the Royal Medical Colleges 

• We will increase teachers’ access to and use of high quality evidence, ensure teachers are trained in understanding and applying evidence, and support the establishment of a new, peer-reviewed British education journal 

• We will continue to work in partnership with the Education Endowment Foundation to expand its role in improving and spreading the evidence on what works in education.

• We welcome moves to establish a portal for teachers to access education journals. We will also work with teachers to set up a bank of research questions which, updated annually, will focus funders of research and academics on generating evidence in areas which directly inform classroom teaching. 

• We will continue the pupil premium, and improve its effectiveness by encouraging schools and virtual school heads to adopt evidence-based strategies, drawing on evidence from the Education Endowment Foundation.

References

Dawes, M., Summerskill, W., Glasziou, P., Cartabellotta, A., Martin, J., Hopayian, K., Porzsolt, F., Burls, A. and Osborne, J., 2005. Sicily statement on evidence-based practice. BMC medical education, 5(1), p.1.

Department for Education (2016) Educational Excellence Everywhere. London CM 9230