The School Research Lead - Criticisms of Research and Evidence-Informed Practice

One of the tasks of an evidence-informed practitioner is to continually challenge the assumptions underpinning your work.  As during this year I have advocated the use of evidence-informed practice, it is right and proper to offer a critique of evidence-informed practice.  This task is made easier by Brown (2015) who identifies a number of areas of controversy or debate:

• the epistemological differences between academic researchers and policy-makers in terms of what counts as evidence, the quality of evidence and what evidence can or can't tell us;
• whether the evidence-informed movement serves to work agains the practitioners' professional judgement;
• issues in relations to how formal academic knowledge and professional or tacit knowledge might be effectively combined;
• differentials in power that can affect or limit interactions between teachers or policy-makers and research/ers;
• controversies in relation to to some of the methods commonly associated with enhancing evidence uses;
• how the capacity to engage with academic research might be enhanced;
• issues such as the inaccessibility of research to teachers and policy-makers, both in terms of where it is published and the language that is typically used in such publications. (adapted from p1)

Hargreaves and Fullan (2012) also provide a number of reasons why the case for the use of research-based evidence in decision-making can be overstated:

• evidence-based decisions can be tainted with self-interest;
• cast-iron evidence can get rusty later on;
• evidence-based principles are used very selectively;
• evidence isn't always self-evident;
• evidence on what to changes isn't the same as how to change;
• positive initiatives based on evidence in one area can inflict collateral damage;
• people can cook the data; 
• evidence-based teaching is only somewhat like evidence-based medicine;
• evidence comes from experience as well as research. (adapted from p47)

As an advocate of evidence-informed practice the temptation is to attempt to refute each of the objections or challenges.  However, I am not going to do this. I would rather you to take the opportunity to reflect on these challenges to evidence-informed practice and decide for yourself whether they are sufficiently robust to fatally undermine the case for the use of evidence-informed practice within schools and colleges.  Alternatively, can these issues be challenges been as issues to be addressed through well thought out counter-strategies

That said, I intend to end this post, and with it my last post of the academic year, with the words of Hargreaves and Fullan (2012)

So-called evidence can be unclear, ambiguous, compromised, out of date, indecipherable, contested or jus plain wrong.  This is not a reason to fall back on intuitions or personal preference as the sole based for teaching. We just need to be a bit more humble and careful about what we are claiming.  Teachers with professional capital are not driven by data or overly dependent on measurable evidence - but they inquire into, adapt the best ways for moving forward, making intelligent, critical and reflective use of measurable evidence and considered experience alike.  And they are committed to knowing and showing what impact they have on their students, and to fulfilling their responsibility for making this transparent to the public they serve. (p48)

References

Brown, C (2015) Evidence-Informed Policy and Practice in Education : A sociological grounding.  Bloomsbury. London

Hargreaves, A and Fullan, M. (2012) Professional Capital: Transforming, teaching in every school. Routledge, Abingdon

Developing Great Teachers - Lessons from the evidence

Recently we saw the publication of the Teacher Development Trust’s report Developing Great Teachers: Lessons from the international reviews into effective professional development.  In this post I will:

• Summarise the main findings of the report
• Re-classify the findings using six basic questions: “who” “what” “where” “when” “how” and “why”
• Discuss the report’s implications for School Research Leads

Developing Great Teachers – The Main Findings

The report’s main finding was that well designed professional development opportunities which focus on pupil on pupil can have a significant impact on pupil achievement.   Now in many ways this finding is not particularly surprising, given that you would not expect poorly designed professional development to have a positive impact on pupil learning and achievement.  As such, ‘the devil is in the details’ in both the design and implementation of CPD.  Table I provides a brief summary of these design elements and associated features.

Table 1 Design elements and associated features.

Design elements	Associated features
Duration	At least 2 terms if not at least 1 year
Rhythm	Process incorporates follow-up, consolidation and support activities within the school, and often repeats this cycle
Designing for participants needs	Relevance to teachers’ day to day working experience Opportunities for individual teachers both to reveal and discuss their beliefs and to engage in peer learning and support. Reflects different starting points of individual teachers
Alignment	Consistent with the school’s approach to student learning, although this may need further development
Sense of purpose	A shared sense of purpose is created
Content of effective professional development	Effective professional development should be underpinned by a number of “key building blocks”: Subject knowledge; Subject-specific pedagogy; Clarity around learner progression, starting points and next steps.; Content and activities dedicated to helping teachers understand how pupils learn, both generally and in specific subject areas. Programmes should also put forward: Alternative pedagogies for pupils with different needs.. A focus on formative assessment, to allow teachers to see the impact of their learning and work on their pupils. Input should allow for the consideration of participants’ existing theories, beliefs and practice, and for opportunities to challenge these in a non-threatening way which leads to positive developments
Activities associated with effective professional development	Discussions about the use of the CPD within classroom settings Experimentation Analysis of assessment data Discussions about embedding any learning into future practice
The role of external providers	Facilitate CPD Act as coaches and mentors
Specialists	Specialist support, should lead to successful outcomes. Successful facilitators employed activities that aim to: Introduce new knowledge and skills to participants.. Help participants access the theory and evidence underlying the relevant pedagogy, subject knowledge, strategies. Raise expectations. Link professional learning and pupil learning. Take account of different teachers’ starting points.  Support teachers through modelling, providing observation and feedback, and coaching.
Collaboration  and peer learning	What makes collaboration effective is still contested, for example, who should be involved and how much is necessary Reciprocal vulnerability through shared risk taking and engaging with evidence together about how pupils respond  to teacher learning are seem to be core components
Leadership around professional development	The review identified four core roles for school leaders in effective professional development. Developing vision Managing and organising Leading professional learning Developing the leadership of others

In other words, CPD can make a difference to pupils learning and pupil achievement but it has to be well designed and implemented.

Six Basic Questions - The who, what, why, where, when and how of effective CPD

Given the diverse nature of both schools and the teaching experience and expertise of teachers within these schools, it will be difficult to provide a generic answer to the SIX basic questions.  Nevertheless, I hope the following captures the main essence of the report and can be used as starting point for discussions within individual schools, groups of staff and with individual teachers.

Why – Because effective CPD makes a difference to both pupil learning and achievement.  Individual teachers have a career-long obligation to continuously improve his or her practice

Who – Everyone needs to be involved in some of well-designed and implemented CPD, from the NQT to the head-teacher.  Likely to involve some form of peer support. May, where appropriate, involve external specialists who have offer support consistent with best practice.  May not necessarily be the local HEI as needs to be alignment between the school’s needs and resource availability and the HEIs capacity and capability.

What –  A focus on subject-specific  knowledge, a range of pedagogic approaches  learner progression recognising the differing needs and starting points.  Provide opportunities for practitioners to review existing theories and beliefs.

When – Over a period of at least two terms if not a year.  Activities such as launch-events, consolidation, follow-up and evaluation need to be timed so they are consistent with the ebb and flow of the school year.   Attention to be given to the available school ‘energy’ levels and other competing priorities

How - May involve collaborative activity though only when there are clear and direct benefits, activities focussing on pupil needs and which reflect the different requirements of individual teaching staff, whilst at the same time aligned to be school’s approach to teaching and learning.  Specialist staff to provide both mentoring and coaching.   Focus on transferring learning to classroom through experimentation and assessment of data.

Where – if at all possible to be school-based, with a focus on what is going on in teachers’ classrooms

What are the implications for developing an evidence-informed school culture?

For me there would appear to be three implications for those school leaders and teachers wishing to promote evidence and research informed school practice.

First, patience – developing effective evidence-informed practitioners is going to take time, and is not a quick fix.  It’s going to take time to develop colleagues’ skills as evidence-informed practitioners and then it will take time for that feed into the hoped for improvements in pupil learning and outcomes. 

Second, energy levels – given that CPD is hard-work, not only will supporting activities have to be built into the school year, but those activities will need to take into account the ebb and flow of a school’s ‘energy’ levels both within a school year and between school years.  This to me suggests avoiding ‘big’ resource and energy hungry initiatives.  Focus on activities, which slowly build and embed evidence-informed into the day-to-day work of the school.

Third, avoid the collaboration trap – I have posted previously about the collaboration trap, though I think the message is clear.  Don’t feel compelled to build collaborative activities into your work – especially with other schools.  If you do engage in collaborative activity be very clear not only about the benefits collaboration brings you, but also the costs.

One final comment

Go and read the report Developing Great Teachers and then go on and critically analyse at least one of the cited pieces of research.  You’ll never know what you might learn or whom you might disagree with.

Leading the Use of Research and Evidence in Schools

So what does a school research lead do?  Over recent months I have blogged about the actions a school research lead can take to support colleagues become better evidence-informed practitioners.  However, I have yet to develop for a check-list of tasks for those colleagues leading the use of evidence and research within schools. To help fill this gap I have used Brown's (2015) concluding chapter in the excellent new book Leading the Use of Research & Evidence in Schools. to draw up a provisional check-list for school research leads.  Subsequently, I will identify a number of possible omissions from this check-list which may need to be included if it is to provide a comprehensive guide to action.

A Check-List for Leading the Use of Research and Evidence in Schools

Brown (2015) has produced a check-list for school leaders wishing to develop a research and evidence-informed school culture.  In this checklist Brown distinguishes between actions/factor which are 'transformational' in nature and enable research and evidence to be a core components of the work of the school.  These 'transformational' actions are contrasted with those which are pupil-centred and focus upon using research and evidence to improve teaching and learning, and are summarised in Table 1

Table 1 Leading Research and Evidence Use in Schools - Themes, factors and sub-actions (adapted from Brown, 2015)

Theme	Factors	Sub-actions
Transformational	Does your approach to research and evidence demonstrate your own commitment as well facilitate the efforts of others?	·        Promote a vision of research and evidence-informed school ·        Make resources available ·        Design and implement support structures ·        Create time and space for such work ·        Make it part of everyone’s work (especially leaders) ·        Model the use of research and evidence in decision-making ·        Develop an enquiry habit of mind – look for new perspectives ·       Seek out new information ·       Explore new ways to tackle old problems
	Does your approach to research and evidence use have buy-in throughout the school?	·        Adopt a distributive approach to leadership ·       Attend to the informal aspects of the school organisation ·       Identify and influence key-opinion formers and shapers ·       Seek to be consensual
Teaching and learning	Does your approach to research and evidence use ‘start with the end in mind’ and ensure that progress towards this end is tracked?	·       Articulate what success would look like ·        Consider what will need to be done differently ·        Question how things will be different for pupils and teachers ·        How will you know things are different ·        Evaluate impact of any changes ·        Engage in learning conversations – develop theories of action and develop and trial new actions ·        Constantly refine processes and actions ·        Stop doing some things
	Does your approach to research and evidence have teacher learning and practice at its core?	·       Continue to emphasise the importance of teacher-expertise ·       Use data to help teachers refine their practice ·        Create opportunities for collaborative learning both inside and outside of the school ·       Continually focus on evidence ·       Draw in external experience and knowledge/theory ·       Develop protocols and ways of working ·      Create facilitative arrangements
	Does your approach to research and evidence ensure that the right people are in the room	·       Develop middle leaders who are interested in evidence-informed practice ·       Identify research and evidence champions ·       Involve people with the right mix of skills to support the use of research and evidence

Some initial observations on Brown's checklist

When constructing a check-list, it is almost inevitable that some items may be left-out.  Items may be omitted from the check-list because they are taken for granted and are assumed to be in place.  Other items may be omitted because they reflect a different stance on the task at hand.   My own experience of check-lists suggest that it is better to have too much on a check-list than too little, so below are a few thoughts as to what could also be included.

• Does your approach to research and evidence ensure the use of an appropriate ethical framework?
• Are teachers explicitly aware of an appropriate ethical framework for the use in evidence-ifnromed practice?
• Do teachers give active consideration to the application of that ethical framework prior to any changes in practice arising from an investigation of the best current available evidence?
• Are ethical considerations explicitly noted in some form; be it a note on a lesson plan; referenced to within a scheme work; or recorded in some way, be it a reflective diary or log?
• Are the processes for the recording of ethical issues need to be both, proportional and fit for purpose?
• Does your approach to research and evidence demonstrate a clear understanding of the difference between research and evidence?
• Are teachers aware of the difference between evidence-informed practice and research and development?
• Is the priority of school research activity on disciplined inquiry and joint-practice over and above research and development?
• Is there a recognition that to role of research and evidence-informed activity is to help teachers improve rather than prove
• Does your approach to research and evidence build capacity and capability for future learning?
• Are teachers doing things right (single loop learning)?
• Are teachers doing the right things (double loop learning)?
• Are teachers participating  in making well-informed choices regarding strategy, objectives, etc. (e.g. triple loop learning)? George, Romme and Van Witteloostujin (1999)

Some concluding comments

Drawing up a check-list is never easy, particularly in an area where practice is a continued state of development and innovation. Furthermore, even if a comprehensive is check-list is created, it is the skill of the implementor which is paramount in whether the use of the check-list leads to success.  What matters is the extent to which the School Research Lead and other senior leaders display 'phronesis' or 'practical wisdom' in the promotion of a research and evidence-informed agenda within a school.  As such, the skilful school research lead will take the check-list and amend its use in a way which meets the specific needs and context of their school.

References

Brown, C. (2015)  Conclusion, in Brown C. (ed) Leading the Use of Research and Evidence in Schools, IOE Press, London
Georges L. Romme, A., and Arjen Van Witteloostuijn. "Circular organizing and triple loop learning." Journal of Organizational Change Management 12.5 (1999): 439-454.