The School Research Lead - Do we need to be vaccinated against the sharing of good practice virus?

The start of the academic year will see an outbreak of CPD and the sharing of good practice virus. Colleagues will spend hours,  if not days, being exposed to the self-espoused good practice of colleagues.  Unfortunately, like many viruses, the sharing of good practice 'virus' may do a great deal of harm, unless colleagues are vaccinated against infection from the well-meaning but unsubstantiated testimonials of colleagues.  Fortunately, a 'vaccine' has been developed by Daniel Willingham in his 2012 book - When Can You Trust the Experts : How to tell good science from bad in education.  Like all vaccines it can only provide the host with some protection against infection, as it will depend upon how the host responds to the vaccine.  However before we examine in more detail Willingham's vaccine, let's quickly remind ourselves of an activity we often see in a CPD and sharing of good practice day.

Sharing Good Practice - A Typical Activity

Some form of 'good-practice speed-dating' will no doubt form part of many schools start of term CPD events.  Colleagues will be asked to bring an example of what they perceive to be good practice and will be asked to explain how it works - often in less than two minutes -  to another colleague, with this process being repeated may be up to 20 - 30 times in a session. Unfortunately, there are a number of problems with this approach.  First, it gives insufficient time for colleagues to engage in an extended conversation about the 'good practice.  Willingham argues that such conversations are essential to ensure that the 'sharer of practice' fully understands his or her own practice and which is an important component of persuading others of the merit of a proposal.  However, the real problem with the speed-dating approach or others based upon testimonials is that it provides insufficient information to evaluate the practice being shared.  As Willingham states: You need not just stories of success from people who adopted the changes but also stories of failure.  You also need stories of success and failure from people who didn't adopt the change. (p 195)   The problem with testimonials in the sharing of good practice is illustrated in Figure 1

Figure 1 Information required to fairly evaluate a Change (Willingham, 2012 p 196)

	Things improved	Things didn’t improve
Adopted Change
Didn’t adopt Change (or adopted a placebo)

As such, Willingham argues that statements from individual teachers about his or her 'good-practice' only provides information from the top-left hand quadrant.  If we want to know whether some intervention or strategy has the potential to be good-practice than we will need information to allow to complete all four quadrants.

So what are we to do?

First it is essential for School Research Leads and CPD Co-ordinators to be aware of the design elements of effective professional development.  Fortunately, the Teacher Development Trust has recently provided us with a number of evidence-informed design elements, and about which I have written about previously.  Second, it is important that teachers talk about their practice in a 'disciplined' manner to determine whether espoused good practice is worth further discussion and possibly wider use within the school. Willingham suggests that when examining the potential of an educational change, strategy, or intervention teachers undertake a four-stage process.

1. Strip it - get rid of the fluff surrounding the idea/change/strategy/intervention and get to the heart of the actual claim.  What specific intervention, strategy or actions should the teacher adopt and what outcomes, say learning or achievement, are being promised.
2. Trace it - where did the idea come from.  Is the idea supported by a leading educational authority, as ironically in education this can be a weak indicator of validity or reliability 
3. Analyse it - what are you being asked to believe.  What is the evidence to support the claims being made?  How does this evidence relate to your own experience as a teacher?
4. Should I do it - is it something we already do.  Is it an old idea wrapped in new language? Has it failed previously in other settings with other students.  What are the opportunity costs 

'Flipping' Willingham to aid the sharing of practice 

 Willingham's four steps have been designed to help teachers discriminate between good and bad research evidence. Each of the four steps can be amended to help teachers structure the way they share practice with others.   In other words,  Willingham's four steps can be 'flipped' around for use by the teacher who is sharing the practice.  As such, if you intend to share practice with others the following elements needs to be covered.

• By doing X with my pupils there appears to be a Y percent chance that Z will happen
• Where the idea came from - is it supported by research evidence - if so, what are the criticisms of this research 
• State the 'class-based' evidence - what was measured, how was it measured, in comparison to what, how many pupils, how much did it help
• What were the opportunity costs associated with the change?  What negative-side effects were experienced? Who else was impacted upon?

Flipping the process around - with the steps being used by the sharer of practice has two main benefits.  It provides a structure for reflection by the teacher sharing the practice and which should facilitate more robust self-evaluation of practice before that practice is shared.   Second, it provides the 'audience' with a framework which allows them to 'test' through conversation how well the 'sharer'  understands his or her espoused good practice, which will in turn impact upon whether the sharees adopt the change.

In conclusion

As education can be a fashion-led profession and with fashions often based on so-called good practice it is essential that teachers can engage in the structured and disciplined evaluation of evidence.   Over the last 12 months as I have blogged about a evidence-informed practice,  attended events researchED events and followed twitter discussions, the need for teachers to be research literate is continually raised.  Unfortunately, there is very little discussion about what research literacy looks like in practice and how teachers should go about it.  With that in mind, if you are in the process of setting up a 'journal club' or are seeking a book to be read over the year, then Willingham's When Can You Trust the Experts : How to tell good science from bad in education is essential reading.  Future posts will help time-pressed School Research Lead by exploring Willingham's work in more detail, and expand upon each of the the four steps he recommends which will help you make significant progress to becoming a research literate and evidence-informed teacher.

Reference
Willingham, D. (2012) When Can You Trust The Experts: How to tell good science from bad in education, Jossey-Bass, San Francisco.

The School Research Lead - Criticisms of Research and Evidence-Informed Practice

One of the tasks of an evidence-informed practitioner is to continually challenge the assumptions underpinning your work.  As during this year I have advocated the use of evidence-informed practice, it is right and proper to offer a critique of evidence-informed practice.  This task is made easier by Brown (2015) who identifies a number of areas of controversy or debate:

• the epistemological differences between academic researchers and policy-makers in terms of what counts as evidence, the quality of evidence and what evidence can or can't tell us;
• whether the evidence-informed movement serves to work agains the practitioners' professional judgement;
• issues in relations to how formal academic knowledge and professional or tacit knowledge might be effectively combined;
• differentials in power that can affect or limit interactions between teachers or policy-makers and research/ers;
• controversies in relation to to some of the methods commonly associated with enhancing evidence uses;
• how the capacity to engage with academic research might be enhanced;
• issues such as the inaccessibility of research to teachers and policy-makers, both in terms of where it is published and the language that is typically used in such publications. (adapted from p1)

Hargreaves and Fullan (2012) also provide a number of reasons why the case for the use of research-based evidence in decision-making can be overstated:

• evidence-based decisions can be tainted with self-interest;
• cast-iron evidence can get rusty later on;
• evidence-based principles are used very selectively;
• evidence isn't always self-evident;
• evidence on what to changes isn't the same as how to change;
• positive initiatives based on evidence in one area can inflict collateral damage;
• people can cook the data; 
• evidence-based teaching is only somewhat like evidence-based medicine;
• evidence comes from experience as well as research. (adapted from p47)

As an advocate of evidence-informed practice the temptation is to attempt to refute each of the objections or challenges.  However, I am not going to do this. I would rather you to take the opportunity to reflect on these challenges to evidence-informed practice and decide for yourself whether they are sufficiently robust to fatally undermine the case for the use of evidence-informed practice within schools and colleges.  Alternatively, can these issues be challenges been as issues to be addressed through well thought out counter-strategies

That said, I intend to end this post, and with it my last post of the academic year, with the words of Hargreaves and Fullan (2012)

So-called evidence can be unclear, ambiguous, compromised, out of date, indecipherable, contested or jus plain wrong.  This is not a reason to fall back on intuitions or personal preference as the sole based for teaching. We just need to be a bit more humble and careful about what we are claiming.  Teachers with professional capital are not driven by data or overly dependent on measurable evidence - but they inquire into, adapt the best ways for moving forward, making intelligent, critical and reflective use of measurable evidence and considered experience alike.  And they are committed to knowing and showing what impact they have on their students, and to fulfilling their responsibility for making this transparent to the public they serve. (p48)

References

Brown, C (2015) Evidence-Informed Policy and Practice in Education : A sociological grounding.  Bloomsbury. London

Hargreaves, A and Fullan, M. (2012) Professional Capital: Transforming, teaching in every school. Routledge, Abingdon

Developing Great Teachers - Lessons from the evidence

Recently we saw the publication of the Teacher Development Trust’s report Developing Great Teachers: Lessons from the international reviews into effective professional development.  In this post I will:

• Summarise the main findings of the report
• Re-classify the findings using six basic questions: “who” “what” “where” “when” “how” and “why”
• Discuss the report’s implications for School Research Leads

Developing Great Teachers – The Main Findings

The report’s main finding was that well designed professional development opportunities which focus on pupil on pupil can have a significant impact on pupil achievement.   Now in many ways this finding is not particularly surprising, given that you would not expect poorly designed professional development to have a positive impact on pupil learning and achievement.  As such, ‘the devil is in the details’ in both the design and implementation of CPD.  Table I provides a brief summary of these design elements and associated features.

Table 1 Design elements and associated features.

Design elements	Associated features
Duration	At least 2 terms if not at least 1 year
Rhythm	Process incorporates follow-up, consolidation and support activities within the school, and often repeats this cycle
Designing for participants needs	Relevance to teachers’ day to day working experience Opportunities for individual teachers both to reveal and discuss their beliefs and to engage in peer learning and support. Reflects different starting points of individual teachers
Alignment	Consistent with the school’s approach to student learning, although this may need further development
Sense of purpose	A shared sense of purpose is created
Content of effective professional development	Effective professional development should be underpinned by a number of “key building blocks”: Subject knowledge; Subject-specific pedagogy; Clarity around learner progression, starting points and next steps.; Content and activities dedicated to helping teachers understand how pupils learn, both generally and in specific subject areas. Programmes should also put forward: Alternative pedagogies for pupils with different needs.. A focus on formative assessment, to allow teachers to see the impact of their learning and work on their pupils. Input should allow for the consideration of participants’ existing theories, beliefs and practice, and for opportunities to challenge these in a non-threatening way which leads to positive developments
Activities associated with effective professional development	Discussions about the use of the CPD within classroom settings Experimentation Analysis of assessment data Discussions about embedding any learning into future practice
The role of external providers	Facilitate CPD Act as coaches and mentors
Specialists	Specialist support, should lead to successful outcomes. Successful facilitators employed activities that aim to: Introduce new knowledge and skills to participants.. Help participants access the theory and evidence underlying the relevant pedagogy, subject knowledge, strategies. Raise expectations. Link professional learning and pupil learning. Take account of different teachers’ starting points.  Support teachers through modelling, providing observation and feedback, and coaching.
Collaboration  and peer learning	What makes collaboration effective is still contested, for example, who should be involved and how much is necessary Reciprocal vulnerability through shared risk taking and engaging with evidence together about how pupils respond  to teacher learning are seem to be core components
Leadership around professional development	The review identified four core roles for school leaders in effective professional development. Developing vision Managing and organising Leading professional learning Developing the leadership of others

In other words, CPD can make a difference to pupils learning and pupil achievement but it has to be well designed and implemented.

Six Basic Questions - The who, what, why, where, when and how of effective CPD

Given the diverse nature of both schools and the teaching experience and expertise of teachers within these schools, it will be difficult to provide a generic answer to the SIX basic questions.  Nevertheless, I hope the following captures the main essence of the report and can be used as starting point for discussions within individual schools, groups of staff and with individual teachers.

Why – Because effective CPD makes a difference to both pupil learning and achievement.  Individual teachers have a career-long obligation to continuously improve his or her practice

Who – Everyone needs to be involved in some of well-designed and implemented CPD, from the NQT to the head-teacher.  Likely to involve some form of peer support. May, where appropriate, involve external specialists who have offer support consistent with best practice.  May not necessarily be the local HEI as needs to be alignment between the school’s needs and resource availability and the HEIs capacity and capability.

What –  A focus on subject-specific  knowledge, a range of pedagogic approaches  learner progression recognising the differing needs and starting points.  Provide opportunities for practitioners to review existing theories and beliefs.

When – Over a period of at least two terms if not a year.  Activities such as launch-events, consolidation, follow-up and evaluation need to be timed so they are consistent with the ebb and flow of the school year.   Attention to be given to the available school ‘energy’ levels and other competing priorities

How - May involve collaborative activity though only when there are clear and direct benefits, activities focussing on pupil needs and which reflect the different requirements of individual teaching staff, whilst at the same time aligned to be school’s approach to teaching and learning.  Specialist staff to provide both mentoring and coaching.   Focus on transferring learning to classroom through experimentation and assessment of data.

Where – if at all possible to be school-based, with a focus on what is going on in teachers’ classrooms

What are the implications for developing an evidence-informed school culture?

For me there would appear to be three implications for those school leaders and teachers wishing to promote evidence and research informed school practice.

First, patience – developing effective evidence-informed practitioners is going to take time, and is not a quick fix.  It’s going to take time to develop colleagues’ skills as evidence-informed practitioners and then it will take time for that feed into the hoped for improvements in pupil learning and outcomes. 

Second, energy levels – given that CPD is hard-work, not only will supporting activities have to be built into the school year, but those activities will need to take into account the ebb and flow of a school’s ‘energy’ levels both within a school year and between school years.  This to me suggests avoiding ‘big’ resource and energy hungry initiatives.  Focus on activities, which slowly build and embed evidence-informed into the day-to-day work of the school.

Third, avoid the collaboration trap – I have posted previously about the collaboration trap, though I think the message is clear.  Don’t feel compelled to build collaborative activities into your work – especially with other schools.  If you do engage in collaborative activity be very clear not only about the benefits collaboration brings you, but also the costs.

One final comment

Go and read the report Developing Great Teachers and then go on and critically analyse at least one of the cited pieces of research.  You’ll never know what you might learn or whom you might disagree with.

Leading the Use of Research and Evidence in Schools

So what does a school research lead do?  Over recent months I have blogged about the actions a school research lead can take to support colleagues become better evidence-informed practitioners.  However, I have yet to develop for a check-list of tasks for those colleagues leading the use of evidence and research within schools. To help fill this gap I have used Brown's (2015) concluding chapter in the excellent new book Leading the Use of Research & Evidence in Schools. to draw up a provisional check-list for school research leads.  Subsequently, I will identify a number of possible omissions from this check-list which may need to be included if it is to provide a comprehensive guide to action.

A Check-List for Leading the Use of Research and Evidence in Schools

Brown (2015) has produced a check-list for school leaders wishing to develop a research and evidence-informed school culture.  In this checklist Brown distinguishes between actions/factor which are 'transformational' in nature and enable research and evidence to be a core components of the work of the school.  These 'transformational' actions are contrasted with those which are pupil-centred and focus upon using research and evidence to improve teaching and learning, and are summarised in Table 1

Table 1 Leading Research and Evidence Use in Schools - Themes, factors and sub-actions (adapted from Brown, 2015)

Theme	Factors	Sub-actions
Transformational	Does your approach to research and evidence demonstrate your own commitment as well facilitate the efforts of others?	·        Promote a vision of research and evidence-informed school ·        Make resources available ·        Design and implement support structures ·        Create time and space for such work ·        Make it part of everyone’s work (especially leaders) ·        Model the use of research and evidence in decision-making ·        Develop an enquiry habit of mind – look for new perspectives ·       Seek out new information ·       Explore new ways to tackle old problems
	Does your approach to research and evidence use have buy-in throughout the school?	·        Adopt a distributive approach to leadership ·       Attend to the informal aspects of the school organisation ·       Identify and influence key-opinion formers and shapers ·       Seek to be consensual
Teaching and learning	Does your approach to research and evidence use ‘start with the end in mind’ and ensure that progress towards this end is tracked?	·       Articulate what success would look like ·        Consider what will need to be done differently ·        Question how things will be different for pupils and teachers ·        How will you know things are different ·        Evaluate impact of any changes ·        Engage in learning conversations – develop theories of action and develop and trial new actions ·        Constantly refine processes and actions ·        Stop doing some things
	Does your approach to research and evidence have teacher learning and practice at its core?	·       Continue to emphasise the importance of teacher-expertise ·       Use data to help teachers refine their practice ·        Create opportunities for collaborative learning both inside and outside of the school ·       Continually focus on evidence ·       Draw in external experience and knowledge/theory ·       Develop protocols and ways of working ·      Create facilitative arrangements
	Does your approach to research and evidence ensure that the right people are in the room	·       Develop middle leaders who are interested in evidence-informed practice ·       Identify research and evidence champions ·       Involve people with the right mix of skills to support the use of research and evidence

Some initial observations on Brown's checklist

When constructing a check-list, it is almost inevitable that some items may be left-out.  Items may be omitted from the check-list because they are taken for granted and are assumed to be in place.  Other items may be omitted because they reflect a different stance on the task at hand.   My own experience of check-lists suggest that it is better to have too much on a check-list than too little, so below are a few thoughts as to what could also be included.

• Does your approach to research and evidence ensure the use of an appropriate ethical framework?
• Are teachers explicitly aware of an appropriate ethical framework for the use in evidence-ifnromed practice?
• Do teachers give active consideration to the application of that ethical framework prior to any changes in practice arising from an investigation of the best current available evidence?
• Are ethical considerations explicitly noted in some form; be it a note on a lesson plan; referenced to within a scheme work; or recorded in some way, be it a reflective diary or log?
• Are the processes for the recording of ethical issues need to be both, proportional and fit for purpose?
• Does your approach to research and evidence demonstrate a clear understanding of the difference between research and evidence?
• Are teachers aware of the difference between evidence-informed practice and research and development?
• Is the priority of school research activity on disciplined inquiry and joint-practice over and above research and development?
• Is there a recognition that to role of research and evidence-informed activity is to help teachers improve rather than prove
• Does your approach to research and evidence build capacity and capability for future learning?
• Are teachers doing things right (single loop learning)?
• Are teachers doing the right things (double loop learning)?
• Are teachers participating  in making well-informed choices regarding strategy, objectives, etc. (e.g. triple loop learning)? George, Romme and Van Witteloostujin (1999)

Some concluding comments

Drawing up a check-list is never easy, particularly in an area where practice is a continued state of development and innovation. Furthermore, even if a comprehensive is check-list is created, it is the skill of the implementor which is paramount in whether the use of the check-list leads to success.  What matters is the extent to which the School Research Lead and other senior leaders display 'phronesis' or 'practical wisdom' in the promotion of a research and evidence-informed agenda within a school.  As such, the skilful school research lead will take the check-list and amend its use in a way which meets the specific needs and context of their school.

References

Brown, C. (2015)  Conclusion, in Brown C. (ed) Leading the Use of Research and Evidence in Schools, IOE Press, London
Georges L. Romme, A., and Arjen Van Witteloostuijn. "Circular organizing and triple loop learning." Journal of Organizational Change Management 12.5 (1999): 439-454.