Summer reading for evidence-based practitioners and school research leads

Here are five suggestions for summer reading for school research champions and school leaders who are interested in improving their evidence-based practice.   The books are quite deliberately drawn from a number of disciplines including : improvement science, design-thinking, adult learning and evidence-based practice - so as to show that ideas for the improvement of evidence-based practice within schools can be found from a range of sources.  Indeed, they highlight the need for writers,  scholars and practitioners of/on evidence-based educational practice to look outside the field of education, as many of the problems we are grappling with have been addressed in other areas.  That does not mean we have to slavishly follow what is put forward, but rather provides us with the opportunity to learn from others. Enjoy!

BRYK, A. S., GOMEZ, L. M., GRUNOW, A. & LEMAHIEU, P. G. 2015. Learning to improve: How America's schools can get better at getting better.

Taking ideas from the field of improvement science, Learning to Improve, identifies six broad principles of improvement.  It then goes onto shows how a process of disciplined inquiry can be used  with networks to identify, adapt, and successfully scale up promising interventions in education.   In doing so, it provides an incredibly useful description of 'disciplined inquiry' i.e. it is informed by three simple questions. What specifically are we trying to accomplish? What change might we introduce and why?  How will we know that change is actually an improvement?

KEGAN, R. & LAHEY, L. L. 2016. An Everyone Culture: Becoming a Deliberately Developmental Organization, Harvard Business Review Press.

The authors look at the simple but radical idea that organisations prosper when they are deeply aligned with peoples' strongest motive i.e to grow and show how a developmental culture can be developed in organisations.  This fascinating book also explores why many individuals are immune to change, and provides an analytical model to help understand others' theories of action.

LANGLEY, G. J., MOEN, R., NOLAN, K. M., NOLAN, T. W., NORMAN, C. L. & PROVOST, L. P. 2009. The improvement guide: a practical approach to enhancing organizational performance, John Wiley & Sons.

Now in its' second edition, The Improvement Guide, offers an integrated approach to learning and improvement. Using simple examples to illustrate core ideas, the authors develop a flexible model for improving quality and productivity in diverse settings.  In doing to they draw upon research conducted in a variety of areas manufacturing, healthcare, government, and schools.   This is a must read for School Research Leads looking for ways to both implement and evaluate changes within their schools.

           

                                                                

                                                                 

            MINTROP, R. 2016. Design-Based School Improvement

            A Practical Guide for Education Leaders, Cambridge, Harvard Education Press

Drawing on work on design thinking by Tim Brown, Tom Kelley and David Kelley - this book provides a step by step guide to the design development process and how it can be applied in schools.  In doing so, it integrates both theory and practice through the use of case studies.

ROUSSEAU, D. M. 2012. The Oxford handbook of evidence-based management, Oxford University Press.

The Oxford Handbook of Evidence-based Management (EBMgt)  provides an overview of key Ebmgt ideas and puts them in context of promoting evidence-based practice. Divided into three sections (research, practice, and education), this handbook examines the realities of everyday management practice and the role Ebmgt can play in improving managerial decision making and employee well being, and is an area largely ignored by writers on evidence-based education.  Indeed, it handbook highlights how relatively underdeveloped our thinking is about evidence-based education.

School Research Leads and the Use of Theories of Action to Bring about Change

If you are an evidence-based practitioner, school research champion or school senior leader and interested in how to ‘overcome’ the resistance to the use of evidence within your school, then this post is for you.   Indeed, this post may be especially relevant to those 400 plus delegates  #researchED York who are returning to their schools and positively enthused about how research can improve teaching, learning and pupil outcomes.  Drawing upon the work Robinson (2011) this post articulate two quite different theories of action about how research evidence can impact upon teaching and learning.  I will then go onto explore two different strategies which can be used to address (or not) those differences.  Finally, I will consider the implications of this analysis and discussion for your role as school research lead.  But first, let's unpack what is meant by the term 'theory of action.'

A theory of action

Robinson – drawing upon the work of Argyris and Schon – defines a theory of action as: the values and beliefs that explain people’s actions, together with the consequences of those actions. When we understand a person’s theory of action we understand why he behaved as he did and we can work with him to evaluate whether or not the theory in his action matched his intentions(p115)

The following figure - which amends a similar figure in Robinson’s work (Fig 1 p117)  –  seeks to outline two competing theories in action about the use of research evidence and the impact on pupils’ outcomes.  However, it is important to note that these are not the only theories of action which could have been developed.  As such, they have been developed to help understand the differences in competing theories of action.

Competing theories about the use of research evidence and pupil outcomes

	School Research Champion/Leader	Teachers
Beliefs and values	Use of research evidence will improve the quality of teaching Teachers are not using research evidence because of a lack of awareness and understanding	Research evidence is not relevant to our teaching
Actions	Plan and lead  staff meetings on how to understand and use research for lesson planning and the development of schemes of work	Attend staff meetings Do not use research for purposes of lesson planning and development of schemes of work
Consequences	Teachers will use research for lesson planning Pupil outcomes will improve	There is no change in teaching practice Pupil outcomes remain unchanged

As such teachers’ resistance to use of research evidence can be seen as a product of two competing theories about the relevance of research evidence to teaching and pupil learning.  The theory of the school research champion explains his model of the implementation of change and associated expectation of outcomes.  Whereas, the teachers’ theory of action explains why they continue not to use research evidence in lesson planning.   The importance of this is that Robinson argues - once we understand someone’s theory of action, we can begin to understand the reasons for the actions they are taking and create the conditions for productive dialogue and discussion.

Contrasting approaches to leading change

Robinson argues that when leading change there are two contrasting approaches.  In the aforementioned example the first approach involves the School Research Champion bypassing the teachers’ theory of action, and focussing on the actions they want to change.  As Robinson states: In short, they don’t ask enough questions of teachers about why they are doing what they want them to stop doing (p119).  In other words, no attempt is made to try and understand the underpinning theory of action held by teachers.  This strategy of bypassing the teachers’ theory of action is in all likelihood is going to lead to teachers continuing not to use research evidence in lesson planning, as it does not explore the underpinning reasons teachers choose not to use research evidence.

The second strategy, involves the School Research Champion/School Leaders seeking to engage and understand the teachers’ theory of action.  Summarising Robinson they listen to what teachers have to say about the relationship between research and practice. They challenge teachers when they disagree.  They summarise and show demonstrate understanding of the teachers’ theory of action.  They check whether they can agree on areas of difference between the different theories of action.  They collaborate with teachers to explore the consequences of these difference.   They keep talking and until there is agreement about a new theory of action, or agree that now new theory of action is required.   By engaging in this dialogue school leaders will gain a real sense of the implicationa of differing theories of action, and begin to understand what may or may not be possible

Implications for you and your role

For me there would appear to be several implications:

• School research champions and senior leaders should articulate their own theories in action about how research can influence teaching and pupil outcomes.
• School research champions and senior leaders wishing to bring about the increase use of research evidence within their schools, could be well-advised to start with researching their colleagues’ theories of action relating to research evidence.   
• School research champions and senior leaders will need to develop their active listening skills to ensure they are properly engaging with colleagues’ theories of action.  
• The key resource in this process may well be time – ensuring that colleagues have the time necessary to undertake a genuine process of engagement with their colleagues. 
• A recognition that to bring about lasting and real changes in colleagues’ practice will require persistence and patience.  With the important factor being the the ability to keep the conversation going (Willingham, 2015)

Some final words

Daniel Dennett in his 2013 book Intuition Pumps And Other Tools for Thinking suggests that at times of disagreement we should use Rapoport’s Rules:

1. You should attempt to re-express your target’s position so clearly, vividly, and fairly that your target says, “Thanks, I wish I’d thought of putting it that way.”

2. You should list any points of agreements (especially is they are not matters of general and widespread agreement).

3. You should mention anything you have learned from your target.

4. Only then are you permitted to say so much as a word of rebuttal or criticism

Further reading

Dennett, D.C., 2013. Intuition pumps and other tools for thinking. WW Norton & Company.

Robinson, V., 2011. Student-centered leadership (Vol. 15). John Wiley & Sons.

Stone, D., Patton, B. and Heen, S., 2010. Difficult conversations: How to discuss what matters most. Penguin.

Updated - When the evidence is not all that it seems and the case of the 0.84 effect size for leading teaching learning and development

As we approach the end of term, we enter the season of INSET days and a tsunami of PowerPoint presentations by consultants and other experts.  In these presentations we often see the same research referenced and used to justify a particular position or course of action, though without reference to the research's limitations  Indeed, one of the most frequently recycled set of findings is Robinson et al's (2009) research on the impact of different dimensions of leadership on student outcomes, and as illustrated in the following figure (Robinson et al 2009, p39)

Robinson (2011) states this figure illustrates that : The most powerful way that school leaders can make a difference to the learning of their students is by promoting and participating in the professional learning and development of their teachers.  The average impact (0.84) of these leadership practices on student outcomes was twice that of any other leadership dimension (p104)

With headteachers in high achieving schools headteachers and principals more likely to :

• participate actively in teacher learning and development than their colleagues in low-achieveing schools;
• promote and participate in staff discussions on teaching and learning;
• to be seen as a source of instructional advice.

Limitations

Unfortunately, there are a number of limitations to these findings and which are not widely reported.  So here goes:

• The effect size for the 'leading teacher learning and development' dimension was generated from only six of the twelve studies used to calculate the effect sizes elsewhere in the study
• In these six studies the vast majority of the evidence came from primary/elementary schools (Robinson et al 2009)
• None of the studies used in the calculation of the effect size came from the United Kingdom, with the majority of schools being based in the United States
• It would appear that the most recent study used in the calculation of the 0.84 effect size came from 1994 
• The findings do not explain why the dimensions is associated with higher achievement or describe the skills and knowledge needed to be an effective leaders of professional learning (Robinson, 2011)

Robinson (2011) states that when asked - school leaders talk about the symbolic importance of leaders involvement in professional learning. Yet Robinson argues that there are other reasons; first, leaders who collaborate and work with their staff are better placed to engage in professional discussions with teachers; second, headteachers' endorsements of external support/consultants are likely to be more credible as teachers know that their leaders are experiencing what they are endorsing.  However, Robinson surmises that perhaps the most important reason for effect is that the involvement of leaders in teacher professional learning make them more aware of the problems generated by the learning and what is necessary for success, which allows the headteacher to ensure the resources are in place for success.

So what are the implications of this analysis?

At first blush, there would appear to be several implications about what actions school leaders should take to improve student outcomes

• It would appear that we don't know what works in secondary schools, particularly in UK secondary schools.
• Given the importance that Robinson that places are leaders making resources available to support professional learning, it maybe that in secondary schools that resourcing strategically is the dimension which has the most impact.
• Indeed, it may be that within elementary/primary schools strategic resourcing is the key factor but it's impact has manifested itself in the 'leading teacher learning and development'.  So possibly there is a misattribution in the original analysis.

However, despite these reservations Robinson et al's research can still provide some guidance for school leaders.

1. If you are a primary school leader - leading and participating in teacher learning and development - may be your best bet in improving pupil outcomes .  
2. If you are a secondary school leader - you may wish to focus on the resourcing of teaching learning and development and well as trying to create a school culture which explicitly supports such activity
3. If you are secondary school leader - the nuts and bolts of leading teacher learning and development will rest with other colleagues - be it Deputy Heads and Heads of Department - and it will be important to appoint individuals into such positions who have the capacity and capability to undertake the leading of teaching learning and development.

Note - since I originally wrote this post I have been in contact with Professor Robinson, who has provided me with some clarification.  As Professor Robinson states:

These findings are NOT about what heads should be doing. They are about the impact  of different types of leadership practice on student outcomes and I state that, while  half of the studies used teacher ratings of principals, the other half were rating  school-wide leadership. There were not enough studies to do separate  meta-analyses. So they should be interpreted as indicative of the impact of type of leadership practice not of the impact of principalship. (email correspondence, 17 August, 2016)

And finally

If you are at a session where a consultant or external expert uses the above figure in one of their PowerPoint presentations, please remember to ask them the following: 

• How many studies were included in Robinson et al's research ?(27).  
• How many studies were used to calculate the effect sizes ? (12).  
• How many studies were used in calculating the 0.84 effect size? (6) 
• How many studies included data from secondary schools? (the minority as not possible to be specific)
• How many studies were conducted in the UK? (none)
• How many studies were conducted in the 21st century? (none)

References

Robinson, V.M., Hohepa, M. and Lloyd, C., 2009. School leadership and student outcomes: Identifying what works and why ; Best Evidence Synthesis, New Zealand Ministry of Education

Robinson, V., 2011. Student-centered leadership (Vol. 15). John Wiley & Sons.