The Biology of School Survival

Last weekend I had the huge privilege of attending the Canons Park Teaching School Alliance conference.  Keven Bartle and Emlyn Lumley gave a fascinating presentation on 'Chaos, complexity and ecological system leaders' and how schools are going to have to respond to an environment of increasing complexity with schools being complex adaptive systems (CAS).   In this post, I will draw upon Reeves, Leving & Ueda's recent article in the Harvard Business Review  (Jan-Feb 2016) to examine in more detail; first, what is meant by a complex adaptive system; second,  how school leaders can attempt to increase resilience within a complex adaptive system; third, some of the limitations of Reeves et al's model when applied to a school-setting.

What is a complex adaptive system?

Reeves at al state that like biological species, companies are 'complex adaptive systems' that continually evolve in hard-to-predict ways.  Local interactions cascade and reshape the entire system; the new structure then influences individual agents - people - resulting in further changes in the system.

They go onto argue that these systems ... are often nested in broader systems.  A populations is a CAS nested in a natural eco-sytems, which itself is nested in a broader biological environment.  So a school is a CAS which is nested within the overall school ecosystem, the education ecosystem, as well as the broader societal system.  As such, not only is there complexity within a school's boundaries, there is tensions between an individual schools and the wider school system-wide ecosystem. As such,what may be good for an individual school may not be good for the broader ecosystem of schools.

Reeves et al argue that there are three broad implications for business leaders, which I will adapt for a school setting.

1. School leaders need to be realistic about what they can predict and control - both within the school and outside of the school. 
2. School leaders need to look beyond their own schools and ensure that their schools contribute positively to the school system as a whole, particularly if they are receiving benefits from the system as a whole.
3. School leaders need to embrace the inconvenient truth that attempts to staff may have negative unintended consequences, which may threaten the school itself.

How can school leaders increase resilience in schools within a complex adaptive system?

Reeves et al identify six principles which can be applied, which may increase the resilience of an organisation (school).  However, they make clear these principles may be in tension with one another and that prioritising one principle may lead to the de-proritisation of another.  As such, Reeves et al argue that the six principles should be seen as a set, which are applied collectively rather than seeing each principle as a standalone strategy or tactic.  With that in mind, we will now examine each of these six principles.

Maintain heterogeneity

School leaders need to ensure their schools are diverse only three axes : people, ideas and endeavours. One way of doing is to hire people from a range of backgrounds, educational qualifications and personality,  and which may be inevitable given the current recruitment crisis.  However, recruiting a diverse range of  individuals, is not enough.  Those individuals need to be given the permission to express their opinions, develop new initiatives and sometimes, to fail.  A safety-first approach - trying to play 'error free football' - may ironically lead to the school becoming even more fragile.

Modularity

In the past there has been much talk about whole-school and integrated approaches. However, it may be necessary to design a school, multi-academy trust or academy chain as a modular system, which will prevent shocks from moving from one part of the system to another.  Yet one of the problems with a modular system is that it potentially gets in the way of networking and collaborative activity.  It may be tempting to increase levels of integration for purposes of integrations, but in increasing interdependence it also leads to an increase in risk.

Redundancy

In systems with redundancy, different aspects of the system often have overlapping roles.  So if one part of the system fails, another part of the system can pick up the slack and carry-out the role.  In the current environment, built in redundancy could be seen as a sign of bloated and inefficient school or organisation.  However, schools are always experiencing shocks - be it new unexpected intitiatives, unanticipated staff turnover or absence -  and there needs to be capacity within the school or MAT to be able to respond to these shocks.

Expect surprise, but reduce uncertainty

In complex adaptive system it is not possible to predict the future shape of the organisation/school.  However, it is possible to spot 'signals', for example, changing patterns of enrolments with feeder schools, changing patterns of pupil applications, changing numbers applications for posts to responsibility - and begin to imagine possible scenarios, and in doing so identify the action to be taken to maximise opportunity and minimise risk.  Reeves et al argue that there are a number of practices which could be adopted:

• Schools may need to accept that a currently successful business model, will at some stage be superseded by some alternative.  Indeed, ways of working that may have been successful in delivering 5 GCSES (inc E&M) at C and above, may not work in the world of the EBacc
• Understand that change comes from the periphery - with incomers having no choice but to do something different to the current dominant players - this can be seen in the way that 'free schools' operate
• Identify the signals from the activities of those organisations and schools who are in effect betting against your continued success - in other words, how well are the scrappy insurgents doing compared to your more established way of working
• Practice contingent thinking - if 'fair funding" becomes a reality - what are you going to do about it. How will it impact upon your way of working and school structures.
• Finally, if possible,  reduce  the threat or idea, by acquiring it or building defences against it.

Now given the context of schools - they key things to do would appear to be to try and spot the signals and engage in contingent thinking

Create feedback loops and adaptive mechanisms

Often the signals that school-leaders may need to be aware that what may be 'hitting' the school, is doing so at some 'organisational' distance from the senior leadership team.  With that in mind - school senior leaders need to engage in real and genuine dialogue with front-line staff - be it school administrators, caretakers, teaching assistants, teachers and others - to keep in in touch with what is happening on the ground.  Second, it's not enough just to get the feedback, it's necessary to act upon it. Reeves talks about out organisations (and schools) having to be ambidextrous - i.e the ability to run the existing school, whilst at the same time reinventing the school in response to the new environment.

Foster trust and reciprocity

Complex adaptive systems require co-operation if they are to survive.  The interests of different schools with a MAT or TSA  or staff who work in them may at times be in conflict.  However, if schools and individuals purse their own interests - then then system as a whole becomes weaker.  Incentives need to be put in place to foster trust and reciprocity.  One way might be ensure that promotion opportunities are given to colleagues who an can not only lead their functional area but contribute to the good of the system as a whole.  The flip side of this - is no matter how 'good' a HOD maybe in leading their department - if this is at the cost of the school as whole, other actions may need to be taken.

Limitations of the model

Keven Bartle kindly reviewed this post before publication and suggested that Reeves et al's model has a number of limitations within a school setting.  Keven argues : One of the inherent tensions in the Reeves approach (and of leveraging complexity in a school environment generally) is that it rather assumes that the leader can see the whole leadership landscape of the school ecology from the outside, as some kind of impartial objective observer and coordinate the very different dynamics to organisational effect. In doing so there is almost the assumption that leadership is a fixed property in charge of the non-fixed properties of agency and interaction and so forth. In doing so it possibly risks reifying the notion of heroic leadership even as it suggests the opposite is true elsewhere in the organisational architecture. 

Perhaps this is inevitable in hierarchical organisations, or maybe there's something that needs to underpin this about the need to de-leader oneself that comes before the "headteacher needs to ensure THEIR schools" bit of the post. It may be fraught with inherent contradictions but such is the nature of chaos and complexity. The key bit before anything else is accepting paradoxes (including the unleading leader). If I follow this logic all the way then it leads to a position that is utterly cooperative in which "OUR school" replaces "MY school" in every element of the hierarchical leadership domains, recruitment (people), professional development (ideas) and strategic planning (actions).  (private email correspondence)

And finally ....

If we accept the notion of the school being a complex adaptive system, being nested in a range of other systems, then this probably has a number of implications for headteachers and senior teams in how they go about their work leading their schools.  Reeves et al may not have all the answers or even the beginning of the questions, but they do provide at the very least is both a framework and agenda for discussion, both within and between schools.

New - Distinguishing the practical from the puerile - a simple 2x2 model

If you are school research lead or someone interested in evidence-informed practice and want to be able to identify research with practical relevance, this post is for you.  Using the work of Anderson, Herriot & Hodgkinson (2001) I will describe a simple 2 x 2 model, which uses the dimensions of relevance and rigour, to help you distinguish between - popularist, puerile, pedantic and pragmatic - research.  I will then consider some of the implications for you, either as a school research leads and/or evidence-informed practitioner.

What do we mean by both practical relevance and methodological rigour?

As Anderson et al state, defining precisely what is meant by these terms in not un-problemmatic.  When thinking of relevance, are multitude of questions to ask, for example, for whom is the research relevant for - TAS, NQTS, experienced teachers, HODs, SLTs and headteachers.  Is the research relevant for pupils, parents or other stakeholders within the school.   However, for our purposes, research can be deemed relevant if, in your view, it can directly improve how and what you do in your current role.  As for methodological rigour, well that will all depend upon your view ontological and epistemological standpoint. and I'm certainly not qualified to summarise the debate within a short-blog spot.  That said, regardless of your epistemological or ontological standpoint, the research needs to be a form of 'disciplined inquiry.'  For as Cronbach and Suppes (1969) state:  Disciplined inquiry has a quality that distinguishes it from other sources of opinion and belief. The disciplined inquiry is conducted and reported in such a way that the argument can be painstakingly examined. The report does not depend for its appeal on the eloquence of the writer or on any surface plausibility, (p. 15).

With that in mind, Hood (2003) usefully identifies a number of the characteristics of disciplined inquiry, which include:

• Meaningful topics are addressed
• Systematic, clearly described procedures are employed and described so that readers can follow the logic of the study and assess the validity of the study's conclusion
• There is sensitivity to the errors that are associated with the methods employed and efforts are made to control the errors or consider how they influence the results
• Empirical verification and sound logic are valued: and
• Plausible alternative explanations are considered (p2)

In other words, are the inner workings of the study transparent, with an inherent recognition of the limitations of the study.

Table 1 illustrates  Anderson's et al simple 2 x 2 working model of how to classify research.

Table 1 Combining rigorous practice and relevant research

		Methodological Rigour Low                                        High
Practical Relevance	High	Quadrant 1 ‘Popularist Science’	Quadrant 2 ‘Pragmatic Science’
	Low	Quadrant 4: ‘Puerile Science’	Quadrant 3 ‘Pedantic Science’

Where studies have high levels practical relevance but low levels of methodological rigour, 'Popularist Science' is created and can be found in Quadrant 1.  Examples of this may include research which may be addressing an extremely important and pressing issue, though the studies lack the rigour to warrant any kind of reliance upon their findings.  It could also be used to describe where new educational initiatives or innovations have led to poorly designed studies being conducted, with the need to publish rapidly being more important than the ensuring the quality of the underlying study.  Another attribute of 'popularist science' is the lack of any form of peer-review.    Examples of 'Popularist Science' within education could include those advocating Brain Gym, Learning Styles and possibly Multiple Intelligences.

Where, we have both high practical relevance and methodological rigour, we can describe this as 'Pragmatic Science' and is located in Quadrant 2.   For those interested in closing the gap between researchers and teachers, this is the type of research which should be most common, and be used to influence conversations within staff-rooms.  Examples of this type of research can be found in The Education Endowment Foundation Teaching and Learning Toolkit.  However, there is research which you might think would definitely fall into this category, for example, John Hattie's Visible Learning - where there are significant doubts about the methodological rigour of the research.

As for Quadrant 3, this is where methodological rigour is high and practical relevance is low and can be described as 'Pedantic Science'.  In this situation, we have studies which are almost flawless in design, yet seek to address questions which have little or no relevance to schools, headteachers and teachers.  These studies have a focus on theory not practice, and have been written for a very small group of researchers who specialise in this area.  And I will leave it to you to identify your own favourites in this category.

And finally, Quadrant 4, this is where we have what is termed 'Puerile Science', where researchers have investigated matters of little practical relevance and have done so with research designs, which are sloppy and ill-conceived.

What are the implications of the 2 x 2 model for school research leads and evidence-informed practitioners?

For me there are three main  implications of the model, which need to be considered.

• School research leads are often keen users of social media and read a wide-range of blogs.  Given the very nature of social media, very little if any, of what comes across your timeline will in be in Quadrant 2 i.e relevant and rigorous.  Social media - be it twitter or blogs - should be seen as either entertainment or information rather than education.  In other words, social media may set you off to search out 'Pragmatic Science' but in all likelihood the social media is in itself at best 'Popularist' Quadrant 1 (felt it was a bit of a stretch to call social media science).
• Even if 'Pragmatic Science' can be identified, that does not mean that whatever intervention has been tested, or effect size has been calculated, will necessary work in school or classroom. And remember working out what works is in itself not enough, rather we need to ask the questions :  'how and why does this work and/or not work, for whom, to what extent, in what respects, in what contexts and over what period?'.
• By skilful use of the 2x2 matrix it should be possible to save both yourself and your colleagues a lot to time and resources - by making sure where at all possible you spend your time discussing and implementing interventions which have the characteristics of 'Pragmatic Science.

And finally

T he only real way of knowing whether you have got your classification of 'research' right, is to specialise in a particular field of research.  This blogpost outlines a simple organising framework for thinking not only about academic research, but also other forms of evidence.  As such, it should be as what Daniel Willingham (2012) would describe as a work-around or heuristic - as you seek to bring practically relevant research into your school and classroom.

Reference

Anderson, N., Herriot, P. and Hodgkinson, G.P., 2001. The practitioner‐researcher divide in Industrial, Work and Organizational (IWO) psychology: Where are we now, and where do we go from here?. Journal of Occupational and Organizational Psychology, 74(4), pp.391-411.

Cronbach, L. J., & Suppes, P. (Eds.). (1969). Research for tomorrow’s schools: Disciplined inquiry for education. New York: MacMillan. This is a report of a special committee of the National Academy of Education. It includes a detailed discussion of disciplined inquiry, a number of historical case studies of educational research programs and a set of policy recommendations.

Hood, P. (2003) Scientific Research and Evidence-Based Practice : WestED

Willingham, D. (2012) When Can You Trust The Experts: How to tell good science from bad in education, Jossey-Bass, San Francisco.

The School Research Lead - Making the most of research evidence and school data - an integrated approach

If you are school-research lead then this post is for you.  In particular, if you are interested in using evidence - be it research, school data, stakeholder views and practitioner expertise - within your school, then this post will help you understand the different stages of the evidence-informed inquiry process.  To help with this,  I will be drawing upon the work of Chris Brown, (UCL Institute of Education), Kim Schildkamp and Mireille D. Hubers (both of the University of Twente) and their presentation COMBINING THE BEST OF TWO WORLDS: INTEGRATING DATA-USE WITH RESEARCH INFORMED TEACHING FOR SCHOOL IMPROVEMENT which they made at the recent International Congress for School Effectiveness and Improvement conference held earlier this year in Glasgow.   The rest of this post will: first, summarise the thesis underpinning the development of Brown et al's model; second look in more detail at the proposed cycle of evidence-based inquiry; third, identify a number of potential limitations/design flaws within the proposed cycle; fourth, put forward an alternative model which is based upon evidence-based practice.

The underpinning thesis 

The main 'thesis' underpinning Brown et al's presentation was a call to integrate data based decision-making (DBDM) and research informed teaching practice (RITP) into a comprehensive professional learning based approach that is designed to enhance teaching quality, and a result increased student achievement.  As such, the model is attempting to use the strengths of each approach to offset the weaknesses of the other.  Brown et al summarise the offsetting benefits of each approach as follows:

• RITP is not based on a real need in the field vs DBDM starts with the vision and goals of a specific school, focusing on a context specific problem. 
• DBDM data can inform educators about problems in their school, but what causes this problem vs  RITP educators can draw upon a variety of effective approaches to school improvement 
• RITP is the ‘one size does not fit all’ vs DBDM based on data, schools develop a context specific solution 
• DBDM is that data can be used to pinpoint possible causes of a problem, but educators may still not know the best available course for school improvement vs RITP picking a promising solution, based on an existing evidence base (slide 23). 

An eight stage cycle of evidence-based inquiry

Brown et al then go onto integrate DBDM and RITP into the following eight stage cycle.

On initial reflection this model provides an incredibly useful way of thinking about the relationship between data and research evidence within the inquiry cycle, with with the local data setting the scene for the necessary research evidence.   Indeed, one of the things I have been thinking about lately is where 'local data-collection' fits within  the models of evidence-based practice and evidence-based medicine which I have previously described and advocated.  And this model, clearly places local data prior to seeking the research evidence.

Potential limitations

However, for me, the model two limitations to be taken into account: first,  the determination of  causes coming before the collection of data; second, the potential of for increasing the risk of confirmation bias.    So lets now examine each of these limitations in more detail.

Causes before data or vice versa

The eight stage model presented places the stage of determining possible causes of the problem before collecting data about causes,  which to me seems to be out of sequence and in reverse to order to what they should be . Indeed, there are other models of data led inquiry where data collection comes before diagnosis - and this illustrated in the four stage model of inquiry popularised by Roger Fisher and William Ury  in the their classic book Getting to Yes

In this model the data stage focuses on identifying what's wrong, what are the current symptoms, and then moves to the identifying of possible causes in the diagnosis phase, with the next two phases being direction setting and action planning.  For me, and I know when you use these cycles you move back and forth between the stages - it seems far more sensible to place the data-collection before the diagnosis phase.   In addition, and I thank Rob Briner for this observation, the proposed process seems to separate use of the of research evidence from the diagnosis phase.  Yet, why would you not use research evidence to help you identify possible causes of the symptoms being experienced.

The potential for cognitive bias

By placing both vision and goal setting next to determining causes and the drawing of conclusions prior to the search for the solution in the research evidence, this may lead to confirmation bias i.e the tendency to selectively search or interpret information in a way that confirms your perceptions or hypotheses.  So determining the vision and goals may lead to certain problems to be highlighted as they are consistent with the already established, whereas other problems or data inconsistent with the 'vision' may be ignored.  Furthermore, given that conclusions are drawn before searching for research evidence - this may lead to research evidence being sought which is consistent with the conclusions that have been already been drawn.

A possible alternative

Barends, Rousseau and Briner (2014) provide an extremely useful definition of evidence-based practice, which identifies a six-stage process in making decisions which are based upon evidence.  In this model the search for and acquiring of evidence happens at stage two, with research evidence, organisational data and stakeholder views being accessed.  This leads to the subsequent appraisal and aggregation of the evidence resulting in the incorporation of the evidence into the decision-making and action planning process.

However, this process does not fully incorporate all the elements of included in the proposed eight-stage cycle of evidence-informed inquiry i.e vision and goal setting.  This 'omission' can be rectified if so required, by the inclusion of another 'A' representing : Ambition, aims and goal setting and which relates to the vision, mission and values of the school, and which gives us a 7 stage cycle of evidence-based inquiry. 

A seven stage cycle of evidence-informed inquiry

Some final words

Brown et al make a powerful case for combining DBDM and RITP and have developed a potentially useful initial synthesis of the two processes.  However, the proposed model has some inherent limitations, be it diagnosis taking place before data collection and increased possibilities of confirmation bias taking place.  Finally, an alternative model - the 7 A's of evidence-informed inquiry - is put forward as way to think about the process of linking DBDM and RITP.

Uplifting Leadership - Transforming schools in tough times

If you are a senior, middle or aspiring leader within a school or college, then this post is for you.  In particular, if you work in a school where you are having to do more with less; where the pupil achievement is not all that it could be; or where teacher morale is poor,  then this post may provide some guidance on how transform what you do and at the same time stay resilient in the face of difficult times.  In doing so, I will be using Andy Hargreaves, Alan Boyle and Alma Harris's 2014 book Uplifting Leadership : How organisations, teams, and communities raise performance, which draws upon research in eighteen organisations and systems - in business, sports and and public education - and identifies what these organisations did to dramatically improve their performance, often against overwhelming odds.  For Hargreaves at al in the end it all came down to one word : Uplift. 

What do we mean by Uplift?

Hargreaves et al define uplift within organisations as .... the force that raises our performance, our spirts, and our communities to attain higher purposes and reach unexpected levels of achievement (p1)

As such uplift is about emotional and spiritual engagements, social and moral justice and higher levels of performance, both in work and life.  Let's now briefly look at each in turn.

Emotional and Spiritual Uplift  .... the beating heart of effective leaders.  It raises people's hopes, stirs up their passions, and stimulates their intellect and imagination.  It inspires them to to try harder, transform what they do, reach for a higher purposes, and be resilient when opposing forces threaten to defeat them. Uplifting leadership makes spirits soar and pulses quicken in a collective quest to achieve a greater good for everyone because we feel drawn to a higher place as well as to the people around us as we strive to reach it (p3)

Social and Community Uplift and .... creating a collective force which is designed to raise everyone's opportunities for achievement and success - with particular reference to those in our communities who are the least well-advantaged.

Uplifting Performance ....raises performance by creating, spiritual, emotional, and moral uplift throughout an organisation and wider community that it influences (p4)

What does Uplifting Leadership involve?

Uplifting leadership involves six inter-related factors which come together to bring about transformations within organisations and these include:

1. Dreaming with Determination - this involves identifying and articulating a clear, challenging destination, and which is informed by a moral imperative.  Furthermore, this dream is firmly connected with the organisation's past and building upon what the very best of what that organisation has been in the past. 
2. Creativity and Counter-Flow - this requires creating the new pathways necessary to reach the desired 'dream'.  However, it also goes against the flow - in that it is not about following the predictable, it involves the counterintuitive - things that don't seem to make sense or that others may already have rejected.
3. Collaboration with Competition- uplifting leadership is at times a counter-intuitive process and at times this will require working alongside current or future competitors.  Competition and collaborative are not mutually exclusive and it is possible for both to co-exist within the same context. 
4. Pushing and Pulling - this necessitates using the power of the group to both push and pull things forward.  Colleagues when faced with difficulties are picked up and supported by others, whilst the higher purpose to which team members are committed pushes them onto higher levels of achievement
5. Measuring with Meaning  - .  this  involves the extensive use of data  allows leaders to identify the direction the organisation is heading and what still needs to be done, yet is done in such a way which is both meaningful and owned by the people who work in the organisation
6. Sustainable Success  - this involves working at a pace that is sustainable.  It's not about leading at a pace which people cannot sustain for any substantive period of time.  It's about recognising the ebb and flow of energy within an organisation and making sure that is managed in such a way as to bring about years and years or continuous improvement and development

What's the evidence base to support Uplifting Leadership?

The evidence-based which informed Uplifting Leadership was drawn from a number of interconnected projects.  First, a project conducted between 2007 an 2010 that looked into public an private sector organisations which performed beyond expectations.  Over 200 in-depth interviews were conducted across 18 projects sites ( five in business, four in sports, and nine in English education).  Within-case and cross-case analysis was to used to identify underlying themes.  Within the original 18 cases - fifteen factors were identified which seemed to explain performance beyond expectation. In order to reach a broader audience an additional sports team was included in the analysis alongside two educational cases - based in Canada and Singapore. Other work being undertaken by the authors was also drawn upon.  This work  included research on successful turnarounds, reform of special education in and a case-study of a London borough.  As a result of combining these research efforts - the analytical framework was reduced from fifteen to six factors.  Additional cases from secondary sources were also drawn upon.

What does this mean for me, a school or college leader?

Hargreaves et al identify a whole range of practical actions that school leaders and their colleagues can take to be uplifting leaders: be become great storytellers; surprise yourself; benchmark relentlessly; avoid cliques and elites; measure what you values; invest for the long-term.  Nevertheless, when it is as all said and done - and as Hargreaves et al argue - uplifting leadership begins with the self, and go onto to cite Manuel Kets de Vries who argues 'if we want things to be different, we must start by being different ourselves'  (Hargreaves et al p 161).   In other words, if we want to lead others, we'd better start by leading ourselves.

New - Why certain practices reach the classroom and others don't

If you are interested in how educational research and ideas reach the classroom, then this post is for you.  Based on Jack Schneider's 2014 book From the  ivory tower to the school house this post see to explain why some educational research and ideas become common practice in schools, and why others do not.  Schneider identifies four attributes - perceived significance, philosophical compatibility, occupational realism, and transportability-  that educational research must have ... if teachers are to notice, accept, use and share (Schneider, p7).  Finally, I will consider the implications of Schneider's analysis for the implementation of evidence-informed practice.

However, before I look into these four attributes in more detail, I will quickly recount Schneider's summary of the arguments as to why much educational scholarship has little impact on the practices of teachers and pupil learning.  First, Schneider argues  there is little or no 'practice ready scholarship' with academic writing being for each other academics and researchers rather than teachers.  Second, teachers are antagonistic to new ideas: teachers have worked to make the profession comfortable for themselves and are determined to protect the status quo.   However, Schneider argues that the real reason why the is a separation between academic scholarship and teaching practice is the separation of capacities and influence required to shift research into the classroom.  Teachers may be able to influence what goes on in the classroom, they normally lack the capacity and capability to engages with academic research.  Whereas policymakers may be well positioned to connect with the 'educational HE academy' they are not best positioned to directly influence the classroom.

Nevertheless, Schneider argues that educational research, ideas and scholarship can move from the ivory tower/academy to the classroom if the research/idea possesses a set of specific attributes.  So what are these attributes:

1. Perceived significance - the research or idea is relevant to an issue that matters to teachers and there would appear to be some evidence to support the idea.  On the one hand, the research is signalling that it matters to the profession and schools, and on the other the research would appear to have an evidential justification and/or is backed by an educational authority.
2. Philosophical compatibility - is the research compatible the with common values, interests and concerns of teachers and headteachers 
3. Occupational realism - is the idea practical and can be easily put into immediate use in the classroom the very next day or does it require a significant physical investment in the school 
4. Transportability - how easily can the idea move from research into practice and from teacher to teacher.  Is it an idea that does not require some form of extensive training or  it can be picked up in a 30 minute CPD session.  Alternatively can the idea be transmitted by social media, and for exampe, Twitter and the Twitterati.

Schneider argues that the possession of these four attributes is a necessary but not sufficient condition for research or an idea to move from the ivory tower to the classroom and illustrates this with reference to a a number of other innovations, for -  the taxonomy for the affective domain, Sternberg's triarchic theory, Wittrock's generative learning model; and finally, the behaviour analysis model - which have not made the jump from the academy to the classroom.   Furthermore, just because an educational idea/research has these four attributes does not in itself guarantee that it has merit or worth, examples of this being the popularity of Brain Gym and Learning Styles.

Applying the four attributes to the development of evidence-informed practice within schools.

If we use these four attributes as a check-list to help us identify the chances of 'success' for the evidence-informed practice movement with schools, then it could be argues that evidence-informed practice still has a long way to go to before it can be said to have all of four attributes.  Indeed, it could be argued that at best the evidence-informed practice currently has only one of the attributes necessary for success i.e transportability - witness the success of researchED which has used Twitter as a main form of communication.  On the other hand, evidence-informed practice - which for some means research-informed practice - is be seen by many as occupationally unrealistic - due to a lack of access to research journals.  For others, an evidence-informed approach is philosophically incompatible with both teacher autonomy and relationship between evidence and practices.  Finally, given that much research is seemed irrelevant to schools and not addressing real school-based problems, research is perceived to be insignificant to the immediate needs of pupils, teachers and schools.   

The above analysis would suggest that the success of the evidence-informed practice movement is by no means certain, and if anything the movement is more likely to fail than succeed.  

If you believe in evidence-informed practice in schools - what is to be done?

In some ways the answer is quite simple : supporters of evidence-informed practice within schools need to improve the extent to which evidence-informed practice exhibits these characteristics.  This could be done by the following :

• Philosophical Compatibility - ensure absolute conceptual clarity, so as not to confuse evidence-informed practice with research-based practice.  This is can be done by proponents of evidence-informed practice continually articulating the role of practitioner expertise just as this was done in evidence based medicine with Sackett et al defining evidence-based evidence as is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence based medicine means integrating individual clinical expertise withthe best available external clinical evidence from systematic research.
• Occupational Realism - emphasising that evidence-informed practice can be put into immediate effect within schools bye supporting colleagues develop well-formulated and answerable questions.  It's not just about accessing journals and research, it also involves teachers accessing and using school data or even just talking to pupils about their perceptions of their learning experience.  Stakeholder views and values are an essential component of evidence-informed practice.  

As for perceived significance - this is where advocates of evidence-informed practice will need to be both patient and open-minded as the 'jury' is still out as to impact of evidence-informed practice on pupil outcomes.  In the specific context of schools, although not comprehensively or systematically established, there are numerous reported benefits to practitioners engaging in evidence informed practice.   However, to help address this lack of academic evidence there are a number of EEF funded studies, for example – The Rise Project: Evidence – Informed School Improvement; Research into Practice : Evidence Informed CPD; Research Learning Communities; and Evidence for the Frontline – which considering the impact of evidence-informed practices on teacher development and pupil outcomes.  These projects will begin to report from 2016 onwards.

To conclude:

This is a 'very quick and dirty analysis' of the evidence-informed practice movement.  However, what it does make abundantly clear is that there is still much to be done to ensure evidence informed practice is perceived to be significant, is philosophically compatible with teacher values and beliefs, and occupationally realistic.  Even if this is done, school leaders will need to ensure that teachers have the capacity and capability to implement evidence-informed practice within their schools.

Why the delusional voodoo of grading lesson observations is not going stop any time soon.

Despite the impassioned words of headteachers, such as Tom Sherrington and John Tomsett on the reliability and validity of graded lesson observations, they are still being used in around 50 per cent of schools.   Drawing upon the pioneering work of Pawson and Tilley (1997) I will use two core ideas/concepts from realist evaluation i.e ‘programme theory,’ and ‘mechanisms’  to help explain, why despite evidence that lesson observations should only be used for low-stakes development purposes, some head-teachers and senior leadership teams still wish to continue with the practice of grading lessons.

What do we mean by a ‘programme theory’?

Any intervention - in this case graded lesson observations - introduced or used by head-teachers and senior leaders - should be informed by some form of programme theory i.e. a statement along the lines of .   if we do X then this will deliver change Y.   In other words, a programme theory is simply an ‘if… then....." statement.  So in the case of head-teachers and senior leadership teams who wish to use to continue to graded lesson observations within their school, they may be being guided by a programme theory - which goes along the following lines : ‘If we use graded lesson observations this will then deliver an increase in the quality of teaching and learning within the school.’

Given the diverse nature of headteachers and senior leadership teams it is unlikely that there is only one programme theory being ‘used' in schools who are continuing to use graded lesson observation.  Other programme theories which could be being used include:

‘If we use graded lesson observations this will lead to greater control over teachers and how well they perform.’

‘If we use graded lesson observations this will then lead to fewer poorly performing teachers within the school.’

"If we use graded lesson observations then we will be able to measure the quality of teaching and learning within the school.'

Of course, this is not a comprehensive list of programme theories and I'm sure you will be able to think of programme theories which include either OfSTED or governing bodies.

In the context of ‘realist evaluation’ what do we mean by a ‘programme mechanism?’

The important thing to remember is that interventions do not in themselves work, they work because people make them work. So an intervention provided an opportunity/constraint X and this prompted me to do Z.    As Wong et al (2013) explain - interventions work by changing the decision-making of the subjects of an intervention. (Now I know describing teachers as 'subjects' of intervention may be troubling for some readers, but stay with me).  With this in mind, the question becomes: how does the use of graded lesson observations influence the thinking and decision-making of teachers?

What programme mechanisms might be considered to be active when considering whether to either to implement or continue to use graded lesson observations?

Drawing upon the work of Pawson and Tilley (1997) on the use of CCTV systems in car parks to reduce car crime, (and which is scarily useful), these are some of the mechanisms head-teachers and senior leadership teams maybe seeking to influence through the use of graded lesson observations.

The 'gotcha mechanism’ graded lesson observations could reduce the number of 'inadequate /requires improvement' lessons as it make more likely that persistently 'poor' teachers are identified and competency and capability procedures are used to to ensure the 'poor' teacher leaves the school.

The 'you’ve been framed mechanism' graded lesson observations could reduce the number of inadequate lessons by encouraging teachers to raise their game on a day to day basis, so as to avoid being graded 'requires improvement/inadequate’.

The 'effective deployment mechanism' graded lesson observation may facilitate the effective deployment of professional development to support those teachers who have been identified as requiring support in order to improve teaching and learning

The 'publicity mechanism' graded lesson observations may indicate to teaching staff that the SLT takes seriously the improvement of teaching and learning. The teachers may then decide to 'up their game' in order to avoid being judged 'requiring improvement/inadequate'

The 'memory jogging mechanism' notices in the staffroom about graded lesson observation protocols may remind teachers of their 'vulnerability' and prompt them to take greater care their lessons meet the required standard

And I'm sure there are others you can think of such as:

The 'get out before they catch me mechanism' graded lesson observations may encourage some teachers who think they are weak practitioners  decide to leave before they are deemed inadequate.

The ‘the compliance mechanism’ graded lesson observations may demonstrate who is ‘in-charge’ within the school and encourage teachers to comply with the desired model of teaching, learning and assessment.

What are the implications of the above analysis the use of graded lesson observation in schools?

First, if we want to understand why headteachers and senior leaders may be extremely reluctant to give up on an established intervention, then we need to take into account both the programme theories they hold and causal mechanisms they are seeking to influence. If the reliability and validity of graded lesson observations is irrelevant to either the programme theory or causal mechanism, then the ‘evidence’ is unlikely to have much impact.

Second, proponents of non-graded observation may need to adopt a slightly different take in disagreeing with colleagues. In the first instance we need to understand colleagues ‘programme theories’ and views of ‘causal mechanisms.’ Relying on evidence, may not be sufficient to bring about changes in those ‘programme theories’ or ‘causal mechanisms’. Daniel Dennett in his 2013 book Intuition Pumps And Other Tools for Thinking suggests that at times of disagreement we should use Rapoport’s Rules:

1. You should attempt to re-express your target’s position so clearly, vividly, and fairly that your target says, “Thanks, I wish I’d thought of putting it that way.”

2. You should list any points of agreements (especially is they are not matters of general and widespread agreement).

3. You should mention anything you have learned from your target.

4. Only then are you permitted to say so much as a word of rebuttal or criticism.

And finally

Any intervention – be it graded or ungraded lesson observations - must be seen within the context within which it is applied, so in other words there is no such thing as a context free intervention. Indeed, context is an essential component of ‘realistic evaluation'. So in the case of schools, much will depend upon; the relationships between staff; the values, beliefs and culture of the school; the impact upon the school of external shocks – be it funding or an Ofsted inspection; or the stance that senior leadership take on the use of external evidence, as to how an intervention such as graded lesson observations play out. Remember interventions – be it graded or ungraded lesson observations - do not in themselves work, they work because teachers and headteachers make them work.