The school research lead and '57' definitions of research

If you work in a school and have anything to do with research, evidence-informed teaching or practitioner inquiry, this post is for you.  In particular, this post is aimed at anyone who finds the current use of these and other similar terms - evidence-based practice, action research, enquiry-based learning -  confusing and a barrier to both your own professional development and the bringing about of improved pupil outcomes within your school.

So what do we mean by the terms : research, evidence-informed teaching, evidence-based practice, action research, teacher/practitioner inquiry?

Accepting that a range of definitions which could be used for each term, I propose the following as a starting point for subsequent discussion.

Research* is systematic and focused equiry seeking truths that are transferable beyond the setting in which they are generated (Greenhalgh, p191)

Evidence-informed teaching involves the 'conscientious, explicit and judicious use of best evidence in making decisions about teaching and learning' (adapted from Sackett, 1995)

Evidence-based practice as 'best-practice' is considered as any practice that has been established as effective through scientific research and according to a set of explicit criteria (Mullen, 2002)

Action research is the systematic collection of information that is designed to bring about social change (Bogdan and Bilken, 1992 p223)

Enquiry-based learning Much of the kind of activity that the professional research community might define as “action research” is more commonly referred to by teacher-researchers as enquiry-based practice (BERA/RSA 2014 p40)

Teacher/practitoner inquiry is defined as systematic, intentional study of one’s own professional practice.

So what are the differences between the terms?

Dana and Yendol-Hoppey (2009) quite rightly point out that the terminology used around teacher inquiry all carry a degree of baggage.  For example,  research is associated the notion of impersonal objectivity and creation of generalisable knowledge.  Action research is associated with critical practice designed to bring about social justice. Evidence-informed practice is associated with evidence-based medicine and the use of RCTs.  Evidence-based practice is associated with bureaucratic rules which remove the discretion and autonomy of the teacher.   So where do we go from here?  Wiliam (2014) argues that education can never be a research-based profession and states:

Unfortunately, Wiliam does not describe 'disciplined inquiry' so we need to look elsewhere for guidance.

So what is disciplined inquiry?

Cronbach and Suppes (1969) state:

Disciplined inquiry has a quality that distinguishes it from other sources of opinion and belief. The disciplined inquiry is conducted and reported in such a way that the argument can be painstakingly examined. The report does not depend for its appeal on the eloquence of the writer or on any surface plausibility, (p. 15).

Hood (2003) usefully identifies a number of the characteristics of disciplined inquiry, which include:

1. Meaningful topics are addressed
2. Systematic, clearly described procedures are employed and described so that readers can follow the logic of the study and assess the validity of the study's conclusion
3. There is sensitivity to the errors that are associated with the methods employed and efforts are made to control the errors or consider how they influence the results
4. Empirical verification and sound logic are valued: and
5. Plausible alternative explanations are considered (p2)

So what are the implications for colleagues interested in school based inquiry?

1. The purpose of inquiry within schools should be to support the development of 'practical wisdom' within the school rather than attempting to emulate the 'research' undertaken in universities.  In other words, inquiry is about making better decisions and improving practices for the benefit of pupils, teachers, the school and the community 
2. Be aware that although all these terms - evidence-informed teaching, action research, practitioner inquiry -  may be describing some form of disciplined inquiry, they are not the same thing, and they have different meanings which may or may not get in the way of your approach to inquiry
3. Whatever form of inquiry you adopt it should seek to address questions that matter to pupils, teachers, the school and the community.
4. Seek to ensure that the processes of inquiry are open and transparent and that the 'logic' of the inquiry is clearly articulated, particularly any underlying assumptions
5. All members of the school-community need to work together in such a way as to create the conditions in which colleagues are prepared to have a go at 'inquiry' and knowing that colleagues will both be supportive and challenging.
6. Ensure that the outcomes of inquiries are communicated to colleagues in ways which are both meaningful and informs practice.   

And finally

To become a virtuoso inquirer or supporter of inquiry,  will take time and will involve a process of going from novice to higher levels of expertise.  This in turn will require going through many cycles of inquiry.  With this in mind  maybe the best we can hope for is to become competent and proficient as school-based inquirers rather than expert.

*Note

I have deliberately chosen not to use the BERA/RSA definition of research as
 .. any deliberate investigation that is carried out with a view to learning more about a particular educational issue. This might take a variety of forms and be concerned with a range of issues, for example: the secondary analysis of published data on school exclusions, interviewing a range of colleagues about examination performance in the English Department, taking part in a national Randomized Control Trial concerned with the teaching of Mathematics, responding to a survey about teachers’ use of the internet to inform curriculum planning, working with a university department of education on a study into teachers’ use of new technology. (BERA/RSA 2014 p40)

As you can see this definition describes types of research activity rather than a provide any guidance as to what generic processes are associated with inquiry.  Indeed, in my view the the definition is particularly unhelpful as it lacks precision, and may lead to teachers believing that it is easy to become a researcher.

References

BERA/RSA (2014) RESEARCH AND THE TEACHING PROFESSION : Building the capacity for a self-improving education system : Final report of the BERA-RSA Inquiry into the role of research in teacher education

Bogdan, R & Biklen, S (1992) Qualitative Research for Education : In introduction to theory and methods (second edition) : London, Allyn and Bacon.

Cronbach, L. J., & Suppes, P. (Eds.). (1969). Research for tomorrow’s schools: Disciplined inquiry for education. New York: MacMillan. This is a report of a special committee of the National Academy of Education. It includes a detailed discussion of disciplined inquiry, a number of historical case studies of educational research programs and a set of policy recommendations.

* cited in Shulman, L.  (1997) Disciplines of Inquiry in Education: An Overview in Jaeger, R. (ed) (1997) Complementary Methods in Complementary Methods for Researchers in Education, American Education Research Association, (pp 3-19)

Dana, N F and Yendol-Hoppey, D. (2009) The Reflective Educator’s Guide to Classroom Research: Learning to Teach and Teaching to Learn Through Practitioner Inquiry (Second Edition) Thousand Oaks, CA: Corwin Press

Greenhalgh, T. (2014). How to Read a Paper: The Basics of Evidence-BasedMedicine (5th ed.). United States: John Wiley & Sons.

Hood, P. (2003) Scientific Research and Evidence-Based Practice : WestED

Mullen, E (2002) Evidence-Based Knowledge : Designs for Enhancing Practitioner Use of Research Findings - Presentation at the 4th International Conference on Evaluation for Practice, University of Tampere

Strategic Inquiry - An Alternative to Lesson Study?

If you want to explore alternatives to Lesson Study, this post is for you.  Nell Scharf Panero and Joan Talbert's 2013  book   - Strategic Inquiry : Starting small for big results in education -  describes a model of school-based inquiry which is explicitly linked to improvements in pupil learning and school improvement.    So use this post  as a stimulus to critically examine whether Strategic Inquiry (SI) is worth including in your school's portfolio of developmental activities.

Background

The SI model has its' origins in the Scaffolded Apprenticeship model (SAM) programme of school improvement, and was launched in 2005 by the School of Public Affairs at Baruch College and New Visions for Public Schools.  Originally developed in New York City public high schools its use has now been extended to Boston, Rochester and Oakland

What is strategic inquiry?

Panero and Talbert (2013) describe SI as

... teams study(ing) the whole school in microcosm, assuming that a part represents the whole.  They identify the essential skills gap for a group of struggling students, investigate how school-wide learnings conditions allow the this gap to persist, and design and implement strategic changes, the effectiveness of which is measured by the closing of the gap.  Getting small is essential in the model's theory of change.  It makes the otherwise overwhelming task of change manageable, scaffolds and ensures the development of new skills and experiences that shift team members' thinking, and defines a direct line between action and results so that actions can be continually improved and results ensured.  (p4)

Why is strategic inquiry (allegedly) successful?

Panero and Talbert argue that SI works because it leads to improvement in student achievement by addressing the learning needs of students who are struggling, whilst at the same time developing leadership capacity for inquiry-based reform.   Panero and Talbert argue that SI is successful for three reasons

• SI starts with a small group of staff who become skilled in the process, who then help other teachers within the school do the same.
• SI focusses on key changes in teachers' thinking, practice and school cultural norms necessary for on-going improvements.
• SI invests in training facilitators to support and coach groups of teachers through the key stages of the process.

What are the assumptions underpinning strategic inquiry?

There are a number of key assumptions which underpin SI, including:

• All schools have a zone of success, where there are students for whom the current ways of working are effective.
• All schools have a zone of failure, where there are students for whom current practices are not working.
• The practices that lead to a lack of student success are a product of school structures and cultures.
• If school leaders can work out what is working and why, and what is not working and why - they will be able to work out what needs to be done to improve outcomes for students and subsequently be able to implement them.
• By getting small it is possible to identify where specific students are struggling and identify what actions to be taken.
• Teachers beliefs about what brings about student improvement are changed as result of adopting new practices i.e behaviour influences values.
• Once learning has taken place it can be applied across the rest of the school.

In other words, SI is about teachers understanding school practices and processes from the perspective of the struggling students and subsequently removing barriers that get in the way of student success.

What are the elements of strategic inquiry?

SI  consists of - what Panero and Talbert describe as the 4 Ts - teams, targets, tasks and training.

Teams

Teacher collaboration is deemed as essential for a number of reasons; the diffuculties of the challenges faced by schools; teams create space in which teachers can practice and develop new skills; by developing a sense of collective responsibility for the success of students creates a sense of shared accountability.

Targets 

An essential element of SI is the setting of small and measurable targets by which progress is measured.  In other words, it's not about setting broad targets about improving maths skills in underachieving students but rather identifying the specific skills and knowledge that those students lack, for example, how to divide a fraction by a fraction.  Panereo and Talbert - argue that this precision in target-settings one of the hardest things for teachers to grasp.  Furthermore, by having precise targets this helps to maintain a laser-like focus in the gaps in current student learning and what is needed to close that gap.

Tasks 

SI requires a number of cycles of inquiry, with inquiry tasks being organised into three distinct phases.   Phase 1 is to  move the students.  Here a specific high impact skill-gap is identified for those students who are outside the school's zone of success.  Having really drilled down into the data, a specific student learning target is identified and trained facilitators are used to redesign teaching practices to meet the needs of this specific group of students.  This phase may lead to teachers changing their beliefs about not only causes gaps in student learning, but also what can be done to close those gaps.

In Phase 2 teams consider what they can to change and improve a decision-processes about teaching and learning within the school.  This may include looking at what is taught, how it is taught and who does the teaching.  However, each of those decisions are considered from the perspective of students with the specified skill gap.  As a result of this analysis, one small change is identified which will help those students close the specified skills gap by getting them support they require.  By being disciplined and focussing on small changes, it is possible to increase shared accountability with an emphasis on measuring what the student has learned in response to the new approach. Teams also analyse other systems from the perspective of students who are not in the zone of success and work to remove to barriers which are in the way of student success.

In Phase 3 - in this phase members of the strategic inquiry team work, in a leadership capacity,  with other colleagues to help those colleagues understand the process and to increase shared learning within the school.  Members of the SI team also get back together to supported one another, as they develop their coaching, mentoring and facilitating skills.  In addition, an important component of this phase is for team members to evaluate the impact of their work in closing gaps in student learning gap.

Training - Facilitators

Panero and Talbert argue that experience within schools using SI found that there was a relationship between teacher ratings of facilitator support and their success with strategic inquiry.  They found that facilitators were necessary to help inquiry teams learn how to overcome challenges in the inquiry process, and in particular learning how to make it work.  External support is deemed necessary to help colleagues push through difficulties, which might otherwise cause the inquiry to stall if not fail.

Limitations to strategic inquiry?

Inevitably, there are limitations of the SI model.

• SI assumes that even if teachers can identify what are the problems facing pupils, the school has the ability to bring about the required change.  As such, the model could be described as being US-centric in the belief in self-improvement.  
• SI is based upon the availability of trained facilitators (and associated funding)  who are able to support colleagues close the identified gap in student learning. Given the current pressures of our 'age of austerity' this may be an important limiting factor.  
• SI could be described as giving a new fangled name to the simple process of finding out what doesn't work and do something else instead. 
• There is lack of evidence examining the effectiveness of the SI process, within so we don't know whether it works or not within the UK's various educational contexts.

Why is strategic inquiry worth considering?

Despite these limitations, there are several reasons why school leadership teams may wish to give SI further consideration.   SI:

• focuses on the evidenced needs of students and reducing gaps in student learning; 
• emphasises improving teacher practice rather than generating research outcomes;
• starts small, which given the resource pressures on both schools and school research leads, is probably a necessary starting point;
• has the potential for demonstrable short-term success, which will create the conditions for further evidence-informed practice.

To conclude:

SI could play an important role as part of and evidence-informed school's repertoire of strategies for school improvement.  By focussing on gaps in student learning, SI  has the potential to align evidence-informed practice with school improvement.  SI may be right for you and your school, or it may not.  The choice is yours to make.


Reference

Scharff Panero, N. & Talbert, J.E. (2013) Strategic Inquiry : Starting small for big results in education.

Lesson Study - The New Brain Gym?

If you are interested in teaching and learning, teacher professional development or school improvement, this post is for you.  Unfortunately, education is a 'fashion-led' profession and we are seeing  'fashion' now raise its head in the form of the widespread adoption of lesson study.  Lesson study is now being widely used as a mechanism to support the development teachers as inquirers.  Yet, the EEF's current view is There is some evidence that Lesson Study could have a positive impact in English schools, but not yet enough to justify scaling it up.  With this view position being  supported by Dylan Wiliam at the 2015 Sunday Times Festival of Education.

With that in mind, there appears to be little evidence of lesson study being critiqued or challenged as a way of supporting a teacher's learning before lesson study is implemented within an individual school.  Furthermore, in the reports published by schools who have used lesson study, there would appear to be little public reflection on either the lesson study process or examples of lesson study not meeting the needs of teacher development.  That does not mean that such reflection has not happened, it just does not appear to be in the public domain.

So what are we to do?   Given this lack of evidence, it would be sensible to look at alternative ways of supporting teachers develop their practice.  One approach is School Based Instructional Rounds (SBIR), which I will now examine in more detail.

Instructional Rounds (IR) :The background

Initially designed and developed by the Harvard Graduate School of Educations to be used within a network of schools, Instructional Rounds (IR) was inspired by the approach adopted by the medical rounds that doctors use as part of their training.   Although there are several versions of medical rounds, the essential elements involve doctors - be it trainees, interns, house officers, residents or consultants - observing and discussing the elements of a patient's diagnosis after a thorough review of the evidence and consideration of the various forms of intervention/treatment.    As Elmore et al (2009) state : The rounds model embodies a specific set of  ideas about how practitioners work together to solve common problems and to improve their practice.  In the education context, we call this practice instructional rounds, or rounds for short 





School Based Instructional Rounds : How does it work?

Although initially designed to work across a network of schools, the IR model has subsequently been developed to work at the level of individual schools.  So how does it work?

1. The school identifies a 'problem of practice' on which observers will focus during the classroom observation.
2. After a brief orientation, observers divide into groups to observer in three or four classrooms, spending about 20 minutes in each.
3. During the observations, observers write down specific non judgemental and non evaluative notes about what teachers and pupils are saying and doing in relation to the problem of practice.
4.  Following the observations, observers and participant teachers then analyse the data looking for patterns, and suggestions for improvements in practice or school systems (Adapted from Teitel, 2014)






What are the advantages of School Based Instructional Rounds?

SBIR have a number of potential advantages over network based IR, which Teitel identifies as follows:

• In-house teacher observers are likely to understand their context better than guest observers, which reduces the time necessary for briefings.
• In-house teacher observers have a more intimate knowledge of both pupils and what is being taught.  This is likely to give the observers a better handle on what is being learnt by pupils
• Given the more detailed nature of the observation and with it being conducted by colleagues, it is likely to lead to a speedier take up of actions required for improvement
• Follow-up - there is an embedded aspect of SBIR which makes it more likely that the outcomes of discussions will be followed-up and discussed.  As such, SBIR are far less likely to be seen as standalone events, with no follow-up.

Possible pitfalls 

Teitel goes onto identify two possile pitfalls of SBIR, and these include: one, teachers who know each other well staying in the 'land of nice;' two, teachers may not be able to notice what's happening as certain things are taken for granted.

School Based Instructional Rounds and School Improvement

Teitel identifies 5 key design questions which schools may want to consider, if they are going to incorporate SBIR into their school improvement work.

• Why develop SBIR?
• Who should be involved at the school, and why?
• What school-based meddles makes the most senses, and how will it work logistically?
• How will SBIR be integrated into existing improvement structures?
• How can educators take advantages of the benefits of SBIR and minimise the downsides.? (Teitel, 2014)

Some final words

Lesson study may have been adopted in your school after an extensive evidence-led evaluative process and works and that's great.  Lesson study may not have worked in your school and setting, and that's a shame for both colleagues and pupils.  Lesson study still requires further study to judge it whether it should be scaled-up.  With that lack of evidence in mind, SBIR offers an alternative to lesson study, which may or may not be right for you school, and that's a decision which is up to you.

Next week, I will be looking at another alternative to lesson study - Strategic Inquiry.

References

Elmore, R. F., Teitel, L., Fiarman, S. E., Lachman, A., & City, E. A. (2009). Instructional Rounds in Education: A Network Approach to Improving Teaching and Learning. Cambridge, MA: Harvard Education Publishing Group

Teitel, L. (2013)  School-Based Instructional Rounds: Improving Teaching and Learning Across Classrooms, Cambridge, MA: Harvard Education Publishing Group

Teitel, L. (2014) School-Based Instructional Rounds : Tackling problems of practice with teachers ; Harvard Education Letter, Vol 30 (1)

The School Research Lead - Can research help reduce teacher fallibility?

Sometimes on Twitter you read a tweet which cannot be answered by using the Uzi like rapid fire of 140 characters, and which deserves a far more considered response.  One such tweet,  recently came across my time-line

Now I had two initial reactions to this tweet.  One, was annoyance that whoever thought up the question for debate had not read my recent posts on when to trust the experts, so would already know the answer  to the question (research can suggest to teachers what to  look at, it cannot tell you whether it should be adopted, as that's the teacher's or school's decision).  More seriously, I was annoyed at the quality of the question as it was unlikely to lead to anything more than a bit of Sunday morning posturing and setting fire to straw-men.   For me, a far more relevant and useful question would be: can research help reduce teacher fallibility?  Now this might not be as 'sexy' or 'roll off the tongue' as well as 'Goodbye Mr Chips all that ..." but on the other hand, addressing the of question how research can reduce teacher fallibility is far more likely to be useful in helping teachers improve their day-to day-practice.  So to help me examine  the relationship between research and teacher fallibility, I have turned to Atul Gawande's 2010 book - The CheckList Manifesto - 

Gawande draws upon Samuel Gorowitz and Alasdair MacIntyre's 1976 essay : Toward a theory of medical fallibility - which sets out to understand why we fail.   Gorowitz and MacIntyre identify three reasons as to why we may fail to achieve what we initially set out to do:

Necessary fallibility

Some things we want to do are beyond our capacity.  Complex systems - such as schools - are beyond all-encompassing generalisations due to the differences in the circumstances of each individual school and complex feedback systems.  In these circumstances, the best possible judgement may turn out to be incorrect - even if it is based on the best available evidence and best-practice implementation.

Ignorance

The second source of fallibility is ignorance, we may not know enough of what works in teaching, learning and education.   There are still things we have yet to learn, examples include: how to make the most of peer-tutoring;  how best to integrate IT into teaching and learning; how best to develop teachers effectiveness in the latter years of their careers; what is best practice in setting and mixed ability teaching.  In other words, teachers will get things wrong because there are certain things we have yet to discover the first principles of getting it right.

Ineptitude

The final cause of fallibility is ineptitude, this is where the underpinning knowledge exists but we fail to apply it.   For example, we know that graded lesson observations lack validity and reliablility for accountability purposes but we still continue, in some cases, to carry them out.  We know that just providing a homework 'mark' is unlikely to lead to pupil learning.

As such, two kinds of teacher fallibility are inevitable; necessary fallibility, which derives from the nature of complex system and ability to understand them; and our current ignorance about how things works.   So what are we to do to reduce unnecessary ineptitude.

The humble checklist

Gawande thought the solution to reducing ineptitude could be found through the use of check-lists and  subsequently led a World Health Organisation study on the development of a Safe Surgery check-list.  A 19 point check-list was eventually developed - which included checks at three stages, pre-aneasthesia, pre-incision and at the end of the operation.  This checklist was subsequently piloted in eight hospitals around the globe, four of which were in high-income countries and four of which were in middle to low-income countries.  As a result of introducing the check-list there was 36 per cent reduction in serious complications post surgery, with deaths falling by 47 per cent.  Based on approximately 4000 patients, more than 150 patients were saved from harm, and 27 of them from death.

So the question becomes :  can checklists be used in educational context to improve teaching and subsequently pupil learning?  Now it could be argued that there is insufficient agreement about what constitutes high quality teaching and learning,  that even is where there is agreement that it is not possible to distil it down to a simple - pre, during and post lesson check-list.  However, even if this view is accepted, I would argue that checklists can help teachers codify their own personal knowledge and experience.  In other words, can teachers reduce ineptitude by using checklists of what they already know derived from their own personal experience.

Checklists and reducing teacher fallibility

The potential of this argument is reflected in Harry Fletcher- Wood's new book - Ticked Off: Checklists for teachers, students, school leaders - which is to be published in January 2016.

Fletcher- Wood argues that on many aspects of teaching, teachers already know what they would like to happen.  However, the day to day pressures associated with teaching means it is possible to forget what is already known, unless that learning is embedded within a series of habits.   Forgetting what we already know is likely to take place in stressful moments, where things have not worked out in the way we would like or unexpected events have taken place.   Fletcher-Wood argues that building in a pause point just before the start of a lesson is a way to ensure that all the things that needed to be done before a lesson, have been done.  Alternatively, teachers may wish to develop a quick check-list of things to consider before issuing a sanction to a student - and in doing so ensure that any sanction is both fair, equitable and proportionate.  In other words, checklists provide a way of doing things that we would do if we were not overtaken by the pressures of the moment.  And of course, checklists can be applied to more than just these two situations, they can cover a range of issues, such as having a difficult conversation with a member of staff.

So going back to our initial question of whether research can help reduce teacher fallibility, my argument is as follows.  Some forms of fallibility are inevitable/necessary given the complexity of school environment.  Some fallibility comes from ignorance, we just don’t know enough of how to make some things work.  On the other hand, some teacher fallibility is the product of ineptitude, that is not doing those things that we know that work.  Research has shown that checklists have the potential to reduce unnecessary ineptitude within medicine.  Even if one accepts that research cannot tell what or how to teach, checklists can help teachers make the most of what they already know and prevent missed opportunities for learning.

Note 

I'd like to thank Harry Fletcher-Wood for contributing to this blogpost

References

Fletcher-Wood, H. (forthcoming) Ticked Off: Checklists for teachers, students, school leaders, Crown House Publishing, London

Gawande, A (2010) The Checklist Manifesto : How to get things right, Profile Books, London

Gorowitz, S. and MacIntyre, A. (1976) Toward a Theory of Medical Fallibility : The Journal of Medicine and Philosophy, 1976, vol. I, no. I., p 51-71