Research Leads Cambridge and What's Love Got to do With It - CUPID and the classification of research activity

On Saturday I attended another great session put on by the research ED dynamic duo of Tom Bennett and Helene Galdin-O’Shea.  There were several great speakers, each of whom made presentations full of opportunities for learning. 

• Philippa Cordingley told us about whole school approaches to using research within individual school, along with evidence about what works.
• Clare Hood, Abi Thurgood-Buss and Bethan Morgan shared their perspectives on leading research in schools, supported by the SUPER Partnership and Cambridge University.
• Vincent Lien made a compelling, and entertaining case for teachers to have free access to research journals.

Unfortunately, due to travel arrangements, I was not able to hear what I’m sure would have stimulating presentations from Caroline Creaby, Jonathan Sharples,  Ffion Eaton and Robert Loe.

Fortunately for me, though probably not for the audience, prior to departure I was able to to squeeze in my own presentation entitled – The School Research Lead and Star Trek:  to boldly go where others have gone before or what the School Research Lead is not Captain Kirk.  The rest of this post will focus one key issue from the presentation: how do you classify research activities. In doing so, I will be aided by the recent BERA-RSA Report on Research and the Teaching Profession. I will then proceed to classify different research activities to help School Research Leads develop a meaningful research agenda for their own school. As such, a sensible place to start is BERA-RSA's definition of the term research 

What do we mean by the term 'research'? 

The BERA-RSA inquiry intentionally devised a general and comprehensive definition of research.

By research, the report’s authors mean any deliberate investigation that is carried out with a view to learning more about a particular educational issue. This might take a variety of forms and be concerned with a range of issues, for example: the secondary analysis of published data on school exclusions, interviewing a range of colleagues about examination performance in the English Department, taking part in a national Randomized Control Trial concerned with the teaching of Mathematics, responding to a survey about teachers’ use of the internet to inform curriculum planning, working with a university department of education on a study into teachers’ use of new technology. 

Setting aside issues arising from the inclusive and wide-ranging nature of this definition of research, there is value in classifying the activities encompassed within the definition. Accordingly, Table 1 classifies different 'research' activities in the following five categories: consume, use, produce, involvement and disseminate .

Table 1 – Classification of Research Activities – The CUPID Model 

Consume –  involves reading texts, searching the Internet, with a critical focus and includes activities such as, the production of critical synopses of texts or discussions at regular Journal Club meetings. Most importantly, this is a state of mind of seeking out research and subsequently engaging in a meaningful and thoughtful critique.

Use –  is where evidence-based practice comes to the fore – where colleagues use the best available evidence from a variety of sources – academic, school, pupils and other stakeholders alongside professional experience – to make judgements to bring about changes in practice for the benefit of pupils, colleagues, and other stakeholders.

Produce – this could involve undertaking action research projects, supported experiments or enrolling on masters or doctoral degree programmes. Alternatively, the collation and presentation of data is also included in this section. However, the main purpose of ‘producing’ research is not the generation of new knowledge, but rather developing the capacity of colleagues to engage in ‘disciplined inquiry’, a topic I will return to in a future post.

Involvement  - this is when a school or individual colleagues engage in a research project either in conjunction with a higher education institution, be a site for a randomised control trial or answer survey requests from various bodies engaged in their own research. This also includes - department, school or college-wide activities designed to look at institution wide issues – where some colleagues are participants rather than active researchers.

Disseminate– an essential part of the research process as evidence, data analysis, discussion and recommendations need to be subject to a process of peer review. This does not mean the ‘research outcomes’ need to be published, but rather there is an open-ness and transparency with allows colleagues and others to explore the assumptions underpinning the research. It is necessary to check- for confirmation bias, and whether the researcher its over-claiming the reliability and validity of the findings.

All classifications, particularly those based on 2x2 boxes, have their limitations. Indeed, the CUPID model does not address the ‘process’ issues associated with creating a research and evidence-based culture (thanks to Lisa Pettifer @Lisa7Pettifer for drawing this to my attention). With that in mind, a future post will explore the process issues associated with creating a research-based school culure.

Nevertheless, the CUPID model provides a simple mechanism for thinking about the range of activities associated with research. If, by allowing colleagues to identify activities fit for their context and school, the model makes research activity more likely to be undertaken, then CUPID will have more than served its purpose.

  

Finally, for those of you who wish to wish to access today's presentation I am sure this will soon be made available on the researchED website.

Next week I will revisit the PICO question format, and the discussions which took place on this topic.

The School Research Lead and Developing Critical Analysis

In a recent post I wrote about the work of Wallace and Wray (2011) who produced a range of resources to support both critical reading and writing.  In particular, I shared their 5 question structure for creating a critical synopsis of a text, which  I suggested could be used as a development activity for a Journal/Book Club. In this post, we will expand upon the work of Wallace and Wray and consider 10 key questions for use in critical analysis.  But first let's recap the 5 questions suggested for producing a critical synopsis of a text

A  Why am I reading this?
B  What are the authors trying to do in writing this?
C  What are the authors saying that is relevant to what I want to find out?
D  How convincing is what the authors are saying?
E  What can I make of this?

Ten Critical Analysis Question(s)
Wallace and Wray suggest 10 further questions to engage in a much more critical and evaluative analysis of the text.

1. What review questions am I asking of this text?
2. What type of literature is this?
3. What sort of intellectual project is being undertaken?
4. What is being claimed that is relevant to answering my review questions
5. To what extent is there backing for claims?
6. How adequately does any theoretical orientation support claims?
7. To what extent does any value stance affect claims?
8. To what extent are claims supported or challenged by others' work?
9. To what extend are the claims consistent with my personal experiences?
10. What is summary evaluation of the text in relation to my review question (Wallace and Wray, 2011 p 109)

The 10 questions can then be mapped against the 5 questions used to create to create the critical synopsis and this allows us to generate a far  richer and more critical analysis of the text.

A Why am I reading this?
1. What review question am I asking of this literature?

B What are the authors trying to do in writing this? 
2. What type of literature is this?
3. What kind of intellectual project is being undertaken?

C. What are the authors saying that’s relevant to what I want to find out?
4. What is being claimed that is relevant to answering my review question?

D How convincing is what the authors are saying?
5. How far is there backing for claims?
6. How adequate is any conceptual or theoretical orientation to back claims?
7. How far does any value stance adopted affect claims?
8. How far are claims supported or challenged by others' work?
9. How far are claims consistent with my experience?

E In conclusion, what use can I make of this?
10. What is my overall evaluation of this literature in the light of my review question?

As such, the major change is in the level of detailed critical analysis relating to questions D – changing from one general evaluative question in the critical synopsis, with little or indication of what to look for, to five questions you could be checking the text for if you do the full critical analysis. 

Wallace and Wray then go onto identify a range of sub-questions for each of our 10 critical analysis questions, for example, for question 1 - 'what review questions am I asking of this text' - has the following subquestions:

• What is my central question?
• Why select this text?
• Does the critical analysis fit in with my investigations with a wider focus?
• What is my constructive purpose ( Wallace and Wray, p237)

Furthermore, in order to help colleagues develop their skills levels , Wallace and Wray have produced a template for conducting a critical analysis which can be found using this link. In addition, they have produced a number of work examples of critical analyses.

However, what I hope becomes clear from both this post and my previous post is the need to have a well-formulated and answerable question, as this will provide a clear framework and starting point for any critical analysis.   Accordingly, these resources provide an extremely useful starting point to support School Research Leads in their task developing colleagues's skills as critical consumers of research.   Indeed, by jointly undertaking critical analysis of key texts - say in a Journal Club or through personal reading - this will not only build capacity in consuming research but will also make it more likely that formal academic research can be put to good use.   This type of work may not be as exciting as initially conducting school research projects, though it will lay the foundation for much more productive future school evidence-based activity.

In a future post I will be building on the both 'critical synopses' and critical analysis' of texts  to explore is meant by a Critically Appraised Topic.

Reference
Wallace, M. and Wray, A (2011) Critical Reading and Writing for Postgraduates (2nd edition), Sage, London

Declaration of interest
Mike Wallace was my doctoral supervisor and remains a  close personal friend and has also contributed to this blog post.

The School Research Leads and Educational Prescriptions - a tool to support the development of evidence-based practice

As noted in previous posts, asking well-formulated questions is fundamental to effective evidence-based practice.  In supporting the development of a school cultures which have the effective use of research and evidence at their core, school research leads need to engage in processes which identify opportunities for generating well-formulated questions, with the answering of such question supporting the needs of individuals or groups of students.  In evidence-based medicine Strauss et al (2011) have identified a number of ways in which asking of good questions and in this post I will be looking at the use of educational prescriptions in aiding the development of evidence-based practitioners.

What is an Educational Prescription?

One of the challenges in developing evidence-based practice is to ensure that naturally occurring opportunities for learning are taken advantage of.  In this instance, a School Research Lead or other member of staff - may identify a particular problem or issue to be addressed - but there is a danger that both the question and answer are lost in the day to day business of the school.  One way of capturing the 'question' is through the use of an educational prescription which records the question, who is responsible for developing the answer and the deadline for the completion of the task. Accordingly, an educational prescription consists of the following:

• A specification of the learning problem that generated the question.
• A re-statement of statement of  the problem, as a well formulated and answerable question.
• A clear statement as to who is responsible for answering the question.
• A deadline for answering  the questions - bearing in mind the learning needs of students or staff (taking into account the urgency of the clinical problem that generated it).
• A clear articulation of the steps involved in answering the question. (adapted from Strauss et al 2011, page 23)

Possible Educational Prescription Template

Pupil/Class/Year Group :	Member of staff:
Problem : How would you describe the situation?
Intervention : What are you planning to do with your pupils/class/year group?
Comparison. What is the alternative to the intervention (e.g.different intervention)?
Outcome : What are the anticipated effects of the intervention?
When will educational prescription be followed up?
Check-list prior to future discussions: How did you go about searching for relevant evidence? How did you take into account pupil/student views? What were the outcomes of your search strategy? How valid and reliable is the evidence you found? Can this evidence be applied to the current problem? How useful was this process ?

Adapted from Straus, S.E., Glasziou, P., Richardson, W. S. & Haynes, B.R. (2011)  Evidence Based Medicine : How to practice and teach it, (4th edition), Churchill Livingston page 24 

How can  Educational Prescriptions be used?

Depending upon where the school is the development of an evidence and research-informed culture these educational prescriptions could be used in a number of differing settings: be it case-reviews of individual students, departmental meetings, or end of year review and evaluations.   Alternatively, they may be used by individual members of staff as a presentation device in order to share with colleagues the essence of the approach.

Tips for using Educational Prescriptions

• Produce 2 or more copies - one for the School Research Lead - and other for those staff who undertaking the review
• If connected to a local higher educational institution- use the prescription to engage in meaningful dialogue with the HEI librarians
• Try and use them as part of day-to-day teaching and learning rather than one-off events.
• In a Journal Club or other School Research Leads sessions ask colleagues to write out educational prescriptions to help develop their skills in developing well-formaulated questions.

And finally

There are no short-cuts in the developing the skills in developing and asking well-formulated questions.  It will invariably take time and both the school research leads and colleagues will no doubt make mistakes.  However, it is that very process of making mistakes and learning in the process which enhance colleagues skills and expertise at being evidence-based practitioners.

Reference
Straus, S.E., Glasziou, P., Richardson, W. S. & Haynes, B.R. (2011)  Evidence Based Medicine : How to practice and teach it, (4th edition), Churchill Livingston

researchED Sydney - Some initial reflections

On Saturday I had the privilege of both attending and speaking at ResearchED Sydney.  As with all these events the best part was meeting some wonderful individuals.  Indeed some of the speakers were just staggering in their enthusiasm, commitment and dare I say it professionalism.  Some of the highlights of the day included:

• The fabulous Deborah Netolicky and Janelle McGann from Western Australia sharing their work on a research-based model of teacher growth and creating a community of self-directed learners.
• The wonderful Michaela Pinkerton from Auckland, New Zealand talking about the work of her school in developing a research focused model of professional learning
• The boundless enthusiasm of Corinne Campbell from Castle Cove Public School who talked through an approach to developing a school homework policy, which showed how different sources of evidence - research, personal experience, stakeholder views and school data - could be brought together to bring about meaningful and agreed change.
• The larger than life Brett Salakas (and part-time magician)  from Sydney who showed how the classroom was without physical limits and talked about how to make global classroom real.
• The quietly authoriative Pamela Ryan who shared a very personal account of her experience as a leader and what this meant for her as she articulated a personal theory of leadership.

Of course at these events it's not just the presentations and speakers who provide much of interest but also other attendees who provide fascinating insights into other educational systems.  However, as with every researchED event I have attended, researchED Sydney provided an incredible stimulus for my own thinking about research and its' relationship to classroom practice and school leadership, so here goes.

1. There is much work to do to de-toxify evidence-based practice 'brand'.  Too often critics of evidence-based practice or evidence-based education fail to recognise, or deliberately chose to ignore,  that evidence-based practice draws on four sources of evidence - academic research, stakeholder views, organisational evidence and professional judgement.  
2. With the above in mind, there is also a lack of a shared understanding amongst teachers and educators about what we mean terms such as research, disciplined inquiry or evaluation.  This lack of a shared understanding, or at the very least a shared understanding of differences in approach, makes constructive dialogue on a range of issues more difficult
3. It's important to distinguish between the following : consuming research for understanding; using research for improvement; and finally conducting research for the purposes of theory testing and development.  Given the knowledge, skills and experience required for all of the above - for me  the role of school research lead should focus of supporting colleagues consume research for understanding and applying research for improvement.   If 'research' is conducted within schools the emphasis should be on evaluation and improvement of practice, rather than attempting the development theories or generalisations for broader use.

Nevertheless, the researchED movement is at a beginning of a process - I was going to use the  Jword - and there are bound to be challenges as the movement matures from a scrappy insurgent to an essential player in educational change in countries all over the world.    However, before I set off back home on my own journey the least I can do is to thank everyone involved in researchED Sydney for making the event so memorable, and which I'm sure will make the 24 hours watching film after film home all the more bearable.

PS
Tom Bennett put me on as one of the last speakers on the Saturday afternoon, he probably thought due to jet lag I would think it was a morning session!

Making the work of the School Research Lead matter

Recently I wrote about some of the misconceptions and pitfalls that could be associated with the role of the school research lead.  In this post, I'm going to focus on how we can make the work of the school research lead matter.  In doing, so I will be drawing upon the work of Bent Flyvbert and his 2001 book Making Social Science Matter : Why social inquiry fails and how it can succeed again.  In this short blog post I will not be able to capture the full-range of Flyvbert's stance but I hope I will do enough to encourage you to read his fascinating book.

Flyvberg develops a conception of social science based on the Aristotelean concept of phronesis which can be translated as practical wisdom or sound judgement.  Phronesis involves more than both knowledge (episteme) and know-how (techne) but involves making decisions and judgements in a manner of virtuoso social and political actor (page 2).  As such 'phronesis' involves an analysis of values and "things that are good or bad for 'man'.  Phronesis require an undertstanding of both general principles and the specific context, and in doing so requires individuals to make judgements and choices, balancing differing perspectives, considerations and informed by experience. Accordingly, trying to reduce social science theory to just two components - knowledge and know-how is inappropriate, given the inherent involvement of phronesis - judgment in our day to day lives. Given the increased discussion we have on using  'educational evidence' the notion of phronesis is essential in our role as practitioners.

At the very core of Flyvberg's model is the notion of virtuosity or expert behaviours, as such, Flyvberg goes onto explore in some detail the so-called Dreyfus model of human-learning which identifies five levels of the human learning process:

• Novices act on the basis of context-independent elements and rules
• Advanced beginner also use situational elements, which they have learned to identify and interpret on the basis of their own experience of a particular situation
• Competent are characterised by the involved choice of goals and plans as a basis for their actions.  Goals and plans are used to structure and store masses of both context-dependent and context-independent information
• Proficient performers identify problems, goals and plans intuitively from their own experientially perspective.  Intuitive choice is checked by analytical evaluation prior to action
• Finally 'expert's behaviour is intuitive, holistic and synchronic, understood in a way that a given situations releases as picture of the problems, goal, plan, decisions and action in one instant and with no division into phases.  This is the level of true human expertise.  Experts are characterised by effortless performance, unhindered by analytic deliberations. (Flyvberg p21)

Flyvberg argues there would be appear to be a significant leap from levels one, two to the higher levels of proficiency and expertise.  In doing so, there is a change from a reliance on rules to a greater reliance upon intuition.  Flyvberg subsequently argues that the main aim of an approach to social science using a phronetic stance is carry out analyses and interpretations of social situations - which take into account values and the interests of society as whole and which provides a guide for action.  The consequence of this stance is that the core questions for a phronetic approach to social science - and dare I say education - are the following four questions.

1. Where are we going?
2. Is this desirable?
3. What should be done?
4. Who gains and loses; by which mechanisms of power? (Flyvberg p 61)

Flyvberg argues that no individual is going to have the knowledge, know-how and practical wisdom in order to provide all the answers.   Flyvberg concludes his book with a call to arms to make social science matter, suggesting that social scientists need to do three things.

1. Forget trying to emulate the natural sciences by building cumulative and predictive theory.
2. Social scientists must take up the problems that matter at a global, regional, national and local community levels.  
3. Communicate the outcomes of the research in a manner which is meaningful to a wider audience, in particular the community as a whole.

So what does this all mean for the school research lead, well for me I think there are a number of implications.

• Research leads need to recognise that the purpose of research and evidence-based practice within schools is to support the development of 'practical wisdom' within the school rather than attempting to emulate the 'research' undertaken in universities.
• Research leads should give thought as to what are the aims of a research or evidence informed school, is this desirable and what should be done?
• Research leads should acknowledge that through the process of 'research/evidence-based activity there will be winners and possibly losers - which may include pupils and colleagues. 
• Research leads should strive to ensure that the outcomes of research/evidence-based activity is communicated to colleagues in a way which is meaningful and which informs practice, 
• Research leads should focus school research efforts on things that matter to their pupils, colleagues, school and local community.
• Research leads need to recognise that to become a virtuoso in such a role will take time and will involve a process of going from novice to higher levels of expertise, and maybe the best we can hope for is for research leads to be competent rather than proficient or expert as researchers.

I hope this post will stimulate some discussion about both the role of the school research lead but also what we can expect a school-based research activities to achieve i.e. better judgements with improved outcomes for pupils, colleagues and the school as a whole.

Next week I will be reporting from ResearchED Sydney which I am lucky enough to be able to attend 

The School Research Lead and Setting up a Journal Club

In my last post I began to explore the use by School Research Leads of Journal Clubs to help colleagues engage in the critical appraisal of texts.  In this week's post I will again be using the work of Straus et al (2011) in the realm of evidence-based medicine to help School Research Leads set up and successfully run a Journal Club.

In setting up and running a Journal Club there would appear to be three broad sets of issues to address.

1. What are the learning goals of the Journal Club?
2. Strategies to support the introduction of the Journal Club?
3. How should an individual Journal Club session be conducted?

What are the learning goals of the Journal Club

There are at least three core learning goals for a Journal Club

• Learning about the best evidence available to support the needs of pupils/staff in addressing a particular learning need.
• Learning about important new evidence that may lead to a change in pedagogy, systems or approach to leadership and management
• Developing skills at being better consumers of research in order to become more effective evidence-based practitioners.

None of the above are mutually exclusive, although the focus will very much depend upon the needs of the pupils, department, school or individual member of staff.

Strategies to support the setting up of the Journal Club 

There are a number of different strategies and tactics that can be used to make the Journal Club effective and these include:

• Building a coalition of senior colleagues and peers who will help you achieve your aims.
• Securing some appropriate resources to allow the Journal Club - be it a small-budget to access materials, tea, biscuits, a decent room and time slot which is not 4.30pm on a Friday afternoon.
• Be clear that this is a collaborative endeavour which will engage colleagues in group learning activities.
• Preparing some basic materials for the first few sessions e.g.  materials which would easily lend themselves to a critical appraisal.
• Developing your own skills in facilitating discussions with colleagues.

Running a Journal Club session 

Whether all of the following can be accomplished in a single session will depend upon the time and other resources which have been made available. Indeed, it may be possible to run a cycle of meetings which where in any one meeting only two of the three components are covered.

• Part One - establish the learning needs to be covered in the future - be it a specific pedagogic, assessment or leadership and management issue (please note I don't regard research/evidence-based practice the exclusive preserve of front-line teachers).  This could involve accessing existing evidence or reviewing new evidence which has been published.  Alternatively, a colleague may have specific research/evidence-based practice skill issue which they wish to develop.
• Part Two - this involve the sharing of the outcomes of the search for evidence which was based on the learning need identified in the previous session - this can take the form of photocopies of summaries, original articles or other evidence.  Remember, evidence-based practice does not just rely on 'scientific' research evidence, but also draws upon internal school data and the views of stakeholders.
• Part Three - this is the main part of the session and involves the critical appraisal of the evidence presented.  As mentioned last week this might involve discussions around:
• Why are we reading this?
• What are the authors trying to do in writing this?
• What are the authors saying that is relevant to what we want to find out?
• How convincing is what the authors are saying?
• What can we make of this? (adapted from Wallace and Wray, 2011)

Of course, this is not a strict menu or template about how to run and conduct a Journal Club. On the other hand, I do hope it provides some food for thought and a useful stimulus in how to run effective sessions.   Finally, if I was to offer one piece of advice it is essential to make sure colleagues have sufficient access to articles, evidence or books etc - for without that -  all you do is meet with frustration.

References

Straus, S.E., Glasziou, P., Richardson, W. S. & Haynes, B.R. (2011)  Evidence Based Medicine : How to practice and teach it, (4th edition), Churchill Livingston.
Wallace, M. and Wray, A. (2011) Critical Reading and Writing for Postgraduates (2nd edition), Sage, London

The School Research Lead and Journal Clubs - Writing a Critical Synopsis of a Text

In my recent posts on the role of the school research lead I have argued that one of the key tasks of such a role is to help colleagues become critical consumers of research.  Indeed, the critical appraisal of evidence, both research texts and other sources -is an essential component of becoming an effective evidence-based practitioner.  One of the ways in which these skills can be developed is through is the creation of a Journal Club, where colleagues regularly get together to critically review journal articles.  In the rest of this post I will look at one particular method of developing skills in the critical appraisal of evidence, which can be used as an integral part of a Journal Club or independently by colleagues looking to become more critical consumers of research.

WRITING A CRITICAL SYNOPSIS
Mike Wallace and Alison Wray in their book Critical Reading and Writing for Postgraduates (second edition) identify 5 basic questions which form the basis of a critical synopsis, which will also form the starting point for a subsequent critical analysis. The FIVE basic questions are:

1. Why am I reading this?
Am I trying to formulate an answerable question.? Is this reading going to help me gain a better understanding of background or foreground questions?  Is the reading for school-based action or for the purpose of academic study? Is this reading about helping me develop my skills as a critical reader.

2. What are the authors trying to do in writing this?

The authors my have a range of differing purposes including : criticising policy and practice; providing a synthesis of others' work in the area; reporting on their own research activities; or they maybe engaged in theory development.

3. What are the authors saying that is relevant to what I want to find out?

What is the text actually about and what do the authors say about it?  How does text the relate to your own interests: is the text directly linked to your interests or key question(s); is the text indirectly related to your key interests/questions; or does the text provide a tangential but interesting perspective on the issues/questions at hand?

4. How convincing is what the authors are saying?

How has the data been collected and analysed?  Does the data directly lead to the analysis or conclusion being drawn, or are there 'leaps' in logic of the argument?

5. What can I make of this?

Is this text worthy of further of further reference or deeper analysis, or is it something which will only be referred to in passing.  Alternatively, is the text interesting but on the other hand not directly relevant to your core interests.

At first, this process may appear to make the reading of appropriate texts more time consuming.  On the other hand, by going through a structured process it make the reading itself more beneficial. In the beginning, it may be necessary to produce a written critical synopsis,  particularly if the reading is being undertaken for more formal processes, such as preparation for the writing of an academic essay or dissertation.

To conclude, being a school-research lead or evidence-based practitioner is about developing the skills to ask better questions, and in doing so become a more proficient consumer of educational research.  A Journal Club which engages in the critical appraisal of a text or text(s) is one approach which can assist colleagues in meeting in this aim.  In my next two posts I will look in more detail about the mechanics of running a successful Journal Club.

USEFUL RESOURCES
A link to resources produced by Wallace and Wray which will help you produce a critical synopsis can be found using this link.