The school research lead and making the most of abstracts

There are a number of reasons why it is particularly important for both school research champions and evidence-based school leaders to make good use of the abstracts found at the beginning of published papers.  First, the full text of the majority peer-reviewed published papers sit behind pay-walls, which makes it expensive for the evidence-based school leader to access research.  As such abstracts provide a summary of the paper indicating the topics covered, the approach adopted and the main claims made, without the expenses of paying for access to the full-paper.   Second, because so much research is published each and abstracts are very short, often no more than 150-200 words, skilled reading of research abstract makes it much easier to keep up with the latest research in your area of interest.  Finally, and most importantly as (Wallace and Wray, 2016) note the very brevity of abstracts, encourages the development of critical reading.  The relative lack of information in an abstract, will facilitate the development of your skills as a critical reader, by forcing you to ask questions about what additional information is required for you be convinced by the argument underpinning the research.

To help us make the best of research abstracts  (Wallace and Wray, 2016)  have identified  a number of questions that may wish to ask of a research article, which uses statistics.

Yes, but what did mean by that term?

You have told me what sort of informants you used, but how many were they, and why did you choose them?

You have told me what you wanted to find out, but why was it interesting and important to find it out

You said that a lot of research has been done on this topics, but what are the key studies I should be checking out?

You said that you have found a significant differences between your two experimental groups, but what statistical test were you using? p30

It may well be that when you read the full paper all of your questions are answered. On the other hand, it may be that a few of your questions are not answered.  (Wallace and Wray, 2016) argue that you are more likely to identify these unanswered questions when you are looking for answers.  Furthermore, as your questions reflect your current knowledge, skills and expertise, then it is likely that these questions are extremely relevant to your problem of practice and associated well formulated and answerable question.

A worked example

(Wallace and Wray, 2016) go on to suggest that the best way of illustrating of how to critically read an abstract is through a worked example. So here is an example using an article which looks at value, ethics and decision-making.

Text	Comments
Source: Educational Management Administration and Leadership 2015 Vol 43 (2) 198 - 213	This journal focuses on educational leadership, with original contributions from educational researchers from around the world
Title: Value(s)-Driven Decision-Making: The Ethics Work of English Headteachers within Discourses of Constraint	What does the author mean by value(s) driven decision-making What is the ‘ethics work’ of headteachers? What sector(s) are the headteachers drawn from
Author: Linda Hammersley-Fletcher, Institute of Education, Manchester Metropolitan University, Crewe Green Road, Crewe, Cheshire, CW1 5DU, UK.	What’s the author’s main purpose in writing the paper?
This article considers the experiences and perceptions of practising English headteachers and the tensions that they face when juggling government prescription and government initiatives, which may be antagonistic to their educational values and beliefs	How long have the headteachers been practising What are the tensions arising from government prescription and initiatives What do we mean by educational values and beliefs. How might these values and belief be at at odd with government initiatives
Managerial control over teachers work has been particularly acute and destructive to ‘human flourishing’.	What is meant by managerial control? What is the work of teachers? What is meant by ‘human flourishing’?
Headteachers have a moral and ethical responsibility for the welfare and education of pupils.	What are the moral and ethical responsibility of headteachers? Is there a difference between the education and welfare of pupils?
Such professional ethics oblige the professional to seek the good of the pupil and therefore good is viewed as intrinsic to the work of an educator.	Is there a difference between professional and educational ethics? Is the intrinsic role of headteachers confined to seeking the good of the pupil?
Thus headteachers are directly involved in negotiating between sometimes contradictory imperatives and drivers.	What sort of imperatives are there? Do these imperatives changes over time? What scope do headteachers to negotiate contradictory pressures?
How then does the headteacher cope with what Colley refers to as ‘situated ethics work’?	What is meant by ‘situated ethics work’ What coping strategies do headteachers adopt
This article presents data derived from written responses from 10 headteachers that begin to open up this question.	How were the headteachers selected? Were the headteachers given a proforma to respond to? How were the written responses analysed?
I argue that it is not uncommon for people to weaken in their values-driven stance when under great pressure	How frequent is ‘uncommon’ ? What types of pressure are headteachers under? How does the pressure change the decision-making of headteachers? What other decisions would they be taking?
It is however important to recognize the extent to which educational values are constrained by neo-liberal value-based market agendas in order to continually question and re-evaluate what is happening within education rear- ticulating this for the benefit of pupils.	What are neo-liberal value based market agendas? How are headteachers re-evaluating what is happening? In what ways are these values being re-articulating the meet the need of pupils

It should be noted that this a fairly eclectic list of questions, which could be added or subtracted from depending your particular problem of practice.  As such, you should not see the above as a definitive list, rather it should be seen as an initial stimulus to develop your own thinking.  

Some final words
Often when thinking about research evidence we mention that teachers haven't got the time to undertake research, or they don't have access to current research, or maybe, they don't have the skills to critically consume and engage with research. The use of structured approaches to the critically reading of abstracts is something that should be possible for all teachers to engage, and would be a worthwhile activity in school journal clubs. And if it whets the appetite for further reading so much the better.

Reference

WALLACE, M. & WRAY, A. 2016. Critical reading and writing for postgraduates, Sage.    

Teacher research literacy and the components of an argument

The question of what is meant teacher research literacy often appears in my Twitter timeline, and recently this tweet caught my attention.

Beth’s tweet got me thinking again about research literacy, as I had written about it earlier in the year http://evidencebasededucationalleadership.blogspot.com/2016/09/teacher-research-literacy-what-does-it.html  where I came up with a pretty long-list of cognitive strategies that research literate teachers could and maybe should use.  However, as I reflected on the feedback on that post, it became clear  that we need to avoid coming up with long lists, as no one will either remember what’s on the list or subsequently use it.  So it struck me was that if we are to ensure ALL teachers are research literate, then we need to make research literacy as simple as we can, but no simpler (to probably misquote Albert Einstein)

So with that in mind, I thought it worthwhile to turn to the work of  (Wallace and Wray, 2016)and their book Critical Reading and Writing for Postgraduates (3rd edn).  As for me they clearly and concisely get to the heart of what is meant by research literacy i.e. the ability to identify the components of an argument, and how to spot incomplete or flawed arguments

What are the components of an argument?

(Wallace and Wray, 2016) state:

An argument consists of a conclusion (comprising one or more claims that somethings is, or should be, the case) and its warranting (the justification for why the claim or claims in the conclusion should be accepted).  The warranting is likely to be based on the evidence from the author’s research or professional experience, or it will draw up on others’ evidence as reported in the literature. p36

An example of an argument, which is increasingly accepted within English schools is Coe’s (2014) view of graded lesson observation, which can be summarised as follows:

• Graded lesson observations should not be used for high stakes accountability purposes as there are substantive issues with both the reliability and validity of lesson observation grades.

There are also other examples of where we can identify the conclusion and warranting of the current educational arguments

• Research shows that pupils as they get older are much more heterogeneous, as such studies should take this into account when using effect sizes to judge the impact  of an innovation.

• Evidence shows that children of poorer parents tend to perform less well academically, than children of wealthier parents.  Therefore, any school accountability system should take this into account.

•  Evidence shows that there is a correlation between how school leaders spend their time and pupil outcomes.  Studies into effective school leadership should take into how school leaders spend their time on different aspects of leadership and management.

Illogical or incomplete arguments

So having identified the argument, you will now need to work out whether argument is flawed through, for example, drawing conclusions without evidence or insufficient warranting for the conclusion.  Table 1 adapted from (Wallace and Wray, 2016) gives examples of a range of flaws in arguments, which as critical consumer and uses of research you will need to be on the look-out for?

Table 1 Identifying flaw in arguments derived from different sources of evidence

Type of flaw in argument	Examples
Conclusion without warranting	The best teachers make the best heads of department
Potential warranting without a conclusion	In-house research shows that many parents often sign ‘consent’ forms without reading them.  School correspondence to parents can be confusing.
Warranting leading to an illogical conclusion	Early career teachers are not very experienced.  This indicates that they are not very good teachers.
Conclusion not explicitly linked to warranting	Evidence shows that out KS3 pupils are spending insufficient time on homework.  KS3 pupils should be set more homework.
Conclusion with inadequate warranting	Experienced teachers learn more effectively when they are given positive feedback.  An in-house school survey of  teachers over the age of 50,  indicated that the vast majority said they preferred praise to criticism

To conclude

This post is only a very brief guide to arguments and warranting.  As such, strongly I recommend that you have a look at Wallace and Wray's Critical Reading and Writing for Postgraduates and that you put on your Christmas list a  copy of Booths et al  The Craft of Research (4th end.).  After which you may want to set yourself the challenge of trying to Tweet both an argument and a warrant in less 140 characters.  I think it can be done, and it might make Twitter a much more informative space.