The school research lead and the FINER questions in life

In last week's blog we looked at how school research leads could help colleagues answer  So What questions. i.e.  Does your question really matter? What would happen if we didn’t answer your question?  Would it make a difference to the life chances of pupils or the well-being of colleagues?  Hopefully colleagues will now have given you a range of different questions that they wish to be answered. Unfortunately, - given competing demands for attention and resources - you will only be able support colleagues answer a relatively small number of these questions.  So in this post, we will look at a couple of ways of trying the prioritise the questions that matter.  We will then seek to answer the question, why it is important to take the time to formulate questions clearly?

How to ask the most appropriate question?

Straus et al (2011) have suggested a series of filters which could be used to identify the most appropriate question to ask in a particular situation.  I have adapted the suggested filters so they can be easily transferred to the setting of a school.

•       Which questions, if answered, will be most useful for your pupils/students/learners' well being academic or personal?

•       Which questions will be most useful for your subject leaders, heads of department in gaining a better understanding of the issues at hand?

•       Which questions will be most useful in helping to improve the department, school or college?

•        Which questions is are most likely to re-occur and will need to be revisited in the future
      Which question is most interesting to your as an evidence-based practitioner and contribute most to your personal professional development?

The FINER questions in life

Alternatively you may wish to use the FINER mnemonic developed by (Hulley et al., 2013) to help you think through what are the most important questions to answer.

•       Feasibility are there sufficient resources, be it capacity and capability – to adequately answer the question
•       I: Interesting : Is the question interesting to those given the task of researching the answer to the question.
•       N: Novel Is this a recurring problem/question or something which is new to the school and may become an on-going issue
•       E: Ethical Have ethical issues been identified and considered 
•       R: Relevant is it relevant to the school and is going to influence school policy and practice

And some final words

Being able to develop well formulated and answerable questions is a fundamental skill for an evidence-based practitioner.  However, for school research leads - it's not enough to be able to helping to well formulated questions, you will also need to be able identify those questions that really matter to the school and should be supported with time, attention and resources.

References

Hulley, S. B., Cummings, S. R., Browner, W. S., Grady, D. G., & Newman, T. B. (2013). Designing clinical research: Lippincott Williams & Wilkins.

Straus, S., Glasziou, P., Richardson, S., & Haynes, B. (2011). Evidence-based medicine: how to practice and teach it. (Fourth Edition). Edinburgh: Churchill Livingstone: Elsevier.

The School Research and the So What question

As we approach the end of the academic year, school research leads will not doubt  be pulling together a programme of work for the coming year.  Inevitably,  colleagues will be putting forward a range of research questions which they hope to have included in the school’s research/inquiry agenda.  However, as harsh as it may seem the most appropriate initial response to these requests is to ask as  (Booth, Colob, Williams, J., & Fitzgerald, 2016) suggest: So What? Does this question really matter? What would happen if we didn’t answer your question?  Would it make a difference to the life chances of pupils or the well-being of colleagues?

In order to help your colleagues answer the - So What – question, we will borrow a three step process developed by (Booth et al., 2016) in helping researchers develop questions that matter.

1. Name the topic:  I am trying to learn/find out about ………..
2. Ask an indirect question about the topic in order to identify what you do not know about the topic
3. Answer So What? by motivating your question by asking a second indirect questions that explains why you asked your first indirect questions. 

Let’s have a look at a few worked examples

• I am interested in working on marking strategies because I want to find out the most time effective way of carrying out marking because I want to reduce unnecessary work load on teaching staff

• I am interested in children’s play because I want to find out the best way of incorporating play into my teaching as I want to increase children’s independence as learners

• I am interested in understanding the nature of parental engagement with their children’s learning so I can  enlist parents help in increase their children’s effectiveness as readers.

• I am interested in looking into incremental coaching as I want to find a way to develop teachers’ pedagogical skill so as to improve pupil outcomes.

• I am interested in understanding the role of school research champions because I want to find out how best to access academic research because I want teaching and learning to be informed by best currently available research knowledge.

Now just because a colleague cannot immediately answer these three questions, does not mean that his or her topic of interest is without merit.   However, what it does suggest that the idea or question is not yet robust enough to be allocated resources in its support.  Moreover, further refinement of the response to - So What – may lead to a situation where senior leaders care sufficiently about the question – as to make sure it becomes a future priority. 

Reference

Booth, W. C., Colob, G., G, Williams, J. M., J., B., & Fitzgerald, W. T. (2016). The Craft of Research (Fourth Edition). Chicago: The University Of Chicago Press.

The school research lead, bullsh.t and what to do about it

One of the many challenges facing school research leads and other teachers who use social media – be it Twitter, Facebook and/or blogs – is to try and make sure they do not fall victim to bulls..t.   So in this post we will use James Ball’s recently published book Post-Truth: How bullshit conquered the world to help us both what identify bulls..t is and the strategies to adopt so we are less likely to fall victim to it.  However, it should be noted that bullsh.t is not confined to social media and may often be heard, dare I say it, on a regular basis in school staff rooms and senior leadership team meetings

What do we mean by bullsh.t?

Ball draws upon the work of Harry Frankfurt in order to distinguish between lies, untruths and bullsh.t and summarises Frankfurt’s argument as: to tell a lie, you need to care about some form of absolute truth or falsehood, and increasingly public life is run by people who don’t care much either way – they care about their narrative.  (Ball, 2017) p6

Ball goes onto cite Frankfurt who concludes

Someone who lies and someone who tells the truth are playing on opposite sides, so to speak, in the same game. Each responds to the facts as he understands them, although the response of the one is guided by the authority of the truth, while the response of the other defies that authority, and refuses to meet its demands.

The bullshitter ignores these demands altogether.  He does not reject the authority of the truth, as the liar does, and oppose himself to it.  He pays no attention to it all, by virtue of this, bullshit is a greater enemy of the truth than lies are.

Ball argues that a bullshitter will say what works to get the result they want, regardless of whether it is true or not.  Ball goes onto say that in his view that this serves as an accurate description of many political campaigns.   No doubt there are some Tweeters and other social media commentators who would say that given the performative nature of external school accountability regimes this is also an accurate description of much educational leadership in schools and multi-academy trusts.

So how are you to protect yourself from bullsh.t?

Ball suggests a number of strategies – aimed at social media – which you could use to prevent yourself falling victim to bullshit.   However, I argue that these strategies can also be extended to the ‘real’ world and be used in other settings.

Burst your bubble

Ball argues observes that many of us live in an online bubble – where we follow and communicate with people who have the same views as ourselves.   Maybe we need to make sure we follow and meaningfully engage with people who have different views than ourselves.  With the aim of this engagement being to understand the views of others, rather than seeking to convince other as to veracity of our own arguments.   Indeed, in the physical world, the could involve secondary teachers engaging in dialogue and conversation with colleagues in either or both the primary or further education sector.  Alternatively, it may be as simple as just spending some time with a colleague who specialises in a subject to completely different to your own, say a science teacher with an art teacher.

Engage system two

Ball makes reference to System One – instinctive reaction and System Two – more deliberative thinking.  With System One thinking Ball argues that we are far more likely to quickly RT and share Tweets and other social media which are consistent with how we see things.  However, this risks spreading bullsh.t – as we have not subject these Tweets or blogs to appropriate critical scrutiny.  On the other hand, Ball argues, that if we subject some of this material to more critical scrutiny and asking - is there evidence to back up the claims being made; where has the evidence come from; do the authors recognise the limitations of the claims being made – then it means we are far more likely not so share bullshit.

Learn some stats

Ball argues that if you have a very basic grasp of statistics then you are far less likely to fall prone to bullsh.t.  In the context of your work as a school research lead it would be useful to get to grips of the notion of effect sizes and their limitations.   A useful introductory text can be found in the work of (Ellis, 2010) It would probably also be worth dipping into an introductory texts on statistics such as (Cumming & Calin-Jageman, 2017) – which is particularly useful at looking at p-values and confidence limits  and which is supported by a range of videos which can be found on YouTube.

Treat narratives you believe in – just as sceptically as those you don’t

Just because a narrative you believe in is being promoted in social media does not mean that it is right.  We may see tweets or posts which sees coaching as the magic bullet of school improvement or tweet which supports the use of Lesson Study or that the use of research evidence is the key to improving practices in the classrooms.  On such occasions, we may need just to step back and say – is there an alternative view or views on this?  If so, what is it?  Are there any elements of these alternative views which are robust.

Try not to succumb to conspiratorial thinking

Ball argues that in a world of bullsh.t there is a tendency for there to be a rise in conspiratorial thinking – and that others are ‘out to get us’.  Where you are tempted to thinking conspiratorially – it is always worth remembering Hanlon’s Razor – which can be described in a number of ways. For example “Never attribute to malice that which is adequately explained by stupidity" or "Don't assume bad intentions over neglect and misunderstanding."

And some final words

School research leads have a particular responsibility for rooting out ‘bullsh.t’ in schools.  On the other hand, school research leads need to make sure they too are not proponents of other people’s bullsh.t and need to adopt an appropriately self-critical stance of their own work.  This may not be easy, but that does not make it any less vital.

References

Ball, J. (2017). Post-Truth: How bullshit conquered the world. London: Biteback Publishing

Cumming, G., & Calin-Jageman, R. (2017). Introduction to the New Statistics: Estimation, Open Science, and Beyond. Abingdon: Routledge.

Ellis, P. D. (2010). The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results: Cambridge University Press.