Friday, 24 August 2018

The school research lead, the three legged stool and how to avoiding falling on your backside

As a school research lead you will have no doubt spent some of your summer reading research articles – be it systematic reviews or randomised controlled trials RCTS - and thinking about whether the interventions you’ve read about will work in your school and setting.  Indeed, you may have been working on a PowerPoint presentation making the case for why some well researched and evidenced teaching intervention which has shown positive outcomes in other schools, should be adopted within your school.  So to help with you with the task of developing an argument - which begins with ‘it worked there’ and which  concludes ‘it’ll work here’ -   I am going to lean on the work of Cartwright and Hardie (2012).  However, before I do that, it’s necessary to: define a number of terms, the understanding of which are central to getting out the most of out of this blogpost; then, clarify the causal claim being made when we say something ‘works’.

Some useful definitions
  • Causal claim -  a claim in  the form  of "A was a cause of B". 
  • Causal role – how a change in A leads to a change in B
  • Support factors – what factors need to be in place so that a change in A will lead to a change in B
Clarifying what is meant by saying something works

Cartwright (2013) states that when arguing that ‘something works’ three quite different claims are often conflated
  • The intervention works somewhere (there); the intervention causes the targeted effect in some individuals in some settings
  • The intervention works: the intervention causes the target effect ‘widely’
  • The intervention will work here: the intervention will cause the target effect in some individuals in this setting (adapted from Cartwright, p 98)
As such, evidence that supports a claims something works somewhere/there, is not evidence for saying that something will work here.  Cartwright and Hardie (2012) claim that what connects ‘it works somewhere’ with ‘it will work here’ is an argument that looks like this.
  • The policy worked there (i.e. played a positive causal role in the causal principles that held there and the  support factors necessary for it to play this positive role there were present for at  least some individuals there).
  • The policy can play the same causal role here as there
  • The support factors necessary for the policy to play a positive causal role here are in place for at least some individuals here post-implementation p54 
In other words, if you want to show what worked (A) in another school or other schools in bringing about (B), you are going to need to show that the intervention can play the same causal role in your school – in other words, will a change in (A) trigger the same causal mechanism as it did in the other school or schools.  Second, you are going to have to show which of the support factors that were work, and which created the conditions for changes in (A) to bring about changes in (B) are in place and available in your setting.

As such, Cartwright and Hardie argue that the effectiveness argument is like a three-legged stool – it doesn’t matter how strong one leg of the stool maybe, if either of the other legs fails – then the stool will fall over.  Or put another way, it does not matter whether an intervention worked somewhere or causes the target effect widely – if either the causal mechanism or support factors are not in place – the intervention will not work here.

 Cartwright and Hardie (2012) p55 and the three legged stool


























What are the implications for you and your role as a school research lead?

The role of the school research lead/knowledge broker is not without significant challenges,  difficulties and which also has a ‘dark-side’ -  Kislov, Wilson, et al. (2016).  So anything that can help you to be clear about the evidence and what it means, and which stops you over-claiming what the research tells you -  should be useful.

Second, when colleagues come up to you and say ‘the research says intervention works – so let’s do it’ – it might be worth pausing and replying –‘that’s great news – so help me to be able to best help you – can you let me know more about – how the intervention worked and what was the context, within which it worked?’

Third, when reading research where there is little or no detail provided about the causal mechanism or support factors at work – then this research could be judged as being interesting but not necessarily that useful.  Further reading into the what support factors are necessary for the intervention to succeed will be required.

What next 

This is the first of a series of posts which are going to cover ground which I hope is relevant to the start of the academic year.  These posts will be on topics, such as; causal cakes; causal roles and mechanisms;  what do we mean by evidence-based practice; and, when does evidence travel.

References

Cartwright, N. (2013). Knowing What We Are Talking About: Why Evidence Doesn't Always Travel. Evidence & Policy: A Journal of Research, Debate and Practice. 9. 1. 97-112.

Cartwright, N. and Hardie, J. (2012). Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford. Oxford University Press.

Kislov, R., Wilson, P. and Boaden, R. (2016). The ‘Dark Side’of Knowledge Brokering. Journal of Health Services Research & Policy. 1355819616653981.







My new book due out October 2018 Evidence-Based School Leadership and Management: A practical guide

https://us.sagepub.com/en-us/nam/evidence-based-school-leadership-and-management/book257046

28 comments:

  1. I am very to read your blog. Thanks for the information.
    Education Consultant In Melbourne

    ReplyDelete
  2. Earn Online
    If you want to earn online then I will provide you Authentic platform without any Investment.

    ReplyDelete
  3. Hello this is John this information is really beneficial for everyone. i have also a information that i want to share with you all i found a website according to business analysis and there is Quality Analyst Job Placement in New York as well in that institute for more information visit here Quality Analyst Job Placement in New York

    ReplyDelete
  4. Great Blog and such a great information we love to read about traveling because we also belong to travel industry have look our services bike on rent in udaipur : visit here:- https://www.padharo.co/

    ReplyDelete
  5. Bangalore is so big city, and in this city many of the sex desire person live apart from their wife's, and
    due to their busy schedule they even don’t have time for their wife's, then escorts girls are the one ultimate option for them .
    and Jungligirl Bangalore escorts is one of the those agency where many these types of men come for sexual enjoyment.
    Bangalore Escorts Agency

    ReplyDelete
  6. Dedicatedhosting4u.com offers decent and persistent dedicated servers with fastest communication over the internet. The customers are provided with two choices, one is self-Managed and the other one is Managed hosting.
    https://www.dedicatedhosting4u.com/
    premier dedicated hosting services

    ReplyDelete
  7. Most of the blogs pretend themselves as most usable and updated blogs with new information but sometime truth might different. I want to share some facts related to this subject which will help people to enhance their skills.
    หนังสือพัฒนาตนเอง

    ReplyDelete
  8. These are pieces of very useful information that will be of great use for someone in the future. You can read progressively about Our inherent includes here.

    Thank you so much for Sharing an Informative Article!

    ReplyDelete
  9. thank you for sharing this blog and here I find honest thoughts .i enjoy reading this amazing blog .keep sharing such helpful information.
    vist

    ReplyDelete
  10. B.ED ONLINE ADMISSION FORM BY KAPOOR STUDY CIRCLE B.ED ONLINE ADMISSION MDU ROHTAK, KURUKSHETRA UNIVERSITY B.ED ONLINE ADMISSION, KASHMIR UNIVERSITY B.ED ONLINE REGISTRATION, B.ED ADMISSION FORM LAST DATE, IGNOU B.ED ONLINE ENTRANCE EXAM FORM.
    B.ED ONLINE ADMISSION MDU ROHTAK
    KURUKSHETRA UNIVERSITY B.ED ONLINE ADMISSION
    KASHMIR UNIVERSITY B.ED ONLINE REGISTRATION
    B.ED ADMISSION FORM LAST DATE
    IGNOU B.ED ONLINE ENTRANCE EXAM FORM
    B.ED ADMISSION HELPLINE NUMBER
    B.ED COURSE ADMISSION
    B.ED ADMISSION CONSULTANT DELHI
    B.ED COLLAGAE
    DU B.ED ADMISSION DATES

    ReplyDelete
  11. You explain everything briefly, Great job, Informative post, Thankx Shane Warne

    ReplyDelete
  12. Thanks for sharing, nice post! Post really provice useful information!

    Hương Lâm chuyên cung cấp bán máy photocopy và dịch vụ cho thuê máy photocopy giá rẻ, uy tín TP.HCM với dòng máy photocopy toshiba và dòng máy photocopy ricoh uy tín, giá rẻ.

    ReplyDelete
  13. Buy weed online Uk, order medical cannabis UK, Buy high Potent THC Cartridges, THC Vape Oils for sale in UK, buy plug play vape cartridges UK, Stiiizy vape cartridges for sale UK , Buy hash online UK, buy wax Online UK, Buy THC marijuana edibles UK, Buy weed UK, Buy Marijuana UK.

    Contact us for more information.

    Website: https://skunktree.co.uk

    ReplyDelete
  14. GRW Board now ready for the results and here get results. Bise gujranwala Board 12th Class Result 2019

    ReplyDelete
  15. Very Nice Article with Useful information
    Visit website to know about Properties for Sale in Mumbai

    ReplyDelete
  16. Very good contend love this article it helps a lot learn so many things here thanks for your submission looking forward of more of this stuff buy weed online

    ReplyDelete
  17. This is a great inspiring article.I am pretty much pleased with your good work. Please share good post on Best cbse schools in dubai also. Keep it up. Keep blogging. Looking to reading your next post.

    ReplyDelete
  18. buy alprazolam,ketamine,mdma,crystal meth,fentanyl at https://shop4herbalincense.com

    Order Fentanyl, Oxycodone, Buy Mephedrone online, Xanax Bars , Percs , Roxicodone at low cost..With us , you get meds discreet with no prescription. We've worked to create a fast,
    efficient and fair system for order processing. We know you want your order quickly, at the lowest possible cost.Check some of our highly purchased meds;

    We are the best supplier of pain medications and research chemicals .our prices are affordable .
    For more details contact our website through any of the details provided below
    Management

    Visit: https://shop4herbalincense.com


    Call: +1 (925)-338-9583 (USA/Canada)

    whatsapp: +1 (925)-338-9583

    ReplyDelete
  19. This article is nicely written. A very well written guide to take care of your furnace.
    medicine graduate courses in abroad

    ReplyDelete
  20. How does one get notified by email when there is an article written.

    This is good stuff!

    Thanks

    ReplyDelete
  21. This article was written by a real thinking writer without a doubt.I agree many of the with the solid points made by the writer.I’ll be back day in and day for further new updates. Research Chemicals

    ReplyDelete