Sunday 24 May 2015

Do all good ideas need to be researched? - A response to David Didau

Recently David Didau posed the question : do all good ideas need to be researched?  In doing so, Didau suggested that rather than researching good ideas, teachers would be better off engaged in a form of  'research'  which involved the following:

Step 1 – have a good idea

Step 2 – try it out with some students

Step 3 – think about what happened – is it worth doing again?

Step 4 – think about why it worked. Maybe dig into the reams of existing research to find out what others think. Come up with a theory which provides a post hoc justification for your good idea’s success.

Step 5 – share your idea with other teachers. Ask them to tell you what they liked and what they didn’t like.

Step 6 – improve the idea.

Step 7 – resist, with all your might, the temptation to slap numbers on to your idea in an attempt to justify why it’s good; this is cargo cult science.

Whilst this approach may appear appealing, the proposed model of 'teacher research' is flawed as it is packed with opportunities for cognitive bias.  Soyer and Hogarth (2015) argue that what we think we have learned from experience may well indeed be wrong.  By using experience as our guide to 'what is a good idea', we will have viewed the past through a number of different filters, examples includes; a focus on outcomes not processes; what our closest and not so closest colleagues tell us; and, our own limited powers of reasoning.  These limitations lead to our interpretations being biased, and as a result our current actions may be misguided. Accordingly, a 'good idea' may be a 'bad idea.'  Nevertheless, if we are aware of those biases we can then use techniques to overcome them.  Soyer and Hogarth identify a number of techniques to overcome these biases which include: seeking evidence to disconfirm your 'good-idea'; looking at situations where  a failure was disguised as a success; studying failure by providing 'safe' environments to discuss what went wrong; provide opportunities for disagreement; pursue prevention by engaging in activities such as pre-mortems.

The Spiral of Inquiry 

Fortunately for teachers wishing to engage in evidence-based teacher inquiry, Timperley, Halbert and Kaser (2014) have developed a process of investigation known as the 'spiral of inquiry', which explicitly acknowledges the potential for cognitive bias.    Timperley et al describe the spiral of inquiry process as a multi-tiered and multi-connected process, which is framed by two central questions :What’s going on for our learners?  How do we know? (p7) and is illustrated in the following figure

    The individual phases of the spiral of inquiry

    Phase 1 Scan what’s going for our learners

    Phase 2 Focus on where our energies will potentially make the most difference

    Phase 3 Develop and check-out your hunches of what might be a good idea

    Phase 4 How and where can we learn more about what to do?

    Phase 5 What can we do differently to make enough of a difference?

    Phase 6 Have we made ‘enough’ of a difference ?

    Let's now look at the developing a hunch phase in more detail.

    Developing a hunch

    In this phase the focus is on How are WE contributing to this situation? (p13).  Hunches are often based on intuition and are not grounded in evidence.  In this phases, we must have the confidence to bring these hunches to the fore, and explore the underpinning assumptions and evidence.   However, to explore deeply held views, will require 'emotionally safe learning environments for teachers, so these hunches can be explored without fear.   As such developing hunches is concerned with with: getting values and beliefs about our own teaching practice out into the open; identifying those practices that are within our domain and we can do something about; and finally, it's about checking our assumptions, our evidence, and our initial interpretations for validity before taking the next step.

    To conclude
    Although the model of  'teacher-research/inquiry' proposed by David Didau may appear attractive, it has fallen into the trap of being both simple and simplistic.  Unfortunately, the model pays little or no attentions to the possibility of cognitive bias, which can distort both what has been learnt from both past-experience and the current process of inquiry.  Fortunately, Timperley et al's 'spiral of inquiry' provides a potentially more effective model of teacher-inquiry.  In doing so,  the model explicitly acknowledges the role of cognitive bias, and provides mechanism by which it may possibly be counter-acted.  Finally, if we want teachers to be evidence-informed practitioners, we must not shy-away from the challenges involved.  Rather than provide overly simple solutions we must help teachers gain the skills and expertise they require for genuine evidence-informed/based teacher inquiry, and which can make a real difference to both professional and student learning.


    Halbert, J and Kaser, L (2013) Spirals of Inquiry for Equity and Quality, BCPVPA Press, Vancouver. Further information available at www.bcpvpa.

    Kaser, L and Halbert, J (2009) Leadership Mindsets: Innovation and Learning in the Transformation of Schools, Routledge, London.

    Soyer, E. and Hogarth, R. (2015) Fooled by Experience : What you think you've learned may be wrong.  A guide to finding out the real lessons.  Harvard Business Review May 2015  pp 73 - 77

    Timperley, H (2011) Realizing the Power of Professional Learning, Open University Press, London.

    Timperley, H, Halbert, J and Kaser, J. (2014) A framework for transforming learning in schools: Innovation and the spiral of inquiry, Centre for Strategic Education, Victoria


    1. This comment has been removed by the author.

    2. I've just written a book about cognitive bias in education - particularly as it applies to research. The point I was making in the post you cite is that in some cases bias doesn't really matter. If I as a teacher have what I consider to be a 'good idea' I will biased in its favour. This being the case I will make it work. Any research short of a double blinded RCT will find in my favour because I, and the teachers who agree to trial my idea will believe in it.

      The other point I'd make is that Timperley et al's spital whatsit is exceedingly unlikely to rid teachers of their biases. Believing it will is a bias :)

    3. Thanks for your comments - I think we are in agreement about the challenges and limitations of big R research and serious issues associated with using 'statistics' with individual teacher based enquiry action/research. And I agree you cannot eliminate cognitive bias, though you can certainly adopt processes which will limit their negative. impact., or at least raise them to the surface Where I think we are in disagreement in the rigour required to engage in teacher-based enquiry. The 'research' process you propose does not start with the pupil or pupils, it starts with what a teacher thinks is a 'non-evidenced 'good-idea,' and has the potential to exacerbate unhelpful 'group-think' within schools. :)

      1. All I can say in my defence is that everything I do is aligned against exacerbating "unhelpful 'group think' in schools". :)

        Research is important, as is uncovering bias. Teacher based enquiry is utterly fraught with bias by its very nature. Telling teachers there's a way of avoiding this is dishonest. Instead I propose this:

        1. Read the research
        2. Consider your pupils
        3. Use your judgement to teach your pupils as well as you can
        4. Resist the efforts of anyone to change what you've decided to do: they are no less biased than you and will have far less knowledge of your pupils.

    4. I think we both agree that we want teachers to engage in enquiry, though I'm not sure that the model you propose could be described as disciplined inquiry. As Cronbach and Suppes (1969) state -
      Disciplined inquiry has a quality that distinguishes it from other sources of opinion and belief. The disciplined inquiry is conducted and reported in such a way that the argument can be painstakingly examined. The report does not depend for its appeal on the eloquence of the writer or on any surface plausibility, (p. 15).
      In other words, you are arguing for a model of sharing practice within schools which is based upon: teacher infallibility; teacher eloquence; and, surface plausibility. For me, this is neither a desirable or justifiable

    5. Just to say I completely share Gary's concerns with David's informal model of teacher inquiry. It strikes me as odd that David would follow up his recent, excellent piece on Hattie's dismissal of the idea of teachers as researchers, by lending support to a model of 'research' - if you can call it that - that would support the return of VAK - i.e. if teacher believe in it then they will make it work. Brain gym likewise. These things seemed like good ideas at the time - they have "surface plausibility" as you say - that's what makes such practices so insidious.

    6. Now we can even bring around more of the possible concerns which are indeed considered to be of utmost importance and the value. here