Lesson Study and the 6 As of effective research use.

In recent years Lesson Study (LS) has become an increasingly popular form of professional development.  Indeed, there is a large body of evidence of the positive impact of lesson study and there is evidence that teachers enjoyed and engaged with lesson study as an approach to professional learning. As yet, there is less extensive evidence of the professional learning from research lessons making a permanent difference to daily teaching and learning, nor to pupil learning outcomes, although there is no evidence of any negative impact. The robustness of this evidence is reduced due to the fact that studies that do demonstrate positive outcomes are frequently (and often very) small scale case studies, making generalisation more difficult. However, all studies found a positive impact and the number of small scale studies finding positive impact suggests a general positive trend across many different settings and contexts

The research base on LS as recently been added to by (Lewis & Perry, 2017) who have published research about a randomised control of lesson study supported by mathematical resources (fractions) which has led to improvements in both teachers’ (+0.19 SD effect size) and students’ (+0.49 SD effect size) knowledge of fractions (see this post on how to interpret the size of effect sizes).  As such, and in light of these findings there is a temptation to argue that the results of this trial provides further supporting evidence for those schools wishing to adopt LS as a major component of their programme of professional learning.  

However, before any decisions are made by schools as to whether to adopt LS or not, it is necessary to consider whether Lewis and Perry's research meet the conditions for effective research use.  To help with this task, I will use the 6 As  framework developed by @profstig and outlined in this tweet.

Accessible

Unfortunately Lewis and Perry's work sits behind a 'paywall'.  Moreover, the Journal for Research in Mathematics Education is not included in the 2000 journals you can access via your membership of the Chartered College of Teaching.  Even if you can get hold of a copy of the article, the statistics used in the paper are advanced, and would not necessarily be accessible even by colleagues who hold advanced degrees in education.  That said,  - if you have an understanding of effect sizes, then it may be possible to access a basic understanding of the claims being made i.e. LS combined with an appropriate resource pack would appear to lead to increases in both teachers' and pupils knowledge of fractions.   On the other hand, Lewis and Perry provide a very clear conceptual map on the inter-relationship between LS, the resources made available and the outcomes for pupils.

Figure 1 taken from (Lewis and Perry, 2014)

Accurate

Given my relative lack of statistical knowledge, I am not in a position to say whether the findings are accurate or not.  However, if you focus on effect size, then it is possible to make a couple of observations.  First, as (Simpson, 2017) notes effect sizes are vulnerable to researcher manipulation in a number of ways.  Now again, my statistics are not strong enough to examine whether in this research this is the case.  However, it does mean we need to make sure we do not take effect-sizes at face value.   Second, there is significant debate about is meant by the size of an effect size.  Indeed with an ES +0.49 SD being claimed, this would suggest we have a result which is larger than the average effect size.

Applicable - specific context and level of use 

The research was conducted mainly in elementary schools in the United States, with volunteer staff, of whom the majority had previous experience of LS.  As such, this does not mean the research is generalisable to a secondary school in England, where there is little or no experience of LS and it is being adopted as a mandatory school-wide approach to professional learning. 

Acceptable to views and beliefs

Given that LS has in recent years become a widely adopted form of professional learning, the research would appear to be outwardly consistent with current views and beliefs re teacher professional learning.

Appropriate to context

This will of course depend upon, the needs of school, staff and pupils, and whether there is a need to develop pupils' knowledge of fractions.  Within the school, there may be other, more pressing problems of practice.  Accordingly, we need to ask : What is the problem that lesson study is the answer to? Fractions are notoriously difficult to teach well, so are a “good bet” or perhaps an “easy win”.   That said, if you chose something you are already reasonably good at, then the gains will be far more marginal. In other words, the focus of improvement efforts should be on areas where there is scope for impact.

Actionable 

There would appear to be two inter-related issues as regards whether the research is actionable or not.  First, LS is potentially extremely resource intensive and there are questions as to whether in current climate of austerity schools have the resources to support the implementation of LS.    However, LS can be managed without significant expense if time and resources spent on less effective forms of PD (e.g. whole school one off speakers, teachers being sent on one day outside programmes, etc) are deployed to support lesson study.  And if done so,  and all PD time is given over to the LS process, there is potential for LS to be a very powerful form of PD.  However, for this to work, schools must be using a model of lesson study that incorporates all of the key features as seen in Japan, and not leaving bits out because they don’t suit them or don’t have the time for them

Second, even if schools have the resources to support LS, the effectiveness of LS with mathematical resources does not meant that LS will be effective without such resources (Lewis and Perry, 2017).   As such, whether this research is actionable depends upon whether appropriate specialist resources are available to support its implementation.    Maybe the fractions resource pack could be adapted to reflect any priority theme for the school, but some form of outside expertise/knowledge is a prerequisite to making lesson study successful, and this is one key feature of lesson that schools in England routinely leave out (see lessonstudy.co.uk for an example of this).  Accordingly, many schools seem to feel that having a group of teachers talk about their practice is enough, with a danger that  this can lead to 'happy talk' if there is not some external external/expertise knowledge from outside the school.

To conclude


On balance, Lewis & Perry's work is a welcome contribution to the evidence-base on the effectiveness of LS.   However, the real value for me of Lewis & Perry's work occurs when it is subject to scrutiny by using @profstig's  6 As model for effective research use.  By doing so, it makes clear some of  the challenges of trying to apply research findings to the context of an individual school.  Even though some research findings may get some favourable publicity and is consistent with current views and practice - this does mean it should be adopted without a degree of structured challenge.

Note

I would like to thank both @sarahselezynov and @profstig for commenting on an earlier draft of this post.

References

Lewis, C., & Perry, R. (2014). Lesson Study with Mathematical Resources: A Sustainable Model for Locally-Led Teacher Professional Learning. Mathematics Teacher Education and Development, 16(1), n1.

Lewis, C., & Perry, R. (2017). Lesson study to scale up research-based knowledge: A randomized, controlled trial of fractions learning. Journal for Research in Mathematics Education, 48(3), 261-299.
Simpson, A. (2017). The misdirection of public policy : Comparing and combining standardised effect sizes. Journal of Education Policy.

Wiliam, D. (2016). Leadership for teacher learning. West Palm Beach: Learning Sciences International.