What does research tell us about effective marking in maths?

There is remarkably little high-quality, relevant research evidence to suggest that detailed or extensive marking has any significant impact on pupils’ learning.

Sean Harford, 2016

Since Hattie’s (2009) review of effect sizes on student outcomes, with feedback (0.73) in the top 10, marking has begun to receive greater attention from teachers and policymakers. However, written marking is only one form of feedback and possibly not the most time efficient one. In the below diagram, from the Education Endowment Foundation (EEF, 2016, p. 6), the place of written marking is shown within a wider context of feedback along two dimensions.

Notwithstanding, marking continues to dominate teachers’ lives. More than two thirds (68%) of UK teachers said the amount of time they spent marking, negatively affected classroom time with pupils (TES, 2016). The DfEDepartment for Education - a ministerial department responsible for children’s services and education in England Marking Policy Review Group (2016) found that providing written feedback on pupils’ work had become ‘unnecessarily burdensome for teachers’ and suggested that school leaders ‘challenge emerging fads’, which lead to excessive marking (TES, 2016). With this in mind, can research help to identify effective practice and cut down on workload?

If you’ve come looking for answers, I’m afraid you may be disappointed. There is a lamentable research gap when it comes to marking (not just in mathematics), which is only just beginning to attract interest and funding. Findings are mostly partial and/or tentative, with ‘a striking disparity between the enormous amount of effort invested in marking books, and the very small number of robust studies that have been completed to date’ (Elliot et al., 2016, p.4).

Teachers across the world face what is termed ‘the problem of enactment’  – they are given policies and initiatives to improve their practice, but without enough evidence or explanation as to why (Dixon & Haigh, 2008). While teachers may report reasonable levels of understanding of the role of feedback and marking, one study found that they struggled to articulate this clearly when asked (Dixon & Haigh, 2008).

A changing narrative of feedback

During the 1990s there was a shift in the narrative around formative assessment. It went from something the teacher did and part of the act of teaching, to giving a more central role to the learner, who should be encouraged to reflect and given the tools and information to self-regulate through a metacognitive process (Dixon & Haigh, 2008).

Pupils – even those as young as five years old – also benefit from marking their own work as long as they have enough information about their targets (Black & Wiliam 1998). This is when language shifted to the term Assessment for LearningKnown as AfL for short, and also known as formative assessment, this is the process of gathering evidence through assessment to inform and support next steps for a students’ teaching and learning (AfLAssessment for Learning (also known as formative assessment) - the process of gathering evidence through assessment to inform and support next steps for a students’ teaching and learning).

If there’s a single principle teachers need to digest about classroom feedback, it’s this: The only thing that matters is what students do with it. No matter how well the feedback is designed, if students do not use the feedback to move their own learning forward, it’s a waste of time.
Wiliam, 2014

Ironically, there has been a mythical feedback loop at play in the last 10 years between schools trying to second-guess what OfstedThe Office for Standards in Education, Children’s Services and Skills – a non-ministerial department responsible for inspecting and regulating services that care for children and young people, and services providing education and skills is looking for, and even Ofsted itself being unable to shift these fixed ideas as thoughts and research change. As author Sue Cowley (2014) explains: ‘The teachers I work with worry most about this – how do they provide evidence of progress in a way that Ofsted will find acceptable? This then seems to get translated into “Write a lot in the children’s books to show it.”‘

We have all heard of triple marking, the purple pen of progress, marking stickers, three stars and a wish, no more red pen, lengthy marking abbreviations – the list goes on, proving that schools are often still seeking to show rather than do when it comes to marking.

Grades, comments, or both?

When we provide written feedback in terms of comments and explanations for judgements in students’ books, they often report finding this useful – but tellingly, may not always be acting on it.

Researchers in one case study note: ‘While 75% of the class indicated that they appreciated receiving the… feedback, not many indicated that they had acted on this information. Furthermore, when [the teacher] spent a considerable length of time writing comments indicating how a grade had been awarded she found that the students were not particularly interested in her comments.’ (Dixon & Haigh, 2008).

Teachers may relate to this idea that their extensive written comments are both time-consuming and not used in a useful way by students. As Dylan Wiliam (2014) writes:

When teachers pair grades with comments, common sense would tell us that this is a richer form of feedback. But our work in schools has shown us that most students focus entirely on the grade and fail to read or process teacher comments. Anyone who has been a teacher knows how many hours of work it takes to provide meaningful comments. That most students virtually ignore that painstaking correction, advice, and praise is one of public education’s best-kept secrets.

So should we stick to just providing grades or marks?

Interestingly, there is evidence to suggest this approach affects different groups in the classroom in different ways. As Elliot et al say:

A large, longitudinal study from Sweden found that boys and lower-attaining pupils who received grades… made less progress than similar pupils who did not. However, grading had a positive long-term effect on girls. No definitive explanation of this effect is known, but the researchers hypothesised that boys and low-attaining pupils were more likely to overestimate their level of performance, and hence be demotivated by the grade, while for girls the converse was often true. – Elliot et al, 2016, p. 10

A fresh outlook?

Pupils who were in a test group that had only comments, not marks given back to them in year 7, performed slightly worse than the control group in mathematics; some reported frustration at not getting this information (Smith & Gorard, 2005).

But marking in mathematics isn’t just about deciding on grades and/or comments – what else is out there?

Audio feedback

Can primary and secondary maths teachers learn anything from higher education? There is also research exploring the use of recorded audio feedback techniques in mathematics, suggesting that ‘providing audio files in lieu of written remarks on graded assignments is arguably a more effective means of feedback allowing students to better process and understand the critique and improve their future work’ (Weld, 2013, p513). Through audio feedback, teachers can quickly and appropriately correct and explain the why; it also allows teachers to explain the relative importance of grammar or presentation corrections versus misunderstandings and misconceptions.

Weld uses the following phrase as an example: ‘I am not as concerned with these series of spelling and grammar mistakes, however they are a distraction to the professionalism of your report and the points you are trying to make supporting your mathematical model.’ He writes: ‘Although it would be awkward, time consuming, and cumbersome to write, on a student’s paper, it is an easy statement to say at the beginning of a voice recorded segment.’ (Weld, 2013, p516) Although he also notes that ‘no gains in effectiveness are possible without students first taking the time to listen to the audio feedback provided to them.’ (Weld, 2013, p520)

Checklist model

In her paper, How to Grade 300 Mathematical Essays and Survive to Tell the Tale, Crannell (1994) gives some advice: always give students a checklist for each piece of work, to be used as a reference for the students and a marking structure for the teacher.  This checklist is devised by the teacher doing the piece of work first and then articulating the necessary elements, serving as a ‘teaching tool as well as a grading tool’ (Crannell, 1994). Using a checklist model can also be the catalyst towards changing formative assessment practices entirely, for example, in moving towards more extended writing in mathematics (Haver & Hoof, 1996). The idea of moving through multiple drafts and only handing in the final one for thorough marking may also be useful here.

Comparative judgement

More recently, the idea of comparative judgementAn approach to marking where teachers compare two students’ responses to a task and choose which is better, then repeat this process with other pieces of work in mathematics has been cited as a way to mark student work in a way that is less time-consuming, more efficient and allows for more open-ended questions to be assessed. Comparative judgement rests on the principle that comparing two answers and elevating one as better than the other is easier and more instinctive than the traditional idea of following a mark scheme or a simple correct/incorrect model; and that multiple judges (including pupils) can therefore moderate judgements easily. Repeating this process for a class set of books, one can use a statistical model to give marks to students. Interestingly, comparative judgement has also been described as a successful way to assess problem-solving in mathematics (Jones & Inglis, 2015).

Impossible search for a silver bullet

In short, ‘we are all searching for a method which will allow us to grade a large quantity of [work] in a way that is (a) meaningful, (b) equitable to all students, (c) helpful to the students’ writing, and (d) time efficient’ (Crannell, 1994). Perhaps we should add (e) ensures we are still awake for the lesson too.

While there is no sense in searching for a silver bullet, so far research provides an incomplete picture even as to the range of options we have and their relative use in different contexts.  Finally, as teachers of mathematics, we may also do well to reflect that marking is only one element of feedback and, in a rush to collect and disperse data, we may forget the human interactions that we have are also highly significant.

As one maths teacher in the case study remarked:

As a mathematics teacher I always wanted to “play” with quantitative data and I tended to forget the human element. The action research project has made me consider that the rich conversations that I have with the students is making more of an impact into what I do and why I do it. … I feel that students are more aware of the formative value of assessment. – Dixon & Haigh, 2008, p177.

Further information

You can find more information on feedback in mathematics teaching in my Espresso (filtered research review) here.