Introduction to research: Judging the quality and trustworthiness of research evidence

An essential part of being an evidence-based practitioner is being able to judge the quality and trustworthiness of research. However, not being an expert in research, and without some kind of aide-memoire or heuristic, this can be difficult. Thankfully, there are frameworks in place to help you do this. For example, Professor Steve Higgins of Durham University has developed a framework based on the ‘6As of the usefulness of research’ which can help you to do this. Professor Higgins poses the following question: is the research: accessible, accurate, applicable, acceptable, appropriate and actionable?

What does it mean?

In order to help you make the most of them, Jones (2018) has identified a number of sub-questions for each of the 6 As:

Accessible

• Is the research physically accessible on: Google scholar, The Chartered College of Teaching, the EBSCO database, Open source journals or via direct contact with the author?
• Who is the audience: practitioners, policymakers or researchers?
• How and why are the authors making this contribution?
• Is it a contribution to policy, theory or practice?
• What do the authors assume about the knowledge of the readers?
• What about other issues of intellectual accessibility – easy to read, jargon free, clear messages?

Accurate

• How robust is the evidence?
• Are the methods used suitable for the research aims?
• To what extent are the claims made supported by others’ work?
• What evidence that challenges their claims is not mentioned?
• Does the knowledge generated meet the specific standards of that type of knowledge?

Applicable

• What groups of learners might benefit from the findings?
• What degree of certainty do the authors make for their claims?
• How generalisable are their claims?

Acceptable

• What values stance is being adopted – are they implicit or explicit?
• How might the values stances taken by the authors affect their claims and acceptability by colleagues?
• To what extent are the claims consistent with my experience?
• Is the research relevant to the problem which is the most interesting to you and your colleagues?
• Are there any ethical issues arising from the research?

Appropriate

• Is the research relevant to the most recurring problems in your department, key-stage, school or multi-academy trustAbbreviated to MAT, a group of schools working in collaboration, governed by a single set of members and directors?
• Is the research related to a problem within your sphere of influence most relevant to your sphere of influence?
• Is the research relevant to problems for which – resources – be it staff, time, expertise and finance – are available ?

Actionable

• Does the research specify causal statements – If this … then ..?
• Are concrete behaviours specified to bring about the intended outcomes?
• Do the teaching staff have the skills required/or can they be taught the skills required to put the research into effect?

What are the implications for teachers?

Willingham (2012) has come up with the following four-step process:

• Strip it. Get rid of the fluff surrounding the idea and get right to the heart of the claim being made.
• Trace it. Where did the idea come from?
• Analyse it. What are you – the evidence-based school teacher – being asked to believe?
• Should I do it? Is it something which is already being done? Is it an old idea wrapped in new language and terminology?

Both of the above approaches are short-cuts and can’t replace the development of high levels of research literacy. Nevertheless, the more you use these frameworks the better you will become at making efficient and effective use of research evidence to bring about improvements in both teaching and learning.

Want to know more?

• Jones G (2018) Evidence-Based School Leadership and Management: A Practical Guide. London: SAGE Publishing.
• Wallace M and Wray A (2016) Critical Reading and Writing for Postgraduates (3rd ed). London: SAGE Publishing.
• Willingham D (2012) When Can You Trust the Experts: How to Tell Good Science from Bad in Education. San Francisco: John Wiley & Sons.