Researching the use of research evidence

DECI-3 Blog

Researching the use of research evidence

By Ricardo Ramirez, DECI-3.

An April 2016 report entitled The Science of Using Science: Researching the use of research evidence in decision-making (by Laurenz Langer, Janice Tripney & David Gough, EPPi Centre, University College, London) provides a ‘review of reviews’ of the literature of evidence-informed decision-making  – from now on referred to a EIDM – , plus a scoping study of the research reported in the reviews.

For DECI-3, this study works like a backdrop for both evaluation planning (probable evaluation uses) and for communication (lessons, principles, bad habits to avoid).

The following provides an excerpt of the findings of Review 1 on decision-makers use of research (a summary of 23 existing reviews):

Evidence of effects.

  • Interventions facilitating access to research evidence, for example through communication strategies and evidence repositories.
  • Interventions building decision-makers’ skills to access and make sense of evidence.
  • Interventions that foster changes to decision-making structures and processes.

Evidence of no effects.

  • Interventions that take a passive approach to communicating evidence such as simple dissemination tools·  Interventions that take a passive approach to building EIDM skills.
  • Skill-building interventions applied at a low intensity (such as a once-off, half a day capacity-building programme).
  • Unstructured interaction and collaboration between decision-makers and researchers tended to have a lower likelihood of success.

Among the first set, the reference to communication strategies and to building decision-makers’ skills, is welcome. Something that the DECI team documented in the Evaluation Report of Research-ICT-Africa.  Under the evidence of no effects column, the case of passive approaches to dissemination is important to note, one of the ‘9 bad things’ to avoid in research communication that was reported on in the first blog in this series.

Review 2 of the same report focused on 67 interventions where social science knowledge supported EIDM interventions and mechanisms including the following:

  • Promote and behavioural norms such as social marketing, or social incentives.
  • Engage in advocacy and awareness raising to communicate and popularize the concept of EIDM, its benefits as well as the risks of not doing them.
  • Effectively frame and formulate communicated messages, tailoring communication including audience segmentation, and regular use of reminders.
  • Enhance the design of evidence repositories and other resources, and ensure they are user- friendly.
  • Build a professional identity with common practices and standards of conduct through, communities of practice, mentoring, and inter-professional education
  • Foster adult learning theories and principles as they are of direct use and relevance to EIDM capacity-building.
  • Build institutional capacities and support organizational change and cultures, management and leadership techniques.
  • Use behavioural techniques, including nudges to support the use of evidence during decision-making
  • Harness the potential of online and mobile technologies.
  • Enhance institutional frameworks and mechanisms that nurture structural changes at all levels of decision-making and increase attention to evaluation in this area.

The study then summarizes the implications of the two reviews, which read like recommendations:

“Interventions that support the communication of and access to research evidence were only effective to increase evidence use if the intervention design simultaneously tried to enhance decision-makers’ opportunity and motivation to use evidence…Similarly, interventions building decision-makers’ skills were only effective to increase evidence use if the intervention design simultaneously tried to enhance both capability and motivation to use research evidence” (this quote appears in page 4 of the report cited at the beginning of this blog).

The University College research project used a framework that is of interest to DECI, as it addressed

  • levels of intervention,
  • mechanisms of change, and
  • capability, motivation and opportunity to change behaviour.

…to help understand (a) what the interventions are trying to achieve and (b) the process or theory of change that explains how change is expected to unfold.  The authors suggest that the framework may assist future evaluation EIDM interventions.