By Ricardo Ramirez and Dal Brodhead
Those of us working at the research-to-policy interface, often talk about the importance of evidence-based policymaking. Is our compulsion to bombard policy makers with evidence akin to the advertising industry’s strategies to convince us to buy shampoo? While advertisers show images of a model’s flowing, shiny hair as evidence, we send policy makers research reports, or polished policy briefs if they are lucky. Furthermore, we emphasize the importance of doing some prior audience research asking: do the policy makers prefer a video, a webinar, or a full report? At what moments in their policy work are they able to access and make use of evidence?
While this process sounds straight forward; do we actually know what their daily slog is like? Does our ‘audience research’ enable us to get a glimpse of their work habits, routines, and moments when they need evidence? This blog explores these questions by summarizing parts of a recent article by Louise Shaxson and Annette Boaz.
For starters, the term ‘policy-maker’ deserves to be unpacked as it includes a wide range of individuals and units within an organization. Senior staff members in government receive advice from multiple sources including specialized advisors and lobbyists, and in turn bring recommendations to directors general and often to elected officials. The staff members and advisors are the ones expected to be able and available to locate, sometimes commission, and digest evidence from research. But how and when does this process work?
Shaxson and Boaz (2020) suggest that policymakers’ daily work practices include:
- developing documents,
- overseeing ongoing programs of work,
- advising ministers,
- responding to ad-hoc enquiries including questions in Parliament, and
- supporting the work of expert committees.
We add that they are also expected to anticipate and plan for upcoming issues and propose future priorities.
In turn, these practices lead to outputs and outcomes such as:
- defining what the problem is,
- keeping issues on the agenda,
- identifying drivers of change,
- challenging assumptions and design, and
- selecting interventions.
We add that they usually propose, but do not select.
In performing the practices, policymakers must negotiate different ways of framing problems, interests and perspectives on key issues. They use a range of techniques to do this work:
- making complex phenomena more manageable by visualizing, conceptualizing, defining; and
- categorizing what policies aim to do and who they aim to benefit;
- testing out ideas and proposals on colleagues and external advisers;
- using evidence tactically to persuade others of the validity of ideas, to mobilize support or to provide an independent voice to defuse conflicts over policy proposals.
Shaxson and Boaz (op.cit.) argue that the aim of policymakers’ practices is to create policy narratives to:
- build, test and communicate a coherent policy ‘narrative’: a storyline about the policy issue that helps communicate a particular perspective on which aspects of the issue are important and an appreciation of why they are important
- help people make sense of, organize and transmit information about the past, present and possible future of the issue;
- shape the decisions that are subsequently taken
- create narratives that could be defended against internal and external critiques including from colleagues in other departments, stakeholder groups, Parliament and the media.
Shaxson and Boaz (op.cit.) add that policy narratives and the ideas they contain are malleable, as policymakers seek to keep key issues on the table while simultaneously responding to critiques and attempting to align these narratives with other policy agendas. How effective policymakers are at this process of alignment depends on enabling conditions such as:
- their ‘policy know-how’:
- the technical, organizational and networking skills they need to do their jobs effectively, and
- the rules, procedures and historical context from which each issue has arisen.
Different types of evidence are used to inform the detail of policy narratives, such as:
- the use of evidence from program evaluations;
- internal and external stakeholder consultations;
- data from state, national and international assessments; meetings with frontline staff;
- reports from consultants and advisors and evidence from researchers who are considered by policymakers to be particularly authoritative.
These different types of evidence are combined to shape the narrative, test ideas and challenge the internal coherence of emerging proposals. In some studies, Shaxson and Boaz (op.cit.) found that the range of evidence sources used by policymakers was wider during the policy development phase than in subsequent phases where policymakers placed more emphasis on testing and communicating the narrative with stakeholders. They argue that when policymakers begin to communicate with key stakeholders to negotiate their buy-in to a particular policy proposal, having robust evidence became less important than having a coherent narrative.
In summary, Shaxson and Boaz (op.cit.) argue that one fruitful avenue for enquiry is to analyze policymakers’ perspectives on evidence use by developing an empirical understanding of what policymakers do on a daily basis; how policy narratives and evidence narratives develop and the organizational systems and processes that shape how evidence is used in the policy process. Further work is needed to better understand how policymakers’ demands for evidence are shaped and what these requirements imply for how researchers (and other providers of evidence) can engage more productively with policymakers. Finally, they argue that the proposed concept of a ‘evidence narrative’ needs further examination. They propose the following questions:
- How do policymakers negotiate various normative positions on evidence?
- How do organizational systems shape these negotiation processes?
- Where and how do evidence narratives interact with policy narratives and to what effect?
- What does this information imply for the roles that researchers and/or policymakers could play in strengthening research-to-policy relationships?
This reflection leads us to wonder how much of the above is part of our DECI partners’ current ‘practical wisdom’ where they already have the pulse of the policy process. Perhaps much of the above is done by intuition? Knowing contacts inside policy circles who can direct them to policy windows of opportunity would be one example. Directing evidence to these time sensitive, contextualized moments would be part of an art of timely influence.
All the above aspects of the policy development process become tangential if those who wished to effect policy change failed to understand that policy advisors faced with often short notice to produce advice are compelled to use whatever research is at hand. If it readily available, well-grounded and crafted in an appropriate format to meet the immediate need, it is used. Here practical wisdom is an essential element in the policy-makers daily slog.
References and suggested readings
 Shaxson, L. & Boaz, A. (2020). Understanding policymakers’ perspectives on evidence use as a mechanism for improving research-policy relationships. Environmental education research. https://doi.org/10.1080/13504622.2020.1764505