By Ricardo Ramírez and Dal Brodhead, DECI-3.
The DECI-3 project name is “Designing Evaluation and Communication for Impact”. So what is impact?
This blog entry tries to move beyond the numerous recommendations and well-tested practices that we have reported in this blog series. We suspect that when our partners read these blogs, the natural reaction may sound like: ‘oh sure, good in theory, but good luck — in our context what has worked is…” And they are right, ‘scoring a goal’ in policy influence is much like in soccer: there is an opening and you take a shot – and it is very hard to replicate.
You need that window of opportunity, matched with practical wisdom on how to respond, combined with solid evidence that you have made sure is accessible. It mostly happens once.
So in this blog we change gears; instead we address the larger question about what success looks like, specifically the impact of research for development.
Just by chance, an incomplete scan of three books from the fields of evaluation of capacity development, planning & complexity, and research communication yielded the following insights:
- “There is a growing need to put country participants at the heart of the evaluation process. Extractive, third-party ‘expert’-dominated efforts do not contribute much to a legacy of evaluative thinking and acting.” (As stated by Morgan, 2013, page 81).
- “The components for resilient governance include: diversity and independence to allow for many faceted perspectives, collaborative dialogues, collaborative development of knowledge, networks as the core of adaptive governance, ’boundary spanning’ for cross-scale efforts, monitoring and feedback, and small diverse groups to develop innovative approaches.” (As stated by Innes & Booher, 2010, pages 209 to 211).
- A key implication for the evaluation of research communication “…is the tendency to rely on the types of outcomes and impacts (whether short term or long term) which are most straightforward to map. The impact agenda implies a sense of co-production, ‘research with rather than about’.” (As stated by Wilkinson & Weitcamp, 2016, pages 228 and 231).
Our take-way is that “research impact” may need to begin with a community of diverse stakeholders who agree to collaborate on a research-change agenda. This idea is the essence of Collective Impact (as proposed by Cabaj & Weaver, 2016), which requires a significant, intentional, long term shared learning contract. A collective of projects or organizations that is able to adjust their course as the learning advances, as the context evolves, as disagreements emerge across multiple world-views; that has the opportunity to define and redefine trajectories, way stations, and change. Is the word “impact” as a term or concept too stationary, let alone linear?
If the question is: “What does success look like?” We venture to respond as follows.
Success is defined by a cluster of mixed stakeholders who agree to work collaboratively and systematically on an issue [often complex or emergent], with an agreement to provoke change at multiple levels and in ways that affect a diversity of citizens, their communities, organizations, programs, institutions and policies. In practice, this approach may mean structuring pro-poverty research with coalitions of stakeholders as opposed to starting only with researchers. Easier said than done… many topics are controversial; so experimentation will be required on how the clusters of organizations interact. More certain is the fact that those research teams will need facilitation to encourage systematic multi-stakeholder learning and collaboration. It will also mean investing in strategic coordination among multi-disciplinary teams as so often researchers, funders, and evaluators come from silo-ed institutional traditions.
There is relevant experience mentoring in utilization-focused evaluation and research communication, now complemented with theory of change and adaptive management dimensions. There is potential to broaden the approach to introduce elements of Collective Impact –where there has been important experimentation that one can learn from. There is room for concepts from Social Innovation that provide a framework from which to learn about scale and about the challenge of project transformation as work advances and stretches to new levels and scales. A safe prediction is that there will no such thing as ‘good practices’…. it is more likely that a handful of shared principles will emerge, as the three books quoted seem to be suggesting.
References
Cabaj, M. & Weaver, L. (2016). Collective impact 3.0: An evolving framework for community change. Waterloo, ON: Tamarack Institute.
Innes, J.E. & Booher, D.E. (2010). Planning with complexity: An introduction to collaborative rationality for public policy. London & New York: Rutledge.
Moore, M.-L.; Riddell, D. & Vocisano, D. 2015. Scaling out, scaling up, scaling deep: Strategies of non-profits in advancing systemic social innovation. The Journal of Corporate Citizenship: 58(June): 67-84.
Morgan, P. (2013). Evaluating capacity development. pp. 75-104. In: Donaldson, S.I; Azzam, T. & Conner, R.F. (eds). Emerging practices in international development evaluation. Charlotte, NC: Information Age Publishing.
Westley, F. R. and Antadze, N. (2010). Making a difference: Strategies for scaling social innovation for greater impact. The Public Sector Innovation Journal, 15, (2): art 2.
Wilkinson, C. & Weitkamp, E. (2016). Creative research communication: Theory and practice. Manchester University Press.