By Ricardo Ramírez and Dal Brodhead, DECI-3.
Years ago, I joined a team of consultants who asked to prepare their final report in a story format. When the team presented the report to an audience, they did so by reading the first chapter of a mini novel. During the question and answer period most of the audience wanted to know what had happened to Paloma, the main character in the story. We told them to read chapter 2. The story had connected – and only a few questions were asked for clarification. More recently, we began an article with a story (Ramírez et al., 2018) – and subsequently at a conference a stranger approached me and recalled “the piece that started with a story”. At that very conference, the closing keynote speaker – a novelist – encouraged the evaluation community to embrace story telling in our work. These experiences all confirm what Annie Neimand wrote recently in the Stanford Social Innovation Review: “People are far more likely to remember information if it reaches them in the form of a story. Good stories also have an incredible ability to reduce ‘counterarguing’ on divisive issues.” (2018: 1)
The idea of using stories to convey complex ideas first came to us from a 2006 report by the Dag Hammarskjöld Foundation entitled What next: Trendlines and alternatives (Mooney, 2006) that was written as a collection of stories, all based on facts and thorough research.
“To deal with the complexity and unpredictability of the task, the artifice of a fictional scenario was adopted to explore the processes that might be involved in taking humanity from 2005 to 2035. … If the world continues on its present course, it was concluded, the fictional scenario China Sundown that follows is unnervingly reasonable and not at all sensational… In developing and debating the scenario, the group fought for accuracy in detail and for logic in analysis. Although technology is clearly at the centre of the discussion, there was a deliberate effort to highlight the central issues of power and control. The events related to in the story that took place before September 2006 have happened. Nothing is invented. Many of the story’s events that occur after September 2006 have also already taken place. Others are based on the understanding of trends or situations that can reasonably be anticipated.” (Mooney: 2006: 4)
We have recently come across several publications that confirm how Narrative and story telling are gaining prominence, we think of this approach as a meeting place for communication and evaluation. For instance, in a recent article that explores how to measure knowledge management, the authors state:
“So, one way of addressing measurement questions is, when the time is right, to retrospectively look at what is successful. Being always on the lookout for success stories and fostering a culture of telling stories can surface much called for ‘evidence’. It is good practice to look out for trends and develop an open-ended collection of views that can be used to tell a story. Methods such as Most Significant Change (Davies and Dart 2004) can be used for harvesting stories.” Cappezzuoli, S. & Jolly, R. (2019: 12-13).
In the economics field, there is an emerging recognition about the importance of narrative. The author makes reference to the influence of narrative and the media in shaping human behaviour; he provides examples of the ‘epidemic’ spreading of stories leading to changes in behaviour by thousand of people during the depression of the 1930.
There is a daunting amount in the scholarly literature about narratives, in a number of academic departments, and associated concepts of memetics, norms, social epidemics, contagion of ideas. While we may never be able to explain why some narratives “go viral” and significantly influence thinking while other narratives do not, we would be wise to add some analysis of what people are talking about if we are to search for the source of economic fluctuations. We economists should not just throw up our hands and decide to ignore this vast literature. We need to understand the narrative basis for macroeconomic fluctuations, and to think about how narrative economics ought to be more informing of policy actions now and in the future. (Shiller, R. J., 2017: 972).
He goes on to add that: “There should be more serious efforts at collecting further time series data on narratives, going beyond the passive collection of others’ words, toward experiments that reveal meaning and psychological significance.” (Shiller, R. J., 2017: 998).
The value of story is the integration of many dimensions:
“Stories must follow the narrative arc to gain the cognitive and psychological benefits of storytelling. Stories include characters and settings, a beginning, middle and end, and conflict and resolution. A quote, vignette or message is not a story. In stories about complex issues, situational factors — like a flood, drought, political turmoil, ethnic conflict — should be the setting of the story. We should step into the world of the characters, experience the world from their perspectives, and watch as they make decisions and grapple with a particular challenge.” (Neimand et al., 2019:3)
These authors suggest that stories about an organization or a cause should include the following dimensions: origin of the story, what will be different, what was learned in defeat, how success was achieved, and the people involved. In the area of climate change communication, they suggest that stories provide a better context than data alone; stories leave space for audiences to insert their assumptions as to what the data means. “Stories provide anecdotal evidence participants can connect with and return to. Coupling diagrams with stories provides the evidence needed for those who prefer diagrams, but illustrates the complexity of the issue for those who do not connect to graphs and charts.” (Neimand et al., 2019:7)
Finally, van Wessel, M. & Ho (2018) address narrative as a means of tracking “the unseen nature of advocacy work”. “Narrative Assessment helps advocates, their organizations and donors to bring to light tacit knowledge and experience for sharing and wider learning and practice.” (van Wessel & Ho, 2018: 4).
The same authors add that: “By co-constructing advocates’ stories in the light of a program’s Theory of Change, Narrative Assessment assesses whether and how outcomes are relevant steps in a pathway of change, and how challenges are to be understood. It thereby helps to revise a program’s Theory of Change and adjust future action.” (van Wessel & Ho, 2018: 5). This experience appears to be compatible with the DECI-3 approach, and especially to developmental evaluation associated with pilot projects in the making, that require quick feedback for course correction.
“A Narrative Assessment story is built up from the following elements:
- It contains an element of transformation;
- It presents this transformation as a movement over time;
- It contains actions by which this transformation happens, carried out by characters;
- These actions take place in a specific setting
These four elements are brought together in a plot (possibly involving crises and turning points). This plot has a point: a key message to take away from the story.
These are human interest stories, like ‘brilliant failures’, there [sp.] are more credible and real.” (van Wessel & Ho, 2018: 8-9). They add a note about quality control, or plausibility testing, also referred to as ‘narrative substantiation’. This step includes a review of detail about claims regarding outcomes; context descriptions that are familiar and do not bring up rival interpretations; and consistency with the internal and external environment.
For our own DECI process, these different literatures remind us that anecdotes are a form of data. Collecting narrative has not been a major focus of evaluation methodology, possibly due to the over-emphasis on ‘indicators’. Indicators have a numeric connotation, while stories bring a subjective one. With such complex efforts such as measuring advocacy outcomes, it is clear that evaluators need both, and each situation will dictate in what proportion and what is realistic to collect.
Finally, these papers also remind us about the power of story telling as a reporting tool. As was key to the 2006 What Next report, each chapter while appearing to read like a novel, was a sticky (Hearn & Hearn, 2006) way of packaging a great deal of hard evidence. Come to think of it, can we think of a graphic novel as an evaluation report? Chances are even senior decision-makers would be curious.
Cappezzuoli, S. & Jolly, R. (2019). Measuring knowledge management: evidence essentials in purpose-driven organizations. Knowledge Management for Development 14(2): on-line https://www.km4djournal.org/index.php/km4dj/article/view/385 (accessed 18 Sep. 2019)
Heath, C. & Heath, D. (2007). Made to stick: Why some ideas survive and others die. New York and London: Random House.
Mooney, P. (2006). What next: Trendlines and alternatives. The What Next Report 2005-2035. Ottawa and Uppsala: ETC Group and the Dag Hammarskjöld Foundation. (pre-publication)
Neimand, A. (2018). How to tell stories about complex issues. Stanford Social Innovation Review. https://ssir.org/articles/entry/how_to_tell_stories_about_complex_issues?platform=hootsuite&mc_cid=115b27ad05&mc_eid=152e49edc1 (accessed 18 Sep. 2019)
Neimand, A.; Christiano, A. & Parater, L. (2018). Chapter 3: Tell stories. In: Bending the Arc. UNHRC Innovation Service. https://medium.com/bending-the-arc/chapter-3-tell-stories-f5069f699007 (accessed 17 Sep. 2019)
Ramírez, R.; Brodhead, D. & Quarry, W. (2018). Readiness in evaluation: Three prompts for evaluators. Canadian Journal of Program Evaluation, 33(2): 258-267.
Shiller, R. J. (2017). Narrative economics. American Economic Review 107(4): 967-1004.
Van Wessel, M. & Ho, W. (2018). Narrative assessment: A new approach to advocacy monitoring, evaluation, learning and communication. The Hague: HIVOS.