By: Ricardo Ramírez, DECI-AM.
For practitioners working towards systems change, tracking impact is a never-ending concern. They face an overwhelming list of systems-oriented evaluation tools (e.g. UNDP Sandbox); sifting through this pile can lead to what Gladwell (2005) calls ‘analysis paralysis’.
If time and resources allow, practitioners can sign up to courses on evaluating systems change (e.g. Clear Horizon Academy). In some instances, they can receive mentoring as they design and adjust their evaluation strategies (as in the case of DECI, and DeSIRA LIFT).
The difficulty is making the resources work for you, as practitioners in your context, at your pace, in your field and under the governance and financial conditions you work under. Engaging external evaluators is one option, but if the focus is on gaining systems evaluation competencies, then there is a need for a mix of support to connect in a timely fashion with the tools that are timely, relevant and affordable. This requirement suggests that there are a number of new roles that evaluators seeking systems change need to be ready to juggle.
- Maven
- Switchboard operator
- General practitioner (GP)
- Broker
- Personal trainer
Maven: the word comes from the Yiddish ‘meyvn’: one who understands, who knows the ins and outs. One needs mavens-on-site. Malcom Gladwell’s book Blink: The Power of Thinking without Thinking (2005) emphasizes “thin-slicing”: the ability to use limited information from a narrow period of time or experience to come to a conclusion. Others refer to this as ‘practical wisdom’, and this skill set is learned through practice.
A good maven knows who or what to go to, whether it be a person, a tool, or a resource; much like the switchboard operators of the past. This step is done without hesitation, it comes naturally. General Practitioners (GPs) in medicine play this role, and can refer patients to a medication, or a specialist. Through years of experience, they thin slice by detecting patterns they have seen before. However, the best GPs listen to the patient to make sure they are understanding the unique circumstance (Compassionomics). This role is not far from being a broker, be it in real-estate or financial services: the function is the same, seamless referral. But what brings real value is when the maven-switch-board operator-GP-broker becomes your personal trainer.
If one person or team can combine these attributes, then they are in the best position to become embedded to help systems change happen: this person is the type of evaluator that we need to help us face the global climate crisis and associated migration and inequity challenges facing the globe.
In Developmental Evaluation, the evaluators are embedded within the project implementation team. They are not external, so-called independent evaluators; rather they are on the bridge, helping navigate as the weather changes. Our challenge then is how to train these jack-of-all trade evaluators.