By Sawsan Al Zatari, Ricardo Ramírez and Dal Brodhead, DECI-4.
Selecting the right Primary Intended Users (PIUs) can be a challenge (Ramírez, 2021). How can we be sure that we are selecting the right users who will commit to both the journey and the learning process? “The selection of PIUs is part art, part strategy, and part intuition by the evaluator and the PIUs themselves” (Ramirez & Brodhead, 2013, p. 25).
Below are questions to help you select PIUs and classify them between heavy, medium, and light users. Some questions, or all, may apply to help you with your selection (Patton & Campbell-Patton, 2022; Ramirez & Brodhead, 2013). Thinking strategically, from the beginning, and managing expectations for both the evaluator and the PIUs is key when working together.
Place the names of your selected PIUs according to their use classification on the diagram below[1]. These categories include:
Ownership
- Are these users the owners and designers of the evaluation?
- Are they knowledgeable about the program, context, and its evaluation needs?
Availability
- Are they willing to put in a lot of time through-out the journey?
Openness
- Are they invested in learning and sharing the knowledge with others?
- Are they adaptive, reflective, and flexible throughout the journey’s context(s)?
Evaluation
- Do they have evaluation/research experience?
- Are they familiar with the UFE approach?
Communication
- Are they good communicators?
- Do they have strong interpersonal skills?
Connections
- Are they connected to important stakeholder constituencies?
- Are they credible in the eyes of other stakeholders?
Heavy PIUs should answer “yes” to most if not all the above categories. They are the owners and the designers of the evaluation and thus, will put a lot of time into the process, while learning and sharing the knowledge with others. Here, you want to ensure that you refer to the Rainbow Diagram and select a balance of committed members and stakeholders with both high and low influence and power. Do not be afraid to bring a mix of diverse stakeholders involved, as long as they can commit, communicate, make decisions and make use of the evaluation and its results.
More often, evaluators slip into the role of PIU where they ask the questions and provide the answers. This could be ‘easier’ during the process but futile once the results come in. PIUs should be the decision makers and the owners of the evaluation collectively. If they are not, then it is likely that they won’t use the evaluation results (Patton & Campbell-Patton, 2022).
Medium PIUs should answer “sometimes” to the categories of Availability and Openness. They may be interested in the evaluation but may not be able to dedicate as much time, thus, may not be able to attend all the meetings and share the learning with others. They need to be kept involved, but their limited commitment and availability levels need to be recognized while focusing on using their input for evaluation use.
At times, heavy PIUs may become medium users. This may depend on many factors, including their time and whether they feel their involvement in the evaluation adds value. If it is the latter, then it is important here to reflect on these issues and resolve them in a mutual agreement where the users are re-engaged. If it is the former, then it is best to respect their time and keep them informed as light PIUs.
Light PIUs should answer “no” to Availability and Openness. These are the users, like top executives and board members, who are interested in the process, but do not have time to be part of the complete journey. They may want to attend one meeting or read the final report. You may want to involve these users, but only at different stops in the journey so as not to overwhelm and lose them. Perhaps, you can keep them up to date with monthly emails and/or newsletters.
Working with the Utilization-Focused Evaluation (UFE) mentor from the beginning can clarify and facilitate the process of PIU selection and categorization. Experienced mentors have a deeper understanding of the process and an appetite to both learn and share their knowledge. Clear communication and reflection with the mentor and the evaluator can make the journey much more innovative and valuable (Ramirez & Brodhead, 2013).
Tips
- Think of the above as soft categories. Throughout the journey some PIUs may move from one category to another depending on their circumstance. It is good to check in with all the PIUs, and revisit and update this chart from time to time.
- Plan for turnover within the PIUs. Always engage new possible PIUs to bring them on board when others leave (Patton & Campbell-Patton, 2022).
- A good number of PIUs should not exceed 10-15 depending on the size of the evaluation. (Patton & Campbell-Patton, 2022).
References
Chevalier, J. & Buckles, D. (2008). SAS2: A Guide to Collaborative Inquiry and Social Engagement. IDRC & Sage.
https://idl-bnc-idrc.dspacedirect.org/bitstream/handle/10625/35977/IDL-35977.pdf
Patton, M. Q., & Campbell-Patton, C. E. (2022). Utilization-Focused Evaluation. Thousand Oaks: Sage.
Ramirez, R., & Brodhead, D. (2013). Utilization Focused Evaluation. A Primer for evaluators. Penang : Southbound.
Ramírez, R. (2021). Some primary evaluation users are more equal than others: Lessons from UFE experiences. DECI Blog. https://evaluationandcommunicationinpractice.net/some-primary-evaluation-users-are-more-equal-than-others-lessons-from-ufe-experiences/
[1] This diagram is based on the ‘rainbow diagram’ used for stakeholder analysis (Chevalier & Buckles, 2008: p.167).