Date of Award
Spring 2023
Degree Type
Open Access Dissertation
Degree Name
Psychology, PhD
Program
School of Social Science, Politics, and Evaluation
Advisor/Supervisor/Committee Chair
Tarek Azzam
Dissertation or Thesis Committee Member
Wanda Casillas
Dissertation or Thesis Committee Member
Stewart Donaldson
Terms of Use & License Information
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Rights Information
© 2023 Ciara C Knight
Keywords
Culturally responsive evaluation, Data visualization, Logic models, Communication tools, Logic Model Knowledge Test
Subject Categories
Biological Psychology
Abstract
Culturally responsive evaluation (CRE) is an approach that centers all evaluation processes around the culture of a program’s secondary stakeholders. Specifically, this entails ensuring shared meanings in a group through communication. However, minimal connections have been made between CRE and logic model designs. Logic models commonly used by evaluators are data visualization and communication tools designed to aid in effectively communicating a program’s theory. Nevertheless, little is understood about the role culture plays in this process. This multiphase mixed methods study explored the integration of CRE to logic model designs using individualism and collectivism (IC) as a construct for culture and as a basis for tailoring the designs. Specifically, Phase 1 utilized an exploratory sequential design to develop the Logic Model Knowledge Test (LMKT) based upon expert evaluation insight and Amazon Mechanical Turk (MTurk). This addressed how learning from a logic model can be assessed. The LMKT was designed to assess the understanding of logic model components and the content of the program theory. It yielded good validity and reliability and was used in the final phase of the study. Phase 2 used a quantitative two-group comparison design to answer the question, which measures of IC would be more likely to help inform how people interpret logic models differently? Of three existing measures, Wagner (1995) was identified as a highly reliable measure that better distinguished IC-oriented people. Accordingly, it was used in the final phase of the study. Lastly, Phase 3’s mixed methods experimental design utilizing the LMKT and Wagner’s (1995) IC measure was implemented to help answer six questions regarding whether culturally tailored language, visualizations, or both impacted participants’ visual efficiency (VE) and perceived credibility of the program (CP). The findings suggest collectivistic MTurk participants had higher VE across seven culturally tailored logic model conditions. Collectivistic-oriented participants could better quickly and accurately answer the LMKT questions with less mental effort across all logic model conditions than individualistic-oriented participants. Yet, these conditions did not impact IC MTurk participants’ CP. Participants rated the program theory as credible across the logic model conditions regardless of their IC orientation. The findings provided partial empirical support for integrating CRE in logic model designs. Moreover, they added to the validity of the IC construct and Wagner’s (1995) measure. The findings also had valuable implications for evaluation practice through measures that could be used to better understand stakeholder cultural values aligned with communication styles and their knowledge of logic models. Finally, these same tools could be used for future research on evaluation concerning CRE and logic model designs.
ISBN
9798379900458
Recommended Citation
Knight, Ciara Cascharelle. (2023). A Culturally Responsive Evaluation Lens to Logic Model Design. CGU Theses & Dissertations, 542. https://scholarship.claremont.edu/cgu_etd/542.
