Date of Award
Fall 2020
Degree Type
Open Access Dissertation
Degree Name
Psychology, PhD
Program
School of Social Science, Politics, and Evaluation
Advisor/Supervisor/Committee Chair
Stewart I. Donaldson
Dissertation or Thesis Committee Member
Michelle C. Bligh
Dissertation or Thesis Committee Member
Michael Quinn Patton
Dissertation or Thesis Committee Member
Huey T. Chen
Terms of Use & License Information
Rights Information
© Copyright Susana Marianne Bonis, 2020
Keywords
graphic conceptual models, program evaluation, program theory, research on evaluation, theory-driven evaluation
Subject Categories
Psychology
Abstract
Theory-driven evaluation has been adopted by numerous philanthropic organizations and government agencies across the world. Many evaluators also have embraced elements of theorydriven evaluation, regardless of their approach to evaluation. In theory-driven evaluation, the beliefs or assumptions behind an intervention are made explicit and used to guide the evaluation. These may be based on the perspectives and experiences of stakeholders, social science theory, observation, and previous evaluation and research. A graphic conceptual model, often in the form of a logic model or theory of change, is commonly developed to show how an intervention is expected to lead to desired outcomes. While there have been various conceptual, methodological, and theoretical writings about theory-driven evaluation since it came to prominence in the early 1990s, empirical research on aspects of theory-driven evaluation are more recent. This study took a comprehensive approach to understanding the development and use of graphic conceptual models in theory-driven evaluation practice. It also investigated issues that have been raised in relation to the use of models, such as the need to adapt them in different cultural contexts and more complex environments, and the possible benefits of identifying program archetypes. The study involved two parts: a content analysis of 116 published evaluations in peer-reviewed journals focused on public health, education, and evaluation and a survey of 141 evaluators. A distinction of this study from other research on theory-driven evaluation is its deeper examination of the graphic conceptual models included in published articles on evaluation studies. Findings reveal several encouraging practices in the design, development, and use of graphic conceptual models in evaluation. Graphic conceptual models were developed using multiple sources, including theory and research, and they often depicted detailed paths of mediation and moderation. While most models reviewed were linear, there was indication that evaluators are exploring alternative ways of describing programs whose path of change is less predictable and describing programs in different cultural contexts. Co-design of models through a participatory process was common and resulted in shared understanding of how an intervention is expected to lead to change. In many instances, program theory was used to develop evaluation questions and the evaluations measured constructs and relationships outlined in the model. Finally, this study took a step towards understanding possible archetypes. While programs may focus on distinct issues and serve different people, there may be similarities across design and implementation characteristics and impact pathways. Archetypes could serve as a heuristic device for practitioners and help build knowledge about programs in a systematic manner. The results indicate that theory-driven evaluation is practiced across various disciplines and cultural contexts in ways that adhere to many of its core principles. The content analysis involved a larger sample than previous empirical research on theory-driven evaluation, allowing for confirmation and contradiction of earlier findings that could add to the knowledge base and point to directions for future research. This study also gathered information on emerging issues and approaches proposed to strengthen theory-driven evaluation that have not been studied empirically on a large scale. The findings around use of graphic conceptual models in complex interventions and in interventions carried out in different cultural contexts, as well as on program archetypes, could serve as part of the foundation for further inquiry. Overall, this study contributes to a better understanding of how evaluators develop and use graphic conceptual models in evaluation practice and suggests possible future directions for research on theorydriven evaluation practice.
ISBN
9798569940707
Recommended Citation
Bonis, Susana Marianne. (2020). The Use of Graphic Conceptual Models in Theory-Driven Evaluation. CGU Theses & Dissertations, 279. https://scholarship.claremont.edu/cgu_etd/279.
