Date of Award

2023

Degree Type

Open Access Dissertation

Degree Name

Psychology, PhD

Program

School of Social Science, Politics, and Evaluation

Advisor/Supervisor/Committee Chair

John LaVelle

Dissertation or Thesis Committee Member

Stewart I. Donaldson

Dissertation or Thesis Committee Member

Brad Cousins

Dissertation or Thesis Committee Member

Tiffany Berry

Terms of Use & License Information

Terms of Use for work posted in Scholarship@Claremont.

Rights Information

© 2023 Marcia P Joppert

Keywords

Emerging Evaluators, Evaluation Education, Professional Preparation, Professionalization of Evaluation, Systems change, Transdisciplinary Field

Subject Categories

Educational Assessment, Evaluation, and Research | Psychology

Abstract

The world has experienced rapid changes, leading to pressing issues such as environmental degradation, social inequality, and resource depletion. As a transdisciplinary field, evaluation has emerged as a crucial tool in addressing these challenges and promoting systemic change. However, concerns have been raised regarding the field's capacity to meet these expectations and the existing gaps in evaluation education. This research aims to address these gaps by exploring how formal evaluation education programs (EEPs) respond to the preparation of professionals for systems change evaluations, prompting discussions on the need to redefine the approach to teaching evaluation. The study adopts a sequential multi-phased mixed-methods design, comprising two qualitative and one mixed-methods study.Study 1 involves semi-structured interviews with 19 international evaluation experts from different roles, locations, and backgrounds to gather their perspectives on the role of evaluation in supporting systems change. Critical topics, approaches, and methods that should be included in formal EEPs to prepare professionals for this challenge are also pointed out. The findings highlight that evaluation can contribute to systems change by providing sound evidence and recommendations for problem understanding, solution design, progress tracking, and strategy adjustments. However, influential contributions require adopting interdisciplinary and systemic approaches that respond to diverse contexts, voices, and considerations of equity and sustainability. Moreover, evaluations must stay abreast of new technologies for data collection, collaboration, and continuous learning. The insights from Study 1 identified desirable traits for EEPs, encompassing curricula design, modalities, strategies for developing soft skills, and connections to the broader professional field of evaluation. Study 2 involves a focus group with 11 young and emerging evaluators (YEEs) from different parts of the globe to explore their perspectives on the role of evaluation in supporting systems change and their challenges, needs, and interests regarding evaluation education. The study reveals that YEEs recognize the importance of evaluation in driving systems change, emphasizing the need to shift from traditional linear models to systemic thinking. YEEs advocate for accessible and inclusive EEPs that provide real-world practical opportunities and instruction in various evaluation theories. They recommend incorporating utilization-focused evaluation, people-centered and culturally responsive approaches, systemic evaluation methods, and foresight techniques to anticipate future transformation scenarios. Including new data collection methods like storytelling and artificial intelligence, as well as incorporating youth perspectives in evaluation processes, is also suggested. Studies 1 and 2 contribute to identifying the anticipated characteristics that will shape the future trajectory of evaluation. Study 3 compiles a comprehensive list of formal postgraduate EEPs offered by academic institutions worldwide. Through descriptive and categorical statistical analysis, these programs are assessed to determine the extent to which they incorporate the characteristics identified in Studies 1 and 2. The analysis reveals significant shortcomings in integrating the relevant topics, approaches, and methods suggested by the previous studies and a weak link with professionalization. A total of 271 formal postgraduate programs from 39 countries across all continents and offered in 9 languages were identified and analyzed, including 45 doctorates, 124 master's, 58 certificates, and 44 diplomas. The data collected in Study 3 were shared with the Global Evaluation Initiative (GEI) to create and publicly disseminate a comprehensive global directory of postgraduate academic programs through the "Better Evaluation" knowledge platform. This research underscores the importance of critically examining the alignment between evaluation trends and EEPs on a global scale. Furthermore, it provides novel insights by offering specific recommendations for integrating emerging content into evaluation education to better prepare students for the demands of the field. The findings offer valuable guidance for future students in selecting EEPs, contribute new elements to research on evaluation education, and encourage EEP leaders to reflect on the relevance and effectiveness of their offerings. Ultimately, this research emphasizes the significance of comprehensive evaluation education.

ISBN

9798381904475

Download

Share

COinS