Date of Award
2024
Degree Type
Open Access Dissertation
Degree Name
Psychology, PhD
Program
School of Social Science, Politics, and Evaluation
Advisor/Supervisor/Committee Chair
Tiffany Berry
Dissertation or Thesis Committee Member
Brad Cousins
Dissertation or Thesis Committee Member
Stewart Donaldson
Dissertation or Thesis Committee Member
Thomas G. Archibald
Terms of Use & License Information
Rights Information
© 2024 Haley E Umans
Keywords
Afterschool, Evaluation, Evaluation Capacity Building, Expanded Learning, Nonprofit, Out-of-School Time
Subject Categories
Psychology
Abstract
Nonprofit organizations, like expanded learning programs, are hungry to engage in evaluation to improve their services to make the greatest impact. To bring this dream to life, individuals in the organization must develop the skills necessary to engage in evaluation and use findings as part of evaluation capacity building (ECB) efforts. The ECB literature has provided several models and tools to support organizations’ quest to engage in evaluation and use finding sustainably; however, these strategies are often hindered when organizations have limited time, resources, and capacity. These barriers have been well documented in expanded learning programs (Block, 2021; Bradshaw, 2015; Chávez, 2021).
Roles of individuals, like other contexts, are noted as key considerations in ECB theories and models because understanding and adapting to context is key to the approach to building evaluation capacity effectively (Cousins et al., 2014; Labin et al., 2012; Preskill & Boyle, 2008; Taylor-Powell & Boyd, 2008). When considering roles and ECB, Taylor-Powell and Boyd (2008) note that not all staff must become evaluation experts; however, it should be a part of everyone’s job in the organization. Additionally, professional development opportunities should be customized to the learner (i.e., knowledge, motivations, responsibilities) (Taylor-Powell & Boyd, 2008). The emphasis on tailoring ECB to roles and responsibilities provides a promising strategy for contexts experiencing limited resources and capacity is ripe for further exploration.
The present mixed methods study examined how considering roles and responsibilities when building capacity could support organizations with limited time and resources in the expanded learning context. The first research question aimed to understand the value of roles and responsibilities in ECB through the lens of evaluators. The second research question examined how the competencies of expanded learning professionals in different roles could be translated into competencies related to evaluation. Expanded learning professionals refers to staff who work for youth serving multipurpose programs that typically include components like homework help, educational enrichment (e.g., arts, STEM), and physical activities before school, after school, or during non-instructional days like summer break (McCombs et al., 2017). This includes staff who work directly with youth, supervisors, and leaders within expanded learning programs.
The final research question assessed the extent to which key precursors to engaging in ECB (e.g., attitudes, motivations) varied by role in the expanded learning sector to provide a deeper analysis of what considering roles may entail. Data collection was conducted with both evaluation and expanded learning professionals (i.e., surveys and interviews for both samples) in two concurrent phases. One phase focused on the perspective of evaluation professionals and the other phase focused on the perspective of expanded learning professionals. There were 292 participants in a survey of evaluation professionals and 32 participants completed follow-up interviews. There were 446 participants in a survey of expanded learning professionals and 12 participants completed follow-up interviews.
Findings from surveys and interviews confirmed that it is valuable to consider roles and different roles in expanded learning are better suited to engaging in specific aspects of evaluation (e.g., staff working directly with students best suited to collect data). Several strategies emerged to support role consideration in any context. This includes strategies around understanding roles and responsibilities, providing professional development opportunities, and building sustainable infrastructure (e.g., integrating evaluation responsibilities in the job description). The empirical development of this strategy clarified benefits, perceived challenges, enablers of success, and strategies that could be used to build evaluation capacity effectively in organizations with limited resources and capacity. Overall, this study provides recommendations to foster the development of evaluation capacity building in more organizations. If more organizations can engage in evaluation and use findings, more organizations will make a greater positive impact on society.
ISBN
9798302167149
Recommended Citation
Umans, Haley Elizabeth. (2024). Promoting Continuous Learning and Organizational Improvement Through Role Based Evaluation Capacity Building. CGU Theses & Dissertations, 920. https://scholarship.claremont.edu/cgu_etd/920.
