Date of Award
2024
Degree Type
Open Access Dissertation
Degree Name
Psychology, PhD
Program
School of Social Science, Politics, and Evaluation
Advisor/Supervisor/Committee Chair
Peggy L. Kern
Dissertation or Thesis Committee Member
Stewart I. Donaldson
Dissertation or Thesis Committee Member
Michelle Bligh
Dissertation or Thesis Committee Member
Jason T. Siegel
Terms of Use & License Information
This work is licensed under a Creative Commons Attribution 4.0 License.
Rights Information
© 2024 Jaclyn Gaffaney
Keywords
checklist, mental health, mixed methods, positive psychology, systems thinking, well-being
Subject Categories
Educational Psychology | Psychology
Abstract
The COVID-19 pandemic shed light on global mental health struggles, underscoring the need to prioritize wellbeing. Many disciplines currently research aspects of wellbeing, ranging from psychology, behavioral economics, public health, and public policy, to prevention, evaluation, and education. However, researchers have rarely integrated insights across disciplines or extracted meaningful nuance from real-world applications. Many wellbeing interventions can also fall short due to poor design and execution as well as inadequate application of systems thinking and culturally responsive lenses. This dissertation addressed these gaps by creating a structured framework for crafting and assessing systems-informed wellbeing initiatives in the form of the Wellbeing Initiative Design & Evaluation Checklist (WIDEC). It uncovered critical factors that contribute to the effectiveness and sustainability of wellbeing initiatives by taking a systems-informed, culturally responsive approach and combining insights from research and practice across disciplines. A Mixed Methods Exploratory Sequential Design was employed to develop and validate the Checklist. Phase 1 ( n = 41) encompassed a literature review and interviews to create a theoretical framework via Reflexive Thematic Analysis and Directed Content Analysis. Phase 2 involved transforming this theoretical framework into a Checklist with guidance from Stufflebeam's Checklist Evaluation Checklist. Lastly, Phase 3 included Western Michigan University’s two-step validation process consisting of Phase 3a’s critical feedback evaluation survey ( n =24) and Phase 3b’s field test ( n = 3). The samples across phases consisted of experts who had been involved in systems-informed wellbeing initiatives as practitioners or practitioner-scholars in organizations, educational institutions, or communities. These samples were representative of the intended users of the Checklist. Findings from Phase 1 identified the critical components of effective wellbeing initiatives including strong leadership, inclusive interdisciplinary participation, and an iterative continuous improvement process. The most common challenges of these initiatives included inadequate funding, leadership turnover, and cultural sensitivity. Aspects unique to wellbeing initiatives included the need to foster a holistic approach, setting realistic expectations that wellbeing is a journey instead of a destination, the practice of embedding wellbeing into every stage of the process, and the common pitfall of focusing on individual-level changes without considering the broader context. Phase 1 culminated in a theoretical framework that was used to build the Checklist in Phase 2. The Checklist was organized into six stages, each with actionable checkpoints: 1. Lay the foundation, 2. Diagnose and define focus, 3. Design the initiative, 4. Plan the evaluation, 5. Implement the initiative and evaluation, and 6. Sustain and scale. The outcomes of Phase 3 determined that the Checklist was suitable for its intended users of practitioners, researchers, and evaluators. Phase 3a’s critical feedback evaluation survey revealed that all checkpoints were deemed at least somewhat important and all but one checkpoint (related to mitigating hedonic adaptation) had been previously employed by the participating experts, on average. In Phase 3a, the overall Checklist was rated favorably using nine evaluation criteria, listed in rank order from highest to lowest: comprehensiveness (M = 6.50), appropriateness (M = 6.48), accuracy (M = 6.41), organization (M = 6.17) clarity of writing (M = 5.91), clarity of purpose (M = 5.74), usability (M = 5.55), accessibility (M = 5.55), and parsimony (M = 5.22). Although comprehensiveness received the highest rating, some found the Checklist overwhelming. Given these findings, the Checklist was shortened, and the copy was edited to bolster parsimony, accessibility, usability, and clarity of purpose. During Phase 3b’s field test, when experts used the Checklist to evaluate a prior wellbeing initiative, usability and clarity of purpose were reassessed and the instructions were further refined as a result. Additionally, the experts gave favorable assessments of their evaluation findings in terms of American Evaluation Association’s four standards of utility, accuracy, feasibility, and propriety. Across all phases, construct, content, face, and consequential validities, as well as fidelity, were all supported. In an era of unparalleled challenges to mental health and wellbeing, the demand for effective solutions has never been greater. The WIDEC now provides a robust, evidence-based tool to aid the design and evaluation of wellbeing initiatives in organizations, educational institutions, and communities. It presents a comprehensive framework that addresses knowledge gaps, enhances rigor, enables collaboration, and bolsters the credibility of wellbeing efforts. Practitioners, including program designers, human resource professionals, and chief wellbeing officers, can navigate the complexities of designing wellbeing initiatives and facilitating planned change. Policymakers and public health officials can leverage the Checklist to refine their strategies, while researchers can explore new avenues for interdisciplinary study. Additionally, the WIDEC equips evaluators with a diagnostic tool and a process evaluation framework, ensuring that wellbeing initiatives are not only well-conceived but also effectively implemented and sustained over time.
ISBN
9798382743035
Recommended Citation
Gaffaney, Jaclyn. (2024). A Checklist to Design and Evaluate Systems-Informed Wellbeing Initiatives: A Mixed Methods Validation Study. CGU Theses & Dissertations, 780. https://scholarship.claremont.edu/cgu_etd/780.
