Date of Award
2025
Degree Type
Open Access Dissertation
Degree Name
Evaluation, PhD
Program
School of Social Science, Politics, and Evaluation
Advisor/Supervisor/Committee Chair
Jennifer Villalobos
Dissertation or Thesis Committee Member
Tiffany Berry
Terms of Use & License Information
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Rights Information
© 2025 Darrel Skousen
Keywords
Equity, Evaluator humility, Cultural responsiveness, Community partnership
Subject Categories
Education
Abstract
Despite evaluation's professed commitments to the values of equity, cultural responsiveness, and community partnership, one overall question remains unexamined: do evaluation reports reflect the values evaluators claim to hold as a profession? This study addresses this question by operationalizing and assessing evaluator humility; a concept frequently mentioned in evaluation ethics literature but rarely examined systematically in evaluation practice. Drawing from intellectual humility, cultural humility, virtue ethics, and evaluator competency frameworks, this study defines evaluator humility as an intentional, ongoing commitment to self-awareness of one's limitations, openness to learning, respect for others' expertise, and critical examination of power dynamics in evaluation practice. The study operationalizes this construct through six theoretical dimensions: 1) self-awareness and reflexivity; 2) valuing others and meaningful dialogue; 3) openness to learning and revision; 4) ethical and intentional knowledge practices; 5) contextual responsiveness and cultural situatedness; and 6) power consciousness and justice orientation. To explore how these dimensions are expressed within evaluation reporting, the study developed the Evaluator Humility Assessment Rubric, converting the six theoretical dimensions into nine observable traits that can be systematically identified in written evaluation reports. These traits include positionality disclosure, inclusion of multiple perspectives, recognition of evaluative partner contributions, acknowledgment of limitations and uncertainty, justification of value judgments, responsiveness to cultural/contextual specificity, attention to power and equity, use of accessible language, and representation of dissent or divergence. Each trait is rated on a three-point scale (0=Not Evident, 1=Somewhat Evident, 2=Clearly Evident) and allows for documentation, enabling both quantitative measurement and qualitative analysis. The study utilized a meta-evaluation approach to analyze 20 publicly available evaluation reports through purposive sampling: 15 equity-focused evaluations and five traditional evaluations. This stratified sample allowed for the assessment of variation in expressions of humility across different evaluation approaches. Reports spanned multiple sectors including education, community health, family strengthening, workforce development, and systems governance, being published between 2012 and 2024. Analysis included methodical rubric coding, reflexive journaling, face validity assessment with five independent raters, and dual scoring comparison between human and AI coding to examine interpretive application of the framework. Findings reveal significant patterns in how evaluator humility is expressed (or constrained) in contemporary evaluation reporting. Quantitatively, humility traits varied widely, spanning from limited visibility (Positionality Disclosure: M=0.30) to a consistent presence (Partner Recognition: M=1.35). Equity-focused reports scored slightly higher than traditional reports (9.67 vs. 7.00 out of 18 points); however, this difference consisted almost entirely of external, relational traits rather than reflexive practices. Both report types demonstrated consistent inclusion of partner voices and methodological transparency. Three overarching patterns emerged: 1) evaluators included expressions of external humility (acknowledging others, recognizing partnerships, communicating accessibly) but not internal reflexivity (disclosing positioning, questioning interpretive authority, preserving dissent); 2) evaluators expressed procedural humility (acknowledging methodological constraints) without epistemic depth (maintaining interpretive confidence about conclusions); and 3) expressions of humility clustered in "safe locations" (Acknowledgments, Limitations sections) rather than being integrated throughout analysis and interpretation. The study documents an enduring "values-practice gap" between evaluation's equity rhetoric and documented practices. Reports explicitly devoted to cultural responsiveness, community voice, and social justice often omitted the very practices that would demonstrate those commitments; that is, evaluator reflexivity about power, preservation of dissenting perspectives, and sharing of interpretive authority. This suggests evaluation has developed language for equity without redeveloping evaluative relationships or redistributing authority in reporting. The study makes several contributions to evaluation scholarship and practice. It provides the first empirical baseline for how humility is currently expressed in evaluative documentation, operationalizes an abstract ethical concept as observable practice, and uncovers systematic patterns in evaluation's professional culture that demonstrate selective expressions of humility, prioritizing relational courtesy over epistemic transparency. The findings challenge equity-focused evaluations to apply these principles throughout the evaluation process, including documentation, and call for field-level shifts in reporting norms to better reflect stated values. This study's small sample size, North American concentration, and single-coder framework reflect deliberate methodological choices rooted in evaluator humility itself; the recognition that all judgment is positioned, that humility manifests differently across cultures, and that understanding is always evolving. These boundaries provide space for future research to build on this work. Larger samples across different contexts, multiple coders with diverse perspectives, cross-cultural validity studies, and triangulation with process observation can further explore whether evaluator humility (as a reporting practice) can strengthen the quality and use of evaluations. This study offers the field its first empirical baseline; subsequent research can deepen and extend it.
ISBN
9798276011424
Recommended Citation
Skousen, Darrel. (2025). Humility in Practice: Examining the Ethical Alignment of Evaluator Values and Reporting. CGU Theses & Dissertations, 1068. https://scholarship.claremont.edu/cgu_etd/1068.
