Date of Award

2023

Degree Type

Restricted to Claremont Colleges Dissertation

Degree Name

Psychology, PhD

Program

School of Social Science, Politics, and Evaluation

Advisor/Supervisor/Committee Chair

Christina A. Christie

Dissertation or Thesis Committee Member

Stewart I. Donaldson

Dissertation or Thesis Committee Member

J. Bradley Cousins

Dissertation or Thesis Committee Member

Wanda Casillas

Terms of Use & License Information

Terms of Use for work posted in Scholarship@Claremont.

Rights Information

© 2023 Jessica Renger

Keywords

Evaluation Training, Evaluator Competencies, Interpersonal Skills

Subject Categories

Psychology

Abstract

With the recent publication of the American Evaluation Association Evaluator Competencies, increasing attention has been paid to understanding the skills evaluators need to be successful in practice. Although much progress has been made in the professional practice, methodology, planning and management, and context domains, relatively little attention has been given to the interpersonal competency domain. Out of all the AEA competency domains, evaluators often report the least confidence in their interpersonal abilities and believe this is an area they need further training to develop (Galport & Azzam, 2017). Employers looking to hire evaluators echo this sentiment stating that although strong interpersonal ability is the most desirable quality for a candidate to possess, it is also the skill most evaluation professionals entering the field lack (Dewey, 2008). This lack of confidence and interpersonal competence is made more pronounced as interpersonal skills are necessary to the conduct of every aspect of the evaluation process. Engaging stakeholders, co-creating a logic model, collecting data, and reporting to leadership, are just a sampling of evaluation activities that center on the effective interpersonal ability of the evaluator. Without this ability, the evaluation process is unlikely to move forward smoothly or effectively. This study sought to remedy this gap by creating a comprehensive interpersonal skills training course for evaluators. A three-stage mixed methods design was used to design, pilot-test, and implement a full-scale interpersonal skills training course for evaluators. In the first stage of the study, expert evaluation practitioners were interviewed to determine which interpersonal skills should be the focus of the course and how the course should be structured. The interviews revealed that the course should target the fundamental interpersonal skills needed of practicing evaluators: listening and communicating, empathy and relationship building, and conflict resolution. Due to the focus on fundamental skills, the course was deemed to be at an introductory level, and thus young and emerging evaluators were the target audience for the course. Additionally, the interviews revealed that the course would be most effective if it consisted of classroom-based content delivery, guided active practice, and opportunities for thoughtful reflection. To ensure the course was accessible and presented the least barriers to participation, all aspects of the course were delivered in a fully online environment. In Stage 2 of the study, the interpersonal skills training course was developed based on the information gathered in Stage 1 of the study and was pilot tested with 10 university evaluation students. A pre-course and post-course quantitative assessment was used to determine the extent to which participants increased their knowledge and awareness of the interpersonal skills needed in evaluation practice (measured through the Knowledge + Awareness subscale), confidence in their interpersonal skills (measured through the Confidence subscale), and ability to use these skills in practice (measured though the Practice subscale). Qualitative post-course interviews were conducted with a subset of the pilot test participants to supplement the data gathered from the pre-course and post-course assessments. Findings from the pilot test revealed that while participants’ scores increased across all three assessment subscales, only the increases in the Knowledge + Awareness subscale and the Practice subscale were significant. Interview findings suggested participants felt they had increased across all three subscales but the impact of the course on their knowledge of specific interpersonal skills as well as on their evaluation practice was mentioned more frequently, corroborating the assessment findings. Participants also suggested several modifications to the course content and structure to enhance the effectiveness of the course. In Stage 3 of the study, the suggested modifications from Stage 2 were incorporated and the course was delivered to a global audience of young and emerging evaluators. Findings from the pre-course and post-course assessments revealed that participants increased significantly across the Knowledge + Awareness, Confidence, and Practice subscales, with findings corroborated by interviews with participants. Taken together the findings suggest the value and importance of creating a dedicated interpersonal skills training course for evaluators. It is the hope of the author that by creating a course with a dedicated focus on interpersonal skills training, it will help highlight the importance of the interpersonal domain, help ensure practicing evaluators are well-equipped to handle the interpersonal aspects of evaluation practice and serve to reshape the current landscape of evaluator training and development.

ISBN

9798381897746

Download

Share

COinS