Weaver, Dax M. (2020). What makes collaborative evaluation approaches collaborative? A Q methodology study on program evaluators’ perceptions. Doctoral dissertation (Educational Leadership), University of North Florida. (Access: https://digitalcommons.unf.edu/etd/953/)

Abstract: The purpose of this study was to explore program evaluators’ perceptions about what they consider to be the essential practices, elements, or attributes of collaborative evaluation approaches (CEAs). This topic was investigated because of the increasing use of CEAs within the field and the ambiguity about what does and does not constitute a CEA. Several researchers have proposed frameworks that attempt to define the essential elements that distinguish CEAs from other evaluation approaches; however, the literature still lacks consensus on their exact nature. It was through an examination of evaluators’ viewpoints on the subject using Q Methodology that this study attempted to provide the field with empirical evidence clarifying the nature of CEAs.

Thirty-two program evaluators sorted 40 statements describing evaluation practices, elements, and approaches on a continuum from “least essential to CEAs” (-4) to “most essential to CEAs” (+4). These 32 sorts were factor analyzed and rotated. Following these procedures, four factors emerged that represented different perspectives on CEAs. Interpretation of these factors yielded distinct themes. These factor themes were named: (a) Culturally Connect with Stakeholders, (b) Talk to Stakeholders and Trust Will Follow, (c) Teach and Empower Stakeholders, and (d) Facilitate Stakeholder Communication.

The results of this study illustrate a varied assortment of perspectives that value culture, trust building, evaluation capacity building and empowerment, and communication as cornerstones to CEAs. Although this study was intended to be exploratory, the viewpoints that emerged help validate the literature about the purposes and dimensions that define and drive CEAs. Furthermore, this study indicates that while the primary goals of CEAs are relatively static, the means for achieving them will differ.

Dax M Weaver <d.weaver@unf.edu> graduated from the University of North Florida, Jacksonville, and is currently president of Health-Tech Consultants, Atlantic Beach, FL.