Evidence-Based Strategies for Improving Project Outcomes

The purpose of this chapter is to provide general concepts, principles and methods for planning and implementing evaluation processes for collaborative team projects. Evaluation is a process that involves collecting and analyzing information about a project or program's activities, characteristics, and outcomes. Its purpose is to make judgments about the outcomes, improve effectiveness, and inform decisions. The terms assessment and evaluation are seen as distinct in some contexts such that evaluation is described as summative, occurring at the end of a project, and designed to pass judgement on the results based on predetermined criteria or standards; and assessment is described as formative, occurring during the project as it is implemented, and designed to improve performance. However, in project and program evaluation practice, these two processes are often intertwined with a focus on continuous improvement and adaptation of a project as it unfolds, as well as collecting evidence of achievement of the goals at specific points in the project timeline and at the end of the project. This type of evaluation is called developmental evaluation and it’s an iterative, ongoing process that combines both formative and summative aspects to support ongoing project design modifications as the project unfolds [19]. There are many approaches to evaluation based on the specific context of the project to be evaluated and the purpose of the evaluation. These can include judging the performance of the project based upon expert or user defined criteria, evaluating the functional characteristics of the project including processes and outcomes, providing evidence to support decision making relevant to the project’s operation and results, and/or supporting the participation and role of stakeholders in the project (Fitzpatrick et al. in Program evaluation: alternative approaches and practical guidelines, Pearson, New York, 2011). To determine the best approach, start with key questions: Developmental evaluation is a type of evaluation often used to support collaborative initiatives, especially in complex and dynamic situations [21]. In this type of evaluation, periodic evaluations are performed to assess the impact or influence of the collaboration, while also continuously and iteratively assessing the strategies and activities designed to achieve the desired outcomes. Developmental evaluation is used to provide feedback on the project design as it is being implemented and is useful for innovative projects. Evaluation and research have different purposes but may overlap. Evaluation supports improvements, judgements, and actionable learning while research generates knowledge about how the world works and why. Research informs evaluation—the more information that exists about the goals of the collaboration and how the various strategies and activities will achieve the outcomes, the more the evaluation can draw on that knowledge. However, the primary purpose of evaluation is to determine the effectiveness of the project activities and strategies in achieving pre-defined outcomes. Evaluation outcomes may include research-related outcomes. Success is determined by results from validated instruments used in evaluation, the participants’ perspectives, other experts in the field, and/or peer-reviewed publications. If the design of the project itself is based on research questions, then evaluation can serve to test specific theoretical frameworks.