Creating Meaningful Assessments for Professional Development Education in Software Architecture

Extensive instructional materials have been developed and used for courses in specific software architecture topics offered at the Software Engineering Institute (SEI) at Carnegie Mellon University, to support the instructional goals laid out by the creators of the SEI's professional education program and the designers of the individual courses. However, to date, these courses have lacked any assessment component, certification for the course being granted solely on attendance. For an assessment component to be meaningful, it must derive from and support these instructional goals, determine which goals must be assessed and how to assess them, and determine how best to assess whether those goals have been achieved through application of the instructional materials, lectures, and activities which are included in each two-day course. In order to ensure that the course assessments target the intended learning goals, we developed content for low-stakes assessment components grounded in education theory, combining current knowledge of educational psychology and the software engineering domain to create evaluations that will effectively determine whether participants in this course have learned what the curriculum developers and the instructors intend them to learn.