Evaluating Software Engineering Technologies

Many new software development practices, tools, and techniques have been introduced in recent years. Few, however, have been empirically evaluated. The objectives of this study were to measure technology use in a production environment, develop a statistical model for evaluating the effectiveness of technologies, and evaluate the effects of some specific technologies on productivity and reliability. A carefully matched sample of 22 projects from the Software Engineering Laboratory database was studied using an analysis-of-covariance procedure. Limited use of the technologies considered in the analysis produced approximately a 30 percent increase in software reliability. These technologies did not demonstrate any direct effect on development productivity.