Towards an Ecologically Valid Study of Programmer Behavior for Scientific Computing

This paper presents the motivation, design rationale and implementation detail of a study of programmer behavior in scientific computing circa 2002. We discuss the constraints of creating a retrospective baseline—the methods used, the necessary conditions—and what we have learned. We examine the problems of doing such a study and the difficulties of ecological validity in a partly controlled setting.