Consensus design of a calibration experiment for human fear conditioning

Measurement practice in experimental psychology is heterogeneous. Psychometric properties of measurement methods are sometimes difficult to establish, leaving researchers with few criteria to select between methods. To overcome this obstacle, calibration is a standard metrological procedure in which well-defined values of a latent variable are generated in an established experimental paradigm. These intended values can then serve as validity criterion, to rank methods by their measurement error. This approach requires community consensus on which calibration experiment to use. Here, we develop a consensus process and exemplify it in the field of human fear conditioning. Based on a literature review and survey of N = 96 experts, we propose a calibration experiment and settings for 25 design variables to calibrate the measurement of "learned unconditioned-stimulus (US) expectation". This latent variable and the calibration procedure were chosen to be as theory-free as possible, and to have the widest experimental applicability among the surveyed researchers. Besides establishing a specific calibration procedure for fear conditioning, the general calibration process we outline may serve as a blueprint for calibration efforts in other subfields of experimental psychology that are in need of measurement refinement.