Towards New Ways Of Evaluating Methods Of Supporting Requirements Management And Traceability Using Signal-To-Noise Ratio

Developing contemporary software solutions requires many processes and people working in synergy to achieve a common goal. Any misalignment between parts of the software production cycle can severely impede the quality of the development process and its resulting products. In this paper, we focus on improving means for measuring the quality of methods used to support finding similarities between software product artifacts, especially requirements. We propose a new set of measures, Signal-to-Noise ratios which extends the commonly used precision and recall measures. We test the applicability of all three types of SNR on two methods for finding similar requirements: the normalized compression distance (NCD) originating from the domain of information theory, and the Vector Space Model originating from computer linguistics. The results obtained present an interesting property of all types of SNR, all the values are centered around 1 which confirms our hypothesis that the analyzed methods can only limit the search space for the analysis. The analyst may still have difficulties in manually assessing the correct links among the incorrect ones.