Leveraging Transformer-based Language Models to Automate Requirements Satisfaction Assessment
CoRR(2023)
摘要
Requirements Satisfaction Assessment (RSA) evaluates whether the set of
design elements linked to a single requirement provide sufficient coverage of
that requirement -- typically meaning that all concepts in the requirement are
addressed by at least one of the design elements. RSA is an important software
engineering activity for systems with any form of hierarchical decomposition --
especially safety or mission critical ones. In previous studies, researchers
used basic Information Retrieval (IR) models to decompose requirements and
design elements into chunks, and then evaluated the extent to which chunks of
design elements covered all chunks in the requirement. However, results had low
accuracy because many critical concepts that extend across the entirety of the
sentence were not well represented when the sentence was parsed into
independent chunks. In this paper we leverage recent advances in natural
language processing to deliver significantly more accurate results. We propose
two major architectures: Satisfaction BERT (Sat-BERT), and Dual-Satisfaction
BERT (DSat-BERT), along with their multitask learning variants to improve
satisfaction assessments. We perform RSA on five different datasets and compare
results from our variants against the chunk-based legacy approach. All
BERT-based models significantly outperformed the legacy baseline, and Sat-BERT
delivered the best results returning an average improvement of 124.75% in Mean
Average Precision.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要