An LLM-Powered API Testing Framework Based on Structural Similarity Analysis

In the context of the accuracy, dependability, and security of application programming interfaces (APIs), the importance of API testing is undeniable. Nevertheless, the effort required to comprehensively test APIs is significant, especially when creating valid API test cases. This study seeks to develop a framework that automates the generation of API test cases while ensuring the structural validity and functional correctness of the API test cases. The study aims to reduce the effort required by human testers to generate API test cases. The study proposes a framework that utilizes a large language model (LLM) to generate API test cases based on the concept of structural similarity. The proposed framework utilizes the LLM to generate API test cases while ensuring the functional correctness of the API test cases. To validate the structural similarity between the API test cases generated by the proposed framework, the study introduces the Jaccard similarity-based API test case structural validation mechanism. The study utilizes a multi-domain API ecosystem that contains more than 40 endpoints to validate the effectiveness of the proposed API test case generation framework. The study found that the proposed API test case generation framework achieved a high success rate of 84% in generating valid API test cases for different API domains. The study found that the proposed API test case generation framework using the LLM achieved a high mean Jaccard similarity coefficient of 0.827, indicating the high level of structural alignment between the API test cases generated by the LLM and the API specifications. Therefore, the study concludes that the proposed API test case generation framework using the LLM is effective in automating the generation of API test cases while ensuring the accuracy, dependability, and security of the API test cases.