Analysis and Design of Document Similarity Using BiLSTM and BERT

In this paper, we propose a deep learning-based approach to measure document similarity using bidirectional encoder representations (BERTs) from transformers and its implementation as an application programming interface (API). BERT has recently shown significant improvements in natural language processing and is widely used in various applications such as question answering and text classification. We trained and refined a BERT model on a large corpus of documents to measure document similarity. The proposed API receives two text arguments and returns the degree of similarity between them. On several benchmark datasets, we demonstrated that our approach outperforms conventional state-of-the-art similarity measures. Our experimental results show that the proposed method of measuring document similarity using BERT and API is effective and efficient, and that API implementations can be used in a variety of real-world scenarios is visible.