OSAP‐Loss: Efficient Optimization of Average Precision Via Involving Samples after Positive Ones Towards Remote Sensing Image Retrieval

Abstract In existing remote sensing image retrieval (RSIR) datasets, the number of images among different classes varies dramatically, which leads to a severe class imbalance problem. Some studies propose to train the model with the ranking‐based metric (e.g., average precision [AP]), because AP is robust to class imbalance. However, current AP‐based methods overlook an important issue: only optimising samples ranking before each positive sample, which is limited by the definition of AP and is prone to local optimum. To achieve global optimisation of AP, a novel method, namely Optimising Samples after positive ones & AP loss (OSAP‐Loss) is proposed in this study. Specifically, a novel superior ranking function is designed to make the AP loss differentiable while providing a tighter upper bound. Then, a novel loss called Optimising Samples after Positive ones (OSP) loss is proposed to involve all positive and negative samples ranking after each positive one and to provide a more flexible optimisation strategy for each sample. Finally, a graphics processing unit memory‐free mechanism is developed to thoroughly address the non‐decomposability of AP optimisation. Extensive experimental results on RSIR as well as conventional image retrieval datasets show the superiority and competitive performance of OSAP‐Loss compared to the state‐of‐the‐art.