SILIC: A cross database framework for automatically extracting robust biodiversity information from soundscape recordings based on object detection and a tiny training dataset

1. Passive acoustic monitoring (PAM) offers many advantages comparing with other survey methods and gains an increasing use in terrestrial ecology, but the massive effort needed to extract species information from a large number of recordings limits its application. The convolutional neural network (CNN) has been demonstrated with its high performance and effectiveness in identifying sound sources automatically. However, requiring a large amount of training data still constitutes a challenge. 2. Object detection is used to detect multiple objects in photos or videos and is effective at detecting small objects in a complex context, such as animal sounds in a spectrogram and shows the opportunity to build a good performance model with a small training dataset. Therefore, we developed the Sound Identification and Labeling Intelligence for Creatures (SILIC), which integrates online animal sound databases, PAM databases and an object detection-based model, for extracting information on the sounds of multiple species from complex soundscape recordings. 3. We used the sounds of six owl species in Taiwan to demonstrate the effectiveness, efficiency and application potential of the SILIC framework. Using only 786 sound labels in 133 recordings, our model successfully identified the species' sounds from the recordings collected at five PAM stations, with a macro-average AUC of 0.89 and a mAP of 0.83. The model also provided the time and frequency information, such as the duration and bandwidth, of the sounds. 4. To our best knowledge, this is the first time that the object detection algorithm has been used to identify sounds of multiple wildlife species. With an online sound-labeling platform embedded and a novel data preprocessing approach (i.e., rainbow mapping) applied, the SILIC shows its good performance and high efficiency in identifying wildlife sounds and extracting robust species, time and frequency information from a massive amount of soundscape recordings based on a tiny training dataset acquired from existing animal sound databases. The SILIC can help expand the application of PAM as a tool to evaluate the state of and detect the change in biodiversity by, for example, providing high temporal resolution and continuous information on species presence across a monitoring network.