Training of Deep Learning Models Using Synthetic Datasets

In order to solve increasingly complex problems, the complexity of Deep Neural Networks also needs to be constantly increased, and therefore training such networks requires more and more data. Unfortunately, obtaining such massive real world training data to optimize neural networks parameters is a challenging and time-consuming task. To solve this problem, we propose an easy-to-use and general approach to training deep learning models for object detection and instance segmentation without being involved in the generation of real world datasets. In principle, we generate and annotate images with open-source software and 3D models that mimic real life objects. This approach allows us significantly reduce the effort required to gather pictures as well as automatize data tagging. It is worth noting that such synthetic datasets can be easily manipulated, e.g. to reduce the texture bias that often occurs in the resulting trained convolutional networks. Using the Mask R-CNN instance segmentation model as an example, we demonstrate that a network trained on the synthetic dataset of kitchen facilities shows remarkable performance on the validation dataset of real-world human-annotated photos. We show that our approach helps to bridge the domain gap between pre-trained models and their specific applications. In summary, such synthetic datasets help overcome the problem of acquiring and tagging thousands of images, while reducing the time and labor costs associated with the preparation of an appropriate real dataset.