Noninvasive Diagnosis of Seedless Fruit Using Deep Learning in Persimmon

Noninvasive diagnosis of internal traits in fruit crops is a high unmet need; however it generally requires time, costs, and special methods or facilities. Recent progress in deep neural network (or deep learning) techniques would allow easy, but highly accurate diagnosis with single RGB images, and the latest applications enable visualization of "the reasons for each diagnosis" by backpropagation of neural networks. Here, we propose an application of deep learning for image diagnosis on the classification of internal fruit traits, in this case seedlessness, in persimmon fruit (Diospyros kaki). We examined the classification of seedlessness in persimmon fruit by using four convolutional neural networks (CNN) models with various layer structures. With only 599 pictures of 'Fuyu' persimmon fruit from the fruit apex side, the neural networks successfully made a binary classification of seedless and seeded fruits with up to 85% accuracy. Among the four CNN models, the VGG16 model with the simplest layer structure showed the highest classification accuracy of 89%. Prediction values for the binary classification of seeded fruits were significantly increased in proportion to seed numbers in all four CNN models. Furthermore, explainable AI methods, such as Gradient-weighted Class Activation Mapping (Grad-CAM) and Guided Grad-CAM, allowed visualization of the parts and patterns contributing to the diagnosis. The results indicated that finer positions surrounding the apex, which correspond to hypothetical bulges derived from seeds, are an index for seeded fruits. These results suggest the novel potential of deep learning for noninvasive diagnosis of fruit internal traits using simple RGB images and also provide novel insights into previously unrecognized features of seeded/seedless fruits.