The multi-learning for food analyses in computer vision: a survey

With the rapid development of food production and health management, analyses of food samples have been essential for preventing diseases and understanding human culture. Recently, food analyses have become increasingly complex and are not limited in food categorization. They also contain many advanced tasks (e.g., nutrition estimation and recipe retrieval). From existing works, two points can be concluded. First, food features are much more comprehensive and sophisticated than general samples. Second, for food analyses, multiple learning strategies (MLSs) usually achieve outperformance over general deep learning methods. However, there are few survey papers reporting food analyses with MLSs, and the main factors lead to difficulty of operation. Therefore, we intend to conduct a survey for applications of MLSs to food analyses. In this survey paper, three types of common MLSs, which are multi-task learning (MTL), multi-view learning (MVL) and multi-scale learning (MSL) strategies, are presented in terms of their guidance, typical works, algorithms and final aggregation methods. Additionally, food characteristics are proposed to be closely related to the difficulty of food analyses. We comprehensively conclude food characteristics as nonrigid, complex in arrangement, and large (small) in intraclass (interclass) variance. Moreover, some experimental results of MLSs are also presented and analyzed in this paper. Based on these results, insightful suggestions for MLSs implementation are proposed. Finally, the promising tendency of MLSs applications in the future is discussed.