This paper presents a specialized tool for film producers and managers to extract useful information from images of different locations. The resulting framework can be used for image retrieval and other applications related to film production and location scouting. The framework uses deep learning models to extract labels, colors, seasons, emotions, and building architecture from input images. The methodology is described in detail, outlining the different steps involved in image processing and feature extraction. The label extraction module uses a transfer learning approach with pre-trained CNN models fine-tuned on a custom dataset of 106 classes. The color extraction module uses the LAB color space, a device-independent and perceptually uniform color space, in image processing applications, to accurately identify the dominant color, while the season extraction module extracts seasonal information from input images. The emotion extraction module identifies the emotion portrayed in an image, and the architectural style module automatically identifies the style of buildings in photos. Finally, the embedding module combines various vectors representing different image aspects to create a general embedding for each image.
Noise is always inevitably appeared in images, which directly affects the performance of machine vision applications. Currently, the commonly used denoising methods can be divided into three strategies: the filtering-based, model-based, and deep learning-based methods. However, they are always difficult to get the considerable accuracy and efficiency simultaneously. In this study, a novel denoising method based on autocorrelation function is investigated, which improve the image quality by utilizing the independence of useful periodic information and noise. Simulations and experiments compared with the current denoising methods confirm that the investigated method has a good comprehensive effect on noise reduction and efficiency improvement.
Multi-model data can improve the performance of image-based models for computer-aided education (CAE) by utilizing features from more dimensions rather than images only. The spatial temporal graph convolution network (ST-GCN) is a two-stream model for classifying human actions based on their skeleton data. This work proposes a novel intelligent data model (MMST-GCN) based on ST-GCN for multi-model image-based CAE. MMST-GCN integrates image and sensor data into a unified framework by introducing a sensor skeleton that provides stronger feature representation and classification abilities. Moreover, a newly modified network structure (ST-GCN+) is developed to percept the slow actions more accurately, promoting the classification performance for representative actions occurring during CAE analysis. MMST-GCN+ is evaluated on two multimodal human action datasets (MHADs) and one homemade teacher action dataset constructed by investigating the research focus on CAE. It achieves better results of 96.21%, 96.33%, 96.21% on the accuracy, macro F1-score and micro F1-score, respectively, on the Berkeley-MHAD, 91.55%, 91.93%, 91.55% of the corresponding indices on the UTD-MHAD and 79.18%, 80.38%, 79.18% on the same indicators for the self-made dataset Teacher-MHAD.
Semantic segmentation can provide spatial information about urban landscapes for autonomous driving. For safety consideration, segmentation methods should adapt to all day conditions including day and night. In this paper, a multilevel alignment in unsupervised domain adaptation network (MADA) is proposed for nighttime semantic segmentation. MADA aligns data distribution at multiple levels, utilizing the information of daytime source domain labels to improve the accuracy of nighttime semantic segmentation. Specifically, MADA includes two steps: 1) style-level alignment: twilight is chosen as intermediate domain and daytime and nighttime images are both transferred to twilight style; 2) feature-level and prediction-level alignment: discriminators are introduced after feature extractor and classifier in the segmentation network to align the distribution of features and predictions between the source and target domain images. The proposed MADA has achieved excellent nighttime segmentation results and minimized the training time. We conducted experiments on nighttime semantic segmentation datasets including Dark Zurich test set, ACDC nighttime validation set and Nighttime Driving test set. The mIoU reaches 44.3%, 37.59% and 44.92% respectively.
Myopia is a prevalent ocular disorder that often develops in early childhood and affects children's vision as well as potential cognitive ability throughout crucial developmental stages. While earlier studies have shown how myopia affects cognitive processes such as intermittent memory in adults, there are currently few straightforward scientific explanations for how myopia affects visual cognitive performance. In order to examine the relevant visual behavior through the patterns of children's eye movements, this study used eye tracking technology to collect eye movement data from children aged 8 to 10 years during a natural reading task and an image recognition task. The cognitive abilities of the control group and children with varied degrees of myopia were quantitatively measured using the MoCA. The results revealed no significant differences in the ability to interpret textual materials throughout the natural reading task. However, in the visual image recognition task, there were significant differences between the control group and the moderate myopia group in the first saccade start time to target pictures (p = 0.047 by independent sample Kruskal-Wallis test), indicating that children with moderate myopia might have slower recognition than their peers with normal vision. These results offered additional evidence that myopia could have a detrimental effect on children's vision-related cognitive function.
