Deep learning presents promising solutions for the restoration and preservation of visual arts, including old color photographs or paintings, which are prone to degradation over time, enabling the vibrant imagery to be effectively revived and maintained. In this paper, we propose a methodology for restoring visual arts based on deep learning techniques purely trained on synthetic data, thereby involving the generation of a dataset that incorporates respective defects, the training of a respective defect segmentation model, and the inpainting using predicted segmentation maps. Through qualitative and quantitative analysis, we demonstrate the potential of our approach in addressing the scarcity of ground truth data and effectively restoring old visual arts by synthetic training on specific defects observed in historical artworks.
Tabletop scene synthesis is an important but underexplored area in scene generation. Existing methods typically focus on larger furniture pieces, often neglecting the fine-grained details of smaller objects positioned on surfaces like tabletops. To address this gap, we propose the first method specifically designed for tabletop scene synthesis using a diffusion model. Our diffusion model, trained on a large dataset of tabletop scenes, achieves state-of-the-art performance compared to current scene generation methods.
In addressing the issue that the Thing of Interest (ToI) in a photograph appears significantly smaller than that being directly perceived from that in the physical world, image editing techniques have been developed to enlarge ToI while maintaining appearance of other areas. However, these methods can suffer from issues such as loss of spatial perception, distortions of objects, the lack of automation, etc. To address these issues, we propose a novel ToI enlarging method based on Seam Carving method. The proposed method is composed of seam insertion and deletion processing, and introduces wide area seam energy mechanism, which considers impact of inserted or deleted seams on adjacent areas, in addition to preservation to salient objects, image structure, gradient and mask energies. Moreover, the proposed method introduces the Segment Anything Model to automatically generate masks representing the ToI. To validate the effectiveness of the proposed method, we conducted two subjective evaluation experiments in this study. The experimental results demonstrate that the proposed method can produce resulting images that reflect perceptual impressions, and can preserve area other than ToI.
Japanese Sign Language (JSL) and Manually Coded Japanese (MCJ) are the two main forms of sign language used in Japan. The former differs significantly from spoken Japanese, while the latter closely aligns with spoken syntax. Pidgin Sign Japanese (PSJ) is an intermediate form that heavily relies on nonmanual signals such as facial expressions and head movements for grammatical nuances. Current sign language recognition (SLR) systems predominantly focus on MCJ, neglecting the challenging properties of PSJ. This paper proposes an annotation assistance tool designed to automate the annotation of non-manual and spatial elements in PSJ. Our tool significantly reduces the manual effort required for annotation by using state-of-the-art methods for tracking human pose, hand, and face landmarks, along with recognizing facial action units (FAUs). Validation on a preliminary dataset of 30 videos containing over 90 instances of nonmanual elements demonstrated a $\mathbf{4 0 \%}$ reduction in annotation time, highlighting our proposal’s efficiency and effectiveness in handling the complexities of PSJ.
Individuals facing challenging and threatening situations significantly strain their minds and bodies. As a result, they may experience emotional, physical, or psychological stress. However, people perceive stress differently depending on how long or how severe their exposition to traumatic events is. Prolonged stress can cause significant and often inexplicable damage to both the body and mind. Because of that, early detection of human stress levels has become a prominent means of early diagnosis. Therefore, the major goal of this study is to investigate how human stress detection using Electroencephalography (EEG) which has been applied to EEG channel selection using Mutual Information (MI) with Minimum Redundancy Maximum Relevance (mRMR) (Mi$m R M R$) and the ensemble learning algorithm mainly using the method of bagging, boosting and stacking works in classifying different stress levels. On the other hand, the features were extracted using time-domain, frequency domain and timefrequency domain analysis containing relevant features related to stress data. Based on the experiment results, the ensemble learning method bagging performs better compared to the boosting method and can classify the data with the highest accuracy, highest F1 score, and highest precision, of 88 percent and highest precision of 94 percent. Hence, this shows the capability of the ensemble learning algorithms to classify stress EEG data that has been selected using (Mi-mRMR) algorithms as a means of early stress detection and diagnosis model.
This paper proposes a method for controlling disaster victim avatars in nursing education materials using the metaverse. The method combines animation with control by a wearable motion capture device to achieve natural movements. While metaverse-based simulators have been developed for various fields, they often lack the naturalness and fidelity of human models. To address this, we developed a prototype system that enhances avatar movements by integrating animation with motion capture, and evaluated the avatars’ impressions.
Currently, skilled farmers judge the ripeness of the Shine Muscat grape variety by looking at the color on the surface of the grapes. However, the color of Shine Muscat grapes does not change much as they grow, and there are individual differences in the way the color is perceived. Furthermore, the same color can look very different depending on the exposure to sunlight and shadows. Therefore, there is a need for a system that can quantitatively determine the color of Shine Muscat grapes to pass on the harvesting techniques of experienced farmers to amateurs and inexperienced farmers. This research aims to improve the accuracy of the color estimation of Shine Muscat grapes using deep learning. We propose a method to estimate the color of individual grapes using a color estimation model with a self-attention mechanism, from which the color of the whole bunch is estimated. A Vision Transformer model with a self-attention mechanism was found to improve the color estimation accuracy to $96.9 \%$. Furthermore, by eliminating outliers using the interquartile range, a color estimation accuracy of $97.2 \%$ could be achieved, demonstrating the effectiveness of the new color estimation model.
Rescue UAVs can support important tasks such as rapid response and manoeuvrability, search and rescue, monitoring and assessment, and delivery of relief materials and communication. However, in natural disasters and accidents scenarios, there are difficulties in the autonomous selection of UAV landing sites. This study proposes a landing point detection method that considers UAV attitude and fuses visual depth estimation with matrix computation. Considering image local visual features and global features, the highlight is to propose a multi-layer feature fusion framework, including visual depth estimation module based on Depth-Anything-V2, planar matrix estimation based on Sobel operator, and optimal landing point detection considering UAV attitude. At present, the initial validation has been completed at UseGeo, and the landing point selection is as expected.
EEG data have significant academic and clinical value because of their non-invasive nature which allows for easy extraction. At the same time, they are also prone to distortion from noise, making quality interpretation challenging. Meanwhile, the advancements of Large Language Models (LLMs) have made them applicable to complex tasks. In this study, we propose a fine-tuned LLM classifier that is able to classify motor imagery EEG data. The research compares the performance of fine-tuned LLM with that of Support Vector Machine, Random Forest, and Multi-Layer Perceptron models. The results show that the fine-tuned LLM can achieve comparable performance compared to traditional machine learning methods and demonstrate the possibility that the finetuned LLM could be applied to more diverse tasks based on EEG data.
The emergence of sophisticated variant botnet attacks like Okiru poses significant security challenges to the proliferating IoT devices. This study proposes a reinforcement learning (RL)-based defense system for IoT networks. Our system employs an RL agent to analyze network packets in real-time and implement adaptive security measures. By leveraging AI techniques, this research offers a proactive approach to addressing complex cyber threats in IoT environments.
This paper presents a method to analyze table utilization and customer movements in fast food courts through video surveillance data using methods from machine learning. The system that has been proposed YOLOv8 for object detection, DeepSORT for object tracking and a custom status order algorithm to classify the table activities. A real-world implementation on existing infrastructure in SA such as the Tapah Southbound RSA (Rest and Service Area) Food Court were undertaken to address challenges posed by non-optimal camera angles, occlusion issues etc. Based on people (people) and objects (bowls, plates, bottles or cups), the system classifies table statuses to eat, drink, eat_drink sit-down empty. To improve the classification accuracy and to address this unstable detection issue, a status order hierarchy was introduce. This paper summarizes the system performance of occupancy rates, activity durations and customer patterns analysis while highlighting its limitations by potential embedding in started a business intelligence for food court operations. Consistent with our confirmation, we find high table utilization rates and clear behavioural patterns that can provide insights for service optimization or layout improvement. It also points to avenues for further improvement in data collection methods that cater to improved detection of nuanced activity under challenging visual settings.
Physical inactivity is one of the major health issues in modern society. Music has shown great potential in enhancing exercise performance and motivation, offering an economical and non-invasive way to improve the exercise experience. However, the number of music tracks available to enhance workout performance is limited, and frequent listening to these tracks can lead to fatigue, diminishing their effectiveness. To address this issue, we explored methods for generating personalized workout music. First, we conducted a detailed analysis of the musical and acoustic features of existing workout music to identify which features are most crucial for enhancing exercise performance and motivation. Based on these features, we developed a GANbased music style transfer model that incorporates these highly relevant workout music features, allowing these characteristics to be embedded into music from different genres. Our results show that our new model generates music that has a higher similarity to workout music compared to the baseline model.
Soft biometric de-identification is an emerging field in biometrics, offering a balance between privacy protection and recognition accuracy. In this work, we present a novel identity-preserving soft biometric obfuscation method that uses the latent feature space of a trained generator and employs deep neural networks. The proposed method aims at preserving the identity of individuals while de-identifying their soft biometric attributes. Specifically, a novel feature space transformation network is designed to preserve identity while modifying facial attributes while minimizing the disclosure of identity. The proposed feature transformation network is the first of its kind developed specifically for controllable adaptive de-identification. Furthermore, we implemented an identity preservation mechanism, utilizing the FaceNet architecture to compute embedding vectors for both the original and deidentified images. Through extensive validation on benchmark datasets such as VGGFace2 and CelebA, we have demonstrated the effectiveness and robustness of our method.
Brain-Computer Interface (BCI) make it possible to identify the physiological changes that are unnoticeable to the normal eye. BCI helps improve the diagnostic capacity to identify the drivers’ emotional states. Previous studies on the emotional states of drivers mostly concentrated on the drivers’ attentiveness, fatigue, and aggressive driving style. When driving a vehicle, drivers with negative emotions may make errors of judgment. One potential preventive measure is to examine the emotions that exist when operating a car. This paper discusses the architecture of the classification methods for the EEG signal when the drivers encounter several situations in the simulated environment. The classification focuses only on KNN and Fuzzy classifiers. The classifier will classify the emotions into five classes: fear, nervous, relax, surprise, and focus.
Cross-subject classification is of great practical value in the mental monitoring. The trained model on a person can be transferred to another person without retraining. To date, it is achieved by global domain adaptation without considering differences in subdomain distributions. In this case, there is a lack of sensitivity to specific information associated with each category. To solve this problem, we proposed a deep subdomain adaptation network (DSAN) to estimate mental workload levels across different persons. In the proposed DSAN, the first temporal and spatial layers were designed as a feature extractor. The features extracted by the feature extractor were aligned between the source samples and target samples in each subdomain separately. The alignment loss calculated by local maximum mean discrepancy (LMMD) was back-propagated to update the weights of the feature extractor to enhance the feature extraction performance. Subdomain adaptation was achieved over iterations during the model training. The proposed subdomain adaptation is not specialized for a particular feature extractor, as shown in this paper. It is universal and can be applied after any feature extractors. Two datasets (Dataset MATB and Dataset SFE) were used to evaluate the proposed DSAN. The results showed that the proposed DSAN outperformed the compared methods in terms of classification accuracy, showing an elevation of $3 \% \sim 7 \%$. This study provides an effective solution for the cross-subject mental workload classification and will promote practical applications of mental workload monitoring.
Messenger ribonucleic acid (mRNA) vaccines, despite their rapid degradation, play a critical role in pandemic response due to their high efficacy and swift production capabilities. Accurate prediction of mRNA vaccine degradation rates is vital for determining their shelf life and maintaining efficacy. This study presents a tetra-nitrogenous-base label encoding approach (4-ntb-lbA) integrated with a novel hyperparameter optimization (HPO) technique, named the Hyperparameter Optimization Penalizer (HOPeR), aimed at enhancing prediction precision. The state-of-the-art hybrid Dense-BiGRU-BiLSTM-BiLSTM (Hybrid_LGSS) model underwent rigorous testing to assess both the model’s performance and the proposed methodologies, broadening its interdisciplinary applications. Results indicate that the 4 -ntblbA method, which leverages bioinformatic data, substantially improves prediction reliability and accuracy while reducing error rates (Set_I: training loss $={0. 0 9 0 4}$, validation loss $=$ 0.0938; Set_II: training loss $=0.0141$, validation loss $=0.0145$), assessed with mean column-wise root mean square error (MCRMSE). The HOPeR approach demonstrated effectiveness across various models and HPO algorithms, including Particle Swarm Optimization (PSO), Bayesian Optimization with Gaussian Process (BOGP), and the RIME optimization algorithm (RIME), by minimizing the risk of suboptimal hyperparameter configurations while promoting fast convergence through penalization strategies. These findings validate the proposed approaches’ flexibility, applicability, and robustness across different algorithms. Additionally, beyond advancing mRNA vaccine degradation prediction, this research introduces a versatile framework for HPO with broad applicability. By fostering interdisciplinary integration and innovation, this paper significantly enhances the precision and efficiency of predictive models in bioinformatics, machine learning (ML), and beyond, thereby paving the way for future advancements across various scientific and engineering disciplines.
In the field of object detection, particularly in medical imaging, the scarcity of data often poses a significant challenge to model performance. To address this issue, this study proposes a semi-supervised learning approach based on a generative model. We begin by fine-tuning a pre-trained generative model using our dataset to better adapt the generative model to our specific data distribution. The fine-tuned generative model is then used to generate additional unlabeled data. These generated unlabeled data, combined with the original dataset, are employed in a semi-supervised training process. Experimental results demonstrate that our method significantly enhances the performance of the object detection model, especially in scenarios with limited labeled data, such as medical imaging. By incorporating the generated unlabeled training data into the semi-supervised framework, we observed a notable improvement in model accuracy. Specifically, our experiments showed an increase of up to $6.92 \%$ after adding the generated iamges. Moreover, it is foreseeable that incorporating a higher proportion of generated unlabeled data could lead to even more significant improvements in performance.
The objective of this research is to ascertain the posture of an object, which is one of the essential pieces of information required for grasping an object. This will be achieved by capturing images of the object with a single camera mounted on an autonomous mobile robot. As a single camera is unable to observe an object from multiple directions, data from the other side, which is not captured by the camera, cannot be obtained. As a result, it is challenging to obtain a comprehensive understanding of the object’s shape. Therefore, we propose a methodology for fitting a three-dimensional shape model of the target to the point cloud data obtained via a depth camera capable of observing the surrounding three-dimensional shape. This paper demonstrates the efficacy of the proposed method for estimating the attitude of empty cans on the road, which are the target of refuse collection operations. This is achieved by modeling the shape of a typical empty can and measuring the similarity between the captured 3D shape and the model shape.
Eye-writing, the act of drawing letters using eye movements, offers a promising avenue for human-computer interaction, especially when coupled with electrooculogram (EOG) recognition techniques. However, achieving high accuracy in eye-writing recognition using deep learning requires large datasets, which is challenging due to the time-consuming nature of data collection and privacy concerns. In this paper, we introduce a diffusion model capable of producing synthetic EOG signals, demonstrating the feasibility of generating high-quality bio-signal data. This approach not only mitigates the challenges of data collection but also facilitates the improvement of pattern recognition accuracies in small bio-signal datasets.
In digital interfaces, companies often exploit psychological vulnerabilities to increase user engagement, particularly on platforms like YouTube. This study explores these manipulative design tactics, known as dark patterns, and their underexplored effects on attention and psychological well-being. We introduced a feature that issues warnings when certain criteria, like content loads or video views, are met. A user study with ten participants showed that these warnings helped reduce unintended content consumption and time-wasting behaviors, though the impact varied. The research suggests AI-enhanced warnings could offer a more adaptive solution to combat dark patterns.
