This paper proposes an image recognition system that recognizes gender by interfacing with an infrared camera and a thermal image sensor. After acquiring image data for object recognition, image pre-processing is performed inside the IoT module to generate an input image. Finally, an object recognition deep learning model is trained through the generated image. Then, the model is optimized to obtain the performance of the object recognition model using IR images and thermal image sets. The model’s performance was evaluated by mAP and IOU, and the optimal overlay ratio was improved by about 15%. Thus, the development system is cost competitive and can robustly recognize gender in various weather and light intensity conditions using IR and thermal images. Furthermore, the developed system makes it possible to improve the recognition rate by combining and analyzing the infrared camera module and the temperature sensor image.

This paper proposes a thermal image-based diagnostic process using explainable artificial intelligence that can interpret 3D printer states. Through this process, it is possible to classify the status of the 3D printer, check the classified active area, and help determine the status of the equipment. First, the proposed methodology stores the thermal image through an acquisition system that visualizes the thermal image of the 3D printer. Second, training a convolution neural network (CNN) model is carried out by defining the state data for the thermal image. Finally, based on the CNN model, the proposed process classifies the thermal images of the 3D printer and visualizes the area that is the basis for the inference classification result using an explanatory artificial intelligence algorithm. Through the proposed process, the operator can check the 3D printers’ condition through the thermal image. By presenting the basis for inference the state, even unskilled operators can quickly check the state of the 3D printer.

This study aims to develop an artificial intelligence-based model for analyzing the condition and detecting anomalies by encoding time-series data from manufacturing processes as images. Deep learning has demonstrated the significance of data analysis and anomaly detection in the vision field, and Convolutional Neural Networks (CNN) models have shown exceptional performance and high applicability in image analysis. Based on this, our study intends to utilize image encoding techniques to perform anomaly detection on time-series data. Data such as force, vibration, and sound from equipment during the manufacturing process are collected and transformed into images using various methods, including Gramian Difference Angular Field, Gramian Summation Angular Field, Markov Transition Field, and Recurrence Plot (RP). The transformed image data is then trained and classified for equipment conditions using various CNN models. Finally, we adopt the RP image encoding method and ResNet50 model, which demonstrated the highest accuracy of 99.6

In the field of occupational health, accurate and efficient ergonomic risk assessment is of paramount importance. This study presents an innovative computer vision-based approach to automate workload assessment with the goal of preventing work-related musculoskeletal disorders (WMSDs). Our methodology, which uses a two-step algorithm, captures 3D skeletal representations from video footage of workers performing tasks such as lifting, lowering, and carrying. We conducted experiments stratified by object weight and task height to simulate different workplace scenarios. Our system uses AlphaPose for human pose estimation and MotionBERT for 3D skeleton extraction to provide a dynamic and accurate assessment of workload. By analyzing spatial and contextual data, this method provides real-time ergonomic risk assessment, which is critical in today’s variable work environments.

These days, many assembly lines are becoming automated, leading to a trend of decreasing defect rates. However, in assembly lines that have opted for partial automation due to high cost of construction, defects still occur. The cause of defects are that the location of the work instructions and the work field are different, which is inefficient and some workers who are familiar with the process tend not to follow the work instructions. As a solution to establishing a system for object detection without disrupting the existing assembly lines, we decided to use wearable devices. As a result, it is possible to solve the problem of spatial constraints and save costs. We adopted the YOLO algorithm for object detection, an image recognition model that stands for “You Only Look Once”. Unlike R-CNN or Fast R-CNN, YOLO predicts images with a single network, making it up to 1000 times faster. The detection point was determined based on whether the pin was fastened after the worker’s hand appeared and disappeared. For the test, 1000 field data were used and the object-detection performance, mAP, was 35%. The trained model was analyzed using seven regression algorithms, among which Xgboost was the most excellent, with a result of 0.15. Distributing labeling and class-specific data equally is expected to enable the implementation of a better model. Based on this approach, the algorithm is considered to be an efficient algorithm that can be used in work fields.

Automating the monitoring of machine tools poses a significant challenge, with previous studies relying on machine vision to address this issue, primarily focusing on measuring specific tool wear using custom algorithms. In this study, we introduce an automated tool wear monitoring process capable of assessing wear across various tools. Utilizing deep neural network models for image classification and segmentation, our proposed process effectively masks areas where tool wear occurs. The image classification model selects the best-performing backbone model based on the highest F1 score. To account for varying depths-of-cut lines among different tools in the masked area, we introduce an algorithm that utilizes the Hough transform to determine the horizontal angle with the cut line. By systematically measuring the maximum flank wear of diverse tools, including indexable, ball, and solid endmills, using this method, we can replace conventional manual processes. This approach can be extended and automated for various machine tools, offering substantial potential for enhancing manufacturing tool monitoring processes.

This study proposes a methodology for detecting anomalies in the manufacturing industry using a self-supervised representation learning approach based on deep generative models. The challenge arises from the limited availability of data on defective products compared with normal data, leading to degradation in the performance of deep learning models owing to data imbalances. To address this limitation, we propose a process that leverages the Gramian angular field to transform time-series data into images, applies StyleGAN for image augmentation of anomalous data, and utilizes a boosting algorithm for classifier selection in supervised learning. Additionally, we compared the accuracy of the classifier before and after data augmentation. In experimental cases involving CNC milling machine data and wire arc additive manufacturing data, the proposed approach outperformed the approach before augmentation, resulting in improved precision, recall, and F1-score for anomaly detection. Furthermore, Bayesian optimization of the hyperparameters of the boosting algorithm further enhanced the performance metrics. The proposed process effectively addresses the data imbalance problem, and demonstrates its applicability to various manufacturing industries.

Plastics, with their versatility and cost-effectiveness, have become indispensable materials across various industries. However, the improper disposal and mismanagement of plastic waste have led to significant environmental issues, including pollution, habitat destruction, and threats to wildlife. To address these challenges, numerous methods for plastic waste sorting and recycling have been developed. While conventional techniques like near-infrared spectroscopy (NIRS) have been effective to some extent, they face difficulties in accurately classifying chemically similar samples, such as polyethylene terephthalate (PET) and PET-glycol (PET-G), which have similar chemical compositions but distinct physical characteristics. This paper introduces an approach that adapts image sensors and deep learning object detection algorithms; specifically, the You Only Look Once (YOLO) model, to enhance plastic waste classification based on the shape of the waste. Unlike conventional methods that rely solely on spectral analysis, our methodology aims to significantly improve the accuracy and efficiency of classifying plastics, especially when dealing with materials having similar chemical compositions but differing physical attributes. The system developed using image sensors and the YOLO model proves to be not only effective but also scalable and adaptable for various industrial and environmental applications. In our experiments, the results are strikingly effective. We achieved a classification accuracy rate exceeding 91.7% mean Average Precision (mAP) in distinguishing between PET and PET-G, surpassing conventional techniques by a considerable margin. The implications of this research extend far and wide. By enhancing the accuracy of plastic waste sorting and reducing misclassification rates, we can significantly boost recycling efficiency. The proposed approach contributes to a more sustainable and efficient plastic waste management system, alleviating the strain on landfills and mitigating the environmental impact of plastic waste, contributing to a cleaner and more sustainable environment.

This paper proposes a methodology to find the hyperparameter that minimizes the compliance of the topology using a univariate dynamic encoding algorithm for searches. The difficulty in topology optimization is that there are various parameters that affects the topology of the modeling design such as the size of the elements to be designed, skip level, Poissions’ ratio, volume fraction limit, penalization power, and filter size. In most studies, these parameters were fixed and topology optimization was performed until the compliance value (cost value) converged. However, the result of the final topology optimization changes according to the change of the value of the hyperparameter in the design stage, it is necessary to study the optimization according to the change of the hyperparameters. To solve this difficulty, this paper pro- poses the methodology for hyperparameter optimization using a univariate dynamic encoding algorithm for searches. The hyperparameters were optimized using the proposed method with three topology optimization problems (both 2D and 3D cantilever beam, and 3D wheel) to show the effectiveness of the proposed methodology.

This study proposes a process for detecting anomalies in the manufacturing industry, where data imbalance is a frequent problem. The labeling of anomalies can be challenging owing to the different types of anomalies. To address this issue, we used clustering based on the distribution of acquired normal data. We extracted latent vector values from normal image data as features using the Style-GAN method, after conversion of the time-series data. Subsequently, we performed dimensionality reduction through Locally Linear Embedding (LLE) using the extracted latent vector values and selected the Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) for anomaly detection. We verified the proposed process using a milling dataset that included measurements of vibration, force, and noise. The evaluation of the process included dimensionality reduction methods such as Locally Linear Embedding (LLE), Principal Component Analysis (PCA), Kernel PCA, Singular Value Decomposition (SVD), and ISOmetric mapping (ISO) produced an F-1 score of 0.86.

This paper proposes a method to find a parameter that minimizes the average compressor power for R134a cycle using a univariate dynamic encoding algorithm for searches. optimizing modeling parameters for the refrigeration cycle is difficult because it has several parameters to conFigure such as the sizes of the compressor, the evaporator, the condenser, the cooler fan, and the internal heat exchange. The univariate dynamic encoding algorithm is a searching algorithm for the optimum point under a multivariate system variable with constraints to solve the aforementioned difficulty. The parameters are assigned to the simulation model and the compressor power is logged, then a univariate dynamic encoding algorithm for searches is applied to find the minimum point of the compressor power, corresponding to the modeling parameters. The refrigeration modeling parameters are optimized by the proposed method with the modeled specifications. The optimization results ensure the average compressor power of 28. 65W with system parameters related to the condenser, and evaporator.

This paper researches an IoT module-based pipe classification system for a flexible manufacturing system that recognizes the size and length of pipes used in the painting process of high-pressure pipes. The proposed system is composed of an IoT module, USB camera, and edge TPU for pipe classification. The proposed system recognizes the type of pipe by three processes; object detection of the pipe, line detection algorithm of the three regions of interest, and pipe classification algorithm based on the line detection algorithm. Furthermore, the proposed system enables web-based real-time monitoring, providing convenience to workers and helping them make quick decisions. The IoT module interfaces with the painting robot and the sequence control that paints for each type of pipe is executed in the painting robot, allowing flexible manufacturing of the painting process.

This paper presents a vibration analysis process using a convolutional neural network (CNN) model and a gradient class activation map (Grad-CAM), based on a model and feature layer selection process. The main problem associated with conventional bearing fault diagnosis is the need to calculate defect frequencies, which requires that all bearing-related information should be known beforehand. The extraction of information regarding the mechanical equipment and the subsequent generation of a database requires significant time and cost. The proposed methodology can detect frequencies without any prior information by visualizing the activation area of the CNN model. The proposed approach helps overcome the drawbacks of the conventional diagnosis using Grad-CAM, and it indicates the rationale of classification, thus explaining the cause of vibration data for a defect diagnosis. The paper presents a process that visualizes and analyzes the activated region through the conversion of vibration signals into spectrogram images and use of CNN models. The proposed method further serves as a process of selecting the appropriate CNN model and its feature layers among several models. The vibration data were collected from a motor using an accelerometer and IoT module. The feasibility of the proposed methodology was verified using the recorded data. Spectrogram images for five situations (normal, inner fault, outer fault, ball fault, and cage fault) were generated from the acquired data. A VGG-19 model with an accuracy of 99.92% was selected based on transfer learning of three CNN models (ResNet50, VGG-16, VGG-19) using these spectrogram images. Finally, the feature layer of dimension 28 × 28 is selected for the Grad-CAM activation model. The visualization model was analyzed and compared to conventional defect frequency analysis methods. The results analyzed through the activation area near the defect frequency region of the conventional frequency analysis show that the use of deep learning in the proposed model helps evaluate the frequency intervals required to determine defects without any prior information regarding mechanical specifications.

In general, an experimental environment for deep learning assumes that the training and the test dataset are sampled from the same distribution. However, in real-world situations, a difference in the distribution between two datasets, i.e. domain shift, may occur, which becomes a major factor impeding the generalization performance of the model. The research field to solve this problem is called domain generalization, and it alleviates the domain shift problem by extracting domain-invariant features explicitly or implicitly. In recent studies, contrastive learning-based domain generalization approaches have been proposed and achieved high performance. These approaches require sampling of the negative data pair. However, the performance of contrastive learning fundamentally depends on quality and quantity of negative data pairs. To address this issue, we propose a new regularization method for domain generalization based on contrastive learning, called self-supervised contrastive regularization (SelfReg). The proposed approach use only positive data pairs, thus it resolves various problems caused by negative pair sampling. Moreover, we propose a class-specific domain perturbation layer (CDPL), which makes it possible to effectively apply mixup augmentation even when only positive data pairs are used. The experimental results show that the techniques incorporated by SelfReg contributed to the performance in a compatible manner. In the recent benchmark, DomainBed, the proposed method shows comparable performance to the conventional state-of-the-art alternatives.

도심지 도로에 발생하는 지하공동은 최근 수년간 사람들의 안전을 위협하는 사회적 현안으로 대두된 바 있다. 이에 따라 도심지 지하공동의 존재를 선제적으로 파악하는 일의 중요성이 증대되고 있다. 지하공동을 탐지하기 위해 일반적으로 지표투과 레이다 시스템을 사용하는데, 이는 지하에 전자기파 펄스 신호를 방사시킨 후, 지하의 불연속면에서 산란되어 돌아온 신호를 수신하여 영상화하는 기법이다. 실제 환경에서 GPR 데이터를 수집하고 지하에 공동이 존재하는지 판단하기 위해서는 많은 인력과 시간이 필요하고 수집된 데이터의 개수가 부족한 한계가 있다. 따라서 파형 영상 분석에 사용되는 인력을 최소화하고 소요되는 시간을 절약하기 위해 실제 데이터를 대신할 가상 데이터를 생성하여 활용하고자 한다. 본 논문에서는 시뮬레이션을 통해 생성된 파형 영상들을 활용해 깊은 신경망 모델인 VGGNet-16을 학습하여, 지하 토양에 공동이 존재하는지 판단하는 방법을 제안한다. 우리는 시뮬레이션을 통해서 공동이 존재하는 지하 토양의 파형 영상과 존재하지 않는 지하 토양의 파형 영상을 생성하여 학습 데이터로 사용하였다. 실험 결과, 공동의 특징을 학습한 VGGNet-16은 공동 모델이 있는 지하 토양과 공동모델이 없는 지하 토양을 분류하는데 92.3%의 정확도를 보였다. 더불어 학습된 모델이 공동의 특징을 적절히 학습했는지 확인하기 위해, 모델이 분류 시 입력 영상에서 어떤 부분을 보고 판단하는지 Score-CAM으로 시각화하였다. 이를 토대로 공동이 존재할 시 나타나는 특징을 깊은 신경망 모델이 적절히 학습하였다고 판단하였다. 향후 연구에서는 시뮬레이션으로 생성된 데이터와 실제 공동 데이터 간의 유사도를 검증하고 실제 공동 데이터를 통해서 학습된 모델의 성능을 검증해볼 필요성이 있다고 판단된다.

객체 검출 기술은 입력 영상의 낮은 화질 및 해상도 등의 한계에 부딪혀서, 현실에서는 성능이 저하될 수 있다. 본 논문은 현실에서의 객체 검출 기술 성능을 실질적으로 향상시키고자, 깊은 신경망 기반 초해상도 (Super-Resolution) 모델을 활용하는 방법을 제안한다. 우리는 실험으로 얼굴 검출과 일반적 객체 검출, 두 조건 모두에 대해 검증하였으며, 현실의 영상 환경을 시뮬레이션하기 위해 복합적인 영상 열화 방법을 사용하였다. 실험 과정으로, 먼저 전통적인 영상 크기 변환 방법인 쌍입방 보간법(Bicubic Interpolation)과 가우시안 노이즈, 가우시안 블러 등을 복합적으로 활용하여 영상을 열화 및 다운샘플링(Down-sampling)한 후, 쌍입방 보간법과 깊은 신경망 기반 최신 초해상도 모델인 EDSR(Enhanced Deep Residual Networks for Single Image Super-Resolution) 을 사용하여 각각 영상을 복원한다. 이렇게 생성한 영상을 얼굴 검출용 깊은 신경망 모델로 RetinaFace, 일반적 객체 검출용 깊은 신경망 모델로는 EfficientDet-D7을 활용하여 평균 정밀도(Average Precision)를 비교 및 분석하였다. 이러한 과정을 Widerface와 COCO 데이터 집합을 기반으로 정량적 실험을 수행하였고, 여러 열화 방법 조건에서 객체 크기가 작은 경우, 얼굴 검출은 최대 54.8%, 일반적 객체 검출은 370.0%만큼 성능이 향상되는 것을 검증하였다. 따라서 깊은 신경망 기반 초해상도 모델을 활용하여, 현실에서 객체 검출 기술의 실질적 성능을 향상시킬 수 있을 것으로 판단된다.

The ReLU(Rectified Linear Unit) function has been dominantly used as a standard activation function in most deep artificial neural network models since it was proposed. Later, Leaky ReLU, Swish, and Mish activation functions were presented to replace ReLU, which showed improved performance over existing ReLU function in image classification task. Therefore, we recognized the need to experiment with whether performance improvements could be achieved by replacing the RELU with other activation functions in the super resolution task. In this paper, the performance was compared by changing the activation functions in EDSR model, which showed stable performance in the super resolution task. As a result, in experiments conducted with changing the activation function of EDSR, when the resolution was converted to double, the existing activation function, ReLU, showed similar or higher performance than the other activation functions used in the experiment. When the resolution was converted to four times, Leaky ReLU and Swish function showed slightly improved performance over ReLU. PSNR and SSIM, which can quantitatively evaluate the quality of images, were able to identify average performance improvements of 0.06%, 0.05% when using Leaky ReLU, and average performance improvements of 0.06% and 0.03% when using Swish. When the resolution is converted to eight times, the Mish function shows a slight average performance improvement over the ReLU. Using Mish, PSNR and SSIM were able to identify an average of 0.06% and 0.02% performance improvement over the RELU. In conclusion, Leaky ReLU and Swish showed improved performance compared to ReLU for super resolution that converts resolution four times and Mish showed improved performance compared to ReLU for super resolution that converts resolution eight times. In future study, we should conduct comparative experiments to replace activation functions with Leaky ReLU, Swish and Mish to improve performance in other super resolution models.

This paper presents an algorithm for the fault detection process using correlation based clustering. Conventional clustering-based fault detection calculates the fault index through dimension reduction and clustering algorithm, and detects when the index exceeds the probabilistic limit. When an abnormality is detected through clustering through dimension reduction, it is difficult to perceive the physical meaning of the original data because the data is transformed. However, when detecting and analyzing anomalies in many engineering problems or data analysis, the physical meaning of the data is one of the important information. This paper proposes an anomaly detection process of correlation-based clustering which could recognize the relationship of data. The proposed anomaly detection algorithm selects highly correlated datasets, generates each clustering model, and calculates a fault index using stochastic distances. The fault detection performance was provided and verified using hydraulic test equipment data, and the results were compared with the conventional methods. (C) 2020 Elsevier B.V. All rights reserved.

Deep learning-based mobile imaging applications are often limited by the lack of training data. To this end, researchers have resorted to using synthetic training data. However, pure synthetic data does not accurately mimic the distribution of the real data. To improve the utility of synthetic data, we present a systematic pipeline that takes synthetic data coming purely from a game engine and then produces synthetic data with real sensor characteristics such as noise and color gamut. We validate the utility of our sensor-realistic synthetic data for multi-frame high dynamic range (HDR) photography using a Samsung Galaxy S10 Plus smartphone. The result of training two baseline neural networks using our sensor realistic synthetic data modeled for the S10 Plus show that our sensor realistic synthetic data improves the quality of HDR photography on the modeled device. The synthetic dataset is publicly available at https://github.com/nadir-zeeshan/sensor-realistic-synthetic-data.