Coal mine personnel tracking is of great significance for realizing intelligent perception of personnel safety situation, as well as improving the level of accurate monitoring and intelligent safety management of mine personnel. Based on the unique advantage of infrared video images in highlighting the visual saliency of personnel targets in coal mine, this paper improves the personnel trackingin infrared monitoring videos of mines by improving the ARCF tracker. In order to identify abnormal tracking of personnel targets in video images, first, the average value of the tracking response map obtained by the ARCF model is calculated, and the parts of the response map that are less than the average value are filtered out. Secondly, the 8-neighborhood comparison method is used to search for the maximum peak point of the response map, and the similarity error between it and the highest peak point of the response map is calculated. Finally, by calculating the rate of change in the similarity error between the peak points of the response map in adjacent two frames of video, it is determined whether the tracking result of personnel target in the current frame of video is abnormal. Experiment results show that compared with the trackers of KCF, BIT, DSST, ARCF and IARCF, the proposed method shows better performance in infrared video tracking of mine personnel.
The High Resolution Range Profile (HRRP) contains rich target information and is commonly utilized in the field of radar automatic target recognition. Most of the key targets of concern in practical applications are non-cooperative, making it challenging to acquire data for such targets. This often results in an imbalance in target categories within datasets, thereby impacting the classification performance of models. To address these issues, this paper proposes the Weighted-SMOTE (Synthetic Minority Over-sampling Technique) algorithm. The algorithm assigns different weights to the number of synthetic new samples for each sample based on the Euclidean distance between minority class samples and the remaining samples. Furthermore, it utilizes the MCNN-TEAN (Multi-scale Convolutional Neural Network – TransEncoder Attention Network) model to extract multi-level features and capture long-term dependencies between feature dimensions, thereby improving the classification accuracy of targets.
Various non-coal foreign objects in raw coal seriously affect the safety and efficiency of coal production. As an attractive object recognition method, object recognition based on computer vision has achieved fruitful results while the accuracy and speed are still great challenges because of the poor image quality results of high conveyor belt speeding and uneven illumination. In this paper, a non-coal foreign object recognition method based on deep learning was proposed. Firstly, based on the noise reduction, image enhancement and image segmentation, a combinatorial optimization method is proposed to preprocess the low-quality images. Secondly, from three aspects including network structure, activation function and decoupling prediction, an improved YOLOv5 algorithm is proposed. Finally, by fusing the visual attention mechanism, the feature extraction quality of the YOLOv5 model is further optimized to enrich the semantic information of foreign objects in the image. The experimental results show that, the proposed method can obtain 88.14% recognition accuracy for gangue objects and 87.10% recognition accuracy for iron objects. Meanwhile, we have achieved as high as 27fps recognition speed, which can meet the real-time recognition on high-speed coal belt conveyor. CCS CONCEPTS • Computing methodologies →Computer graphics →Image manipulation →Image processing; • Computing methodologies →Artificial intelligence → Computer vision; • Computer systems organization →Real-time systems; • Computing methodologies → Machine learning.
The purpose of cross-corpus speech emotion recognition is to transfer the capability of recognizing emotions from speech, learned in a richly annotated corpus, to an unannotated corpus. Due to significant differences between various corpora, this task is highly challenging. Current methods often predict the emotions of unseen samples by learning the correlation between inputs and labels, but their generalization performance is not ideal. Therefore, this paper proposes a cross-corpus speech emotion recognition method based on causal feature disentanglement (CFDM). Specifically, we designed a causal feature disentanglement framework that includes two feature encoders, ingeniously crafting loss functions to learn causal features and confounding features separately. Then, a feature self-enhancement method was developed by disordering and summing the confounding features to further strengthen the disentanglement of causal and confounding features. This framework is versatile and can be adapted to existing emotion recognition models to extract causal features for emotion recognition. Additionally, extended experiments conducted on five widely used speech corpora confirmed the effectiveness of the proposed method.
With the rapid development of deep learning technology, ship target detection plays a crucial role in maritime safety. This paper proposes a ship image fusion method based on convolutional neural networks to enhance detection accuracy and robustness, which provides clear visible visual images during night-time navigation. Firstly, the SCAF (Spatial and Channel Attention Feature Exchange) attention mechanism is incorporated after the first three layers of convolution in the feature extraction module to obtain the spatial salient information and perform the complementary operations on two modal features. Secondly, a light perception network is introduced to calculate the probability of the current scene being daytime or night-time, and this probability is used as the weight of the light loss function in the fusion network to guide the training of the fusion network. Finally, texture detail loss and auxiliary intensity loss are introduced to retain the texture and intensity information of the ship. Experimental results show the fusion effects between proposed model and compared baselines by six evaluation metrics, including Mutual Information (MI), Spatial Frequency (SF), Visual Information Fidelity (VIF), Structural Similarity (SSIM), Fusion Quality (Qabf), and Average Gradient (AG). The compared results prove that the proposed method improves the average performance by 10.4%, 50.9%, 10.8%, 37.6%, 4.0%, and 21.9% respectively compared to the average performance of baseline models.
The transformer-based image captioning models have shown remarkable performance based on the powerful sequence modeling capability. However, most of them focus only on learning deterministic mappings from image space to caption space, which generally leads to common words, repeated phrases and single sentence. To this end, we propose a novel hybrid feature based global variational transformer for diverse image captioning (HF-GVT-IC), which incorporates conditional variational autoencoder (CVAE) into transformer-based captioning model by augmenting transformer with the stochastic latent variables. Meanwhile, we design an adaptive attention module for both grid and region features to learn their interaction with text information at the decoder. Our method is able to learn one-to-many projections and improve the diversity and accuracy of generated captions simultaneously The experimental results demonstrate that our proposal achieves significant gains in both diversity and accuracy compared with the state-of-the-art diverse image captioning models.
Segment-based text detection methods can accurately locate text regions of arbitrary shapes, but there are two problems to be solved in the current segment-based text detection methods: (1) they do not make full use of the feature maps of each stage output by the backbone network, and do not fully extract multi-scale features of spatial semantics; (2) There is no suitable multi-scale feature fusion module designed to make full use of contextual scale information. In this paper, we design the Spatial Relationship Module (SPM) to extract multi-scale features with powerful spatial semantics. Aiming at the text features in different scale feature maps, a Subspace Attention Module (SAM) is introduced. By dividing the multi-scale fusion feature map channel into multiple subspace feature maps, the text feature weights in each subspace are learned respectively. The fused feature map contains more text features of different scales, so as to enhance the representation ability of the fused feature map for text instances, and then improve the detection effect of the network. The experimental results show that SPANet proposed by text can greatly improve the text detection capability, and has excellent performance compared with other methods on public data sets CTW1500 and Total-text.
The complex lighting conditions in the coal mine lead to the problems of noise interference, low definition, lack of color and texture information, which seriously affect the development of computer vision tasks in the coal mine. In addition, most of the existing image enhancement methods require training on low/normal-light paired image datasets. Therefore, in view of the above problems, an image enhancement method based on unsupervised learning model Zero-DCE is proposed, which can effectively overcome the difficulty of collecting low/normal-light paired image datasets in the coal mine. The CBAM attention mechanism is used to express the key features in the low-light image, and enhance the feature information and local contrast of images. The asymmetric convolution block (ACB) is applied to express the local image features and the detail image features, and improve the calculation efficiency of the algorithm. The experimental results show that the proposed method can enhance the texture details effectively and perform well in the image noise reduction. Compared with the Zero-DCE model, the mean of MSE and NIQE are decreased by 6.66% and 2.28%, the mean of PSNR, SSIM and VIF are increased by 1.21%, 1.11% and 17.64%, respectively, which can effectively enhance the low-light images in the coal mine.
Defects such as bird pecking, splitting, and mildew can affect the quality of red jujubes during their growth, transportation, and processing. To ensure the quality of red jujube products, grading the appearance of the jujubes is particularly important. Utilizing computer vision technology for classifying the appearance of red jujubes is a more cost-effective, faster, and more accurate alternative to manual screening. This paper introduces a method for grading the appearance of red jujubes based on YOLOv8. The results include a detection model with an mAP50 of 0.987 and a segmentation model with an mAP50 of 0.986, both based on the YOLOv8s model and a segmentation model with an mAP50 of 0.985 based on the YOLOv8n model. By employing the advanced YOLOv8 algorithm for jujube Surface Defect Detection, this study holds significant implications for improving quality control in the jujube industry. This technology not only enhances detection efficiency but also reduces labor costs, aligning with the trend toward industrial automation and intelligence. It offers scientific support to jujube enterprises, boosts their competitiveness, and contributes to elevating the development of agriculture to a higher standard.
In image enhancement technology, there is a spatial domain method that changes a certain grayscale value or neighboring grayscale value of an image in the spatial domain to achieve the enhancement effect. This article proposes an improvement on the traditional double histogram equalization by using threshold segmentation to perform equalization processing on subgraphs. The values obtained from K-means clustering are used as thresholds to segment images according to different thresholds. After histogram equalization, the two subgraphs are merged into a new image. After simulation testing, the improved algorithm has improved the image enhancement effect, making it more convenient to observe the edge features and contrast of the image.
During the last three decades, meta-heuristics, especially evolutionary algorithms and swarm intelligence algorithms are preferred to be employed for solving more and more complex real-world optimization problems. Among them, variants of three classical algorithms, genetic algorithm (GA), particle swarm optimization (PSO), and diffenrential evolution (DE), consists of most of the state-of-the-art algorithms currently. However, one human music composition inspired harmony search (HS) proposed shortly after them may be undervalued to be further and adequately developed. Due to its different search paradigm and efficient performance, its future advanced variants may have potential consists the state-of-the-art meta-heuristics. In this paper, we propose a differential search integrated harmony search algorithm (DHS) and compare it with original HS, three classical EAs, and one newly proposed artificial hummingbird algorithm, on the CEC2024 competition benchmark suite. The experimental statistical results and convergence analysis show that the DHS outperforms all these 5 algorithms and thus shows a well potential to be advanced in the future to reach and even surpass the current GA, PSO, and DE state-of-the-art variants.
As an effective mini-aircraft, Unmanned Aerial Vehicles (UAV) is popularly increasing with the demand for diverse tasks, in which visual sensor raw resolution is toughly demanded under the limitation about the amount physical constraints such as flight altitude and load capacity. Its raw resolution of UAV images generally lack sufficient representation for some details of the target, especially when the low contrast situation increases the difficulty of target detection and recognition. In order to improve the raw UAV imagery resolution, this paper proposes a deep learning-based image restoration method that simultaneously performs super-resolution in the perspective of couple-domain, which consists of the hybrid modules of frequency-based and spatial-based. For frequency domain, low-frequency and high-frequency sub-bands are decomposed from Low-resolution (LR) with the help of wavelet way. It is able to explore the structure of low-frequency content and the details of high-frequency information, which improves the resolution of UAV image to retain the edge details. For spatial domain, complementary operations are adopted to process different sub-images blocks after the LR decomposition, which can provide the difficulty degrees about their separate parts restoration. This way adopts different restorations on various regions with appropriate network capacities which is able to obtain the mapping consistencies between the LR and high-resolution (HR). Final results prove that above couple-domain structural interaction does not only exploit the powerful feature mapping ability, but also leverages the prior of the observation mode.
The lack of tools for automatically recognizing giant panda behaviors poses challenges to their conservation and management. In this study, we improved the existing human action recognition model YOWO, developing an automated model named YOWO-Bamboo capable of identifying 22 giant panda behaviors. Specifically, we utilized the 2D backbone ConvNeXt network with enhanced feature extraction capabilities to extract spatiotemporal information from input videos. We employed Huber loss and Distance Intersection over Union (DIoU) loss to optimize object localization and prediction. Experimental results on the giant panda action dataset demonstrated the effectiveness of YOWO-Bamboo, with a notable improvement in mean Average Precision (mAP) from 61.5% from 66.5%, achieving higher recognition accuracy. Furthermore, the model sustains its capability for real-time detection, maintaining a Frame Per Second (FPS) rate of 30. Our code is available on https://github.com/detimage123/YOWO-Bamboo.
The video safety monitoring and analysis is a critical problem in underground coal mines. Due to the complex environment in the underground coal mine and the requirement of perception and decision-making abilities from the coal mine robots, single intelligent agent(robot) is difficult to complete complicated tasks. This paper proposes a framework of federated learning for multiple robots(agents) in underground coal mines to achieve video monitor and analysis tasks. Three federated learning models are raised include asynchronous federation optimization algorithm; visual federation method for heterogeneous image data and coal mine video equipment federated model aggregation. The visual federated learning framework can significantly improve the safety monitoring, accelerate the development of automation in the coal mine production field, and constructing and improving smart mines.
Aiming at the complex coal mine environment resulting in impaired image quality and the difficulty of a fixed number of convolutional kernels to capture diverse motion features, coupled with fast-moving personnel deteriorating behavioural coherence, a deep-learning-based method for recognition of violation behaviours of underground personnel is proposed, and a Depth Sensing Wild Attention Convolution Module (DRFA), a Multi-dimensional Feature Extraction Module (MFE), and a Displacement-Aware Module (DAM) are designed, and the method is implemented on the self-constructed Experiments are conducted on the downhole track area violation dataset, and its TOP-1 accuracy reaches 95.10%, which proves that the method can effectively improve the recognition accuracy.
With the development of deep learning, neural network language model has been widely used in speech recognition, however, due to simple and robust, NGram language model is also the most and effective language model in speech recognition and other machine learning tasks, especially in industrial tasks. Unfortunately, NGram model can only capture short distance context dependency while long term dependency is also common in text. In order to solve the NGram's long term dependency deficiency, we propose a novel NGram-Bayes joint model, which can capture context dependency in both short distance and long distance. It is found that NGram-Bayes joint model has slight better performance than NGram only model in general open-source data and reduced about 30% relatively wer, in some special confusion dataset.
The measured signals or collected data offer a practical way to understand and study the world. The anomalies can be generated in the measured signals due to human or instrument failure. The signal quality can be improved by suppressing the anomalies, which is helpful for signal processing. The wavelet coefficient distribution characteristics of the maximal overlap discrete wavelet transform are investigated in the paper, and an adaptive method to suppress the anomalies is presented by combining Grubbs’ test. The criteria of the transform level number, threshold calculation and threshold shrinkage algorithm are specified in detail. The anomalies can be suppressed to the ground noise level by the proposed method, and the signal features can be kept well. The seismic signals and the wind spectra data with the processed results are provided in the paper to show the validity and reliability of the proposed method.
In order to solve the problems of lack of video-level and clip-level feature fusion and insufficient multi-modal feature processing in existing video question answering algorithms, this paper proposes a multi-level video question answering algorithm that uses multi-head attention units to process and fuse the relationship between videos and questions. At the clip level and video level, two layers of multi-head attention units are used to process and fuse the appearance features, motion features of the video and text features of the question, integrate processed clip-level features to the video-level, allowing the model to capture the global context and structure of the entire video. The method proposed in the paper achieved accuracy rates of 76.42%, 82.61% and 57.94% respectively in the action, transition and frame question and answer tasks in the TGIF-QA dataset, which were 1.42%, 1.21% and 2.04% higher than the baseline model respectively. The accuracy rates in the MSRVTT-QA and MSVD-QA datasets are 36.13% and 37.32%, which are 1.22% and 0.53% higher than the baseline model respectively. Experimental results show that the algorithm in this paper effectively improves the accuracy of the video question and answer algorithm, and the research work can provide theoretical support for the implementation of video question and answer technology.
Semi-supervised learning has garnered significant attention, particularly in medical image segmentation, owing to its capacity to leverage a large number of unlabeled data and a limited amount of labeled data to improve performance. However, most existing semi-supervised segmentation methods exhibit shortcomings in supervising unlabeled data, both in the label space due to potential noise in pseudo-labels, and the feature space due to indistinct class boundaries. To address the issues above, we propose Dual-view Label and Feature Supervision Network, termed DLFS-Net, to enhance label and feature supervision for unlabeled data. Our approach is based on a dual-view learning strategy that incorporates two modules: the Confidence Fusion Supervision Module (CFS) and the Dual-view Prototype Learning Module (DPL). Specifically, the CFS module generates weight matrices to integrate complementary information from two model outputs, producing more accurate pseudo-labels in the label space. The DPL module reduces intra-class variations in the feature space from two different views through prototype learning to generate clear class boundaries. Experiments on the LA and Pancreas-CT datasets demonstrate that our framework show solid gains (e.g.,1.76% Dice and 2.37% Jaccard improvement on Pancreas-CT dataset with 20% labeled data) compared with the state-of-the-art methods.
The color of the wool ball is very similar to the color of the fabric, and there is a certain difficulty in the recognition process. Propose an image processing algorithm that combines frequency domain and morphology. Firstly, construct two frequency domain Gaussian filters and subtract them to obtain a bandpass filter; Secondly, the image is subjected to a fast Fourier transform, and a bandpass filter is used to filter the frequency domain of the image. Then, the image is transformed back to obtain a spatial domain image with obvious hairballs; Finally, the hair ball can be segmented through morphological image processing。The experimental results indicate that blended fabrics have a higher number of fuzz balls, making them fabrics that are prone to pilling and pilling; The number of wool balls in the fabric is moderate; Cotton has the least number of wool balls. The experimental results show that the algorithm can effectively identify fuzz balls on various fabric surfaces.
