The rapid development of computer image processing has greatly improved the progress of artificial intelligence. When big data and distributed machine learning interact, the accuracy and efficiency of algorithm recognition can be greatly improved, but its security cannot be guaranteed in terms of privacy data protection. The traditional horizontal federated learning algorithm FedAvg can perform model training under the premise of protecting user data, but the security vulnerabilities in the parameter interaction process and the local data training process may be exploited by internal or external attackers. In order to balance the efficiency and security of the algorithm, this paper proposes an image classification model SGX-FedAvg. The model adds Intel SGX based on hardware encryption mechanism on the basis of horizontal federated learning FedAvg, which loses a small part of the algorithm efficiency of federated learning and adds reliable security to its model. In this paper, two data sets, Cifar-10 and Mnist, are selected for training. The model proposed in this paper is compared with Native local distributed machine learning and FedAvg horizontal federated learning. It is analyzed that SGX-FedAvg has good performance in execution efficiency and security within acceptable overhead, and its feasibility is verified. Finally, it points out which fields the model can be applied to.
In recent years, artificial intelligence applications have been on the rise. Many enterprises have embraced digital transformation and have established new business models based on artificial intelligence and the Internet of Things, such as the telerehabilitation industry. The companies may utilize sensors or cameras to collect user data, and data mining is applied to discover insights for doctors' aids. This paper establishes a novel two-stage data mining model combining gait recognition and sequential pattern mining. In the first stage, a particular computer vision application, gait recognition, identifies possible diseases using the subject's walking postures. The gaits in a video can be converted to a temporal sequence according to user-defined events. For example, (normal gait, Parkinsonian gait, normal gait) is a temporal sequence in which the identified gaits are arranged by temporal orders in the sequence. In the second stage, after collecting a dataset of temporal sequences, the frequent patterns are discovered by sequential pattern mining. Our preliminary experiment collected 30 samples from the real world and demonstrated the model's feasibility.
Abstract—Aiming at the problems of long reasoning time and high hardware requirements of workers ' helmet wearing detection model in existing construction sites, a lightweight helmet wearing detection algorithm based on improved FastestDet is proposed. Firstly, the FASTDET backbone network is optimized to further reduce the number of parameters while ensuring accuracy. Secondly, the spatial pyramid pooling module is changed from SPP to SimSPPF proposed by Meituan YOLOv6, which accelerates the recognition speed. Finally, the EMA attention mechanism is introduced at different locations to enhance the perception ability of the model. The experimental results show that the recall rate and mAP of the improved FastestDet algorithm are 1.3 % and 1.7 % higher than FastestDet, respectively. And the inference speed is faster than other models.
Infrared thermal imaging is capable of operating under various lighting conditions, unaffected by external factors, but it often suffers from issues like low contrast and blurry details. This article introduces an infrared image enhancement algorithm based on multi-scale guided filtering. The algorithm employs guided filtering to split the captured infrared image into a base layer and a detail layer. The base layer undergoes non-linear compression via segmented histogram equalization to enhance image contrast, while multi-scale guided filtering extracts diverse detail information from the image, which is then merged into a new detail layer to amplify fine details. Finally, the base layer and detail layer are combined to complete the enhancement of the infrared image. Experimental results demonstrate that this algorithm surpasses suboptimal approaches by 15.2% in terms of Enhancement Measure Evaluation(EME) and enhances the peak signal-to-noise ratio (PSNR) by 7.9%.
Strong infrared radiation interference is a common type of interference in optoelectronic countermeasures. An optical system exposed to strong light would disrupt the acquisition of image information and even damage the device. In this paper, we aim to design an infrared system for object detection under the circumstances of strong infrared radiation interference in order to enhance the countermeasure capability of the optoelectronic system. Firstly, we introduce the Wave-front Coding (WFC) technique to attenuate the damage caused by strong infrared radiation interference. Secondly, we use L₂ (Tikhonov) regularization to restore the blurred image caused by wavefront coding and obtain a clear low-light image. Thirdly, we use Recursively Separated and Weighted Histogram Equalization (RSWHE) for image enhancement to improve its contrast. Finally, we use the YOLOv8 algorithm for object detection. Experiments on real infrared images show that this scheme has outstanding performance under strong infrared radiation interference.
Video frame interpolation involves synthesizing intermediate frames between two consecutive frames to enhance the smoothness of a video. Nevertheless, generating high-quality interpolated frames in videos featuring substantial motion and complex scenes remains a formidable challenge. To produce superior quality frames, this paper introduces an interpolation method based on multi-scale features and correlation volumes. The multi-scale feature connects the deep features of high-resolution frames with the shallow features of lower-resolution frames, thereby increasing the number of available pixels and feature details for motion analysis. Correlation volumes are employed to construct correlation features for all pairs of pixels, which are utilized to refine the underlying optical flow field. We propose a unified network approach that eliminates the need for additional complex optical flow network integration, simplifying the training process. The experimental results demonstrate that this method outperforms the baseline approach in both objective and subjective evaluations across various datasets. Particularly, this method exhibits advantages on datasets characterized by complex backgrounds and large motions.
At present, in the process of performing the dynamic visual measurement task of the flight test, the method of theoretical analysis and static verification is mainly used to evaluate the accuracy of the measurement data. This method cannot fully reflect the impact of the error on the measurement accuracy in the motion scene, and the measurement data of the whole station is used as the evaluation benchmark, which cannot meet some Requirements for higher-precision (sub-millimeter) measurement tasks. Therefore, this paper designs and builds a dynamic posture measurement accuracy evaluation system. Based on the principle of visual measurement and the typical target motion law, it comprehensively analyzes the error impact factor, plans and designs a variety of test verification scenarios, completes the ground simulation of the target motion, and combines the reference data generated by the mobile robot with the number of position obtained by visual measurement. According to the comparison, error traceability and quantitative analysis are realized to provide effective technical support for the optimization design, feasibility demonstration and precision analysis and evaluation of specific task implementation plans.
The paper provided a brief analysis of video denoising characteristics, discussed and analyzed various existing video denoising methods, and proposed a new video denoising algorithm based on bidirectional time fusion and the W-Net architecture, designed to meet the requirements of real-time video denoising. This algorithm effectively combines past and future information, increases the temporal receptive field, and reduces memory usage. Additionally, by selecting a deeper W-Net backbone network, the algorithm achieves high-fidelity real-time video denoising. Comparative analysis with other video denoising models demonstrated that this approach outperforms others in terms of fidelity.
In the task of emotion recognition from videos, a key improvement has been to focus on emotions over time rather than a single frame. There are many architectures to address this task such as GRUs, LSTMs, Self-Attention, Transformers, and Temporal Convolutional Networks (TCNs). However, these methods suffer from high memory usage, large amounts of operations, or poor gradients. We propose a method known as Neighborhood Attention with Convolutions TCN (NAC-TCN) which incorporates the benefits of attention and Temporal Convolutional Networks while ensuring that causal relationships are understood which results in a reduction in computation and memory cost. We accomplish this by introducing a causal version of Dilated Neighborhood Attention while incorporating it with convolutions. Our model achieves comparable, better, or state-of-the-art performance over TCNs, TCAN, LSTMs, and GRUs while requiring fewer parameters on standard emotion recognition datasets. We publish our code online for easy reproducibility and use in other projects.
In the underwater environment, the color distortion and low contrast of the image are caused by environmental problems such as light absorption and scattering, which leads to the degradation of image quality. In order to improve the visual effect of the image, this paper proposed a color correction underwater image enhancement algorithm based on convolutional neural network. Firstly, a new underwater imaging model was used to correct the color cast problem of underwater images. Then, the convolutional neural network is used to extract the channel features of the corrected image, and the channel features are re-weighted by the multi-scale attention module to enhance the consistency of different feature maps, and the feature fusion is performed with the color corrected image. Finally, the image enhancement effect was improved by the fusion of features through the reconstruction calculation module. Experimental results show that the proposed algorithm can better correct the color distortion of the image and improve the image contrast. The main advantage is that the running speed of the proposed algorithm is two times faster than other advanced underwater image enhancement methods.
Abstract:Aiming at the problem that the existing models are not enough to extract the spatial details and feature channel information of remote sensing images, a super-resolution reconstruction model of remote sensing images is proposed, which integrates the receptive field and the attention mechanism. In the depth feature extraction stage of the model, several cascaded receptive field and coordinate attention blocks (RFCAB) are designed to fully extract the depth features of the image: Firstly, an RFB-CA module is designed inside the residual in residual dense block (RRDB). The model can use convolution of different scales to extract multi-scale spatial features and make both channel and space dimensions get attention. At the same time, in the process of learning features, the model pays more attention to the useful channels for the current task, so as to improve the feature representation ability of the model. In order to further improve the recovery ability of the model to detail information, a multi-scale fusion module (MSFM) was designed to obtain more detailed features by weighted fusion of features at different levels. In the 2x, 3x and 4x overscore reconstruction of DOTA dataset, the PSNR/SSIM value of this model is increased by 0.13dB/0.003, 0.17dB/0.007 and 0.24dB/0.013 compared with ESRGAN, respectively. In the 2x, 3x and 4x over-fraction reconstruction of AID dataset, the PSNR/SSIM value is increased by 0.15dB/0.004, 0.20dB/0.009 and 0.26dB/0.015 compared with ESRGAN, respectively. The experimental results show that the reconstruction effect of this model is better than other classical algorithms, and it has certain practical significance.
The article introduces a novel technique to improve underwater image quality, addressing challenges such as light absorption, poor contrast, scattering effects, and color distortions. The algorithm uses the YCbCr color space to separate image luminance and chrominance, allowing a targeted treatment of these elements. It begins by converting the RGB underwater image to YCbCr, followed by decomposing it into luminance and chrominance components. The luminance undergoes dehazing using the underwater normalized total variation (UNTV) method and superpixel segmentation to improve clarity and reduce haze. Atmospheric light estimation through superpixel segmentation contributes to accurate scene radiance restoration. Chrominance is then refined using the dehazed luminance, enhancing color accuracy and contrast. After independent processing, the luminance and chrominance components are recombined in the YCbCr space, and the final image is converted back to RGB. The efficacy of the algorithm is demonstrated on two benchmark datasets, showing its promise and highlighting the potential of the YCbCr color space for effective underwater image enhancement.
Panoptic segmentation has become a research hotspot in the field of computer vision. In this paper, we propose a novel and efficient method for panoptic segmentation. Based on semantic segmentation architecture DeepLabv3+, we predict a three-dimensional offset vector for each pixel, and produce a unified panoptic segmentation mask by combining semantic prediction. Previous bottom-up panoptic segmentation methods only consider offset vector in a horizontal plane, i.e. a two-dimensional space. In contrast, we consider offset vector in both horizontal plane and vertical direction, and predict an offset vector in three-dimensional space for each pixel. Offset vector in vertical direction is used to determine whether a pixel belongs to a foreground object, while offset vector in horizontal plane is used to determine which foreground object the foreground pixel belongs to. In this way, we can predict offset vectors for both background and foreground object pixels at the same time, and effectively model foreground pixels clustering process. During inference, we can produce a unified panoptic segmentation mask by combining the foreground pixels clustering result and semantic prediction. We demonstrate our method’s effectiveness on Cityscapes dataset, and obtain competitive results.
Infrared and visible dual camera is commonly used in UAV inspection. As a crucial preprocess, accurate registration can promote subsequent multi-source image fusion, which can improve detectability and reduce the false positive rate. Due to significant geometric distortions, gray-scale differences and partial overlap between infrared image and visible image, feature-based methods or joint area-feature based methods cannot obtain satisfactory results. To solve this problem, this paper presents a novel registration method based on UAV imaging characteristics and intensity-structure similarity optimization. The reliable initial registration parameters are obtained by utilizing the UAV imaging parameters and approximate coaxial imaging principle. For further improving the accuracy of registration, this paper proposes an intensity-structure similarity metric and the final rectification parameters are obtained by maximizing the proposed metric via quantum particle swarm optimization (QPSO) method. The experimental results of infrared and visible images in UAV inspection of photovoltaic power station demonstrate that the proposed method is competitive against traditional feature-based methods (FBMs), such as SIFT and SURF, and the joint area-feature based methods (AFBMs) based on SIFT combined with regional mutual information.
In order to improve the efficiency of image stitching and solve the problem of visual artifacts, an image stitching algorithm based on FABEB and parallax tolerance is proposed. Firstly, the FABEB algorithm is proposed to efficiently obtain robust feature points by combining scale-space keypoint detection of FAST (Features From Accelerated Segment Test) and boosted efficient binary local image descriptor (BEBLID). In the feature point registration algorithm, KNN algorithm is first used for general screening. Marginalizing sample consensus algorithm (MAGSAC) is used for further screening, to get the geometric relation between the images. In order to solve the artifact problem in large parallax image stitching, this paper proposes an optimal seam-cutting fusion algorithm based on visual saliency. And it adds saliency feature into the seam-cutting strategy to make the results more consistent with the perception of human eyes. Experimental results show that compared with the traditional algorithm, the proposed image stitching strategy can extract and match feature points more efficiently and accurately, and eliminate stitching artifacts to get a better visual experience.
Visible images are sensitive to light and are easily affected by imaging technology, while infrared images are not affected by illumination and have strong penetration in rain or smog, so the combination of infrared and visible images in the object tracking can significantly improve tracking performance. In this paper, we propose an RGB-T tracking method based on the cross-channel normalization of local response and incorporate some improvements to the ADRNet framework. On the one hand, we improve the structure of the backbone network to obtain richer semantic information and more robust features, and the dropout layer is added to prevent overfitting. On the other hand, we use 3D average pooling to process the feature data convolved with cross-channel operations in the part of local response normalization to extract the timing information between video frames. We conducted a large number of experiments on the corresponding dataset and compared our tracker with some of the most advanced trackers. The results show that our tracking algorithm can achieve better performance on RGB-T object tracking.
As a common image stratification algorithm, Kmeans clustering effect is affected by the initial random clustering center. The same parameter is used for different images, and the clustering effect is not the same. It is difficult to meet the standards of industrial production. Therefore, it is very important to improve the Kmeans algorithm to improve the clustering effect. This paper proposes an improved Kmeans algorithm, which is a combination of Kmeans algorithm and AP(Affinity Propagation) algorithm. This algorithm not only avoids the need of Kmeans to select the appropriate K value in advance, but also improves the overall clustering effect. The experimental results show that the clustering effect of Kmeans-AP algorithm proposed in this paper is better than the average effect of Kmeans in 83% of the whole data set.
Pig detection is vital to pig farms since it is the basis for counting, weight estimation, and behavior recognition functions. Existing methods focused on white pig detection instead of black pigs because the contrast between white pigs and the background is more evident than black pigs, making black pig detection more challenging. Furthermore, pig farms often suffer from insufficient light, and pigs tend to be stacked together, which makes it hard to detect pigs accurately. To this end, we propose a black pig detection method, based on YOLOV4, robust to both crowd scenes and low-illumination conditions. The method consists of the following parts: 1) generate domain adaptation dataset (DAD) based on style-transfer to optimize the original data distribution, therefore improving the performance of the method for low-illuminated conditions; 2) propose a crowd-aware module (CAM), adapted to YOLOV4 backbone architecture, to generate crowd density maps; 3) develop an adaptive attention module (AAM) to fuse YOLOV4 backbone features with corresponding crowd density map allowing the method robust to pigs in the crowd. The experimental results confirm the feasibility of this method. The mAP value in fully-lighting and poorly-lighting increased to 88.95% and 84.74%, respectively.
Autonomous driving systems need to undergo rigorous testing in complex scenarios including a variety of extreme operating conditions before they can be put into use. In this process, digital twin technology can migrate the scenes in the physical world to computer simulation software, so that engineers can comprehensively and safely conduct large-scale test experiments, get well prepared for vehicle tests on real road. The existing related systems in academia are still lacking in the authenticity, diversity and complexity of the scene. To solve the above problems, we propose a digital twin-based test and data acquisition system for autonomous driving under extreme traffic scenes. Based on UE4 engine and CARLA simulation platform, the system includes a Chinese urban style map with a total area of 10.8 square kilometers and 232 multi-site test routes including 29 possible events, 4 sets of environmental parameters, 2 sets of location parameters, and all weather condition. At the same time, a large amount of sensor data can also be collected in the system, which fills the insufficiency of collecting extreme working scene data in the real world. We give a perception-oriented autonomous driving data collection scheme, which can store sensor output while running simulation events, and automatically generate corresponding annotations, which can collect data in emergency situations. We also propose a new evaluation metric of autonomous driving system based on research on accident hazard. This test system covers a variety of extreme working conditions that are not involved in the existing systems, and puts forward higher requirements for the perception and decision-making algorithms related to autonomous driving.
As 3D hand models have attracted extensive attention in human-computer interaction, VR and AR fields, building realistic hand models is becoming a long-term research hotspot. Performances of many existing 3D hand models may be limited due to their low resolution, sparsity and non-linearity. In this paper, we introduce a new linear parametric hand model termed BlendPose for hand reconstruction and prospose a new framework to redefine the relationship between a hand mesh and its 3D joints. BlendPose is a global parametric-driven model, reconstructing hand shape and pose simultaneously. The key idea of BlendPose is a hand mesh can be expressed by the linear combination of several hand posture models and a rest-pose model. Additionally, the new framework first reconstruct personalized hand mesh via BlendPose and then regress 3D joints from mesh vertices using a trainable joints regressor, which is quite different from other skeleton-driven models. We experimentally show that BlendPose can be applied to challenging datasets such as OneHand10k and Freihand while achieving high-quality reconstruction results.
