Existing Low-Light Image Enhancement (LLIE) methods exhibit limitations when applied to complex scenes, primarily due to the lack of adaptation to real-world and synthetic images. To address this issue, we introduce a novel approach that leverages the mean-teacher strategy and contrastive regularization within a Contrastive Semi-Learning Network (CSL-Net). Specially, our CSL-Net employs high-quality pseudo labels to significantly improve model robustness and generalization. The novelties of this paper lie in: (1) We utilize a score- pool to store teacher's and student's outputs during training. After that, we propose a monotonic strategy to determine the most reliable Non-Reference Image Quality Assessment (NR-IQA) metric, which provides effective evaluations and determines the optimal output as the pseudo-label. (2) We integrate this selected pseudo-label with a contrastive loss to provide auxiliary information that mitigates overfitting issues and guides student towards generating realistic images. (3) In network design, we incorporate illumination and gradient priors to refine color appearance and detail preservation, facilitated by several elaborated modules, including Contextual Refinement Blocks (CRBs), Non-Local Dynamic Attention (NLDA) and Illumination Perceptual Enhancer (IPE). Experimental results on popular datasets demonstrate that the enhancement quality can be significantly improved by our proposed method.

Three-dimensional (3D) point clouds are increasingly deployed across various emerging fields, such as autonomous driving, robots, drones, and virtual reality (VR) [1]–[6]. Point-based point-cloud neural networks (PNNs) [3]–[6] have demonstrated superior performance in point-cloud analysis, compared to both sparse 3D convolution-based networks [7], [8] and graph-based convolutional neural networks [9], [10]. Due to the high computational complexity, low parallelism, and frequent irregular external memory accesses, deploying PNNs in hardware is a great challenge. PNN hardware accelerators have been developed [11]–[20]. However, three key challenges remain unsolved in these accelerators, as illustrated in Fig. 23.4.1. 1) The inherent farthest point sampling (FPS) features serial computation and suffers from quadratic growth in inference latency with rising point counts. The existing uniform block-wise FPS techniques [13], [21] fail to achieve a well-balanced block segmentation, due to a typically non-uniform point distribution. 2) A large amount of redundant operations exist for both discarded points (DPs) and retained points (RPs) in FPS. These operations exist in the sampling operations of RPs ① as well as grouping ② convolution ③, and aggregation ④ for DPs, introducing unnecessary energy and latency costs. 3) The irregular memory accesses in the aggregation operation cause significant latency penalties. Channel-wise aggregation in [11] relieves irregularity, yet is unsuitable for large-scale point clouds, as the external memory access of features and the neighbor index table (NIT) is quadratically increased due to the iterative loading of features or the NIT.

Point cloud is increasingly used in a variety of applications. Farthest Point Sampling (FPS) is typically employed for down-sampling to reduce the size of point cloud and enhance the representational capability by preserving contour points in point cloud analysis. However, due to low parallelism and high computational complexity, high energy consumption and long latency are caused, which becomes a bottleneck of hardware acceleration. In this brief, we propose an adjustable multi-stream block-wise FPS algorithm, adjusted by four configurable parameters, according to hardware and accuracy requirements. A unified hardware architecture with one parameter is designed to implement the adjustable multi-stream block-wise FPS algorithm. Furthermore, we present a rapid searching algorithm to select the optimal configuration of the five parameters. Designed in an industrial 28-nm CMOS technology, the proposed hardware architecture achieves a latency of 0.005 (1.401) ms and a frame energy consumption of 0.09 (27.265) μJ/frame for 1 k (24 k) input points at 200 MHz and 0.9 V supply voltage. Compared to the state of the art, the proposed hardware architecture reduces the latency by up to 99.9%, saves the energy consumption by up to 99.5%, and improves the network accuracy by up to 9.34%.

This paper tackles the problem of federated domain generalization in person re-identification (FedDG re-ID), aiming to learn a model generalizable to unseen domains with decentralized source domains. Previous methods mainly focus on preventing local overfitting. However, the direction of diversifying local data through stylization for model training is largely overlooked. This direction is popular in domain generalization but will encounter two issues under federated scenario: (1) Most stylization methods require the centralization of multiple domains to generate novel styles but this is not applicable under decentralized constraint. (2) The authenticity of generated data cannot be ensured especially given limited local data, which may impair the model optimization. To solve these two problems, we propose the Diversity-Authenticity Co-constrained Stylization (DACS), which can generate diverse and authentic data for learning robust local model. Specifically, we deploy a style transformation model on each domain to generate novel data with two constraints: (1) A diversity constraint is designed to increase data diversity, which enlarges the Wasserstein distance between the original and transformed data; (2) An authenticity constraint is proposed to ensure data authenticity, which enforces the transformed data to be easily/hardly recognized by the local-side global/local model. Extensive experiments demonstrate the effectiveness of the proposed DACS and show that DACS achieves state-of-the-art performance for FedDG re-ID.

Automatic detection of traffic signs is crucial for Advanced Driving Assistance Systems (ADAS). Current two-stage approaches consist of a preliminary object detection step, where the traffic signs are categorized within broader families (e.g., speed limits), and then sub-classes (e.g., speed limit 40). However, these cascading methods fail to achieve satisfying performance, especially in more realistic driving scenarios where images are acquired under more challenging conditions. Under such conditions, the first-stage detection step is likely to provide inaccurate predictions, making the subsequent classification step useless. In this paper, we propose a simple yet effective two-stage fusion framework for traffic sign detection. Different from the previous cascading method, our framework directly predicts categories in the first-stage detection and fuse the two-stage category predictions to improves overall robustness. Besides, in order to filter the false detection boxes under low-resolution inputs, we also propose an effective post-processing method called Surrounding-Aware Non-Maximum Suppression (SA-NMS) as an alternative technique for the first-stage detection. After combining the above proposed methods, our framework obtains good detection performance. Experimental results on the widely used Tsinghua-Tencent 100K (TT100K) traffic sign dataset, which contains images of traffic signs collected under a variety of challenging conditions, show that the proposed framework outperforms current approaches in both accuracy and inference speed, achieving 89.7 mAP and 65 FPS for ${608\times608}$ low resolution images.

Transformer-based deep learning networks are revolutionizing our society. The convolution and attention co-designed (CAC) Transformers have demonstrated superior performance compared to the conventional Transformer-based networks. However, CAC Transformer networks contain various nonlinear functions, such as softmax and complex activation functions, which require high precision hardware design yet typically with significant cost in area and power consumption. To address these challenges, SoftAct, a compact and high-precision algorithm-hardware co-designed architecture, is proposed to implement both softmax and nonlinear activation functions in CAC Transformer accelerators. An improved softmax algorithm with penalties is proposed to maintain precision in hardware. A stage-wise full zero detection method is developed to skip redundant computation in softmax. A compact and reconfigurable architecture with a symmetrically designed linear fitting module is proposed to achieve nonlinear functions. The SoftAct architecture is designed in an industrial 28-nm CMOS technology with the MobileViT-xxs network classifying the ImageNet-1k dataset as the benchmark. Compared with the state of the art, SoftAct improves up to 5.87% network accuracy under 8-bit quantization, 153.2x area efficiency, and 1435x overall efficiency.

The use of Internet of Things (IoT) technology for real-time monitoring of agricultural pests is an unavoidable trend in the future of intelligent agriculture. This paper aims to address the difficulties in deploying models at the edge of the pest monitoring visual system and the low recognition accuracy. In order to achieve that, a lightweight GCSS-YOLOv5s algorithm is proposed. Firstly, we introduce the lightweight network GhostNet, use the Ghostconv module to replace the traditional convolution, and construct the C3Ghost module based on the CSP structure, drastically reducing the number of model parameters. Secondly, during the feature fusion process, we introduce the content-aware reassembly of features (CARAFE) lightweight up-sampling operator to enhance the feature integration capability of the pests by reducing the impact of redundant features after fusion. Then, we adopt SIoU as the bounding box regression loss function, which enhances the convergence speed and detection accuracy of the model. Finally, the traditional non-maximum suppression (NMS) was improved to Soft-NMS to improve the model's ability to recognize overlapping pests. According to the experimental results, the mean average precision (mAP) of the GCSS-YOLOv5s model reaches 90.5%. This is achieved with a 44% reduction in the number of parameters and a 7.4 G reduction in computation volume compared to the original model. The method significantly reduces the model's resource requirements while maintaining accuracy, which offers a specific theoretical foundation and technological reference for the future field of intelligent monitoring.

Centipeda minima is a traditional Chinese medicine with wide applications and diverse pharmacological effects. Scholars have conducted extensive studies on its relevant clinical applications, especially its remarkable efficacy in cancer treatment. This paper thoroughly investigates the chemical composition and identification, pharmacological effects, and toxicity, along with the safety of Centipeda minima, so as to lay the foundation for corresponding clinical applications and product development. Furthermore, as global scholars have conducted extensive research on such clinical applications and made significant progress, the future development and utilization of Centipeda minima’s active ingredients to create novel drugs are of great clinical significance.

Object detection is one of the essential tasks in computer vision, with most detection methods relying on a limited number of sizes for anchor boxes. However, the boundaries of particular composite objects, such as ports, highways, and golf courses, are ambiguous in remote sensing images, and therefore, it is challenging for the anchor-based method to accommodate the substantial size variation of the objects. In addition, the dense placement of anchor boxes imbalances the positive and negative samples, which affects the end-to-end architecture of deep learning methods. Hence, this paper proposes a single-stage object detection model named Xnet to address this issue. The proposed method designs a deformable convolution backbone network used in the feature extraction stage. Compared to the standard convolution, it adds learnable parameters for dynamically analyzing the boundary and offset of the receptive field, rendering the model more adaptable to size variations within the same class. Moreover, this paper presents a novel anchor-free detector that classifies objects in feature images point-by-point, without relying on anchor boxes. Several experiments on the large remote sensing dataset DIOR challenging Xnet against other popular methods demonstrate that our method attains the best performance, surpassing by 4.7% on the mAP (mean average precision) metric.

In graph-structured data, the node content contains rich information. Therefore, how to effectively utilize the content is crucial to improve the performance of graph convolutional networks (GCNs) on various analytical tasks. However, current GCNs do not fully utilize the content, especially multi-order content. For example, graph attention networks (GATs) only focus on low-order content, while high-order content is completely ignored. To address this issue, we propose a novel graph attention network with adaptability that could fully utilize the features of multi-order content. Its core idea has the following novelties: First, we constructed a high-order content attention mechanism that could focus on high-order content to evaluate attention weights. Second, we propose a multi-order content attention mechanism that can fully utilize multi-order content, i.e., it combines the attention mechanisms of high- and low-order content. Furthermore, the mechanism has adaptability, i.e., it can perform a good trade-off between high- and low-order content according to the task requirements. Lastly, we applied this mechanism to constructing a graph attention network with structural symmetry. This mechanism could more reasonably evaluate the attention weights between nodes, thereby improving the convergence of the network. In addition, we conducted experiments on multiple datasets and compared the proposed model with state-of-the-art models in multiple dimensions. The results validate the feasibility and effectiveness of the proposed model.

Three-dimensional (3D) point cloud has been employed in a wide range of applications recently. As a powerful weapon for point cloud analysis, point-based point cloud neural networks (PNNs) have demonstrated superior performance with less computation complexity and parameters, compared to sparse 3D convolution-based networks and graph-based convolutional neural networks. However, point-based PNNs still suffer from high computational redundancy, large off-chip memory access, and low parallelism in hardware implementation, thereby hindering the applications on edge devices. In this paper, to address these challenges, an energy-efficient 3D point cloud neural network accelerator is proposed for on-chip edge computing. An efficient filter pruning scheme is used to skip the redundant convolution of pruned filters and zero-value feature channels. A block-wise multi-layer perceptron (MLP) fusion method is proposed to increase the on-chip reuse of features, thereby reducing off-chip memory access. A dual-stream blocking technique is proposed for higher parallelism while maintaining inference accuracy. Implemented in an industrial 28-nm CMOS technology, the proposed accelerator achieves an effective energy efficiency of 12.65 TOPS/W and 0.13 mJ/frame energy consumption for PointNeXt-S at 100 MHz, 0.9 V supply voltage, and 8-bit data width. Compared to the state-of-the-art point cloud neural network accelerators, the proposed accelerator enhances the energy efficiency by up to 66.6× and reduces the energy consumption per frame by up to 70.2×.

Three-dimensional (3-D) understanding or inference has received increasing attention, where 3-D convolutional neural networks (3D-CNNs) have demonstrated superior performance compared to 2D-CNNs, since 3D-CNNs learn features from all three dimensions. However, 3D-CNNs suffer from intensive computation and data movement. In this article, Sagitta, an energy-efficient low-latency on-chip 3D-CNN accelerator, is proposed for edge devices. Locality and small differential value dropout are leveraged to increase the sparsity of activations. A full-zero-skipping convolutional microarchitecture is proposed to fully utilize the sparsity of weights and activations. A hierarchical load-balancing scheme is also introduced to increase the hardware utilization. Specialized architecture and computation flow are proposed to enhance the effectiveness of the proposed techniques. Fabricated in a 55-nm CMOS technology, Sagitta achieves 3.8 TOPS/W for C3D at a latency of 0.1 s and 4.5 TOPS/W for 3D U-Net at a latency of 0.9 s at 100 MHz and 0.91-V supply voltage. Compared to the state-of-the-art 3D-CNN and 2D-CNN accelerators, Sagitta enhances the energy efficiency by up to $379.6\times $ and $11\times $ , respectively.

An energy-efficient convolutional neural network (CNN) accelerator is proposed for low-power inference on edge devices. An adaptive zero skipping technique is proposed to dynamically skip the zeros in either activations or weights, depending on which has the higher sparsity. The characteristic of non-zero data aggregation is explored to enhance the effectiveness of adaptive zero skipping in performance boosting. To mitigate the load imbalance issue after zero skipping, a sparsity-driven data flow and low-complexity dynamic task allocation are employed for different convolution layers. Facilitated further by a two-stage distiller, the proposed accelerator achieves $5.42\times $ , $3.41\times $ , and $3.42\times $ performance boosting for VGG16, AlexNet, and Mobilenet-v1, respectively, compared to the baseline. Implemented in a 55-nm low power CMOS technology, the proposed accelerator achieves an effective energy efficiency of 2.41 TOPS/W, 2.35 TOPS/W, and 0.64 TOPS/W for VGG16, AlexNet, and Mobilenet-v1, respectively, at 100 MHz and 1.08 V supply voltage.

Abstract Pulmonary disease arising from slow‐growing mycobacterial infections has emerged as an increasingly prevalent clinical concern over the past two to three decades. Proteins belonging to the family of ESAT‐6 secretion (Esx) systems play critical roles in the virulence of most pathogenic mycobacterial species and are associated with drug resistance. However, no clinical applications can detect and discriminate the expression of species‐specific variants of these proteins in clinical samples, such as early growth cultures, for rapid diagnosis of specific mycobacterial infections, which may require distinct interventions. Conventional immunoassay approaches are not suitable for this purpose due to the significant degree of conservation of Esx proteins among species. Herein we describe the development of a novel immunoprecipitation‐coupled mass spectrometry assay that can distinguish Esx proteins that are expressed by slow‐growing mycobacterial species commonly detected in clinical isolates. This approach uses custom antibodies raised against single semi‐conserved peptide regions in M. tuberculosis (Mtb) EsxB and EsxN to capture corresponding peptides from protein orthologs of mycobacteria associated with human respiratory infections, including Mtb, M. avium, M. intracellulare, M. kansasii, M. gordonae, and M. marinum, to detect these species in standard clinical cultures at the first sign mycobacterial growth to allow rapid disease diagnosis.

When large training datasets are unavailable in real clinical scenarios, researchers turn to unsupervised learning for under-sampled magnetic resonance image reconstruction. However, unsupervised learning methods suffer from insufficient a priori knowledge. We introduce self-consistency constraint with the calibration and acquisition data to tackle these issues. Specifically, we propose an iterative data correction operator to ensure high fidelity of the reconstructed MR data. Experiments shows that the method is flexible and can reconstruct data from arbitrary k-space sampling patterns and easily incorporates additional image priors.

The emergence of computation intensive automotive applications poses significant challenges on computation capacity of automotive electronic systems, thus vehicular edge computing (VEC) is introduced as a new computing paradigm into the internet of vehicle (IoV) to improve its data processing capability. However, as the computation capacity is limited in VEC servers, efficient task offloading algorithms need to be proposed. This paper first proposes a multi-task offloading model and gives the related task response time analysis method, and then, both a mixed-integer linear programming (MILP)-based algorithm and a simulated annealing-based heuristic algorithm are proposed to minimize the task response time. By comparing with a baseline algorithm, the MILP-based offloading algorithm can reduce the average task response time by 91.45%, and the heuristic offloading algorithm can reduce the average task response time by 70%.

With the booming of artificial intelligence, convolutional neural networks (CNNs) have attracted widespread attention. In this paper, a look-up table (LUT) based multiplier array is proposed for convolution in CNNs, where multiplications play the critical role. By exploring the distinctive features of CNNs based on systolic array, the proposed multiplier array pre-computes all possible partial products and stores them in an LUT. The partial products in the LUT are reused multiple times to reduce the switching activities. The proposed multiplier array is employed in a multiply-and-accumulate (MAC) array for convolutional calculations. Implemented with different bit widths and clock frequencies in an industrial 65-nm CMOS technology, the proposed design achieves up to 23.34% and 33.26% reduction in power consumption as well as power area product (PAP), respectively, as compared to the reference designs with various conventional multipliers.

Due to the variety of categories and uneven distribution of available samples, automatic traffic sign detection and recognition is still a challenging task. For those categories with less training data, existing deep learning methods cannot achieve desirable performance, and the overall detection effect is not satisfactory as well. In this letter, we fully explore the relationship between different traffic signs with digital characters and transform the category objects into multi-level classes to alleviate the uneven distribution of samples. We design a lightweight two-stage object detection framework with high real-time performance. The first stage network is proposed to obtain the category groups of traffic signs, and then we construct another object detection network to identify the digital characters of the detected traffic signs. To make the prediction in the first stage more accurate, we put forward a boxes fusion algorithm in the post-processing process and a refine module to improve the recognition performance. Experimental results show that our approach possesses significantly improved performance compared with the latest object detection networks and other traffic sign detectors. Even some traffic signs that only exist in testset can also be recognized accurately by our method.

Various algorithms based on Convolutional Neural Network (CNN) have achieved great performance in the task of face anti-spoofing (FAS). However, the issue of most approaches is that the robustness in unknown scenes is not strong due to the different quality of attack images and environmental factors. In this letter, we propose a real-time face anti-spoofing method based on stereo matching. We input the left and right views of a pair of infrared face images into our proposed lightweight stereo matching network to get a disparity map. Then, we use the disparity map as input for the classification network to obtain face living information. Experimental results show that the proposed approach has significantly improved performance compared with the latest state-of-the-art face anti-spoofing methods. Considering the real-time requirement, our method is superior to most single-image-based models in inference time and far less than those based on other large scale stereo matching networks in computational complexity. The code is available at https://github.com/lizhishan1997/StereoMatching_FAS.

Three-dimensional convolutional neural network (3D-CNN) has demonstrated outstanding classification performance in video recognition compared to two-dimensional CNN (2D-CNN), since 3D-CNN not only learns the spatial features of each frame, but also learns the temporal features across all frames. However, 3D-CNN suffers from intensive computation and data movement. To solve these issues, an energy-efficient low-latency 3D-CNN accelerator is proposed. Temporal locality and small differential value dropout are used to increase the sparsity of activation. Furthermore, to fully utilize the sparsity of weight and activation, a full zero-skipping convolutional microarchitecture is proposed. A hierarchical load-balancing scheme is also introduced to improve resource utilization. With the proposed techniques, a 3D-CNN accelerator is designed in a 55-nm low-power CMOS technology, bringing in up to 9.89× speedup compared to the baseline implementation. Benchmarked with C3D, the proposed accelerator achieves an energy efficiency of 4.66 TOPS/W at 100 MHz and 1.08 V supply voltage.