Low-light images pose challenges to visual perception and downstream tasks due to poor visibility. Despite the strong performance achieved by existing deep learning-based Low-Light Image Enhancement (LLIE) methods on certain public datasets, these methods still lack sufficient generalization and adaptability to complex low-light environments. To address these issues, this paper proposes an LLIE framework (MoE-EnNet) based on Mixture-of-Experts (MoE) and ensemble enhancement. We design a MoE module that dynamically routes low-light inputs to suitable experts using hierarchical features and a lightweight expert-weight prediction mechanism. The Ensemble Network (EnNet) fuses the experts’ outputs using self-attention and multi-kernel convolutions. By freezing the weights of pre-trained expert models on LOL-V1 and only training the lightweight MoE-EnNet modules on each target dataset, the huge amount of workload to retrain experts repeatedly has been avoided. The proposed MoE-EnNet model achieves competitive PSNR scores of 27.86, 27.58, 28.95 and 27.74 on LOL-v2-synthetic, SDSD-indoor, SDSD-outdoor, and FiveK respectively, demonstrating strong cross-dataset generalization without retraining expert models. The code is available at https://huggingface.co/geruome/ennet.

Automated inspection of Integrated Circuits (ICs) on Printed Circuit Boards (PCBs) is essential for ensuring the reliability of modern electronic systems. However, the inspection process faces significant challenges, particularly data scarcity and low inter-class variance. To address these challenges, we propose SSRNet, a few-shot learning-based framework for precise IC segmentation in complex PCB optical images. Unlike traditional deep learning models, our proposed SSRNet utilizes a similarity-guided approach for initial mask prediction and integrates a region classifier for further refinement. This design allows SSRNet to accurately segment IC components, even with limited annotated data. Experimental results demonstrate that our proposed SSRNet outperforms the state-of-the-art model, achieving a 23.0% increase in IoU and a 13.2% improvement in the Dice coefficient on NTU PCB DSX Dataset (NPDD).

Weakly-supervised Temporal Action Localization (WSTAL) aims to localize actions in untrimmed videos using only video-level supervision. Latest method introduce a pseudo label learning framework to bridge the gap between classification-based training and inference targets at localization. Typically, this framework employs a classification-based teacher model to generate pseudo labels, which are then used to train a regression-based student model for precise boundary prediction. However, the quality of these pseudo label-critical to the student model's performance-has not been systematically investigated, leading to suboptimal localization accuracy. In this paper, we propose a set of simple yet efficient mechanisms for pseudo label quality enhancement to build our FuSTAL framework. Unlike previous one or two stages methods, FuSTAL decomposes the learning process into three stages and enhances pseudo label quality at each one: cross-video contrastive learning for more informative initiative pseudo labels at the Generation-Stage, prior-based filtering to remove the false positive proposals at the Selection-Stage and EMA-based distillation for smoother pseudo labels at the Training-Stage. These designs supplement each other, and enhance action proposals' quality with respect to the accuracy, true positive rate and smoothness. With the help of these comprehensive designs at all three stages, FuSTAL achieves an average mAP of 50.8% on the benchmark data THUMOS'14, outperforming the previous best method by 1.2%.

Vision-Language Models (VLMs) bring powerful understanding and reasoning capabilities to multimodal tasks. Meanwhile, the great need for capable aritificial intelligence on mobile devices also arises, such as the AI assistant software. Some efforts try to migrate VLMs to edge devices to expand their application scope. Simplifying the model structure is a common method, but as the model shrinks, the trade-off between performance and size becomes more and more difficult. Knowledge distillation (KD) can help models improve comprehensive capabilities without increasing size or data volume. However, most of the existing large model distillation techniques only consider applications on single-modal LLMs, or only use teachers to create new data environments for students. None of these methods takes into account the distillation of the most important cross-modal alignment knowledge in VLMs. We propose a method called Align-KD to guide the student model to learn the cross-modal matching that occurs at the shallow layer. The teacher also helps student learn the projection of vision token into text embedding space based on the focus of text. Under the guidance of Align-KD, the 1.7B MobileVLM V2 model can learn rich knowledge from the 7B teacher model with light design of training loss, and achieve an average score improvement of 2.0 across 6 benchmarks under two training subsets respectively.

Adaptive gradient methods, especially Adam-type methods (such as Adam, AMSGrad, and AdaBound), have been proposed to speed up the training process with an element-wise scaling term on learning rates. However, they often generalize poorly compared with stochastic gradient descent (SGD) and its accelerated schemes such as SGD with momentum (SGDM). In this paper, we propose a new adaptive method called DecGD, which simultaneously achieves good generalization like SGDM and obtain rapid convergence like Adam-type methods. In particular, DecGD decomposes the current gradient into the product of two terms including a surrogate gradient and a loss based vector. Our method adjusts the learning rates adaptively according to the current loss based vector instead of the squared gradients used in Adam-type methods. The intuition for adaptive learning rates of DecGD is that a good optimizer, in general cases, needs to decrease the learning rates as the loss decreases, which is similar to the learning rates decay scheduling technique. Therefore, DecGD gets a rapid convergence in the early phases of training and controls the effective learning rates according to the loss based vectors which help lead to a better generalization. Convergence analysis is discussed in both convex and non-convex situations. Finally, empirical results on widely-used tasks and models demonstrate that DecGD shows better generalization performance than SGDM and rapid convergence like Adam-type methods.

The detection and partitioning of standard cells from Scanning Electron Microscope (SEM) images is a crucial step in hardware assurance of Integrated Circuit (IC). Traditional methods may struggle with the noise and complexity of these signals. This paper introduces a novel approach to SEM image processing by framing the standard cell partition problem as a multiple hypothesis testing (MHT) problem. This method enables simultaneous decision-making across many hypotheses, enhancing detection accuracy while controlling the false discovery rate (FDR). We show how MHT can identify partition lines in noisy brightness signals extracted from SEM images. Using the Benjamini-Hochberg (BH) procedure, we achieve effective FDR control, improving detection robustness and providing a clearer understanding of cell structures. This study demonstrates the suitability of MHT for SEM image processing and its potential for other circuit-related challenges.

Despite the progress of Semi-supervised Learning (SSL), existing methods fail to utilize unlabeled data effectively and efficiently. Many pseudo-label-based methods select unlabeled examples based on inaccurate confidence scores from the classifier. Most prior work also uses all available unlabeled data without pruning, making it difficult to handle large amounts of unlabeled data. To address these issues, we propose two methods: Variational Confidence Calibration (VCC) and Influence-Function-based Unlabeled Sample Elimination (INFUSE). VCC is a universal plugin for SSL confidence calibration, using a variational autoencoder to select more accurate pseudo labels based on three types of consistency scores. INFUSE is a data pruning method that constructs a core dataset of unlabeled examples under SSL. Our methods are effective in multiple datasets and settings, reducing classification error rates and saving training time. Together, VCC-INFUSE reduces the error rate of FlexMatch on the CIFAR-100 dataset by 1.08% while saving nearly half of the training time.

Cross-patient automatic epileptic seizure detection through electroencephalogram (EEG) is significant for clinical application and research. However, most automatic seizure detection methods are patient-specific and have poor generalization ability to unseen patients. In this paper, we consider two aspects for solving the cross-patient generalization problem in deep learning approaches. Firstly, we propose a new data augmentation method to largely improve generalization. We analyze the statistical distribution of seizure EEG signals and propose spatio-temporal EEG augmentation (STEA) to generate synthetic training seizure data with spatio-temporal dependencies in EEG. Secondly, we propose patient-adversarial neural network (PANN) to learn a patient-invariant representation for better generalization. We add a discriminator to distinguish the identity of patients and perform adversarial optimization between the feature extractor and identity discriminator, so that only shared seizure features are maintained. We conduct experiments on both public and clinical EEG datasets with various settings of different window lengths and divisions of training and testing sets. Our model significantly improves the performance of cross-patient detection compared with other methods and achieves state-of-the-art performance. Particularly, our proposed method achieves up to 95% sensitivity on CHB-MIT under 5 s window length segments, and achieves about 85% AUC and sensitivity even on the clinical dataset. Our designed methods can enable detection models extended to unseen patients, with performance achieving the clinical application standards.

Despite the recent progress of Semi-supervised Learning (SSL), we argue that the existing methods may not employ unlabeled examples effectively and efficiently. Many pseudo-label-based methods select unlabeled examples into the training stage based on the inaccurate confidence scores provided by the output layer of the classifier network. Additionally, most prior work typically adpots all the available unlabeled examples without data pruning, which is incapable of learning from massive unlabeled data. To address these issues, this paper proposes two methods called VCC (Variational Confidence Calibration) and INFUSE (INfluence-Function-based Unlabeled Sample Elimination). VCC is a general-purpose plugin of confidence calibration for SSL. By approximating the calibrated confidence through three types of consistency scores, a variational autoencoder is leveraged to reconstruct the confidence score for selecting more accurate pseudo-labels. Based on the influence function, INFUSE is a data pruning method for constructing a core dataset of unlabeled examples. The effectiveness of our methods is demonstrated through experiments on multiple datasets and in various settings. For example, on the CIFAR-100 dataset with 400 labeled examples, VCC reduces the classification error rate of FixMatch from 46.47\% to 43.31\% (with improvement of 3.16\%). On the SVHN dataset with 250 labeled examples, INFUSE achieves 2.61\% error rate using only 10\% unlabeled data, which is better than RETRIEVE (2.90\%) and the baseline with full unlabeled data (3.80\%). Putting all the pieces together, the combined VCC-INFUSE plugins can reduce the error rate of FlexMatch from 26.49\% to 25.41\% on the CIFAR100 dataset (with improvement of 1.08\%) while saving nearly half of the original training time (from 223.96 GPU hours to 115.47 GPU hours).

IntroductionEpilepsy is a global chronic disease that brings pain and inconvenience to patients, and an electroencephalogram (EEG) is the main analytical tool. For clinical aid that can be applied to any patient, an automatic cross-patient epilepsy seizure detection algorithm is of great significance. Spiking neural networks (SNNs) are modeled on biological neurons and are energy-efficient on neuromorphic hardware, which can be expected to better handle brain signals and benefit real-world, low-power applications. However, automatic epilepsy seizure detection rarely considers SNNs.MethodsIn this article, we have explored SNNs for cross-patient seizure detection and discovered that SNNs can achieve comparable state-of-the-art performance or a performance that is even better than artificial neural networks (ANNs). We propose an EEG-based spiking neural network (EESNN) with a recurrent spiking convolution structure, which may better take advantage of temporal and biological characteristics in EEG signals.ResultsWe extensively evaluate the performance of different SNN structures, training methods, and time settings, which builds a solid basis for understanding and evaluation of SNNs in seizure detection. Moreover, we show that our EESNN model can achieve energy reduction by several orders of magnitude compared with ANNs according to the theoretical estimation.DiscussionThese results show the potential for building high-performance, low-power neuromorphic systems for seizure detection and also broaden real-world application scenarios of SNNs.

Semi-supervised Learning (SSL) reduces the need for extensive annotations in deep learning, but the more realistic challenge of imbalanced data distribution in SSL remains largely unexplored. In Class Imbalanced Semi-supervised Learning (CISSL), the bias introduced by unreliable pseudo-labels can be exacerbated by imbalanced data distributions. Most existing methods address this issue at instance-level through reweighting or resampling, but the performance is heavily limited by their reliance on biased backbone representation. Some other methods do perform feature-level adjustments like feature blending but might introduce unfavorable noise. In this paper, we discuss the bonus of a more balanced feature distribution for the CISSL problem, and further propose a Balanced Feature-Level Contrastive Learning method (BaCon). Our method directly regularizes the distribution of instances' representations in a well-designed contrastive manner. Specifically, class-wise feature centers are computed as the positive anchors, while negative anchors are selected by a straightforward yet effective mechanism. A distribution-related temperature adjustment is leveraged to control the class-wise contrastive degrees dynamically. Our method demonstrates its effectiveness through comprehensive experiments on the CIFAR10-LT, CIFAR100-LT, STL10-LT, and SVHN-LT datasets across various settings. For example, BaCon surpasses instance-level method FixMatch-based ABC on CIFAR10-LT with a 1.21% accuracy improvement, and outperforms state-of-the-art feature-level method CoSSL on CIFAR100-LT with a 0.63% accuracy improvement. When encountering more extreme imbalance degree, BaCon also shows better robustness than other methods.

In this paper, we develop a novel regularization method for deep neural networks by penalizing the trace of Hessian. This regularizer is motivated by a recent guarantee bound of the generalization error. We explain its benefits in finding flat minima and avoiding Lyapunov stability in dynamical systems. We adopt the Hutchinson method as a classical unbiased estimator for the trace of a matrix and further accel-erate its calculation using a Dropout scheme. Experiments demonstrate that our method outperforms existing regularizers and data augmentation methods, such as Jacobian, Confidence Penalty, Label Smoothing, Cutout, and Mixup. The code is available at https://github.com/Dean-lyc/Hessian-Regularization.(c) 2023 Elsevier B.V. All rights reserved.

Objectives: Laryngoscopy is a medical procedure for obtaining a view of the human larynx. It is challenging for clinicians to distinguish laryngeal neoplasms by human visual observation. Recent deep learning methods can assist clinicians in improving the accuracy of distinguishing. However, existed methods are often trained on large-scale private datasets, while other researchers and hospitals can neither access these private datasets nor afford to build such large-scale datasets. In this paper, we focus on identifying laryngeal neoplasms under the "small data" regime, which is more important for many small hospitals to investigate deep learning models for diagnosis.Material and methods: We build an extremely small dataset consisting of 279 laryngoscopic images of different categories. We found that traditional deep learning models for image classification cannot achieve satisfactory performance for small data, due to the great variability of recording laryngoscopic images and the small area of the neoplasms. To address these difficulties, we propose to employ object detection methods for this small data problem. Concretely, a Faster R-CNN is implemented here, which combines the DropBlock regularization technique to alleviate overfitting additionally.Results: Compared to previous methods, our model is more robust to overfitting and can predict the location and category of detected neoplasms simultaneously. Our method achieves 73.00% overall accuracy, which is higher than the average of clinicians (65.05%) and the recent state-of-the-art classification method (65.00%).Conclusion: The proposed method shows great ability to detect both the category and location of neoplasms and can be served as a screening tool to help the final decisions of clinicians.(c) 2023 AGBM. Published by Elsevier Masson SAS. All rights reserved.

OBJECTIVE:The 730 nm picosecond titanium sapphire laser is a novel laser that shows promising results in treating freckles. This study aimed to further investigate the efficacy and safety of the 730 nm picosecond titanium sapphire laser for treating freckles in Asian patients compared with those of the 755 nm picosecond alexandrite laser. METHODS:Each face of 86 participants was split into two parts and randomly assigned either one session of 730 or 755 nm picosecond-laser treatment each. Efficacy and safety were determined based on blinded visual evaluations and self-reports at each follow-up visit. RESULTS:The treatment outcomes of the 730 nm picosecond laser for the treatment of freckles were comparable to those of the 755 nm picosecond laser, with 68.99 ± 7.42% and 69.27 ± 7.75% clearance, respectively (p > 0.05). Participants achieved similar Global Aesthetic Improvement Scale scores (4.04 ± 0.31 vs. 4.02 ± 0.30, respectively [p > 0.05]). Additionally, the 730 nm picosecond laser was perceived to be less painful than the 755 nm picosecond laser (4.69 ± 1.63 vs. 5.65 ± 1.80 nm, p < 0.0001). CONCLUSION:The 730 nm picosecond laser is safe and effective for the treatment of freckles in Asian patients. Besides, the 730 nm picosecond laser is less painful than the 755 nm picosecond laser.

Hardware assurance (HA) is imperative to ensure the integrity of integrated circuits (ICs) after manufacturing. To verify the integrity of an IC would require the extraction of relevant functional block of interest through circuit partitioning. This is usually done by converting a recovered circuit netlist into a circuit graph and subsequently performing unsupervised clustering on the graph. However, circuit graphs are difficult to cluster due to the existence of inherent hierarchies and clusters of imbalanced sizes. In this article, we propose a novel hierarchical graph neural network (GNN), termed as GraphClusNet, to perform circuit graph clustering in a multistage process to achieve near-optimal results. We analytically derive a normalized-cut-based loss function, which allows for clusters of imbalanced sizes. We train our GraphClusNet in a multistage hierarchical fashion to regularize intermediate node embedding, which greatly alleviates the oversmoothing effect in deep GNNs. We further propose a multiscale location-based node feature and feature concatenation to provide a good initialization to stabilize the training of our GNN. Based on the experiments of synthetic graphs, we show that our proposed GraphClusNet obtained solutions with n-cut values up to $\mathbf {65\times }$ closer to the global optimum. On FPGA circuit graphs, our proposed GraphClusNet outperformed the reported methods with up to 30% improvement in normalized mutual information for circuit graph clustering.

Recent empirical studies observe that even without normalization, a deep residual network can be trained reliably. We call such a structure as normalization-free Residual Networks (N-F ResNets), which add a learnable parameter $\alpha$ to control the scale of the residual block instead of normalization. However, the theoretical understanding on N-F ResNets is still limited despite their empirical success. In this paper, we provide the first theoretical understanding of N-F ResNets from two perspectives. Firstly, we prove that the gradient descent (GD) algorithm can find the global minimum of the training loss at a linear rate for over-parameterized N-F ResNets. Secondly, we prove that N-F ResNets can avoid the gradient exploding or vanishing problem, by initializing the key parameter $\alpha$ to be a small constant. Notably, we demonstrate that the gradients of N-F ResNets are more stable than those of ResNets with Kaiming initialization. Moreover, empirical experiments on benchmark datasets verify our theoretical results.

In this paper, we develop a novel regularization method for deep neural networks by penalizing the trace of Hessian. This regularizer is motivated by a recent guarantee bound of the generalization error. We explain its benefits in finding flat minima and avoiding Lyapunov stability in dynamical systems. We adopt the Hutchinson method as a classical unbiased estimator for the trace of a matrix and further accelerate its calculation using a dropout scheme. Experiments demonstrate that our method outperforms existing regularizers and data augmentation methods, such as Jacobian, Confidence Penalty, Label Smoothing, Cutout, and Mixup.

Epilepsy is a chronic neurological disease that affects many people in the world. Automatic epileptic seizure detection based on electroencephalogram (EEG) signals is of great significance and has been widely studied. The current deep learning epilepsy detection algorithms are often designed to be relatively simple and seldom consider the characteristics of EEG signals. In this paper, we propose a promising epilepsy detection model based on convolutional transformer networks. We demonstrate that integrating convolution and transformer modules can achieve higher detection performance. Our convolutional transformer model is composed of two branches: one extracts time-domain features from multiple inputs of channel-exchanged EEG signals, and the other handle frequency-domain representations. Experiments on two EEG datasets show that our model offers state-of-the-art performance. Particularly on the CHB-MIT dataset, our model achieves 96.02% in average sensitivity and 97.94% in average specificity, outperforming other existing methods with clear margins.

Background:We aimed to establish and validate a deep learning-based hybrid artificial intelligence (AI) model for the objective morphometric and colorimetric assessment of vitiligo lesions.Methods:Two main datasets containing curated images of vitiligo lesions from Chinese patients (Fitzpatrick skin types III or IV) were established, including one with 2,720 images for lesion localization study and the other with 1,262 images for lesion segmentation study. Besides, an additional test set containing 145 images of vitiligo lesions from other Fitzpatrick skin types (I, II, or V) was also generated. A 3-stage hybrid model was constructed. YOLO v3 (You Only Look Once, v3) architecture was trained and validated to classify and localize vitiligo lesions, with sensitivity and error rate as primary performance outcomes. Then a segmentation study comparing 3 deep convolutional neural networks (DCNNs), Pyramid Scene Parsing Network (PSPNet), UNet, and UNet++, was carried out based on the Jaccard index (JI). The architecture with the best performance was integrated into the model. Three add-on metrics, namely VAreaA, VAreaR, and VColor were finally developed to measure absolute, relative size changes and pigmentation, respectively. Agreement between the AI model and dermatologist evaluators were assessed.Results:The sensitivity of the YOLO v3 architecture to detect vitiligo lesions was 92.91% with an error rate of 14.98%. The UNet++ architecture outperformed the others in the segmentation study (JI, 0.79) and was integrated into the model. On the additional test set, however, the model achieved a lower detection sensitivity (72.41%) and a lower segmentation score (JI, 0.69). With respect to size changes, no difference was observed between the AI model, trained dermatologists (W=0.812, P<0.05), and Photoshop analysis (P=0.075, P=0.212 respectively), which all displayed good concordance.Conclusions:We developed a novel, convenient, objective, and quantitative deep learning-based hybrid model which simultaneously evaluated both morphometric and colorimetric vitiligo lesions from patients with Fitzpatrick skin types III or IV, rendering it suitable for the assessment of severity of vitiligo lesions in Asians in both clinic and research scenarios. More work is also warranted for its use in other ethnic skin groups.

Continual learning aims to alleviate catastrophic forgetting when handling consecutive tasks under non-stationary distributions. Gradient-based meta-learning algorithms have shown the capability to implicitly solve the transfer-interference trade-off problem between different examples. However, they still suffer from the catastrophic forgetting problem in the setting of continual learning, since the past data of previous tasks are no longer available. In this work, we propose a novel efficient meta-learning algorithm for solving the online continual learning problem, where the regularization terms and learning rates are adapted to the Taylor approximation of the parameter's importance to mitigate forgetting. The proposed method expresses the gradient of the meta-loss in closed-form and thus avoid computing second-order derivative which is computationally inhibitable. We also use Proximal Gradient Descent to further improve computational efficiency and accuracy. Experiments on diverse benchmarks show that our method achieves better or on-par performance and much higher efficiency compared to the state-of-the-art approaches.