Spiking Neural Networks (SNNs) are recognized as a promising type of neural network due to their biological plausibility and low power consumption. However, for effective deployment in practical scenarios, SNNs still face the challenge of balancing high performance and strong robustness. Although recent studies have utilized knowledge distillation (KD) to construct high-accuracy and deeper SNNs, they often overlook the transfer of robustness from the teacher network. To address this challenge, we propose a Stage-wise Robust Distillation (SWRD) framework that utilizes an Artificial Neural Network (ANN) as the teacher model to enhance the robustness of SNNs in corrupted and noisy environments. Our framework is characterized by the linear incorporation of both response-based and hybrid distillation at different training stages, tailored to the sparsity properties of SNNs. Experimental results demonstrate that our SWRD method not only significantly improves image classification accuracy but also outperforms previous state-of-the-art (SOTA) methods in corruption robustness on CIFAR-10/100. Moreover, it also achieves competitive performance on the ImageNet dataset. Our code is available at https://github.com/fourbeans/SWKD.

Geometric features derived from single static images have the potential to be highly effective for facial emotion analysis, as shape, structure, and spatial relationships are key factors. However, these aspects are rarely explored in existing research. In this paper, we propose a novel approach that utilizes Genetic Programming (GP) to automatically extract geometric features for more effective emotional representation. The proposed GP system uses various evaluation strategies, evolving either a single feature per run or multiple features within a single run. These GP-evolved features capture critical angular and distance-based relationships between facial landmarks, which are then integrated with an existing deep learning model to enhance performance. The results show that the proposed method achieves improved performance in dimensional emotion analysis, providing a more comprehensive understanding of emotional expressions in static images. In addition, our approach is effective in improving the accuracy of emotion predictions, establishing a foundation for more precise facial emotion analysis using geometric information.

End-to-end learning approaches have become one of the mainstream paradigms in embodied navigation research, particularly demonstrating advantages in tasks that integrate perception and decision-making. However, conventionally structured end-to-end models rely on RNNs based on discrete time representations for time-series-modeling. This reliance limits their performance and generalization capability in complex environments. Liquid Neural Networks (LNNs), as continuous-time dynamical systems, offer improved generalization in dynamic environments. We incorporate LNNs into Object-Goal Navigation to enhance existing end-to-end navigation methods. To address the gradient vanishing issues commonly observed in both ODE-based and closed-form approximated liquid models when processing long-horizon sequential data in navigation tasks, we introduce the memory-augmented closed-form continuous-time network (M-CfC). By reconstructing the internal memory mechanism, M-CfC enhances temporal modeling capacity and training stability in long-horizon navigation scenarios. Experiments on challenging object-goal navigation benchmarks show that M-CfC improves the state-of-the-art success rate from 72.0% to 73.5%, significantly enhances path planning efficiency, while reducing the number of parameters and computational consumption compared to standard LNNs, demonstrating superior performance and potential for real-world embodied agents.

Document classification is a challenging task to the data being high-dimensional and sparse. Many transfer learning methods have been investigated for improving the classification performance by effectively transferring knowledge from a source domain to a target domain, which is similar to but different from the source domain. However, most of the existing methods cannot handle the case that the training data of the target domain does not have labels. In this study, we propose a transductive transfer learning system, utilizing solutions evolved by genetic programming (GP) on a source domain to automatically pseudolabel the training data in the target domain in order to train classifiers. Different from many other transfer learning techniques, the proposed system pseudolabels target-domain training data to retrains classifiers using all target-domain features. The proposed method is examined on nine transfer learning tasks, and the results show that the proposed transductive GP system has better prediction accuracy on the test data in the target domain than existing transfer learning approaches including subspace alignment-domain adaptation methods, feature-level-domain adaptation methods, and one latest pseudolabeling strategy-based method.

Language is an advanced cognitive function of humans, and verbs play a crucial role in language. To understand how the human brain represents verbs, it is critical to analyze what knowledge humans have about verbs. Thus, several verb feature datasets have been developed in different languages such as English, Spanish, and German. However, there is still a lack of a dataset of Chinese verbs. In this study, we developed a semantic feature dataset of 1140 Chinese Mandarin verbs (CVFD) with 11 dimensions including verb familiarity, agentive subject, patient, action effector, perceptual modality, instrumentality, emotional valence, action imageability, action complexity, action intensity, and the usage scenario of action. We calculated the semantic features of each verb and the correlation between dimensions. We also compared the difference between action, mental, and other verbs and gave some examples about how to use CVFD to classify verbs according to different dimensions. Finally, we discussed the potential applications of CVFD in the fields of neuroscience, psycholinguistics, cultural differences, and artificial intelligence. All the data can be found at https://osf.io/pv29z/.

Backpropagation-based supervised learning has achieved great success in computer vision tasks. However, its biological plausibility is always controversial. Recently, the bio-inspired Hebbian learning rule (HLR) has received extensive attention. Self-Organizing Map (SOM) uses the competitive HLR to establish connections between neurons, obtaining visual features in an unsupervised way. Although the representation of SOM neurons shows some brain-like characteristics, it is still quite different from the neuron representation in the human visual cortex. This paper proposes an improved SOM with multi-winner, multi-code, and local receptive field, named mlSOM. We observe that the neuron representation of mlSOM is similar to the human visual cortex. Furthermore, mlSOM shows a sparse distributed representation of objects, which has also been found in the human inferior temporal area. In addition, experiments show that mlSOM achieves better classification accuracy than the original SOM and other state-of-the-art HLR-based methods. The code is accessible at https://github.com/JiaHongZ/mlSOM.

The U-Net deep learning algorithm and its variants have been developed for biomedical image segmentation, and due to their success gained popularity in other science domains including remote sensing. So far no U-Net structure has been specifically designed to segment complex tree canopies from aerial imagery. In this paper, a handcrafted convolutional block is introduced to replace the raw convolutional block used in the standard U-Net structure. Furthermore, we proposed a Genetic Programming (GP) approach to evolving convolutional blocks used in the U-Net structure. The experimental results on a tree crown dataset show that both the handcrafted block and the GP evolved blocks have better segmentation results than the standard U-Net. Additionally, the U-Net using the proposed handcrafted blocks has fewer numbers of the learning parameters than the standard U-Net. Also, the proposed GP approach can evolve convolutional blocks used in U-Nets that perform better than the handcrafted U-Net and the standard U-Net, and can also achieve automation.

Most current rolling bearing fault diagnostic approaches presume that sufficient labeled data are available for training. However, labeled fault data collection in realistic engineering is time-consuming and expensive, whereas the majority of data available are unlabeled, restricting diagnostic performance. To alleviate the dependence of feature self-extraction model on rolling bearing fault data and improve the feature capture ability, a bearing fault diagnosis method based on feature-enhanced generative adversarial networks with an auxiliary classifier (AC-FEGAN) is proposed in this paper. Firstly, the non-saturating loss with gradient penalty term is introduced to circumvent the gradient vanishing and improve learning stability. Then, employing identity mapping of residual networks introduced into AC-FEGAN, representative faulty features can be automatically extracted, overcoming the restrictions of shallow convolutional neural networks in feature extraction. Simultaneously, to improve feature learning ability, the self-attention module embedded in the residual network accomplishes refined feature extraction by incorporating appropriate weight matrices into the faulty feature maps. Finally, the auxiliary classifier is employed as a pre-training model for the fault diagnosis model, with generated and unlabeled samples employed to fine-tune the auxiliary classifier to identify faulty bearings. The experimental results show that the proposed method can significantly improve the quality of generated samples while eliminating the reliance of existing diagnosis methods on data volume to achieve more accurate and effective fault diagnosis.

With the fast development of deep learning models, hierarchical convolutional neural networks have achieved great success in image denoising tasks. To further boost the performance of image denoising, a novel non-local hierarchical network (NHNet) is proposed. Unlike existing U-Net-based hierarchical methods, which mainly focus on downsampling operations, NHNet adopts an initial resolution path and a high resolution path. Specifically, the high-resolution features are obtained through upsampling, where the non-local mechanism is adopted to capture the self-similarity properties, which contribute to a better denoising performance. Cross connections and channel attention layers are added between the two paths to integrate features in different resolutions. Compared with other U-Net-based hierarchical networks, NHNet requires fewer parameters. Experiments show that NHNet achieves state-of-the-art performance in Gaussian denoising tasks and gets competitive results when dealing with real image denoising.

Computational neuroscience is interdisciplinary and plays an essential role in facilitating the development of cognitive neuroscience and Artificial General Intelligence (AGI). To explore the computational efficiency of Spiking Neural Network (SNN) on a computational hardware platform, we deploy SNN on the Xilinx Zynq7 Zedboard Field Programmable Gate Array (FPGA). Specifically, we use multi-compartmental Hodgkin-Huxley (HH) neuron model to construct neural circuits via C++ code and adopt a biologically plausible calcium concentration-based plasticity model as the learning rule of SNN. Our experimental results show that the FPGA platform can support parallel computation of the multi-compartment HH model with synaptic plasticity. Compared to other methods, FPGA-based computing has a significant speed advantage, suggesting that FPGA can be a good choice for implementing brain-inspired computation.

The rapid development of the Internet has caused information overload, and recommendation algorithms have emerged. In recent years, graph neural network (GNN) has been increasingly used in recommendation algorithms due to its strong performance in learning the node and relationship representation of networks. Based on the perspective of graph structure, this paper provides a new taxonomy of graph neural networks for recommendation models, and introduces the latest typical models in each category. We compare the differences and commonalities of different models from the step of information aggregation and information combination of graph neural networks, and summarize the challenges and difficulties still faced by GNN recommendation, hoping to provide reference for future research.

Transfer learning has been studied in document classification for transferring a model trained from a source domain (SD) to a relatively similar target domain (TD). In feature-based transfer learning techniques, there is an investigation on the features being transferred from SD to TD. This paper conducts an investigation on an output-based transfer learning system using Genetic Programming (GP) in document classification tasks, which automatically selects features to construct classifiers. The proposed GP system directly generates programs from a set of sparse features and only considers the output change of the evolved programs from SD to TD. A linear model is then used to combine existing GP programs from SD as features to TD. Also, new GP programs are mutated from the programs evolved in SD to improve the accuracy. Via directly utilizing the evolved GP programs and their mutations, the feature extraction and estimation processes on TD are avoided. The results for the experiments demonstrates that the GP programs from SD can be effectively used for classifying documents in the relevant TD. The results also show that it is easy to train effective classifiers on TD when the GP programs are used as features. Furthermore, the proposed linear model, using multiple GP programs from SD as its inputs, outperforms single GP programs which are directly obtained from TD.

Document classification is one of the predominant tasks in Natural Language Processing. However, some document classification tasks do not have ground truth while other similar datasets may have ground truth. Transfer learning can utilize similar datasets with ground truth to train effective classifiers on the dataset without ground truth. This paper introduces a transductive transfer learning method for document classification using two different text feature representations—the term frequency (TF) and the semantic feature doc2vec. It has three main contributions. First, it enables the sharing knowledge in a dataset using TF and a dataset using doc2vec in transductive transfer learning for performance improvement. Second, it demonstrates that the partially learned programs from TFs and from doc2vecs can be alternatively used to “label then learn” and they improve each other. Lastly, it addresses the unbalanced dataset problem by considering the unbalanced distributions on categories for evolving proper Genetic Programming (GP) programs on the target domains. Our experimental results on two popular document datasets show that the proposed technique effectively transfers knowledge from the GP programs evolved from the source domains to the new GP programs on the target domains using TF or doc2vec. There are obviously more than 10 percentages improvement achieved by the GP programs evolved by the proposed method over the GP programs directly evolved from the source domains. Also, the proposed technique effectively utilizes GP programs evolved from unbalanced datasets (on the source and target domains) to evolve new GP programs on the target domains, which balances predictions on different categories.

Recently, with the broad applications of deep learning technology in image denoising, many deep neural networks based on the residual block (ResBlock) have been proposed to improve the performance.This paper presents DC-ResBlock, a ResBlock with an extra dilated convolution, to replace standard ResBlock for image denoising.Our study shows that the deep neural network with DC-ResBlocks, named DRNet, can achieve a very competitive result.We experiment with other residual blocks by adding Rectified Linear Unit, batch normalization, and Dilated Convolution to the standard ResBlock in several different ways.Ablation study shows that DC-Resblock has the best denoising performance.To evaluate the robustness of proposed DRNet, we make a statistical measure using many different random seeds instead of just a single seed, as used by many previous studies.DRNet performs well in denoising gray and color images with additive white Gaussian noise.Furthermore, because of its good performance on the Smartphone Image Denoising Dataset, DRNet is hopeful to be applied in practical tasks.The code of DRNet is accessible at https://github.com/JiaHongZ/DRNet.

The three-dimensional human body feature parameters extracted by the neural network method are coupled in the high-dimensional feature space. The fully connected network processes the three-dimensional human pose and shape feature information synchronously, which cannot handle multifactor features well, which makes the model perform poorly. To solve this problem, on the basis of the benchmark network, a decoupling model is added, which processes the three-dimensional human pose and shape information separately, and finally integrates the features through the bi-linear transformation method. Through experimental verification, the algorithm effectively reduces the reconstruction error, and the improved algorithm performs better.

A reliable wind speed forecasting framework can contribute to handling rational dispatching and safe operation for power system effectively. For this purpose, a novel compound framework coupling decomposition technique, subseries aggregation, synchronous optimization, improved deep network and residual error correction (REC) is investigated in this study. To begin with, time varying filter-based empirical mode decomposition (TVF-EMD) is employed to decompose the raw series into a set of subseries, which are further aggregated based on fuzzy entropy (FE) theory and approximation criterion. Then the synchronous optimization implemented by blended coding-based Harris hawks optimization (HHO) is adopted to optimize the parameters of phase space reconstruction (PSR) and applicable features for each aggregated subseries. Subsequently, quantile regression (QR) is incorporated into an improved deep network, namely convolutional simplified long short-term memory network (QRConvSLSTM), to deduce conditional quantiles for each aggregated subseries, in which the optimized arguments obtained above are applied to construct the optimal input matrixes. Later, the initial point forecasting results can be calculated on the basis of accumulating the conditional quantiles of all the aggregated subseries, while the corresponding error series is deduced therewith. Then the conditional quantiles of the error series are estimated by QRConvSLSTM in the light of REC strategy, after which the final conditional quantiles are calculated by summating the conditional quantiles of the raw series and the error series. Finally, kernel density estimation (KDE) is employed to estimate probabilistic density functions (PDF) of wind speed series in accordance to the final conditional quantiles. To validate the efficiency and effectiveness of the proposed compound framework, nine relevant models are performed on three datasets for comparative experiments, among which the results of point, interval and probability prediction are comprehensively demonstrated and analyzed. The experimental results illustrate that: (1) data preprocessing strategy integrating TVF-EMD and FE-based subseries aggregation contributes to balancing forecasting performance and timing computation properly; (2) the applicable deterministic and uncertainty forecasting results can be obtained by the improved deep network, namely QRConvSLSTM; (3) appropriate parameters of PSR and feature selection can be effectively optimized by the proposed synchronous optimization; (4) the application of REC possesses positive effects on further compensating the ultimate forecasting results.

The integration of transfer learning methods and other machine learning branches can bring a good improvement in speed and performance, it has become a good research topic in the recent years and it is necessary for researchers to understand the integration. This paper focuses on sorting and classifying the integrated results of the transfer learning and the other non-transfer machine learning, such as reinforcement leaning, lifelong learning, adversarial networks. Besides, this paper also organizes the latest work on the transitive transfer learning and categorizes the relevant applications for transfer learning.

In edge detection, designing new techniques to combine local features is expected to improve detection performance. However, how to effectively design combination techniques remains an open issue. In this study, an automatic design approach is proposed to combine local edge features using Bayesian programs (models) evolved by genetic programming (GP). Multivariate density is used to estimate prior probabilities for edge points and non-edge points. Bayesian programs evolved by GP are used to construct composite features after estimating the relevant multivariate density. The results show that GP has the ability to effectively evolve Bayesian programs. These evolved programs have higher detection accuracy than the combination of local features by directly using the multivariate density (of these local features) in a simple Bayesian model. From evolved Bayesian programs, the proposed GP system has potential to effectively select features to construct Bayesian programs for performance improvement.

Document classification is a common but challenging task in text mining, since the feature set used is often high-dimensional and sparse. Transfer learning has been applied to improve the classification performance of a (target) domain by transferring knowledge from a previously learnt (source) domain. When there are no labels provided for documents in target domains, it is challenging to effectively transfer knowledge from source domains to target domains. In this paper, we develop a new Genetic Programming (GP) based transfer learning method for document classification, which utilises the evolved GP programs from the source domain to learn a set of weak GP classification models on the target domain with unlabelled documents, which is called self-taught learning. These weak classifiers are combined with the GP programs transferred from the source domain to predict the labels of test documents in the target domain. The experimental results show that the GP programs from source domains with their weak classifiers can effectively classify documents in the target domain.

For the study of new computer architecture inspired by brain, implementing neuron function on hardware has become an emerging research direction. To study the feasibility of biological neuron implementation and acceleration characteristics on the hardware platform, the basic Hodgkin-Huxley (HH) neuron model and a small scale of synapse connection model is implemented on Xilinx Zynq7 Zedboard FPGA, high-level-synthesis method is used to reduce the burden of hardware design. The Hodgkin-Huxley model simulation result of FPGA are validated through high level software simulation tool in Brian2, furthermore, resource utilization and calculation time cost are provided in this paper, finally, a pipeline design of HH neuron model is used to accelerate calculations and reduce resource utilization, the result proves that this design is suitable for HH model simulation.