Hierarchical temporal memory (HTM) is a promising unsupervised machine-learning algorithm that models key principles of neocortical computation. One of the main components of HTM is the spatial pooler (SP), which encodes binary input streams into sparse distributed representations (SDRs). In this paper, we propose an information-theoretic framework for the performance comparison of HTM-spatial pooler (SP) algorithms, specifically, for quantifying the similarities and differences between sparse distributed representations in SP algorithms. We evaluate SP's standalone performance, as well as HTM's overall performance. Our comparison of various SP algorithms using Renyi mutual information, Renyi divergence, and Henze–Penrose divergence measures reveals that the SP algorithm with learning and a logarithmic boosting function yields the most effective and useful data representation. Moreover, the most effective SP algorithm leads to superior HTM results. In addition, we utilize our proposed framework to compare HTM with other state-of-the-art sequential learning algorithms. We illustrate that HTM exhibits superior adaptability to pattern changes over time than long short term memory (LSTM), gated recurrent unit (GRU) and online sequential extreme learning machine (OS-ELM) algorithms. This superiority is evident from the lower Renyi divergence of HTM (0.23) compared to LSTM6000 (0.33), LSTM3000 (0.38), GRU (0.41), and OS-ELM (0.49). HTM also achieved the highest Renyi mutual information value of 0.79, outperforming LSTM6000 (0.73), LSTM3000 (0.71), GRU (0.68), and OS-ELM (0.62). These findings not only confirm the numerous advantages of HTM over other sequential learning algorithm, but also demonstrate the effectiveness of our proposed information-theoretic approach as a powerful framework for comparing and evaluating various learning algorithms.

One of the main disadvantages of the traditional mean square error (MSE)-based constructive networks is their poor performance in the presence of non-Gaussian noises. In this paper, we propose a new incremental constructive network based on the correntropy objective function (correntropy-based constructive neural network (C2N2)), which is robust to non-Gaussian noises. In the proposed learning method, input and output side optimizations are separated. It is proved theoretically that the new hidden node, which is obtained from the input side optimization problem, is not orthogonal to the residual error function. Regarding this fact, it is proved that the correntropy of the residual error converges to its optimum value. During the training process, the weighted linear least square problem is iteratively applied to update the parameters of the newly added node. Experiments on both synthetic and benchmark datasets demonstrate the robustness of the proposed method in comparison with the MSE-based constructive network, the radial basis function (RBF) network. Moreover, the proposed method outperforms other robust learning methods including the cascade correntropy network (CCOEN), Multi-Layer Perceptron based on the Minimum Error Entropy objective function (MLPMEE), Multi-Layer Perceptron based on the correntropy objective function (MLPMCC) and the Robust Least Square Support Vector Machine (RLS-SVM).

In the field of single image super-resolution (SISR), transformer-based models, have demonstrated significant advancements. However, the potential and efficiency of these models in applied fields such as real-world image super-resolution have been less noticed and there are substantial opportunities for improvement. Recently, composite fusion attention transformer (CFAT), outperformed previous state-of-the-art (SOTA) models in classic image super-resolution. In this paper, we propose a novel GAN-based framework by incorporating the CFAT model to effectively exploit the performance of transformers in real-world image super-resolution. In our proposed approach, we integrate a semantic-aware discriminator to reconstruct fine details more accurately and employ an adaptive degradation model to better simulate real-world degradations. Moreover, we introduce a new combination of loss functions by adding wavelet loss to loss functions of GAN-based models to better recover high-frequency details. Empirical results demonstrate that IG-CFAT significantly outperforms existing SOTA models in both quantitative and qualitative metrics. Our proposed model revolutionizes the field of real-world image super-resolution and demonstrates substantially better performance in recovering fine details and generating realistic textures. The introduction of IG-CFAT offers a robust and adaptable solution for real-world image super-resolution tasks.

Anti-lock braking system is designed to optimize braking procedure while maintaining automobile steerability through controlling wheels slip. However, due to nonlinearity and uncertainty of ABS structure, designing the controller for wheel slip encounters so many problems which necessitate a robust control system. In this paper a hybrid controller is proposed for ABS to address this issue. The designed controller is a combination of sliding mode control and fuzzy control. In fact, the fuzzy system determines switching factor of sliding mode controller proportional to automobile speed employing fuzzy rules. In this way, it would be able to avoid braking command fluctuations in lower speeds. Simulations are performed in ¼ model of automobile in MATLAB environment. The simulation results revealed the capability of proposed system to maintain slip ratio in optimal value as well as avoiding braking fluctuations in low speeds.

Feature selection (FS) is one of the major tasks in data cleansing step in machine learning. However, multi-objective FS is more challenging because it tries to optimize two conflicting objectives, namely minimizing the feature set and classification error. In this way, evolutionary algorithms are promising solutions aimed to obtain more reliable Pareto fronts. However, unfortunately they suffer from consuming much time due to exploration in a large search space. Another issue encountered in multi-objective FS approaches is related to the correlation between features. This challenge arises because choosing such features reduces the performance of the classification. To address these challenges, we introduce a multi-objective FS approach that makes several significant contributions. First, the proposed method deals with the correlation between features through a novel probability structure. Secondly, it relies on the Pareto Archived Evolution Strategy (PAES) method, which offers many advantages, including simplicity and its ability to explore the solution space at an acceptable speed. We enhance the PAES structure in a manner that promotes the intelligent generation of offsprings. Consequently, our proposed approach benefits from the introduced probability structure to generate more promising offspring. Lastly, it incorporates a novel strategy to guide the algorithm to find the optimal subset throughout the evolutionary process. The obtained results on real-world datasets reveal a substantial enhancement in the quality of the final solutions.

Hierarchical Temporal Memory (HTM) is an unsupervised algorithm in machine learning. It models several fundamental neocortical computational principles. Spatial Pooler (SP) is one of the main components of the HTM, which continuously encodes streams of binary input from various layers and regions into sparse distributed representations. In this paper, the goal is to evaluate the sparsification in the SP algorithm from the perspective of information theory by the information bottleneck (IB), Cramer-Rao lower bound, and Fisher information matrix. This paper makes two main contributions. First, we introduce a new upper bound for the standard information bottleneck relation, which we refer to as modified-IB in this paper. This measure is used to evaluate the performance of the SP algorithm in different sparsity levels and various amounts of noise. The MNIST, Fashion-MNIST and NYC-Taxi datasets were fed to the SP algorithm separately. The SP algorithm with learning was found to be resistant to noise. Adding up to 40% noise to the input resulted in no discernible change in the output. Using the probabilistic mapping method and Hidden Markov Model, the sparse SP output representation was reconstructed in the input space. In the modified-IB relation, it is numerically calculated that a lower noise level and a higher sparsity level in the SP algorithm lead to a more effective reconstruction and SP with 2% sparsity produces the best results. Our second contribution is to prove mathematically that more sparsity leads to better performance of the SP algorithm. The data distribution was considered the Cauchy distribution, and the Cramer-Rao lower bound was analyzed to estimate SP's output at different sparsity levels.

So far several types of random neural networks have been proposed in which optimal output weights are adjusted using the Mean Square Error (MSE) objective function. However, since many real-world phenomena do not follow a normal distribution, MSE-based methods act poorly in such cases. This paper presents a new single-layer random constructive neural network based on the regularized Minimum Error Entropy (MEE) objective function. The proposed method investigates the performance of MSE and MEE objective functions in combination using a regularization term to adjust the optimal output parameter for new nodes. Experimental results show that the proposed method performs well in the presence of both Gaussian and impulsive noise. Furthermore, due to the random assignment of the hidden layer parameters, the computational burden of the proposed method is reduced. Incremental constructive architecture of the proposed network helps optimize non-convex objective functions to achieve the desired performance. Computational comparisons indicate the superior performance of our method with several synthetic and benchmark datasets.

Aggregation of flexible loads and power generation of solar photo-voltaic (PV) systems is considered as a valuable power resource in residential demand response (DR). Despite the rapid growth of smart appliances, there are few practical solutions for exploiting their potentials in DR load aggregation. In this paper, we present a practical multi-prosumer framework to enable the aggregator reach a minimum bidding power and participate in the wholesale market. This is attainable through directly rescheduling a large number of smart appliances and utilizing the surplus power generation of residential PVs. An optimization model is designed which maximizes the aggregator's profit while respecting customers convenience. Fairness is a significant component of this model ensuring fair selection of appliances for shifting by the aggregator and not biased toward customers availability. We investigate the model as a hard instance of the 0–1 Knapsack problem and devise a heuristic algorithm to cope with its time complexity and to improve its scalability. The simulation results of two large-scale case studies are presented and discussed. It is demonstrated that the proposed framework is beneficial to both the aggregator and its customers, leading to a greener environment.

This paper proposes a novel decentralize and asynchronous swarm robotic search algorithm integrated with game theory to better disperse robots in the environment while crossing obstacles and solving mazes. This prevents early convergence and improves the efficiency of the searches. In the proposed algorithm, individual robots, while searching, play a sequential game at each iteration, and based on that, choose their velocity update rule. The effectiveness of the proposed strategic game is tested in a specially designed framework. As a validation, the introduced algorithm is compared with the state-of-the-art in simple and complex search environments. The results showed that the suggested algorithm outperforms other methods both in search duration and attained path length to the target, and its success rate is equal to the one of state-of-the-art (i.e., 100% in the conducted experiments). Also, it is shown that the proposed strategic game works well in search environments with different levels of complexity and especially improves search efficiency further in complex environments.

In the field of medical image analysis, there is a substantial need for high-resolution (HR) images to improve diagnostic accuracy. However, it is a challenging task to obtain HR medical images, as it requires advanced instruments and significant time. Deep learning-based super-resolution methods can help to improve the resolution and perceptual quality of low-resolution (LR) medical images. Recently, Generative Adversarial Network (GAN) based methods have shown remarkable performance among deep learning-based super-resolution methods. Real-Enhanced Super-Resolution Generative Adversarial Network (Real-ESRGAN) is a practical model for recovering HR images from real-world LR images. In our proposed approach, we use transfer learning technique and fine-tune the pre-trained Real-ESRGAN model using medical image datasets. This technique helps in improving the performance of the model. We employ the high-order degradation model of the Real-ESRGAN which better simulates real-world image degradations. This adaptation allows for generating more realistic degraded medical images, resulting in improved performance. The focus of this paper is on enhancing the resolution and perceptual quality of chest X-ray and retinal images. We use the Tuberculosis chest X-ray (Shenzhen) dataset and the STARE dataset of retinal images for fine-tuning the model. The proposed model achieves superior perceptual quality compared to the Real-ESRGAN model, effectively preserving fine details and generating images with more realistic textures.

Multi-objective feature selection is one of the most significant issues in the field of pattern recognition. It is challenging because it maximizes the classification performance and, at the same time, minimizes the number of selected features, and the mentioned two objectives are usually conflicting. To achieve a better Pareto optimal solution, metaheuristic optimization methods are widely used in many studies. However, the main drawback is the exploration of a large search space. Another problem with multi-objective feature selection approaches is the interaction between features. Selecting correlated features has negative effect on classification performance. To tackle these problems, we present a novel multi-objective feature selection method that has several advantages. Firstly, it considers the interaction between features using an advanced probability scheme. Secondly, it is based on the Pareto Archived Evolution Strategy (PAES) method that has several advantages such as simplicity and its speed in exploring the solution space. However, we improve the structure of PAES in such a way that generates the offsprings, intelligently. Thus, the proposed method utilizes the introduced probability scheme to produce more promising offsprings. Finally, it is equipped with a novel strategy that guides it to find the optimum number of features through the process of evolution. The experimental results show a significant improvement in finding the optimal Pareto front compared to state-of-the-art methods on different real-world datasets.

As global search techniques, population-based optimization algorithms have provided promising results in feature selection (FS) problems. However, their major challenge is high time complexity associated with the exploration of a large search space and consequently a large number of fitness function evaluations. Moreover, the interaction between features is another key issue in FS problems, directly affecting the classification performance through selecting correlated features. In this paper, an estimation of distribution algorithm (EDA)-based method is proposed with three important contributions. Firstly, as an extension of EDA, the proposed method in each iteration generates only two individuals competing based on a fitness function, evolving during the algorithm using our proposed update procedure. Secondly, we provide a guiding technique to determine the number of features to be selected for individuals in each iteration. As a result, the number of selected features in the final solution would be optimized during the evolution process. These two would lead to increasing the convergence speed of the algorithm. Thirdly, as the main contribution of the paper, in addition to considering the importance of each feature alone, the proposed method can consider the interaction between features, being able to deal with complementary features and consequently increase classification performance. To do this, we provide a conditional probability scheme that considers the joint probability distribution of selecting two features. The introduced probabilities successfully detect correlated features. Experimental results on a synthetic dataset with correlated features proved the performance of our proposed approach facing these types of features. Furthermore, the results on 13 real-world datasets obtained from the UCI repository showed the superiority of the proposed method in comparison with some state-of-the-art approaches. To evaluate the effectiveness of each feature subset, support vector machines are used as classifier. The efficiency analysis of the experimental results using two non-parametric statistical tests proved that the proposed method had significant advantages in comparison to other approaches.

Single image super-resolution (SISR) models are able to enhance the visual quality of underwater images and contribute to a better understanding of underwater environments. The integration of these models in Autonomous Underwater Vehicles (AUVs) can improve their performance in vision-based tasks. Real-ESRGAN is a powerful model that has shown remarkable performance among SISR models. In this paper, we optimize the Real-ESRGAN model for underwater image super-resolution. To optimize and evaluate the performance of the model, we use the USR-248 dataset. The proposed model generates images that demonstrate a higher level of visual quality than the outputs of the Real-ESRGAN model.

This paper presents a novel adaptive one-pass Passive-Aggressive Radial Basis Function (APARBF) for classification problems. The APARBF uses elliptic Gaussian neurons followed by a Softmax layer and cross-entropy loss function. This network tries to overcome the elasticity-plasticity dilemma by using the Passive-Aggressive (PA) algorithm and adapting the hidden layer structure. The weight updates have to be plastic to acquire the most information from each sample and at the same time need to be elastic to retrain the information from the past instances. Inspired by PA, a novel update formula for cross-entropy loss minimization has been derived. The adaptive design of the network lets it start with zero hidden neurons and grow or shrink according to the data. For kernel parameters, the adaptive structure determines the correct number of hidden neurons and updates recursively each neuron’s center and covariance matrix. To evaluate our network, we perform two series of experiments. The first experiments compare the proposed APARBF with other recently developed one-pass algorithms (i.e., OVIG, OBHT, SCW, AROW, OGD, and PA). The subsequent experiments include comparing the proposed algorithm with some online adaptive structures such as FGAP-RBF, C-Mantec, McNN, and PBL-McNN, based on their mean classification error and the number of hidden neurons. Wilcoxon sign rank test and Friedman test clearly show the superiority of the proposed network’s results compared with its competitors in one-pass classification problems.

This paper presents a new method of predicting the P300 component of an electroencephalography (EEG) signal to recognize the characters in a P300 brain-computer interface (BCI) speller accurately. This method consists of a deep learning model and the nonlinear time-frequency features. It is believed that the combination of the deep model network and extracting the nonlinear features of the EEG led this research to a better prediction of the P300 and, therefore, character recognition. Cohen's class distribution is used in order to extract the nonlinear features of the EEG. Evaluating all of the kernels, Butterworth found to be more informative and it produced better results. Based on the differences observed between time-frequency responses of target and nontarget signals, specific subbands are selected to extract seven features. A deep-structured neural network, namely stacked sparse autoencoders, is applied for BCI character recognition. This deep network reduces the dimension of feature space by extracting unsupervised features. Then, the features are fed to a Softmax classifier. Afterward, the whole network passes a fine-tuning phase by a supervised backpropagation algorithm. For evaluating the work, Dataset II of BCI Competition III is utilized. Based on the results, this approach would improve the accuracy in both P300 detection and character recognition. This research results in 82.7% and 93.5% accuracy for P300 classification and character recognition, respectively.

Epilepsy, a brain disease generally associated with seizures, has tremendous effects on people’s quality of life. Diagnosis of epileptic seizures is commonly performed on electroencephalography (EEG) signals, and by using computer-aided diagnosis systems (CADS), neurologists can diagnose epileptic seizure stages more accurately. In these systems, a mandatory stage is feature extraction, performed by handcrafting features or learning them, ordinarily by a deep neural net. While researches in this field commonly show the value of a group of limited features, yet an accurate comparison between different suggested features is essential. In this article, first, a comparison between the importance of 50 different handcrafted features for seizure detection is presented. Additionally, the computational complexity of features is investigated as well. Then the best features based on Fisher scores are picked to classify signals on a benchmark dataset for evaluation. Additionally, a convolutional autoencoder with five layers is applied to learn features in order to have a complete comparison among feature extraction approaches. Finally, a hybrid method is employed, which combines handcrafted features and encoding of autoencoder to reach high performance in seizure detection in EEG signals.

Training a support vector machine (SVM) on large datasets is a slow daunting process. Further, SVM becomes slow in the testing phase, due to its large number of support vectors (SVs). This paper proposes an effective geometric algorithm based on construction of Delaunay triangulation (DT) algorithm using Quickhull algorithm with a novel strategy to exactly identify and extract the boundary data points laid between the two classes of a dataset, and later uses these most informative data points as a reduced dataset to solve various SVM algorithms and proposes new DT-SVM algorithms Two synthetic datasets with the size of 1K incrementally up to 500K datasets are generated to extensively verify the effectiveness of the proposed DT-SVM algorithms over various data sizes and for further assessment, the most efficient version of proposed DT-SVM is applied on well-known benchmark datasets from UCI Machine Learning Repository. Two variant of sequential minimization optimization (SMO) decomposition methods, in addition to Least Square form of SVM are implemented to present the scalability of new DT-SVM algorithms in linear/nonlinear separable/non-separable large low dimensional datasets. Moreover, the most efficient version of the proposed algorithm is compared to RCH-SK as a known geometric approach in the SVM literature. The results demonstrate that while the proposed approach improves the scalability of DT-SVM in large low dimensional datasets, it leads SVM algorithms to maintain the accuracy in an acceptable range with considerably lower time in both training and testing phases with using a noticeably fewer number of SVs.

This paper proposes a novel decentralize and asynchronous robotic search algorithm based on particle swarm optimization (PSO), which has focused on solving mazes and finding targets in unknown environments with minimal inter-swarm communication and without any synchronization or communication center. In the proposed method, robots are advanced particles of the PSO algorithm, enriched with a toolkit, including an angle of rotation to change the course when confronted with obstacles to avoid them (AoR tool), and a memory to remember and reuse their best personal experiences to turn back from dead-ends (Mem tool). This toolkit enables the swarm to avoid obstacles and solve mazes while moving toward the target. The performance of the proposed algorithm is tested in a specially designed framework. As a validation, the proposed algorithm is compared with some recently published methods, including Adaptive Robotic PSO (A-RPSO), Robotic Bat Algorithm (RBA), and Adaptive Robotic Bat Algorithm (ARBA), in simple search environments that they can solve. The results of this comparison show that the introduced search method has the highest success rate (100%) in environments of different sizes and reflects the nature of swarm intelligence better. The proposed method is also tested in various maze-like search environments. The results depict the algorithm's high efficiency to solve mazes in varying complexity levels and locate the target in a reliable time. It is also shown that the performance of the proposed algorithm does not decrease and remains constant as the complexity of search environments increases.

Digital watermarking is a remarkable issue in the field of information security to avoid the misuse of images in multimedia networks. Although access to unauthorized persons can be prevented through cryptography, it cannot be simultaneously used for copyright protection or content authentication with the preservation of image integrity. Hence, this paper presents an optimized multipurpose blind watermarking in Shearlet domain with the help of smart algorithms including MLP and NSGA-II. In this method, four copies of the robust copyright logo are embedded in the approximate coefficients of Shearlet by using an effective quantization technique. Furthermore, an embedded random sequence as a semi-fragile authentication mark is effectively extracted from details by the neural network. Due to performing an effective optimization algorithm for selecting optimum embedding thresholds, and also distinguishing the texture of blocks, the imperceptibility and robustness have been preserved. The experimental results reveal the superiority of the scheme with regard to the quality of watermarked images and robustness against hybrid attacks over other state-of-the-art schemes. The average PSNR and SSIM of the dual watermarked images are 38 dB and 0.95, respectively; Besides, it can effectively extract the copyright logo and locates forgery regions under severe attacks with satisfactory accuracy.

A variety of screening approaches have been proposed to diagnose epileptic seizures, using electroencephalography (EEG) and magnetic resonance imaging (MRI) modalities. Artificial intelligence encompasses a variety of areas, and one of its branches is deep learning (DL). Before the rise of DL, conventional machine learning algorithms involving feature extraction were performed. This limited their performance to the ability of those handcrafting the features. However, in DL, the extraction of features and classification are entirely automated. The advent of these techniques in many areas of medicine, such as in the diagnosis of epileptic seizures, has made significant advances. In this study, a comprehensive overview of works focused on automated epileptic seizure detection using DL techniques and neuroimaging modalities is presented. Various methods proposed to diagnose epileptic seizures automatically using EEG and MRI modalities are described. In addition, rehabilitation systems developed for epileptic seizures using DL have been analyzed, and a summary is provided. The rehabilitation tools include cloud computing techniques and hardware required for implementation of DL algorithms. The important challenges in accurate detection of automated epileptic seizures using DL with EEG and MRI modalities are discussed. The advantages and limitations in employing DL-based techniques for epileptic seizures diagnosis are presented. Finally, the most promising DL models proposed and possible future works on automated epileptic seizure detection are delineated.