Sigma-Pi-Sigma neural network (SPSNN) has gained significant attention due to its exceptional modeling capabilities and superior nonlinear mapping performance. Despite these advantages, the training process of SPSNN poses some challenges. Conventional gradient-based optimization methods often exhibit limitations such as slow convergence and computational inefficiency when applied to SPSNN, negatively impacting the convergence and stability of the model. Therefore, in this paper, a Polak-Ribière-Polyak (PRP) conjugate gradient-based training algorithm is implemented in SPSNN. In contrast to traditional gradient-based training methods, conjugate gradient methods offer advantages in convergence speed and computational efficiency, which significantly enhances model performance. We assume four specific conditions and based on these, results on strong and weak convergence are established. Specifically, the error function presents a monotonic decrease and gradient approaching zero, indicating weak convergence. The weight sequence converges to a fixed point, indicating strong convergence. Detailed proofs of these results are provided. Finally, numerical simulations are conducted across three distinct categories: function approximation problems, parity problems and real-world classification problems. The experimental results not only validate the theoretical findings but also demonstrate the effectiveness of the proposed method. Comparative results also reveal that the proposed method has better performance than other typical methods.

Zero-order Takagi-Sugeno (TS0) fuzzy models are known for their robust interpretability and generalization capabilities, because they feature human-readable IF-THEN rules, semantically clear fuzzy sets, and inherent tolerance of fuzzy logic to uncertainty and imprecision. The TS0 model, based on traditional integer-order gradient descent methods, often struggles to capture the dynamic characteristics of complex nonlinear systems. Therefore, a novel Caputo fractional-order gradient descent method (FGDM) is presented for TS0 fuzzy models, aiming to accelerate convergence and enhance classification accuracy. This paper rigorously validates both the weak and strong convergence of the proposed method through theoretical analysis. The proposed FGDM demonstrates considerable improvements in both training and testing performance across a variety of tasks, including approximation, classification, and regression. These findings illustrate the potential of FGDM to enable broader and more effective applications in fuzzy systems.

This paper introduces an innovative neuro-fuzzy learning algorithm based on the Polak-Ribi & egrave;re-Polyak (PRP) conjugate gradient method. The proposed algorithm effectively integrates the PRP conjugate gradient approach with a first-order Takagi-Sugeno neuro-fuzzy network to enhance its accuracy and precision. Theoretical and empirical analyses prove the algorithm's convergence. They demonstrate that the fuzzy model can accurately approximate the underlying data structure and converge to an optimal solution using the conjugate gradient optimization strategy. In comparative analyses, the proposed algorithm was evaluated against fractional-order and integer-order gradient descent methods using multiple metrics: accuracy, generalization gap, RMSE, and PBIAS. The results show that the conjugate gradient-based method yields more accurate predictions. For instance, on the Wine dataset, it achieved the following testing performance: accuracy = 0.8962, generalization gap = 0.0438, RMSE = 0.2880, and PBIAS = 1.6018. The proposed algorithm consistently demonstrates superior performance and reliability over both comparative Methods. By leveraging the efficiency of conjugate gradient optimization, our approach effectively mitigates common limitations of traditional gradient descent, such as slow convergence and a tendency to become trapped in local minima. Its consistent performance across diverse datasets confirms the algorithm's robustness and its potential for application in complex real-world systems.

This paper presents Adaptive Learning Rate Polak-Ribiere-Polyak Conjugate Gradient Neuro-Fuzzy (ALR-PCGNF), an advanced hybrid optimization algorithm specifically designed for training first-order Takagi-Sugeno fuzzy neural networks. The methodology incorporates a Barzilai-Borwein adaptive learning rate mechanism within a Polak-Ribiere-Polyak conjugate gradient optimization framework. This integrated approach enables automated and dynamic step-size determination throughout the optimization process, effectively addressing the limitations of conventional gradient-based methods, including slow convergence rates and hypersensitivity to learning rate configurations. Comprehensive experimental evaluation conducted on multiple University of California, Irvine (UCI) classification benchmarks demonstrates that the proposed ALR-PCGNF algorithm achieves statistically superior generalization performance and accelerated convergence characteristics compared to state-of-the-art optimization techniques, including Fractional-Order Gradient Descent Method (FOGDM) and Integer-Order Gradient Descent Method (IOGDM).

The higher-order Takagi-Sugeno-Kang (TSK) model, renowned for its interpretability, adaptability, robustness, and ease of training, has been extensively utilized in fuzzy inference and modeling. However, there has been a noticeable scarcity of studies exploring its counterparts in the complex-valued domain, particularly employing fully complex-valued mechanisms. Therefore, this paper introduced an adaptive fully complex-valued fuzzy inference system (AFCFIS). Leveraging Wirtinger calculus, the paper found partial derivatives and updated the network weights according to the gradient descent method, which was easily solved due to the fully complex-valued learning mechanism. Furthermore, the paper provided convergence results of the proposed algorithm under mild conditions. Finally, numerical simulations verified the convergence of AFCFIS, and demonstrated its good performance in both real and complex domain tasks, as well as both regression and classification tasks.

As the simplest structure of interval neural networks (INNs), the single-layer interval perceptron (SIP) has the advantages of uncomplicated structure and fast computation, making it well-suited for handling various uncertain data. While L_0 regularization yields the sparsest solution among all L_n regularization methods, optimizing L_0 regularization poses a challenge as it is an NP-hard problem. Therefore, L_0 regularization is approximated using smoothing functions. The incorporation of smoothing Group L_0 regularization retains the sparse solution characteristics of L_0 regularization and effectively resolves its NP-hard problem. Building upon the aforementioned content, a modified learning algorithm based on smoothing Group L_0 regularization for interval perceptron with interval weights (MIPSG L_0 ) is proposed, where the interval perceptron take real numbers as inputs, weights and outputs are represented as intervals. The radius of each interval weight is expressed through a quadratic term rather than an absolute value function, ensuring a positive radius and preventing oscillations phenomenon. The monotonicity, the strong and weak convergence of the proposed algorithm is rigorously demonstrated under moderate assumptions. Moreover, experimental results on one-class approximation and one-class classification simulations reveal that the proposed algorithm exhibits superior performance in terms of training and testing mean squared error (MSE), pruning weights and accuracy.

Ridge Polynomial neural network have been widely acknowledged for strong nonlinear mapping capability. Nevertheless, conventional training based on integer-order gradient methods often suffers from low efficiency and limited precision, which can undermine model performance. Therefore, this paper presents a novel approach for training Ridge Polynomial neural network using fractional-order gradient descent based on the Caputo fractional-order derivative. The method introduces fractional-order gradient to optimization process, enhancing model’s performance and generalization ability. Numerical simulations validate the proposed method, showing improved performance in comparison with traditional integer-order gradient descent, particularly in terms of accuracy and generalization. This work demonstrates the potential of fractional-order optimization method for Ridge Polynomial neural network training, offering significant improvements over classical method.

Fully Homomorphic Encryption (FHE) facilitates computations on encrypted data without requiring access to the decryption key, offering substantial privacy benefits for deploying neural network applications in sensitive sectors such as healthcare and finance. Nonetheless, programming these applications within the FHE framework is complex and demands extensive cryptographic expertise to guarantee correctness, performance, and security. In this paper, we present ANT-ACE, a production-quality, open-source FHE compiler designed to automate neural network inference on encrypted data. ANT-ACE accepts ONNX models and generates C/C++ programs, leveraging its custom open-source FHE library. We explore the design challenges encountered in the development of ANT-ACE, which is engineered to support a variety of input formats and architectures across diverse FHE schemes through a novel Intermediate Representation (IR) that facilitates multiple levels of abstraction. Comprising 44,000 lines of C/C++ code, ANT-ACE efficiently translates ONNX models into C/C++ programs for encrypted inference on CPUs, specifically utilizing the RNS-CKKS scheme. Preliminary evaluations on a single CPU indicate that ANT-ACE achieves significant speed enhancements in ResNet models, surpassing expert manual implementations and fulfilling our design goals.

As neural network architectures grow deeper, the choice and refinement of optimization algorithms play a pivotal role in enhancing training efficiency and improving generalization performance. Traditional gradient descent methods often suffer from that the convergence is slow and it is easily trapped in local minima, particularly when dealing with high dimensional and complex data. In this study, we propose a gradient descent algorithm based on Caputo fractional-order derivatives to optimize product unit neural networks, and give a comparison of the optimization paths of Caputo fractional order and traditional gradient descent. By addressing the limitations of conventional integer order optimization methods in capturing long term dependencies and non-local features, the introduction of Caputo fractional order derivatives leverages the exponential feature combination properties of product-units, leading to more stable convergence and greater robustness to noise in function approximation tasks. The classification results, computational complexity, and training time of gradient descent and Caputo fractional-order gradient descent are compared on five benchmark datasets, and the experiments show that Caputo fractional order gradient descent improves accuracy at the cost of increased complexity.

To optimize the siting of public charging stations for electric vehicles, a method based on the k-means clustering algorithm (K-Means) and randomly occurring distributedly delayed particle swarm optimization (RODDPSO) algorithm was proposed. By integrating K-Means with a variation randomly occurring distributedly delayed particle swarm optimization (VRODDPSO) algorithm, the approach could determine the optimal locations for charging stations according to the charging demands of electric vehicles. Firstly, the RODDPSO algorithm was enhanced by incorporating an adaptive variation strategy. Then, the VRODDPSO algorithm was used to optimize the positions of the K-Means clustering centers. After clustering, the center points of each region were considered as the optimal locations for charging stations. Compared with using the K-Means algorithm alone for three iterations, the improved clustering model effectively addressed the issue of the K-Means algorithm potentially falling into local minima due to inappropriate initial cluster centers, which could result in suboptimal clustering. Finally, in an empirical study on the optimization of public charging station locations in Nanjing, a new evaluation method was proposed. This method assessed the siting quality of charging stations based on their comprehensive utilization rates. The analysis confirms that the integrated K-Means and VRODDPSO algorithm effectively optimizes the positions of the clustered centers, i.e., the locations of charging piles and stations.

In this study, interval-valued data is processed using an interval neural network (INN) based on a fractional- order extreme learning machine (FOELM). The traditional gradient-based algorithms of INN learn slowly. Extreme learning machine (ELM) achieves optimal weights for the output layer in a single calculation, which is faster than iterative training algorithms. However, due to the nonlinear constraints of INN, ELM cannot compute the output layer weights through the Moore-Penrose generalized inverse. Therefore, FOELM is proposed to train the INN in this paper, which ensures faster learning and significant performance. Specifically, in a single hidden layer INN, the hidden layer weights are randomly generated and fixed, and the output layer weights are trained by the fractional-order gradient descent method (FGDM). The experimental results indicate that FOELM exhibits faster convergence and superior classification performance compared to alternative algorithms.

In the understanding of important edges in complex networks, the edges with larger degree are naturally considered more important, and they will cause greater destructiveness when attacked. However, through simulation analysis, we conclude that this understanding needs to be based on certain preconditions. In this article, the robustness of BA scale-free network and WS small-world network is studied based on the edge-removal attack strategy considering the edge load and cascading failure. Specific attack methods include: High Load Edge-removal Attacks (HLEA) and Low Load Edge-removal Attacks (LLEA). The simulation results show that the importance of edges is closely related to the load parameter {\delta}. When 0<{\delta}<1, attacking the edge with smaller degree will cause greater cascading failure. For this condition, the edge with smaller degree is more important. When {\delta}=1, cascading failure is basically independent of the edge degree. When {\delta}>1, attacking the edge with larger degree will cause greater cascading failure. Therefore, the edge with larger degree is more important for this condition.

This article presents a real-time localization method for Unmanned Aerial Vehicles (UAVs) based on continuous image processing. The proposed method employs the Scale Invariant Feature Transform (SIFT) algorithm to identify key points in multi-scale space and generate descriptor vectors to match identical objects across multiple images. These corresponding points in the image provide pixel positions, which can be combined with transformation equations, allow for the calculation of the UAV's actual ground position. Additionally, the physical coordinates of matching points in the image can be obtained, corresponding to the UAV's physical coordinates. The method achieves real-time positioning and tracking during UAV flight, with experimental results demonstrating that within an acceptable error range, the UAV coordinates calculated using the proposed algorithm are consistent with the actual coordinates. The maximum error along the x-axis, y-axis, and z-axis is 4.501 cm, with the horizontal error exhibiting high stationarity and the vertical error having a low average value of 0.041 cm. The real-time positioning algorithm presented in this article possesses characteristics such as simplicity, ease of implementation, and low error, making it suitable for UAVs with limited computational processing power.

Smoothing interval neural networks (SINNs) are widely recognized for their effectiveness in handling uncertain data across various domains. However, training SINNs using the integer-order gradient method usually leads to lower computational accuracy and instability during parameter updating, which adversely affects the convergence and accuracy of the model. Fractional-order derivatives, in contrast, offer superior non-local memory properties and broader data fitting capabilities, providing more accurate descriptions of complex system dynamics. Therefore, this paper introduces a fractional-order gradient descent (FGD) method that utilizes Caputo fractional-order derivatives to train SINNs. Theoretical validation of the FGD method is achieved by rigorously proving the monotonicity of the error function and the strong convergence theorem of the algorithm. Additionally, various simulations are conducted to assess the enhanced algorithm’s performance on different approximation functions and classification datasets. The experimental results demonstrate that the FGD method significantly accelerates the convergence rate, enhances generalization ability, and improves the overall performance of SINNs.

Aiming at the problem of controlling the excess ratio of oxygen in the air supply subsystem of proton exchange membrane fuel cells.Firstly,a control-oriented fourth-order nonlinear dynamic model of the proton exchange membrane fuel cells system is established,and a fitting curve equation between the stack load current and the optimal oxygen excess ratio is constructed.Subsequently,a sliding mode controller using a new compound reaching law is designed,and the parameters in the sliding mode control are optimized and tuned using the sand cat swarm optimization algorithm.Finally,the improved sliding mode controller is simulated and verified,and compared with PID and other three sliding mode controllers.The simulation results show that when the load current of the stack changes,the improved sliding mode controller can adjust the driving voltage of the air compressor according to the cathode flow deviation parameter.At this time,the real-time oxygen excess ratio of the system will quickly approach the optimal oxygen excess ratio.The deviation between the two can be controlled within 0.1%,and the required average adjustment time and error performance indicators are better than those of the comparison group.

As an essential tool for processing fuzzy or chaotic information, the main feature of the first-order Takagi–Sugeno (T–S) neuro-fuzzy model is utilizing a set of IF-THEN fuzzy rules to represent non-linear systems, showcasing commendable non-linear approximation ability and significant interpretability. However, the coexistence of linear rules and the affiliation function of fuzzy sets makes the integer-order gradient descent method (IOGDM), commonly used in training the first-order T–S neuro-fuzzy model, fail to accurately capture the intricate relationships among weights, resulting in the error function struggling to converge rapidly to low values. To enhance the convergence speed and training accuracy of the first-order T–S neuro-fuzzy model during the training process, a fractional-order gradient descent method (FOGDM) is proposed to update the fuzzy rule parameters and neural network weights of the model in this paper. By subdividing the gradient into fractional orders, FOGDM exhibits heightened flexibility in gradient adjustments, thus better capturing the complex non-linear relationships among parameters during the optimization process. The weak and strong convergence of the proposed approach is meticulously demonstrated in this paper, ensuring that the weight of error functions converges to a constant value and that the gradient of the error functions tends toward zero, respectively. Simulation results analysis indicates that, compared to IOGDM, FOGDM exhibits faster convergence speed and more significant generalization capabilities.

As a universal approximator, the first order Takagi-Sugeno fuzzy system possesses the capability to approximate widespread nonlinear systems through a group of IF THEN fuzzy rules. Although group lasso regularization has the advantage of inducing group sparsity and handling variable selection issues, it can lead to numerical oscillations and theoretical challenges in calculating the gradient at the origin when employed directly during training. The paper addresses the aforementioned obstacle by invoking a smoothing function to approximate group lasso regularization. On this basis, a gradient-based neuro fuzzy learning algorithm with smoothing group lasso regularization for the first order Takagi-Sugeno fuzzy system is proposed. The convergence of the proposed algorithm is rigorously proved under gentle conditions. In addition, experimental outcomes acquired on two approximations and two classification simulations demonstrate that the proposed algorithm outperforms the algorithm with original group lasso regularization and L2 regularization in terms of error, pruned neurons, and accuracy. This is particularly evident in significant advancements in pruned neurons due to group sparsity. In comparison to the algorithm with L2 regularization, the proposed algorithm exhibits improvements of 6.3, 5.3, and 142.6 in pruned neurons during sin(pi x)$(\pi x)$ function, Gabor function, and Sonar benchmark dataset simulations, respectively. A gradient-based neuro-fuzzy learning algorithm with smooth group lasso regularization for first-order Takagi-Sugeno fuzzy system is proposed, in which smooth group lasso regularization can optimize the network structure by inducing group sparsity. The convergence of the proposed algorithm is rigorously proved. Two approximation and two classification simulations illustrate that the proposed algorithm exhibits better sparsity, convergence, and classification ability.image

The first-order Takagi-Sugeno (TS1) fuzzy system harbors an assemblage of IF-THEN fuzzy rules and is capable of approximating nonlinear systems of arbitrary complexity via discerning suitable affiliation functions and rules. As a natural extension of the lasso, group lasso adeptly trims superfluous connections among neurons and disposes the variable selection issue at the group stratum. Nonetheless, a fundamental shortcomings arises when the group lasso regularizer is directly applied during network training, particularly evident as weights converge towards zero, that is numerical oscillations. In addressing this concern, a smoothing function is introduced to approximate the original group lasso regularization (OGLR). Building upon the aforementioned content, a novel gradient-based neuro-fuzzy learning algorithm with smoothing group lasso regularization (SGLR) for the TS1 fuzzy system is proposed. Ultimately, the efficacy of the proposed algorithm is substantiated through three simulations that validate its sparse performance, approximation capability, and generalization potential.

In this paper, we propose a novel variant of the algorithm to improve the generalization performance for Elman neural networks (ENN). Here, the weight decay term, also called L2 regularization, which can effectively control the value of weights excessive growth, also over-fitting phenomenon can be effectively prevented. The main contribution of this work lies in that we have conducted a rigorous theoretical analysis of the proposed approach, i.e. the weak and strong convergence results are obtained. The comparison experiments to the problems of function approximation and classification on the real-world data have been performed to verify the theoretical results.

In harsh environments, wireless sensor networks need to be deployed randomly and unattended. Based on this premise, the network structure is analyzed, and the complex wireless sensor network under random scattering is divided into a giant connected component, a small number of isolated nodes and a small number of isolated areas. The complex network characteristics of the giant connected components are studied, and a connectivity analysis method of wireless sensor networks based on degree, average degree and node density is proposed. After simulation and analysis, the conclusions are as follows: firstly, there is a critical region in the node communication radius. In this region, with the increase of the communication radius, the network connectivity rate will rise rapidly until it is basically fully connected. Secondly, under a certain node density, the critical region can be determined by the communication radius, and then the coverage can be determined. Finally, after predetermined network connectivity and coverage, the more appropriate communication radius and average degree can be estimated.