Spiking Neural Networks (SNNs) are known as a branch of neuromorphic computing and are currently used in neuroscience applications to understand and model the biological brain.SNNs could also potentially be used in many other application domains such as classification, pattern recognition, and autonomous control.This work presents a highly-scalable hardware platform called POETS, and uses it to implement SNN on a very large number of parallel and reconfigurable FPGA-based processors.The current system consists of 48 FPGAs, providing 3072 processing cores and 49152 threads.We use this hardware to implement up to four million neurons with one thousand synapses.Comparison to other similar platforms shows that the current POETS system is twenty times faster than the Brian simulator, and at least two times faster than SpiNNaker.
—In recent years, the rapid development of internet technology brings many severe network security problems linked to malicious intrusions. Intrusion Detection System is considered to be one of the significant techniques to safeguard the network from both external and internal attacks. However, with the fast expansion of the IoT network, cyberattacks are also changing quickly, and many unknown types are showing up in the contemporary network environment. Consequently, the efficiency of traditional signature-based and anomaly-based Intrusion Detection System is insufficient. We propose a novel Intrusion Detection System, which uses an evolutionary technique based feature selection approach and a Random Forest-based classifier. The evolution-based feature selector uses an innovative Fitness Function to select the important features and reduces dimensions of the data, which raise the Ture Positive Rate and reduce the False Positive Rate at the same time. With exceptional high accuracy in multi-classification tasks and outstanding capabilities of handling noise in massive data scenarios, the Random Forest technique is widely used in anomaly detection. This research proposes a framework that can select more steady features and improve the classification results as compared with other technologies. The proposed framework is tested and experimented on UNSW-NB15 datasets and NSL-KDD datasets. Various statistical results and detailed comparison to other methods are presented within this article.
Abstract —An intelligent machine and manufacturing system has a significant role in the near future, especially when the circumstance of manufacturing industries are seriously competitive. New technologies are continuously being developed to serve future manufacturing. CNC turning machine is widely utilized in various advanced manufacturing industries. Straightness is a critical parameter in CNC turning process, which affects the workpiece assembly directly. However, control of straightness of the workpieces during in-process turning is difficult to be measured. Moreover, CNC turning machine cannot be adjusted real-time without stopping the operation. Hence, the aim of this research is to develop the straightness prediction model in the CNC turning process under various cutting conditions for carbon steel and aluminum workpieces in order to improve in-process monitoring and control of straightness. The cutting forces ratio has been adopted to estimate straightness. The Daubechies wavelet transform is utilized to decompose the dynamic cutting forces to remove the noise signals for better prediction. The straightness is calculated by employing the two-layer feed forward neural network, which is trained with the Levenberg-Marquardt back-propagation algorithm. As a result, the in-process straightness could be predicted well with greater accuracy and reliability using the proposed straightness
—Generating network traffic flows remains a critical aspect of developing cyber and network security systems. In this survey, we first consider the history of network traffic generation methods and identify the weaknesses of these. We then proceed to introduce more recent approaches based on machine learning (ML) models. In particular, we focus on Generative Adversarial Network (GAN) models, which have developed from their initial form to encompass many variants in today’s ML landscape. The use of GANs for generating traffic flows that have appeared in the literature are then presented. For each instance, we present the architecture, training methods, generated results, identified limitations and prospects for further research. We thus demonstrate that GANs are key to future developments in network traffic generation and secure cyber and network systems.
Abstract —This paper presents a novel framework for image classification which comprises a convolutional neural network (CNN) feature map extractor combined with a Gaussian process (GP) classifier. Learning within the CNN-GP involves forward propagating the predicted class labels, then followed by backpropagation of the maximum likelihood function of the GP with a regularization term added. The regularization term takes the form of one of the three loss functions: the Kullback-Leibler divergence, Wasserstein distance, and maximum correntropy. The training and testing are performed in mini batches of images. The forward step (before the regularization) involves replacing the original images in the mini batch with their close neighboring images and then providing these to the CNN-GP to get the new predictive labels. The network performance is evaluated on MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 datasets. Precision-recall and receiver operating characteristics curves are used to evaluate the performance of the GP classifier. The proposed CNN-GP performance is validated with different levels of noise, motion blur, and adversarial attacks. Results are explained using uncertainty analysis and further tests on quantifying the impact on uncertainty with attack strength are carried out. The results show that the testing accuracy improves for networks that backpropagate the maximum likelihood with regularized losses when compared with methods that do not. Moreover, a comparison with a state-of-art CNN Monte Carlo dropout method is presented. The outperformance of the CNN-GP framework with respect to reliability and computational efficiency is
Intrinsic motivation is one of the potential candidates to help improve performance of reinforcement learning algorithm in complex environments. The method enhances exploration capability without explicitly told by the creator. This is suitable for the case of multi-agent reinforcement learning where the environment complexity is beyond standard. In this paper, the Random Network Distillation method is applied to implement intrinsic motivation in the multi-agent environment. Two intrinsic motivation architectures are developed and compared with the benchmark in different scenarios. The experiments show an increase in performance of the very complex environments while little to no improvement over the non-complex ones. Although there exists some overhead which results in less sample efficiency, the centralized intrinsic motivation architecture shows a long-term on par or even better optimization performance as it could explore on more states. The performance of the centralized architecture shows a solid improvement in 2s3z environment and achieves almost 70%win rate over the benchmark of 43%.
—Translators are becoming more and more popular and achieving reliable results since deep learning was born. English-Vietnamese machines translation (MT) still have limitations due to Vietnamese contain words with many different meanings, thus resulting in the lower accuracy of automatic MT systems. Our study applied Named Entity Recognition (NER) tool for Vietnamese sentences to determine the category of words in the English-Vietnamese parallel corpus with over 900K sentence pairs. Then, we performed experiments to assess the effect of NER on English-Vietnamese MT systems. The results showed that NER had a positive effect on MT with averagely 1.24 Bi-Lingual Evaluation Understudy (BLEU) scores and averagely 1.8 Translation Error Rate (TER) scores increased comparing to data without using NER.
—As a fundamental transportation service, ride-hailing has greatly improved the city mobility efficiency and served millions of passengers in big metropolitan cities. However, due to the imbalance between the limited supply caused by the strict car-buying policy and the increasing travelling demand, ride-hailing services are far from satisfactory. A better prediction of travel demand is one possible solution of improving ride-hailing service efficiency and quality and the idle drivers can be scheduled to hotspots with more potential ride requests. In this paper, we explore the usage of deep learning technique, i.e., ConvLSTM networks, for ride-hailing service prediction. Experiment results on a real-world ride-hailing dataset provided by Didi Chuxing show the superiority of ConvLSTM over baseline methods including Multi-Layer Perceptron and two simple historical methods.
This paper presents an application of the hybrid Machine Learning (ML) techniques to real-time detection of unsafe personal safety equipment (e.g., helmet and safety vest) of construction workers on site, so that the unsafe behaviors can be corrected timely to reduce safety risks.Three different Convolutional Neural Network (CNN) based Deep Learning (DL) techniques were adopted for worker position locating, object classification, and subtle feature detection, including Faster R-CNN, YOLO and DenseNet.The lab testing showed high detectability with the Recall of 95% and the Precision of 90%.In in-situ implementation of a real-world construction site, a moderately acceptable detectability was achieved, with the Cleanness of 85% and Correctness of 80%.It is concluded that the proposed method quotes profound potentials to enhance the current safety management practice of construction site.
Female breast shape is significantly essential for female healthcare, bra design, etc.However, there is no authoritative standard for breast shape classification.In this paper, we analysis the female breast category by unsupervised clustering the horizontal female breast contours.Specifically, the Elliptic Fourier Descriptors (EFDs), extracted from breast contour, are employed as the contour features.Subsequently, we use PCA to reduce the feature into lower dimensions.Experiments demonstrate that the lower dimensions are enough to present the original features.Then, we employ two widely used clustering algorithms, K-Means++ and FCM, to cluster the female breast contours, and deeply analyze and compare the results of two clustering results in terms of effectiveness.Experimental results demonstrate that the K-Means++ is more suitable for female breast contour clustering, and the results are more reasonable than FCM.
The electricity demand has been steadily increasing throughout the years.A robust predictive model is required to prepare for future electricity consumption.This paper applied the ARIMA models to forecast electricity consumption in the Philippines.Dataset used was retrieved from the Philippine Institute for Development Studies website.It contains 48 data points, of which 43 were used in model building, and the remaining 5 data points were used in forecast evaluation.The order of the ARIMA (p,d,q) model was based on the ACF and PACF plots.The model with the most negative AIC value was chosen among the candidate models, and the best-fitting model was identified.Based on the analysis results, ARIMA (0,2,1) is the statistically appropriate model to forecast electricity consumption in the Philippines.It is predicted that by 2030, the Philippines will consume 163,639.9GWh of electricity.The R statistical software was used to do all of the calculations.
This paper explores the impact of reweighting the minority class of an imbalanced fraud dataset on the performance of an XGBoost binary classifier.Classifier performance is measured here in terms of true positive rate, false positive rate, precision, accuracy, AUC-ROC and AUC-PR.Our results suggest that reweighting the minority class has significant impact on these four key performance metrics when the classification threshold is held fixed and the model bias is not corrected.However, this impact becomes insignificant when (1) classification threshold is held fixed and the bias is corrected, or (2) when the target number of predicted positives is held fixed.Since fraud detection often prescribes a target number of cases for special treatment, these findings suggest that reweighting a dataset offers performance advantage only under very specific conditions for XGBoost-based classifiers.These conclusions can also generalize to problems where certain resampling techniques are used instead of reweighting since the two approaches tend to converge for sufficiently large datasets.
The rapid progress of convolutional neural networks (CNNs) in multiple applications of practical implementation is generally hindered by an upsurge in network size and computational complexity.Currently, engineers focus on reducing these problems through compressing the CNNs by pruning filters and their weights.In this paper, we present a fresh and easy-to-use pruning approach that reduces the model size by eliminating complete filters and filter weights based on the sparse group LASSO (Least Absolute Shrinkage and Selection Operator) method across the convolutional layers.More precisely, it regulates the sparsity at the feature level and the group level.During the process of pruning, the unnecessary filters with their weights eliminate directly without sacrificing accuracy in the test, resulting in much compact and slimmer architectures.We experimentally compute the effectiveness of our methodology with various state-of-art CNN models on various benchmark data sets.Mainly, CIFAR-10 data sets applied on VGG-16 model and reduce the parameters approx.96.1% and saved approx.83.55% float-point-operations (FLOPs) without sacrificing accuracy and have obtained development in state-of-art.
As a solution to the last mile problem in big metropolitan cities, free-floating bike-sharing service is becoming a new choice for short travels all over the world.Unlike the docked bikes which requires the users to borrow and return at fixed stations, free-floating bikes can be used everywhere.However, this feasibility also brings a higher management cost.The bikes should be scheduled from the regions with less demand to those with higher demand, based on a precise demand prediction.In this paper, we use deep learning techniques including Multi-Layer Perceptron and ConvLSTM networks for this task.We find that in the case of the insufficient training data, e.g., one-month data of Mobike, Multi-Layer Perceptron performs better than both ConvLSTM and two simple historical methods.
The 21st century has been witnessing a high growth in technology in every field including the medical sector.Dynamic systems have been designed and implied for better and accurate diagnosis of a large variety of ailments; but, the growing number of patients makes it difficult to provide proper medical attention in time.To overcome this difficulty, Intelligent Systems techniques can be employed in the medical sector and help us overcome the huge difference in the ratio of doctors versus patients; along with reducing the examination and waiting time for the patients.Among all the variety of ailments prevailing in today's world, "Lower Back Pain" has emerged as one of the most prevailing ailments which includes around 80% of the total population once in lifetime, making it to one of the prior concerns of medical sector.To act effectively onto it, many conventional methods have been used to diagnose lower back pain.This study aims to design a non-Conventional technique to classify Lower back pain either Normal or Abnormal using Machine Learning techniques such as Naï ve Bayes, Support Vector Machines, Decision Trees, Gradient Boosted Trees, Fast Large Margin, K Nearest Neighbor, Multilayer Perceptron, Random Forest, and Artificial Neural Networks.This research focuses upon the implementation of the above-mentioned techniques for the proper classification of Spine Dataset and for determining the best technique in terms of Accuracy, Precision, Sensitivity, Specificity, F-measure and Area under Curve.
In this paper, we solve the bi-objective scheduling problem on two dedicated processors with an evolutionary algorithm.The algorithm incorporates a look-ahead-based path-relinking as a learning strategy.The designed algorithm first determines a starting archive set by applying a knapsack procedure tailored for the scheduling.Second, an adaptation of the dominating local search, combined with exchange operators, is considered for generating a series of new non-dominated solutions that enrich the reference archive set.Third, a lookahead strategy-based path-relinking is added to the algorithm for iteratively highlighting the final Pareto front.A preliminary experimental part is given, where the performance of the method is evaluated on a set of benchmark instances extracted from the literature.Its results are compared to those achieved by the best methods of the literature.New results are obtained.
The operating modes in the smelting process of fused magnesium are cyclically shifted, resulting in severe fluctuations in electricity load.Accurate prediction of its operating mode shifting can optimize the power supply curve of an electric furnace, to improve electric energy efficiency and reduce electricity expenses.In this paper, we propose a prediction model of fused magnesium operating mode based on ADASYN-XGBoost.Four supervised machine learning algorithms including a eXtreme Gradient Boosting (XGB), ADASYN-LGB and ADASYN-RF, ADASYN-SVM were compared with the proposed ADASYN-XGB method.The results indicate that the ADASYN-XGB has the best prediction accuracy (92.5%), high average precision (>0.8), low hamming loss (0.03) and low ranking loss (0.075).Based on these results for classification performance and prediction accuracy, the ADASYN-XGB is a solid candidate for a correct classification of operating modes.These findings suggest that ADASYN-XGB systems trained with real data may serve as a new tool to assist in fused magnesium smelting process.
Looking for the spatial co-location that appears frequently in nearby space is widely used in many areas, including mobile phone services and traffic management.To achieve this goal, the SGCT algorithm improves other algorithms which use tables to discover candidate sets.It uses an undirected graph to mine candidates of the maximal co-location patterns first, then uses a condensed-tree structure to store instance cliques of candidates.However, as the amount of data grows, the SGCT algorithm may store large number of nodes in the process of generating the tree.In this paper, we propose a new strategy which will consider the number of instances of each event.We propose a Count-Ordered Instances-tree to record candidates of relation sets.From our experimental results, we show that our approach needs shorter time and costs less storage space than the SGCT algorithm.
Forecasting is the basis of planning, and the key to tourism supply chain.The international tourists visiting Taiwan from Mainland China, Japan, and South Korea is the major international tourist source markets for Taiwan.Recurrent neural network model is a fairly new and promising neural network technology.Genetic algorithms can be optimized the neural network structure.The ARIMAX model is recently utilized time series model for tourism demand forecasting.Accordingly, this work presents recurrent neural network model to forecast numbers of international tourists to Taiwan from Mainland China, Japan, and South Korea to help the Taiwanese tourism industry.This work also compares the forecast accuracy of the ARIMAX model with that of the ARIMAX model.Empirical results demonstrate that the recurrent neural network model with genetic algorithms is better than the ARIMAX model for Mainland China and South Korea, and worse than for Japan.The findings of this study can contribute to management and policy-decision issues related to the tourism industry for Taiwan in tourism supply chains.
In this paper, a new method is proposed to improve hand joint regression in 3D hand pose estimation.The existing methods regress all joints together given a depth map.This causes misallocations of some hand joints, misuse of hand depth information, and have difficulties in estimating 3D coordinates accurately.In this paper, joint regression is performed in stages, such that highly flexible joints e.g.fingertip joints are regressed first followed by less flexible joints to avoid getting some errors while estimating all joints together.In practice, fingertip joints constitute relatively higher estimation errors than all other joints.Thus, we perform fingertip joint localization (2D joint estimation) after obtaining rough pose estimates from the pose estimator to locate fingertip joint positions.We then use these 2D joint estimates to generate the depth coordinates of the pose estimator.To further ensure the accuracy of the absolute pose hypothesis, we integrate a robust implicit shape-based hand detector with the deep regression pose estimator into one pipeline through a shared convolutional layer.Finally, a shared convolutional layer converts the 2D joint location to 3D poses.Consequently, our system can accurately estimate hand pose based on the prior knowledge of a well detected human hand and the properly located joint positions.Experiments were carried out on three publicly available datasets, ICVL, NYU, and MSRA.The proposed hand pose estimation system attains an accuracy of 96.4% at the threshold level of 40mm on the ICVL dataset, 92% on MSRA, and 89% on the NYU dataset illustrating the effectiveness of the proposed system over many state-of-art approaches.
