Predictive compilation and intelligent source code analysis tasks necessitate the development of practical source code embeddings that preserve the contextual and semantic information of the code, allowing to achieve high quality of predictions made by machine learning algorithms. In this research we propose a novel source code embedding based on a Markov chain model which is constructed for a control flow graph (CFG). The code embedding is assessed on 2 classification problems: a problem of heterogeneous mapping of OpenCL programs to either CPU or GPU; and a problem of determining the type of the task implemented in by a given Python program solving unique programming exercise in a massive open online course. The results show that CFG-based Markov chains can improve the quality of classification results when compared to abstract syntax tree (AST)-based Markov chains, histogram-based embeddings and embeddings that are based on complex neural networks. In addition., we show that the use of term frequency-inverse document frequency (TF-IDF) model does not improve the quality of classification results when compared to simple histograms of AST node types or opcodes.

The widespread use of heterogeneous computing platforms, as well as the incorporation of computationally expensive implementations of intelligent data analysis algorithms into modern software systems leads to the demand in moving software fragments to most suitable hardware accelerators that are available on a heterogeneous computing platform. In this research, we propose an approach to the generation of recommendations for improving the performance of software systems by finding candidate algorithm implementations for hardware acceleration, and by suggesting the most suitable hardware accelerator among the specialized processors that are available on a given heterogeneous computing platform. The proposed approach is based on a code-to-code search technique, which extracts code fragments from an abstract syntax tree (AST), converts them into vectors containing program features, and compares the vectors with the query program vector. The obtained results confirm that the use of automatically recommended hardware accelerators for the code fragments identified using the proposed approach indeed allows to increase the performance of software systems solving machine learning tasks.

Modern software systems consist of many software components; the source code of modern software systems is hard to understand and maintain for new developers. Aiming to simplify the readability and understandability of source code, companies that specialize in software development adopt programming standards, software design patterns, and static analyzers with the aim of decreasing the complexity of software. Recent research introduced a number of code metrics allowing the numerical characterization of the maintainability of code snippets. Cyclomatic Complexity (CycC) is one widely used metric for measuring the complexity of software. The value of CycC is equal to the number of decision points in a program plus one. However, CycC does not take into account the nesting levels of the syntactic structures that break the linear control flow in a program. Aiming to resolve this, the Cognitive Complexity (CogC) metric was proposed as a successor to CycC. In this paper, we describe a rule-based algorithm and its specializations for measuring the complexity of programs. We express the CycC and CogC metrics by means of the described algorithm and propose a new complexity metric named Educational Complexity (EduC) for use in educational digital environments. EduC is at least as strict as CycC and CogC are and includes additional checks that are based on definition-use graph analysis of a program. We evaluate the CycC, CogC, and EduC metrics using the source code of programs submitted to a Digital Teaching Assistant (DTA) system that automates a university programming course. The obtained results confirm that EduC rejects more overcomplicated and difficult-to-understand programs in solving unique programming exercises generated by the DTA system when compared to CycC and CogC.

In this paper we consider the research and development of classifiers that are trained to predict the task solved by source code. Possible applications of such task detection algorithms include method name prediction, hardware–software partitioning, programming standard violation detection, and semantic code duplication search. We provide the comparative analysis of modern approaches to source code transformation into vector-based representations that extend the variety of classification and clustering algorithms that can be used for intelligent source code analysis. These approaches include word2vec, code2vec, first-order and second-order Markov chains constructed from abstract syntax trees (AST), histograms of assembly language instruction opcodes, and histograms of AST node types. The vectors obtained with the forementioned approaches are then used to train such classification algorithms as k-nearest neighbor (KNN), support vector machine (SVM), random forest (RF), and multilayer perceptron (MLP). The obtained results show that the use of program vectors based on first-order AST-based Markov chains with an RF-based classifier leads to the highest accuracy, precision, recall, and F1 score. Increasing the order of Markov chains considerably increases the dimensionality of a vector, without any improvements in classifier quality, so we assume that first-order Markov chains are best suitable for real world applications. Additionally, the experimental study shows that first-order AST-based Markov chains are least sensitive to the used classification algorithm.

Modern artificial neural network architectures and training algorithms are able to achieve high accuracy in a wide range of problems. However, training multilayer neural networks using backpropagation or evolutionary algorithms might take a large amount of time. Extreme learning machines (ELMs) are aimed to resolve this problem by excluding training from the neural network model creation process by randomly initializing weights between input and hidden layers and computing weights between hidden and output layers. However, random weights initialization might lead to suboptimal results produced by the network. In this paper, we apply biology-inspired algorithms, including genetic algorithm with tournament selection, particle swarm optimization, and chaotic fish school search with exponential step decay, to the selection of input weights in ELM, the obtained ELM configurations are applied to solve regression problems. The results of the study show, that population-based algorithms can improve ELM accuracy.

This paper presents a dataset containing automatically collected source codes solving unique programming exercises of different types. The programming exercises were automatically generated by the Digital Teaching Assistant (DTA) system that automates a massive Python programming course at MIREA—Russian Technological University (RTU MIREA). Source codes of the small programs grouped by the type of the solved task can be used for benchmarking source code classification and clustering algorithms. Moreover, the data can be used for training intelligent program synthesizers or benchmarking mutation testing frameworks, and more applications are yet to be discovered. We describe the architecture of the DTA system, aiming to provide detailed insight regarding how and why the dataset was collected. In addition, we describe the algorithms responsible for source code analysis in the DTA system. These algorithms use vector representations of programs based on Markov chains, compute pairwise Jensen–Shannon divergences of programs, and apply hierarchical clustering algorithms in order to automatically discover high-level concepts used by students while solving unique tasks. The proposed approach can be incorporated into massive programming courses when there is a need to identify approaches implemented by students.

Dimensionality reduction techniques are often used by researchers in order to make high dimensional data easier to interpret visually, as data visualization is only possible in low dimensional spaces. Recent research in nonlinear dimensionality reduction introduced many effective algorithms, including t-distributed stochastic neighbor embedding (t-SNE), uniform manifold approximation and projection (UMAP), dimensionality reduction technique based on triplet constraints (TriMAP), and pairwise controlled manifold approximation (PaCMAP), aimed to preserve both the local and global structure of high dimensional data while reducing the dimensionality. The UMAP algorithm has found its application in bioinformatics, genetics, genomics, and has been widely used to improve the accuracy of other machine learning algorithms. In this research, we compare the performance of different fuzzy information discrimination measures used as loss functions in the UMAP algorithm while constructing low dimensional embeddings. In order to achieve this, we derive the gradients of the considered losses analytically and employ the Adam algorithm during the loss function optimization process. From the conducted experimental studies we conclude that the use of either the logarithmic fuzzy cross entropy loss without reduced repulsion or the symmetric logarithmic fuzzy cross entropy loss with sufficiently large neighbor count leads to better global structure preservation of the original multidimensional data when compared to the loss function used in the original UMAP algorithm implementation.

Objectives. Recent research in machine learning and artificial intelligence aimed at improving prediction accuracy and reducing computational complexity resulted in a novel neural network architecture referred to as an extreme learning machine (ELM). An ELM comprises a single-hidden-layer feedforward neural network in which the weights of connections among input-layer neurons and hidden-layer neurons are initialized randomly, while the weights of connections among hidden-layer neurons and output-layer neurons are computed using a generalized Moore– Penrose pseudoinverse operation. The replacement of the iterative learning process currently used in many neural network architectures with the random initialization of input weights and the explicit computation of output weights significantly increases the performance of this novel machine learning algorithm while preserving good generalization performance. However, since the random initialization of input weights does not necessarily guarantee optimal prediction accuracy, the purpose of the present work was to develop and study approaches to intelligent adjustment of input weights in ELMs using bioinspired algorithms in order to improve the prediction accuracy of this data analysis tool in regression problems.Methods. Methods of optimization theory, theory of evolutionary computation and swarm intelligence, probability theory, mathematical statistics and systems analysis were used.Results. Approaches to the intelligent adjustment of input weights in ELMs were developed and studied. These approaches are based on the genetic algorithm, the particle swarm algorithm, the fish school search algorithm, as well as the chaotic fish school search algorithm with exponential step decay proposed by the authors. By adjusting input weights with bioinspired optimization algorithms, it was shown that the prediction accuracy of ELMs in regression problems can be improved to reduce the number of hidden-layer neurons to reach a high prediction accuracy on learning and test datasets. In the considered problems, the best ELM configurations can be obtained using the chaotic fish school search algorithm with exponential step decay.Conclusions. The obtained results showed that the prediction accuracy of ELMs can be improved by using bioinspired algorithms for the intelligent adjustment of input weights. Additional calculations are required to adjust the weights; therefore, the use of ELMs in combination with bioinspired algorithms may be advisable where it is necessary to obtain the most accurate and most compact ELM configuration.

The digitalization of modern economic sectors leads to the lack of software developers in many industries. Hence, there is the increasing demand for mass training of IT specialists in colleges and universities. With the increasing students count, the workload for teachers of disciplines connected with IT tremendously increases. The automation of software developers training allows to reduce the burden on teachers by saving them from routine work. In this paper, we provide the details regarding the development of the Digital Teaching Assistant (DTA), the information system used in Federal State Budget Educational Institution of Higher Education “MIREA – Russian Technological University”, integrated into the Python programming course. The considered system consists of two components, namely, the DTA core, and the DTA web application, both components are implemented using the Python programming language. The DTA core generates tasks and tests for students, and the DTA web application allows students to interact with the core. We describe the architecture of DTA and visualize the statistics collected during system operation.

The massive nature of modern university programming courses increases the burden on academic workers. The Digital Teaching Assistant (DTA) system addresses this issue by automating unique programming exercise generation and checking, and provides means for analyzing programs received from students by the end of semester. In this paper, we propose a machine learning-based approach to the classification of student programs represented as Markov chains. The proposed approach enables real-time student submissions analysis in the DTA system. We compare the performance of different multi-class classification algorithms, such as support vector machine (SVM), the k nearest neighbors (KNN) algorithm, random forest (RF), and extreme learning machine (ELM). ELM is a single-hidden layer feedforward network (SLFN) learning scheme that drastically speeds up the SLFN training process. This is achieved by randomly initializing weights of connections among input and hidden neurons, and explicitly computing weights of connections among hidden and output neurons. The experimental results show that ELM is the most computationally efficient algorithm among the considered ones. In addition, we apply biology-inspired algorithms to ELM input weights fine-tuning in order to further improve the generalization capabilities of this algorithm. The obtained results show that ELMs fine-tuned with biology-inspired algorithms achieve the best accuracy on test data in most of the considered problems.

Recent research introduced plenty of approaches to time series forecasting. Long short-term memory (LSTM) is a widely studied and effective recurrent artificial neural network architecture commonly used in time series prediction. LSTM networks are often trained using gradient-based methods. Such methods might be prone to premature convergence, and this affects prediction performance. In this paper, we consider a biology-inspired approach to LSTM loss function optimization. We compare the performance of different LSTM networks trained with backpropagation and using biology-inspired algorithms, including the Genetic Algorithm, Particle Swarm optimization, and Fish School Search. According to the obtained results, the LSTM network trained with the chaotic Fish School Search algorithm with exponential step decay produces the most accurate predictions in the considered time series forecasting problems.

The Fish School Search (FSS) algorithm is a heuristic technique for finding globally optimal solutions. This algorithm is characterized by its simplicity in implementation, and high performance. Since the first mention of FSS, this effective optimization algorithm has been of a great interest among researches and practitioners around the globe. Modifications of FSS exist, applied to solve practical problems, including image reconstruction in electrical impedance tomography, finding optimal solutions in assembly line balancing problems, neural network structure optimization. In this paper, we consider a modification of the FSS algorithm, which uses chaos theory to generate uniformly distributed pseudorandom numbers, and incorporates exponential step decay. The described modified optimization algorithm is known as ETFSS, and is characterized by faster convergence speed and better performance. In order to further investigate the performance of the novel optimization algorithm, we apply ETFSS to neural network loss function optimization. In addition, we compare the described approach with other machine learning techniques, such as the support vector machine (SVM) algorithm, k-nearest neighbors (KNN) algorithm and back propagation-based neural network, trained using the adaptive moment estimation (Adam) optimizer. We visualize classification results using T-distributed stochastic neighbor embedding (TSNE) method, and uniform manifold approximation and projection (UMAP) method, in order to provide more details considering classification performance and dataset shape. The obtained results confirm, that ETFSS can produce slightly more accurate classifications when compared to backpropagation.

Inspired by the collective behavior of fish schools, the fish school search (FSS) algorithm is a technique for finding globally optimal solutions. The algorithm is characterized by its simplicity and high performance; FSS is computationally inexpensive, compared to other evolution-inspired algorithms. However, the premature convergence problem is inherent to FSS, especially in the optimization of functions that are in very-high-dimensional spaces and have plenty of local minima or maxima. The accuracy of the obtained solution highly depends on the initial distribution of agents in the search space and on the predefined initial individual and collective-volitive movement step sizes. In this paper, we provide a study of different chaotic maps with symmetric distributions, used as pseudorandom number generators (PRNGs) in FSS. In addition, we incorporate exponential step decay in order to improve the accuracy of the solutions produced by the algorithm. The obtained results of the conducted numerical experiments show that the use of chaotic maps instead of other commonly used high-quality PRNGs can speed up the algorithm, and the incorporated exponential step decay can improve the accuracy of the obtained solution. Different pseudorandom number distributions produced by the considered chaotic maps can positively affect the accuracy of the algorithm in different optimization problems. Overall, the use of the uniform pseudorandom number distribution generated by the tent map produced the most accurate results. Moreover, the tent-map-based PRNG achieved the best performance when compared to other chaotic maps and nonchaotic PRNGs. To demonstrate the effectiveness of the proposed optimization technique, we provide a comparison of the tent-map-based FSS algorithm with exponential step decay (ETFSS) with particle swarm optimization (PSO) and with the genetic algorithm with tournament selection (GA) on test functions for optimization.

Inspired by biological systems, swarm intelligence algorithms are widely used to solve multimodal optimization problems. In this study, we consider the hybridization problem of an algorithm based on the collective behavior of fish schools. The algorithm is computationally inexpensive compared to other population-based algorithms. Accuracy of fish school search increases with the increase of predefined iteration count, but this also affects computation time required to find a suboptimal solution. We propose two hybrid approaches, intending to improve the evolutionary-inspired algorithm accuracy by using classical optimization methods, such as gradient descent and Newton's optimization method. The study shows the effectiveness of the proposed hybrid algorithms, and the strong advantage of the hybrid algorithm based on fish school search and gradient descent. We provide a solution for the linearly inseparable exclusive disjunction problem using the developed algorithm and a perceptron with one hidden layer. To demonstrate the effectiveness of the algorithms, we visualize high dimensional loss surfaces near global extreme points. In addition, we apply the distributed version of the most effective hybrid algorithm to the hyperparameter optimization problem of a neural network.