This paper presents an evolutionary algorithm for optimal mapping of the back-propagation learning algorithm onto a parallel heterogeneous processor network. Training-set parallelism is used as the paradigm for parallelizing the backpropagation algorithm, and the processor network is a heterogeneous array of transputers connected in a pipelined ring topology. It is known from earlier studies that finding the optimal mapping (i.e. optimal allocation of training patterns among the processors to minimize the time for a training epoch) involved solving a linear Mixed Integer Programming (MIP) problem. Solving the MIP using the traditional Branch and Bound (B&B) method takes a large amount of computing time. Approaches based on evolutionary algorithms are then investigated as alternatives to the branch and bound method to solve the pattern allocation problem. It is found that a conventional genetic algorithms (GAs) search time taken. However when the crossover and mutation probabilities in the GA are varied over a wide range, the best solution is obtained by an evolutionary algorithm even though the studies were begun with a conventional GA. A new stopping criterion to detect convergence to stop the search is also incorporated in the final algorithm.

This paper analyzes the parallel mapping of the backpropagation learning algorithm, onto a heterogeneous multiprocessor ring architecture. Training set parallelism is used as the parallelizing paradigm. A mathematical model is developed in order to obtain an expression for the training time per epoch. This model is then used to find the optimal mapping that minimizes the training time per epoch. It is shown that the optimal mapping results in a mixed integer programming (MIP) problem which is NP-complete. To solve this problem, the genetic algorithmic approach is used and the optimal distribution of training patterns are obtained. A Monte Carlo study is then carried out which statistically verify the proximity of these optimal solutions to the global optimum, for the MIP problem

This paper analyzes parallel implementation of the backpropagation training algorithm on a heterogeneous transputer network (i.e., transputers of different speed and memory) connected in a pipelined ring topology. Training-set parallelism is employed as the parallelizing paradigm for the backpropagation algorithm. It is shown through analysis that finding the optimal allocation of the training patterns amongst the processors to minimize the time for a training epoch is a mixed integer programming problem. Using mixed integer programming optimal pattern allocations for heterogeneous processor networks having a mixture of T805-20 (20 MHz) and T805-25 (25 MHz) transputers are theoretically found for two benchmark problems. The time for an epoch corresponding to the optimal pattern allocations is then obtained experimentally for the benchmark problems from the T805-20, TS805-25 heterogeneous networks. A Monte Carlo simulation study is carried out to statistically verify the optimality of the epoch time obtained from the mixed integer programming based allocations. In this study pattern allocations are randomly generated and the corresponding time for an epoch is experimentally obtained from the heterogeneous network. The mean and standard deviation for the epoch times from the random allocations are then compared with the optimal epoch time. The results show the optimal epoch time to be always lower than the mean epoch times by more than three standard deviations (3sigma) for all the sample sizes used in the study thus giving validity to the theoretical analysis.

This article investigates optimal and suboptimal pattern allocation schemes when training-set parallelism is used as the paradigm to map a backpropagation neural network on a heterogeneous array of processors. In earlier work, it was shown that finding the optimal allocation of patterns to minimize the time for a training epoch for such a mapping leads to a mixed integer programming problem. Because the solution to the mixed integer programming requires prohibitively large computing time, several suboptimal allocation methods that are computationally less demanding are discussed and their performance compared with the optimal solution for the NETTALK benchmark problem. The processor network used for mapping the neural network is a heterogeneous array of transputers connected in a pipelined ring topology.

Proposes a path-based neural network algorithm called recurrent continuous backpropagation two (RCBP-2) for complex sequential processing using a gradient descent method. Under the path-based approach, the goal weights are a collection of weight states. Coupled with the underlying continuity of training exemplars and sequential nature of the system attributes, RCBP-2 can achieve arbitrarily close approximations of complex trajectories within a fixed and relatively small network topology. The performance of RCBP-2 is also monitored by training and subsequently testing on a 4-orbits problem. The results show that RCBP-2 results in a fast and efficient algorithm for complex sequential processing.

In this paper we investigate parallel implementation of feedforward neural networks on a transputer array when the networks are trained by the backpropagation algorithm. Two methods of transputer implementations are considered: namely, the processor farming method and the pipelining method. The performance of the parallel implementations are compared with a serial implementation on a 486 machine using the N-X-N encoder/decoder as the benchmark problem. Simulation results show that for small networks serial implementation out-performs the parallel implementations, but, as the network and training set size becomes large, parallel implementations produce shorter training times than the serial implementation. Among the two methods of transputer implementations, the pipelining method always produced shorter training times than the processor farming method.

Training set parallelization is an efficient method to optimize the training procedure performance of the backpropagation neural network algorithm. In training set parallelism, the training patterns are distributed `optimally' among a heterogeneous array of processors, optimality criterion obtain the minimum training time per epoch. Earlier studies on heterogeneous transputers connected in a pipeline-ring topology have indicated that the above optimization problem results in a mixed integer programming problem and results in large computation time to find the optimal pattern allocations. In this paper, a genetic algorithm is used as an optimization tool to find the optimal allocation of patterns. The approach is illustrated using two benchmark problems, the 256-8-256 Encoder and NETTALK problems. Results indicate that when `a priori' information is not used, the computation time needed by the genetic algorithm is comparable to that obtained by mixed integer programming. However, when `a priori' information is used, the genetic algorithm results in significant reduction in computation time for finding the optimal solution

Backpropagation algorithm is one of the most popular training algorithms for multilayer feedforward neural networks. However training the network with this algorithm has proved to be computationally intensive for a sequential machine. In this paper, parallel implementation of the backpropagation algorithm is investigated using transputers hosted by a personal computer. Two methods of transputer implementations were considered. One method was the multi-tasking approach and the other the processor farming approach. Results showed that for all test cases, the training time for the neural network with the multi-tasking approach is shorter than the processor farming approach. Comparing with a serial 486-33 PC, it is found that as the problem size scales up, the improvement in training time from the parallel implementation becomes significant.