This paper introduces a novel divide-and-conquer approach to optimize a neural network ensemble. The approach would optimize the available resources to implement multiple neural networks in the ensemble. Three algorithms are proposed to explore the structure of neural networks and to estimate the optimal parallelism configuration for an ensemble; they provide a configuration design guide to facilitate the study of trade-offs between resource usage and run time. Our approach also supports variable widths of input data, further improving the diversity of the ensemble. The proposed approach shows promise in the evaluation against related work and baseline implementations, especially when used to process input data streams, with the ensemble utilizing the available resources to achieve 33 times speed up with 10