Knowledge-Engineered Multi-Cloud Resource Brokering for Application Workflow Optimization

Data-intensive application workflows benefit by leveraging cloud services to decrease execution times and increase data sharing. Cloud service providers (CSPs) have distinct capabilities and policies, and performance/cost of the cloud services are amongst the prime factors for CSP selection. However, workflow users who need brokering of cloud resources often lack expert guidance to handle the problem of overwhelming choice in CSP selection, and optimization to compensate for service dynamics. In this paper, we address the optimal resource selection problem using a multi-cloud resource broker viz., OnTimeURB that uses knowledge-engineering of user requirements and service capabilities across multiple CSPs. OnTimeURB is powered by integer linear programming and a Naive Bayes classifier to recommend optimal cloud template solutions by weighting performance, agility, cost, and security (PACS) factors. We evaluate the OnTimeURB recommendations with a catalog of bioinformatics application workflows using four CSP resources featuring more than 300 different instance configurations. Our evaluation results show the efficacy of OnTimeURB in creating consistently cost-effective and agile solutions compared to a state-of-the-art k-nearest neighbors (k-NN) approach. We also show that OnTimeURB has 91% success rate improvement in workflow execution times via cloud template recommendations over approaches that do not use knowledge-engineered multi-CSP resource brokering.