MLFV: Network-aware Orchestration for Placing Chains of Virtualized Machine Learning Functions

Machine Learning as a Service (MLaaS) platforms enables access to Machine Learning (ML) processing with scalable infrastructure, from anywhere, and at any time, but requires sending large amounts of data to the cloud. ML on the edge is emerging as an option to reduce latency and bandwidth usage, maintaining data privacy. However, the existing edge approaches are not aware of the current network state for orchestrating the tasks. Network-aware orchestration services are supported by the Network Function Virtualization (NFV) architecture, making it a promising approach to manage and place ML tasks. In this paper, we propose Machine Learning Function Virtualization (MLFV), a fully network-aware framework that explores the NFV environment to virtualize ML tasks as virtual network functions. We describe a novel model for placing chains of ML tasks, considering constraints on CPU, memory, the existence of ML libraries, and the network overload, aiming to reduce the overall execution time of a chain. The results showed that MLFV outperformed existing cloud and edge approaches, particularly when network connections present instabilities. MLFV was able to identify the irregularities, allocating the ML tasks on hosts with normal connections, and thus, reducing the time for classifying single and multiple concurrent requests.