Capability Adaptive Elastic Iot Architecture

IoT networks, together with the data that flows between inter-networked smart devices are growing at unprecedented rates. Storing the data arriving in high volume at IoT gateways or transferring it to back-end clusters for timely processing and real-time response generation is not always feasible, and generally requires considerable overprovisioning of resources. In typical configurations, a computationally capable front-end gateway is available for coordinating the operation of smart devices and for linking them to computation and communication rich back-end infrastructure. It is attractive to harness the computational capabilities in front-end gateways to achieve elasticity of deployment so that the total throughput and response time of IoT solutions can be optimized by intelligent division of labor between devices, front-end gateways, and back-end servers. This paper proposes an Elastic Computing Framework (ECF), an architecture in which programmability of front-end machines is drawn upon to dynamically divide data processing tasks into those that see all of the data but perform simple operations on it, and those that see a transformed subset of the data and extract deep insights from it. This results in lifting the pressure on the network that links the front-and back-ends, and achieves capability adaptive deployment of IoT solutions. In experimental evaluation with one production usage scenario, this approach achieves between six fold and fifty fold reduction in data transferred to back-end clusters, and improves responsiveness by roughly sixteen fold.