SensiX: A System for Best-Effort Inference of Machine Learning Models in Multi-Device Environments

Multiple sensory devices on and around us are on the rise and require us to redesign a system to make an inference of ML models accurate, robust, and efficient at the deployment time. While this multiplicity opens up an exciting opportunity to leverage sensor redundancy and high availability, it is still extremely challenging to benefit from such multiplicity and boost the runtime performance of deployed ML models without requiring model retraining and engineering. From our experience of deploying ML models in multi-device environments, we uncovered two prime caveats, device and data variabilities that affect the runtime performance of ML models. To this end, we develop an ML system that addresses these variabilities without modifying deployed models by building on prior algorithmic work. It decouples model execution from sensor data and employs two essential operations between them: a) device-to-device data translation for principled mapping of training and inference data and b) quality-aware dynamic selection for systematically choosing the execution pipeline as a function of runtime accuracy. We develop and evaluate a prototype system on wearable devices with motion and audio-based models. The experimental results show that ML models achieve a 7-13% increase in runtime accuracy solely by running on top of our system, and the increase goes up to 30% in dynamic environments. This performance gain comes at the expense of 3 mW on the host device.