A Reinforcement Learning-Based Service Model for the Internet of Things.

The Internet of Things (IoT) creates environments where devices and users interact. Service-oriented architectures (SOAs) encapsulate devices' capabilities as IoT services which users can request. SOAs manage the scale of IoT services by placing services descriptions about appropriate services in distributed architectures (e.g., a network of IoT gateways). Such distribution increases the chances of responding to users in an efficient fashion as requests are attended locally. However, dynamic IoT environments can easily outdate the distribution of services descriptions, which in turn impacts SOAs efficiency when the required services descriptions are not in place. Current architectures use pre-defined knowledge to adapt the distribution of services descriptions reactively. However, such human intervention is not always available and may be error-prone in dynamic IoT environments. We propose a reinforcement learning model that IoT gateways use to automatically decide how to distribute services descriptions over time. We evaluate the model in a real IoT testbed and results show that its performance in different scenarios compares favourably against a reactive baseline.