Sequential decision-making in healthcare IoT: Real-time health monitoring, treatments and interventions

Internet of Things (IoT) technology and infrastructure have the potential to revolutionize healthcare delivery. Networked body sensing devices coupled with sensors in our living environment enable the real-time and continuous collection of information related to an individual's physical and mental health and related behaviors. Captured in a continual basis and aggregated, such information needs to be effectively exploited to permit real-time, continuous and personalized monitoring, treatments and interventions. However, medical decisions are often sequential and uncertain in nature. Sequential decision-making models such as Markov decision processes (MDPs) and partially observable MDPs (POMDPs) constitute powerful tools for modeling and solving such stochastic and dynamic problems. In this paper, an overview of such models that are expected to support proactive, preventive and personalized healthcare delivery are surveyed along with the associated solution techniques. A set of representative health applications that take advantage of such tools is also described. Finally, various challenges and opportunities that arise during the realization of smart and connected healthcare IoT are highlighted.