TY - GEN
T1 - Continuous-Time Stochastic Differential Networks for Irregular Time Series Modeling
AU - Liu, Yingru
AU - Xing, Yucheng
AU - Yang, Xuewen
AU - Wang, Xin
AU - Shi, Jing
AU - Jin, Di
AU - Chen, Zhaoyue
AU - Wu, Jacqueline
N1 - Publisher Copyright: © 2021, Springer Nature Switzerland AG.
PY - 2021
Y1 - 2021
N2 - Learning continuous-time stochastic dynamics is a fundamental and essential problem in modeling irregular time series, whose observations are irregular and sparse in both time and dimension. For a given system whose latent states and observed data are multivariate, it is generally impossible to derive a precise continuous-time stochastic process to describe the system behaviors. To solve the above problem, we apply Variational Bayesian method and propose a flexible continuous-time stochastic recurrent neural network named Variational Stochastic Differential Networks (VSDN), which embeds the complicated dynamics of the irregular time series by neural Stochastic Differential Equations (SDE). VSDNs capture the stochastic dependency among latent states and observations by deep neural networks. We also incorporate two differential Evidence Lower Bounds to efficiently train the models. Through comprehensive experiments, we show that VSDNs outperform state-of-the-art continuous-time deep learning models and achieve remarkable performance on prediction and interpolation tasks for irregular time series.
AB - Learning continuous-time stochastic dynamics is a fundamental and essential problem in modeling irregular time series, whose observations are irregular and sparse in both time and dimension. For a given system whose latent states and observed data are multivariate, it is generally impossible to derive a precise continuous-time stochastic process to describe the system behaviors. To solve the above problem, we apply Variational Bayesian method and propose a flexible continuous-time stochastic recurrent neural network named Variational Stochastic Differential Networks (VSDN), which embeds the complicated dynamics of the irregular time series by neural Stochastic Differential Equations (SDE). VSDNs capture the stochastic dependency among latent states and observations by deep neural networks. We also incorporate two differential Evidence Lower Bounds to efficiently train the models. Through comprehensive experiments, we show that VSDNs outperform state-of-the-art continuous-time deep learning models and achieve remarkable performance on prediction and interpolation tasks for irregular time series.
KW - Irregular time series
KW - Neural Stochastic Differential Equations
KW - Stochastic recurrent neural network
UR - https://www.scopus.com/pages/publications/85121911553
U2 - 10.1007/978-3-030-92307-5_40
DO - 10.1007/978-3-030-92307-5_40
M3 - Conference contribution
SN - 9783030923068
T3 - Communications in Computer and Information Science
SP - 343
EP - 351
BT - Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings
A2 - Mantoro, Teddy
A2 - Lee, Minho
A2 - Ayu, Media Anugerah
A2 - Wong, Kok Wai
A2 - Hidayanto, Achmad Nizar
PB - Springer Science and Business Media Deutschland GmbH
T2 - 28th International Conference on Neural Information Processing, ICONIP 2021
Y2 - 8 December 2021 through 12 December 2021
ER -