Deep Latent Variable Models for Sequential Data

Due to their deterministic and non-linear nature however, they struggle to model the uncertainty inherent in real world applications and it is generally difficult to interpret their behaviour. State-space models, on the other hand, are a class of probabilistic graphical models for which Bayesian inference and prior domain knowledge can be naturally used to reason under uncertainty and to discover interpretable representations of the data. For inference to be tractable however, the state-space model can only be constructed from a simple family of probability densities, that is not powerful enough to capture the non-linearities typical in high-dimensional sequential data. To combine the best of both worlds we introduce deep state-space models, a class of non-linear and stochastic generative models for high-dimensional sequential data. Deep state-space models combine the expressiveness of deep neural models such as recurrent or convolutional networks and the probabilistic reasoning capabilities of state-space models. Scalable approximate inference procedures for end-to-end learning are developed by exploiting the conditional independence properties of the graphical model and extending ideas from variational auto-encoders to the temporal setting. We develop different architectures belonging to the family of deep-state space models,that represent different trade-offs between the flexibility given by the deep learning component and the probabilistic reasoning of the graphical model one.