\n", " Rahul G. Krishnan, Uri Shalit, David Sontag\n", " \n", "Please note that while we do not assume that the reader of this tutorial has read the reference, it's definitely a good place to look for a more comprehensive discussion of the deep markov model in the context of other time series models.\n", "\n", "We've described the model, but how do we go about training it? The inference strategy we're going to use is variational inference, which requires specifying a parameterized family of distributions that can be used to approximate the posterior distribution over the latent random variables. Given the non-linearities and complex time-dependencies inherent in our model and data, we expect the exact posterior to be highly non-trivial. So we're going to need a flexible family of variational distributions if we hope to learn a good model. Happily, together PyTorch and Pyro provide all the necessary ingredients. As we will see, assembling them will be straightforward. Let's get to work." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The Model\n", " \n", "A convenient way to describe the high-level structure of the model is with a graphical model." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/html" }, "source": [ "

" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/html" }, "source": [ "

\n", " Rahul G. Krishnan, Uri Shalit, David Sontag\n", " \n", "[2] `Variational Inference with Normalizing Flows`,\n", "

\n", "Danilo Jimenez Rezende, Shakir Mohamed \n", " \n", "[3] `Improving Variational Inference with Inverse Autoregressive Flow`,\n", "

\n", "Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, Max Welling \n", "\n", "[4] `MADE: Masked Autoencoder for Distribution Estimation`,\n", "

\n", "Mathieu Germain, Karol Gregor, Iain Murray, Hugo Larochelle \n", "\n", "[5] `Modeling Temporal Dependencies in High-Dimensional Sequences:`\n", "

\n", "`Application to Polyphonic Music Generation and Transcription`,\n", "

\n", "Boulanger-Lewandowski, N., Bengio, Y. and Vincent, P." ] } ], "metadata": { "celltoolbar": "Raw Cell Format", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.10" } }, "nbformat": 4, "nbformat_minor": 2 }