# Probabilistic PCA PCA can also be expressed as the maximum likelihood solution of a probabilistic latent variable model. It is an example of linear-Gaussian framework, in which all of the marginal and conditional distributions are Gaussians. We first define a continuous latent variable model, $ p(\boldsymbol{x})=\int p(\boldsymbol{x}, \boldsymbol{z}) d \boldsymbol{z}=\int p(\boldsymbol{x} \mid \boldsymbol{z}) p(\boldsymbol{z}) d \boldsymbol{z} $ Where, the Gaussian prior distribution $p(\mathbf{z})$ is defined as, $ p(\mathbf{z})=\mathcal{N}(\mathbf{z} \mid \mathbf{0}, \mathbf{I}) $ Similarly, conditional distribution of the observed variable x conditioned on the value of the latent variable z is given as, $ p(\mathbf{x} \mid \mathbf{z})=\mathcal{N}\left(\mathbf{x} \mid \mathbf{W} \mathbf{z}+\boldsymbol{\mu}, \sigma^{2} \mathbf{I}\right) $ where the generative model is defined as the generalized linear model $ \mathbf{x}=\mathbf{W} \mathbf{z}+\boldsymbol{\mu}+\boldsymbol{\epsilon} $ with $\mu \in \mathbb{R}^{D}$ and the continuous latent variable $z \in \mathbb{R}^{M}$. Matrix $\mathbf{W} \in R^{D \times M}$ transforms the latent variables into observed variables. The independant noise is also a Gaussian $p(\boldsymbol{\epsilon})=\mathscr{N}\left(\boldsymbol{\epsilon} \mid \boldsymbol{0}, \sigma^{2} \mathbf{I}\right)$. ## References 1. 12.2 Bishop 2006, [[Lecture 10 - ML1]] ---