PPCA Tutorial #499
PPCA Tutorial #499
Conversation
|
Thanks for writing this. Your description of PPCA is accurate and succinct. I think the ideal for a model tutorial should roughly explain its key ideas through an illustration with a data set; followed by the model; followed by an algorithm to infer it; followed by a check of its fit. (Good references are http://edwardlib.org/tutorials/unsupervised and http://edwardlib.org/tutorials/gan. Some of the other model tutorials are lacking in this regard.) So you could probably scaffold your current writing into sections with the data and output of the script. |
|
Thanks for the quick feedback @dustinvtran and sorry it took this long to revise. Let me know if I can improve this in any way. Thanks! |
docs/tex/bib.bib
Outdated
| year = {1999}, | ||
| volume = {61}, | ||
| pages = {611--622} | ||
|
|
There was a problem hiding this comment.
missing }. adding the ending brace lets it compile successfully for me.
docs/tex/bib.bib
Outdated
| @@ -667,6 +667,14 @@ @article{marin2012approximate | |||
| pages = {1167--1180} | |||
| } | |||
|
|
|||
| @ARTICLE{Tipping99probabilisticprincipal, | |||
There was a problem hiding this comment.
i recommend the citekey format tipping1999probabilistic, following google scholar.
|
|
||
| \subsubsection{Data} | ||
|
|
||
| We simulate our data points below. We'll talk about the individual variables and what they stand for in the next section. For this example, $\mathbf{x}\in\mathbb{R}^2$. |
There was a problem hiding this comment.
"For this example, each data point is 2-dimensional,
|
|
||
| \subsubsection{Model} | ||
|
|
||
| Consider a dataset $X = \{(\mathbf{x}_1, \mathbf{x}_2,\ldots , \mathbf{x}_n)\}$ where $\mathbf{x}_i \in \Bbb{R}^D$. |
There was a problem hiding this comment.
Following the code, I prefer the notation that N denotes the data set size and n denotes an index to one data point.
There was a problem hiding this comment.
Just to clarify, are you saying to change the i's to n? Or something else?
There was a problem hiding this comment.
Ignore - figured it out by looking at http://edwardlib.org/tutorials/unsupervised
|
|
||
| Note here that regular PCA is simply the specific case of Probabilistic PCA, where $\sigma^2 \to 0$. | ||
|
|
||
| We set up our model below. |
There was a problem hiding this comment.
I would also describe in the model section that you're placing a distribution over principle axes, either viewed as a prior or as a regularizer.
|
|
||
| \subsubsection{Inference} | ||
|
|
||
| Since $\mathbf{W}$ cannot be analytically determined, we must use some approximation method. Below, we set up our inference variables and then run the approximation algorithm. For this example, our method is to minimize the $\text{KL}(q\|p)$ divergence measure. |
There was a problem hiding this comment.
"Since ... determined, we must use some inference method." In this model, I think the posterior is actually normally distributed.
|
@timshell: The tutorial looks great. Only minor suggestions above. Happy to merge it once those are made. |
|
Sweet, just made the changes @dustinvtran ! |
Re: discussion in Gitter channel
Not sure what level of detail is expected. I used the other tutorials as my guide. Wanted to get one tutorial done in line with what you expect before I start doing more.
All feedback appreciated!