Transformations for Variational Factor Analysis to Speed up Learning

Reference:

Jaakko Luttinen, Alexander Ilin, and Tapani Raiko. Transformations for variational factor analysis to speed up learning. In Proceedings of the 14th European Symposium on Artificial Neural Networks (ESANN 2009), pages 77–82, Bruges, Belgium, April 2009.

Abstract:

We propose simple transformation of the hidden states in variational Bayesian (VB) factor analysis models to speed up the learning procedure. The transformation basically performs centering and whitening of the hidden states taking into account the posterior uncertainties. The transformation is given a theoretical justification from optimisation of the VB cost function. We derive the transformation formulae for variational Bayesian principal component analysis and show experimentally that it can significantly improve the rate of convergence. Similar transformations can be applied to other variational Bayesian factor analysis models as well.

Suggested BibTeX entry:

@inproceedings{Luttinen09esann,
    address = {Bruges, Belgium},
    author = {Jaakko Luttinen and Alexander Ilin and Tapani Raiko},
    booktitle = {Proceedings of the 14th {E}uropean Symposium on Artificial Neural Networks ({ESANN} 2009)},
    month = {April},
    pages = {77--82},
    title = {Transformations for Variational Factor Analysis to Speed up Learning},
    year = {2009},
}