Targeted learning by imposing asymmetric sparsity

Reference:

Jussi Gillberg. Targeted learning by imposing asymmetric sparsity. Master's thesis, Aalto University, Department of Information and Computer Science, June 2011.

Abstract:

Modern data sets often suffer from the problem of having measurements from very few samples. The small sample size makes modeling such data sets very difficult, as models easily overfit to the data. Many approaches to alleviate the problem have been taken.

One such approach is multi-task learning, a subfield of statistical machine learning, in which multiple data sets are modeled simultaneously. More generally, multiple learning tasks may be learnt simultaneously to achieve better performance in each.

Another approach to the problem of having too few samples is to to prevent overfitting by constraining the model by making suitable assumptions.

Traditional multi-task methods treat all learning tasks and data sets equally, even thought we are usually mostly interested in learning one of them. This thesis is a case study about promoting predictive performance in a specific data set of interest in a multi-task setting by constraining the models for the learning tasks unevenly. The model for the data set of interest more sparse as compared to the models for the secondary data sets.

To study the new approach, the research question is limited to the very specific and popular family of so-called topic models using Bayesian nonparametric priors. A new model is presented which enables us to study the effects of asymmetric sparsity.

The effects of asymmetric sparsity are studied by using the new model on real data and toy data. Subtle beneficial effects of asymmetric sparsity are observed on toy data and the new model performs comparably to existing state-of-the-art methods on real data.

Keywords:

asymmetric multi-task learning, latent Dirichlet allocation, nonparametric Bayesian statistics, small sample size, sparsity

Suggested BibTeX entry:

@mastersthesis{Gillberg11mscthesis,
    address = {Department of Information and Computer Science},
    author = {Jussi Gillberg},
    language = {eng},
    month = {June},
    school = {Aalto University},
    title = {Targeted learning by imposing asymmetric sparsity},
    year = {2011},
}