Discriminative learning of Bayesian latent structure models

Probabilistic topic models (and their extensions) have become popular as models of latent structures in collections of text documents or images. These models are usually treated as generative models and trained using maximum likelihood estimation, an approach which may be suboptimal in the context of an overall classification problem. In this project, we show how to train Latent Dirichlet Allocation (LDA) discriminatively by maximizing the conditional likelihood of side information such as labels. Our empirical study shows that the predictive power of the discriminatively learned LDA improves significantly over that of unsupervised LDA.

Left. 2-d embedding of LDA topics on the 2o-newsgroup data shows mixed clusterings of documents from different (color-coded) newsgroups. Right. Discriminatively trained LDA shows much clearly separated clusters of documents.

Related publications

Learning parts based representation for speech and audios

An auditory scene, composed of overlapping acoustic sources, can be viewed as a complex object whose constituent parts are the individual sources. In this project, we investigate how the technique of nonnegative matrix factorization (NMF) can be used to learn parts from voices. These parts correspond to harmonic stacks of periodic components in voices, which give rise to the perception of pitches.

Related publications