Generative versus discriminative training of RBMs for classification of fMRI images

Tanya Schmah; Geoffrey E. Hinton; Richard S. Zemel; Steven L. Small; Stephen Strother

Generative versus discriminative training of RBMs for classification of fMRI images

Tanya Schmah^*, Geoffrey E. Hinton, Richard S. Zemel, Steven L. Small, Stephen Strother

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

62 Citations (Scopus)

Abstract

Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference
Pages	1409-1416
Number of pages	8
Publication status	Published - 2009
Event	22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 - Vancouver, BC, Canada Duration: 8 Dec 2008 → 11 Dec 2008

Other

Other	22nd Annual Conference on Neural Information Processing Systems, NIPS 2008
Country/Territory	Canada
City	Vancouver, BC
Period	8/12/08 → 11/12/08

Cite this

@inproceedings{1072e62dd841499ea6fb76f55f05a7b5,

title = "Generative versus discriminative training of RBMs for classification of fMRI images",

abstract = "Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.",

author = "Tanya Schmah and Hinton, {Geoffrey E.} and Zemel, {Richard S.} and Small, {Steven L.} and Stephen Strother",

year = "2009",

language = "English",

isbn = "9781605609492",

pages = "1409--1416",

booktitle = "Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference",

note = "22nd Annual Conference on Neural Information Processing Systems, NIPS 2008 ; Conference date: 08-12-2008 Through 11-12-2008",

}

Generative versus discriminative training of RBMs for classification of fMRI images. / Schmah, Tanya; Hinton, Geoffrey E.; Zemel, Richard S. et al.
Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference. 2009. p. 1409-1416.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Generative versus discriminative training of RBMs for classification of fMRI images

AU - Schmah, Tanya

AU - Hinton, Geoffrey E.

AU - Zemel, Richard S.

AU - Small, Steven L.

AU - Strother, Stephen

PY - 2009

Y1 - 2009

N2 - Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.

AB - Neuroimaging datasets often have a very large number of voxels and a very small number of training cases, which means that overfitting of models for this data can become a very serious problem. Working with a set of fMRI images from a study on stroke recovery, we consider a classification task for which logistic regression performs poorly, even when L1- or L2- regularized. We show that much better discrimination can be achieved by fitting a generative model to each separate condition and then seeing which model is most likely to have generated the data. We compare discriminative training of exactly the same set of models, and we also consider convex blends of generative and discriminative training.

UR - http://www.scopus.com/inward/record.url?scp=78149327745&partnerID=8YFLogxK

M3 - Conference proceeding contribution

AN - SCOPUS:78149327745

SN - 9781605609492

SP - 1409

EP - 1416

BT - Advances in Neural Information Processing Systems 21 - Proceedings of the 2008 Conference

T2 - 22nd Annual Conference on Neural Information Processing Systems, NIPS 2008

Y2 - 8 December 2008 through 11 December 2008

ER -

Generative versus discriminative training of RBMs for classification of fMRI images

Abstract

Other

Other files and links

Fingerprint

Cite this