Layer-wise analysis of deep networks with Gaussian kernels

Grégoire Montavon; Mikio L. Braun; Klaus Robert Müller

Layer-wise analysis of deep networks with Gaussian kernels

Grégoire Montavon, Mikio L. Braun, Klaus Robert Müller

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Deep networks can potentially express a learning problem more efficiently than local learning machines. While deep networks outperform local learning machines on some problems, it is still unclear how their nice representation emerges from their complex structure. We present an analysis based on Gaussian kernels that measures how the representation of the learning problem evolves layer after layer as the deep network builds higher-level abstract representations of the input. We use this analysis to show empirically that deep networks build progressively better representations of the learning problem and that the best representations are obtained when the deep network discriminates only in the last layers.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 23
Subtitle of host publication	24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010
Publication status	Published - 2010
Externally published	Yes
Event	24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 - Vancouver, BC, Canada Duration: 2010 Dec 6 → 2010 Dec 9

Publication series

Name	Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Other

Other	24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010
Country/Territory	Canada
City	Vancouver, BC
Period	10/12/6 → 10/12/9

ASJC Scopus subject areas

Information Systems

Cite this

Montavon, G., Braun, M. L., & Müller, K. R. (2010). Layer-wise analysis of deep networks with Gaussian kernels. In Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 (Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010).

Layer-wise analysis of deep networks with Gaussian kernels. / Montavon, Grégoire; Braun, Mikio L.; Müller, Klaus Robert.
Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010. 2010. (Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Montavon, G, Braun, ML & Müller, KR 2010, Layer-wise analysis of deep networks with Gaussian kernels. in Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010. Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010, 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010, Vancouver, BC, Canada, 10/12/6.

Montavon G, Braun ML, Müller KR. Layer-wise analysis of deep networks with Gaussian kernels. In Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010. 2010. (Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010).

Montavon, Grégoire ; Braun, Mikio L. ; Müller, Klaus Robert. / Layer-wise analysis of deep networks with Gaussian kernels. Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010. 2010. (Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010).

@inproceedings{0f784a31352943e28e943db01e70e2b0,

title = "Layer-wise analysis of deep networks with Gaussian kernels",

abstract = "Deep networks can potentially express a learning problem more efficiently than local learning machines. While deep networks outperform local learning machines on some problems, it is still unclear how their nice representation emerges from their complex structure. We present an analysis based on Gaussian kernels that measures how the representation of the learning problem evolves layer after layer as the deep network builds higher-level abstract representations of the input. We use this analysis to show empirically that deep networks build progressively better representations of the learning problem and that the best representations are obtained when the deep network discriminates only in the last layers.",

author = "Gr{\'e}goire Montavon and Braun, {Mikio L.} and M{\"u}ller, {Klaus Robert}",

year = "2010",

language = "English",

isbn = "9781617823800",

series = "Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010",

booktitle = "Advances in Neural Information Processing Systems 23",

note = "24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 ; Conference date: 06-12-2010 Through 09-12-2010",

}

TY - GEN

T1 - Layer-wise analysis of deep networks with Gaussian kernels

AU - Montavon, Grégoire

AU - Braun, Mikio L.

AU - Müller, Klaus Robert

PY - 2010

Y1 - 2010

N2 - Deep networks can potentially express a learning problem more efficiently than local learning machines. While deep networks outperform local learning machines on some problems, it is still unclear how their nice representation emerges from their complex structure. We present an analysis based on Gaussian kernels that measures how the representation of the learning problem evolves layer after layer as the deep network builds higher-level abstract representations of the input. We use this analysis to show empirically that deep networks build progressively better representations of the learning problem and that the best representations are obtained when the deep network discriminates only in the last layers.

AB - Deep networks can potentially express a learning problem more efficiently than local learning machines. While deep networks outperform local learning machines on some problems, it is still unclear how their nice representation emerges from their complex structure. We present an analysis based on Gaussian kernels that measures how the representation of the learning problem evolves layer after layer as the deep network builds higher-level abstract representations of the input. We use this analysis to show empirically that deep networks build progressively better representations of the learning problem and that the best representations are obtained when the deep network discriminates only in the last layers.

UR - http://www.scopus.com/inward/record.url?scp=84860614044&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84860614044&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84860614044

SN - 9781617823800

T3 - Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

BT - Advances in Neural Information Processing Systems 23

T2 - 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Y2 - 6 December 2010 through 9 December 2010

ER -

Layer-wise analysis of deep networks with Gaussian kernels

Abstract

Publication series

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this