Regularizing AdaBoost

Gunnar Rätsch; Takashi Onoda; Klaus R. Müller

Regularizing AdaBoost

Gunnar Rätsch, Takashi Onoda, Klaus R. Müller

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Boosting methods maximize a hard classification margin and are known as powerful techniques that do not exhibit overfitting for low noise cases. Also for noisy data boosting will try to enforce a hard margin and thereby give too much weight to outliers, which then leads to the dilemma of non-smooth fits and overfitting. Therefore we propose three algorithms to allow for soft margin classification by introducing regularization with slack variables into the boosting concept: (1) AdaBoost_reg and regularized versions of (2) linear and (3) quadratic programming AdaBoost. Experiments show the usefulness of the proposed algorithms in comparison to another soft margin classifier: the support vector machine.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998
Publisher	Neural information processing systems foundation
Pages	564-570
Number of pages	7
ISBN (Print)	0262112450, 9780262112451
Publication status	Published - 1999
Externally published	Yes
Event	12th Annual Conference on Neural Information Processing Systems, NIPS 1998 - Denver, CO, United States Duration: 1998 Nov 30 → 1998 Dec 5

Publication series

Name	Advances in Neural Information Processing Systems
ISSN (Print)	1049-5258

Other

Other	12th Annual Conference on Neural Information Processing Systems, NIPS 1998
Country/Territory	United States
City	Denver, CO
Period	98/11/30 → 98/12/5

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

Regularizing AdaBoost. / Rätsch, Gunnar; Onoda, Takashi; Müller, Klaus R.
Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998. Neural information processing systems foundation, 1999. p. 564-570 (Advances in Neural Information Processing Systems).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Rätsch, G, Onoda, T & Müller, KR 1999, Regularizing AdaBoost. in Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998. Advances in Neural Information Processing Systems, Neural information processing systems foundation, pp. 564-570, 12th Annual Conference on Neural Information Processing Systems, NIPS 1998, Denver, CO, United States, 98/11/30.

@inproceedings{a69086517e254fe7b324eb26d9df37b6,

title = "Regularizing AdaBoost",

abstract = "Boosting methods maximize a hard classification margin and are known as powerful techniques that do not exhibit overfitting for low noise cases. Also for noisy data boosting will try to enforce a hard margin and thereby give too much weight to outliers, which then leads to the dilemma of non-smooth fits and overfitting. Therefore we propose three algorithms to allow for soft margin classification by introducing regularization with slack variables into the boosting concept: (1) AdaBoostreg and regularized versions of (2) linear and (3) quadratic programming AdaBoost. Experiments show the usefulness of the proposed algorithms in comparison to another soft margin classifier: the support vector machine.",

author = "Gunnar R{\"a}tsch and Takashi Onoda and M{\"u}ller, {Klaus R.}",

year = "1999",

language = "English",

isbn = "0262112450",

series = "Advances in Neural Information Processing Systems",

publisher = "Neural information processing systems foundation",

pages = "564--570",

booktitle = "Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998",

note = "12th Annual Conference on Neural Information Processing Systems, NIPS 1998 ; Conference date: 30-11-1998 Through 05-12-1998",

}

TY - GEN

T1 - Regularizing AdaBoost

AU - Rätsch, Gunnar

AU - Onoda, Takashi

AU - Müller, Klaus R.

PY - 1999

Y1 - 1999

N2 - Boosting methods maximize a hard classification margin and are known as powerful techniques that do not exhibit overfitting for low noise cases. Also for noisy data boosting will try to enforce a hard margin and thereby give too much weight to outliers, which then leads to the dilemma of non-smooth fits and overfitting. Therefore we propose three algorithms to allow for soft margin classification by introducing regularization with slack variables into the boosting concept: (1) AdaBoostreg and regularized versions of (2) linear and (3) quadratic programming AdaBoost. Experiments show the usefulness of the proposed algorithms in comparison to another soft margin classifier: the support vector machine.

AB - Boosting methods maximize a hard classification margin and are known as powerful techniques that do not exhibit overfitting for low noise cases. Also for noisy data boosting will try to enforce a hard margin and thereby give too much weight to outliers, which then leads to the dilemma of non-smooth fits and overfitting. Therefore we propose three algorithms to allow for soft margin classification by introducing regularization with slack variables into the boosting concept: (1) AdaBoostreg and regularized versions of (2) linear and (3) quadratic programming AdaBoost. Experiments show the usefulness of the proposed algorithms in comparison to another soft margin classifier: the support vector machine.

UR - http://www.scopus.com/inward/record.url?scp=0001102148&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0001102148&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:0001102148

SN - 0262112450

SN - 9780262112451

T3 - Advances in Neural Information Processing Systems

SP - 564

EP - 570

BT - Advances in Neural Information Processing Systems 11 - Proceedings of the 1998 Conference, NIPS 1998

PB - Neural information processing systems foundation

T2 - 12th Annual Conference on Neural Information Processing Systems, NIPS 1998

Y2 - 30 November 1998 through 5 December 1998

ER -

Regularizing AdaBoost

Abstract

Publication series

Other

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this