Support Vector Data Descriptions and k-Means Clustering: One Class?

Nico Gornitz; Luiz Alberto Lima; Klaus Robert Muller; Marius Kloft; Shinichi Nakajima

doi:10.1109/TNNLS.2017.2737941

Support Vector Data Descriptions and k-Means Clustering: One Class?

Nico Gornitz, Luiz Alberto Lima, Klaus Robert Muller, Marius Kloft, Shinichi Nakajima

Research output: Contribution to journal › Article › peer-review

27 Citations (Scopus)

Abstract

We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.

Original language	English
Article number	8052229
Pages (from-to)	3994-4006
Number of pages	13
Journal	IEEE Transactions on Neural Networks and Learning Systems
Volume	29
Issue number	9
DOIs	https://doi.org/10.1109/TNNLS.2017.2737941
Publication status	Published - 2018 Sept

Bibliographical note

Funding Information:
Manuscript received August 7, 2016; revised December 8, 2016 and May 4, 2017; accepted August 1, 2017. Date of publication September 27, 2017; date of current version August 20, 2018. This work was supported by the German Research Foundation under Grant DFG MU 987/6-1 and Grant RA 1894/1-1. The work of N. Görnitz was supported by BMBF ALICE II under Grant 01IB15001B. The work of L. A. Lima was supported by Petrobras. The work of K.-R. Müller was supported by the National Research Foundation of Korea, Ministry of Education, Science, and Technology, through the BK21 Program. The work of M. Kloft was supported by the German Research Foundation under Grant KL 2698/1-1 and Grant KL 2698/2-1. The work of S. Nakajima was supported by the German Ministry for Education and Research as Berlin Big Data Center under Grant 01IS14013A. (Corresponding author: Nico Görnitz; Klaus-Robert Müller.) N. Görnitz and S. Nakajima are with the Machine Learning Group, Berlin Institute of Technology, 10587 Berlin, Germany (e-mail: nico.goernitz@tu-berlin.de).

Publisher Copyright:
© 2012 IEEE.

Keywords

Anomaly detection
clustering
k-means
one-class classification
support vector data description (SVDD)

ASJC Scopus subject areas

Software
Computer Science Applications
Computer Networks and Communications
Artificial Intelligence

Access to Document

10.1109/TNNLS.2017.2737941

Cite this

@article{ecd0ebd17fb841798bee5c8993314b32,

title = "Support Vector Data Descriptions and k-Means Clustering: One Class?",

abstract = "We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.",

keywords = "Anomaly detection, clustering, k-means, one-class classification, support vector data description (SVDD)",

author = "Nico Gornitz and Lima, {Luiz Alberto} and Muller, {Klaus Robert} and Marius Kloft and Shinichi Nakajima",

note = "Funding Information: Manuscript received August 7, 2016; revised December 8, 2016 and May 4, 2017; accepted August 1, 2017. Date of publication September 27, 2017; date of current version August 20, 2018. This work was supported by the German Research Foundation under Grant DFG MU 987/6-1 and Grant RA 1894/1-1. The work of N. G{\"o}rnitz was supported by BMBF ALICE II under Grant 01IB15001B. The work of L. A. Lima was supported by Petrobras. The work of K.-R. M{\"u}ller was supported by the National Research Foundation of Korea, Ministry of Education, Science, and Technology, through the BK21 Program. The work of M. Kloft was supported by the German Research Foundation under Grant KL 2698/1-1 and Grant KL 2698/2-1. The work of S. Nakajima was supported by the German Ministry for Education and Research as Berlin Big Data Center under Grant 01IS14013A. (Corresponding author: Nico G{\"o}rnitz; Klaus-Robert M{\"u}ller.) N. G{\"o}rnitz and S. Nakajima are with the Machine Learning Group, Berlin Institute of Technology, 10587 Berlin, Germany (e-mail: nico.goernitz@tu-berlin.de). Publisher Copyright: {\textcopyright} 2012 IEEE.",

year = "2018",

month = sep,

doi = "10.1109/TNNLS.2017.2737941",

language = "English",

volume = "29",

pages = "3994--4006",

journal = "IEEE Transactions on Neural Networks and Learning Systems",

issn = "2162-237X",

publisher = "IEEE Computational Intelligence Society",

number = "9",

}

TY - JOUR

T1 - Support Vector Data Descriptions and k-Means Clustering

T2 - One Class?

AU - Gornitz, Nico

AU - Lima, Luiz Alberto

AU - Muller, Klaus Robert

AU - Kloft, Marius

AU - Nakajima, Shinichi

N1 - Funding Information: Manuscript received August 7, 2016; revised December 8, 2016 and May 4, 2017; accepted August 1, 2017. Date of publication September 27, 2017; date of current version August 20, 2018. This work was supported by the German Research Foundation under Grant DFG MU 987/6-1 and Grant RA 1894/1-1. The work of N. Görnitz was supported by BMBF ALICE II under Grant 01IB15001B. The work of L. A. Lima was supported by Petrobras. The work of K.-R. Müller was supported by the National Research Foundation of Korea, Ministry of Education, Science, and Technology, through the BK21 Program. The work of M. Kloft was supported by the German Research Foundation under Grant KL 2698/1-1 and Grant KL 2698/2-1. The work of S. Nakajima was supported by the German Ministry for Education and Research as Berlin Big Data Center under Grant 01IS14013A. (Corresponding author: Nico Görnitz; Klaus-Robert Müller.) N. Görnitz and S. Nakajima are with the Machine Learning Group, Berlin Institute of Technology, 10587 Berlin, Germany (e-mail: nico.goernitz@tu-berlin.de). Publisher Copyright: © 2012 IEEE.

PY - 2018/9

Y1 - 2018/9

N2 - We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.

AB - We present ClusterSVDD, a methodology that unifies support vector data descriptions (SVDDs) and k-means clustering into a single formulation. This allows both methods to benefit from one another, i.e., by adding flexibility using multiple spheres for SVDDs and increasing anomaly resistance and flexibility through kernels to k-means. In particular, our approach leads to a new interpretation of k-means as a regularized mode seeking algorithm. The unifying formulation further allows for deriving new algorithms by transferring knowledge from one-class learning settings to clustering settings and vice versa. As a showcase, we derive a clustering method for structured data based on a one-class learning scenario. Additionally, our formulation can be solved via a particularly simple optimization scheme. We evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data, as well as on real-world problems, and provide a Python software package comprising various implementations of primal and dual SVDD as well as our proposed ClusterSVDD.

KW - Anomaly detection

KW - clustering

KW - k-means

KW - one-class classification

KW - support vector data description (SVDD)

UR - http://www.scopus.com/inward/record.url?scp=85030751821&partnerID=8YFLogxK

U2 - 10.1109/TNNLS.2017.2737941

DO - 10.1109/TNNLS.2017.2737941

M3 - Article

C2 - 28961127

AN - SCOPUS:85030751821

SN - 2162-237X

VL - 29

SP - 3994

EP - 4006

JO - IEEE Transactions on Neural Networks and Learning Systems

JF - IEEE Transactions on Neural Networks and Learning Systems

IS - 9

M1 - 8052229

ER -

Support Vector Data Descriptions and k-Means Clustering: One Class?

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this