On-line learning in changing environments with applications in supervised and unsupervised learning

Noboru Murata, Motoaki Kawanabe, Andreas Ziehe, Klaus Robert Müller, Shun Ichi Amari

    Research output: Contribution to journalArticlepeer-review

    49 Citations (Scopus)

    Abstract

    An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. The framework is applied for unsupervised and supervised learning. Its efficiency is demonstrated for drifting and switching non-stationary blind separation tasks of acoustic signals. Furthermore applications to classification (US postal service data set) and time-series prediction in changing environments are presented.

    Original languageEnglish
    Pages (from-to)743-760
    Number of pages18
    JournalNeural Networks
    Volume15
    Issue number4-6
    DOIs
    Publication statusPublished - 2002 Jun

    Bibliographical note

    Funding Information:
    We thank the participants of the 1997 on-line learning workshop at the Newton Institute in Cambridge for interesting discussions. K.R.M., M.K. and A.Z. gratefully acknowledge partial financial support from DFG under contracts JA 379/91, MU 987/11 and the BMBF BCI project and the EU in the Neurocolt 2 and the BLISS project (IST-1999-14190).

    Keywords

    • Adaptive learning rate
    • Blind source separation
    • ICA
    • On-line learning
    • Stochastic gradient descent
    • Supervised learning
    • Unsupervised learning

    ASJC Scopus subject areas

    • Cognitive Neuroscience
    • Artificial Intelligence

    Fingerprint

    Dive into the research topics of 'On-line learning in changing environments with applications in supervised and unsupervised learning'. Together they form a unique fingerprint.

    Cite this