Specmix: a mixed sample data augmentation method for training with time-frequency domain features

Gwantae Kim; David K. Han; Hanseok Ko

doi:10.21437/Interspeech.2021-103

Specmix: a mixed sample data augmentation method for training with time-frequency domain features

Gwantae Kim, David K. Han, Hanseok Ko

School of Electrical Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

20 Citations (Scopus)

Abstract

A mixed sample data augmentation strategy is proposed to enhance the performance of models on audio scene classification, sound event classification, and speech enhancement tasks. While there have been several augmentation methods shown to be effective in improving image classification performance, their efficacy toward time-frequency domain features of audio is not assured. We propose a novel audio data augmentation approach named "Specmix"specifically designed for dealing with time-frequency domain features. The augmentation method consists of mixing two different data samples by applying time-frequency masks effective in preserving the spectral correlation of each audio sample. Our experiments on acoustic scene classification, sound event classification, and speech enhancement tasks show that the proposed Specmix improves the performance of various neural network architectures by a maximum of 2.7%.

Original language	English
Title of host publication	22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021
Publisher	International Speech Communication Association
Pages	6-10
Number of pages	5
ISBN (Electronic)	9781713836902
DOIs	https://doi.org/10.21437/Interspeech.2021-103
Publication status	Published - 2021
Event	22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021 - Brno, Czech Republic Duration: 2021 Aug 30 → 2021 Sept 3

Publication series

Name	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
Volume	1
ISSN (Print)	2308-457X
ISSN (Electronic)	1990-9772

Conference

Conference	22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021
Country/Territory	Czech Republic
City	Brno
Period	21/8/30 → 21/9/3

Bibliographical note

Funding Information:
This material is based upon work supported by the Air Force Office of Scientific Research under award number FA2386-19-1-4001.

Publisher Copyright:
Copyright © 2021 ISCA.

Keywords

Acoustic scene classification
Data augmentation
Deep neural networks
Sound event classification
Speech enhancement

ASJC Scopus subject areas

Language and Linguistics
Human-Computer Interaction
Signal Processing
Software
Modelling and Simulation

Access to Document

10.21437/Interspeech.2021-103

Cite this

Kim, G., Han, D. K., & Ko, H. (2021). Specmix: a mixed sample data augmentation method for training with time-frequency domain features. In 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021 (pp. 6-10). (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH; Vol. 1). International Speech Communication Association. https://doi.org/10.21437/Interspeech.2021-103

Specmix: a mixed sample data augmentation method for training with time-frequency domain features. / Kim, Gwantae; Han, David K.; Ko, Hanseok.
22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021. International Speech Communication Association, 2021. p. 6-10 (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kim, G, Han, DK & Ko, H 2021, Specmix: a mixed sample data augmentation method for training with time-frequency domain features. in 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, vol. 1, International Speech Communication Association, pp. 6-10, 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021, Brno, Czech Republic, 21/8/30. https://doi.org/10.21437/Interspeech.2021-103

Kim G, Han DK, Ko H. Specmix: a mixed sample data augmentation method for training with time-frequency domain features. In 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021. International Speech Communication Association. 2021. p. 6-10. (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH). doi: 10.21437/Interspeech.2021-103

Kim, Gwantae ; Han, David K. ; Ko, Hanseok. / Specmix : a mixed sample data augmentation method for training with time-frequency domain features. 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021. International Speech Communication Association, 2021. pp. 6-10 (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH).

@inproceedings{f99bee6fc9364cb6a2b834af00652358,

title = "Specmix: a mixed sample data augmentation method for training with time-frequency domain features",

abstract = "A mixed sample data augmentation strategy is proposed to enhance the performance of models on audio scene classification, sound event classification, and speech enhancement tasks. While there have been several augmentation methods shown to be effective in improving image classification performance, their efficacy toward time-frequency domain features of audio is not assured. We propose a novel audio data augmentation approach named {"}Specmix{"}specifically designed for dealing with time-frequency domain features. The augmentation method consists of mixing two different data samples by applying time-frequency masks effective in preserving the spectral correlation of each audio sample. Our experiments on acoustic scene classification, sound event classification, and speech enhancement tasks show that the proposed Specmix improves the performance of various neural network architectures by a maximum of 2.7%.",

keywords = "Acoustic scene classification, Data augmentation, Deep neural networks, Sound event classification, Speech enhancement",

author = "Gwantae Kim and Han, {David K.} and Hanseok Ko",

note = "Funding Information: This material is based upon work supported by the Air Force Office of Scientific Research under award number FA2386-19-1-4001. Publisher Copyright: Copyright {\textcopyright} 2021 ISCA.; 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021 ; Conference date: 30-08-2021 Through 03-09-2021",

year = "2021",

doi = "10.21437/Interspeech.2021-103",

language = "English",

series = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",

publisher = "International Speech Communication Association",

pages = "6--10",

booktitle = "22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021",

}

TY - GEN

T1 - Specmix

T2 - 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021

AU - Kim, Gwantae

AU - Han, David K.

AU - Ko, Hanseok

PY - 2021

Y1 - 2021

N2 - A mixed sample data augmentation strategy is proposed to enhance the performance of models on audio scene classification, sound event classification, and speech enhancement tasks. While there have been several augmentation methods shown to be effective in improving image classification performance, their efficacy toward time-frequency domain features of audio is not assured. We propose a novel audio data augmentation approach named "Specmix"specifically designed for dealing with time-frequency domain features. The augmentation method consists of mixing two different data samples by applying time-frequency masks effective in preserving the spectral correlation of each audio sample. Our experiments on acoustic scene classification, sound event classification, and speech enhancement tasks show that the proposed Specmix improves the performance of various neural network architectures by a maximum of 2.7%.

AB - A mixed sample data augmentation strategy is proposed to enhance the performance of models on audio scene classification, sound event classification, and speech enhancement tasks. While there have been several augmentation methods shown to be effective in improving image classification performance, their efficacy toward time-frequency domain features of audio is not assured. We propose a novel audio data augmentation approach named "Specmix"specifically designed for dealing with time-frequency domain features. The augmentation method consists of mixing two different data samples by applying time-frequency masks effective in preserving the spectral correlation of each audio sample. Our experiments on acoustic scene classification, sound event classification, and speech enhancement tasks show that the proposed Specmix improves the performance of various neural network architectures by a maximum of 2.7%.

KW - Acoustic scene classification

KW - Data augmentation

KW - Deep neural networks

KW - Sound event classification

KW - Speech enhancement

UR - http://www.scopus.com/inward/record.url?scp=85119245236&partnerID=8YFLogxK

U2 - 10.21437/Interspeech.2021-103

DO - 10.21437/Interspeech.2021-103

M3 - Conference contribution

AN - SCOPUS:85119245236

T3 - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

SP - 6

EP - 10

BT - 22nd Annual Conference of the International Speech Communication Association, INTERSPEECH 2021

PB - International Speech Communication Association

Y2 - 30 August 2021 through 3 September 2021

ER -

Specmix: a mixed sample data augmentation method for training with time-frequency domain features

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this