UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection

Bumsoo Kim; Taeho Choi; Jaewoo Kang; Hyunwoo J. Kim

doi:10.1007/978-3-030-58555-6_30

UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection

Bumsoo Kim, Taeho Choi, Jaewoo Kang, Hyunwoo J. Kim

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

76 Citations (Scopus)

Abstract

Recent advances in deep neural networks have achieved significant progress in detecting individual objects from an image. However, object detection is not sufficient to fully understand a visual scene. Towards a deeper visual understanding, the interactions between objects, especially humans and objects are essential. Most prior works have obtained this information with a bottom-up approach, where the objects are first detected and the interactions are predicted sequentially by pairing the objects. This is a major bottleneck in HOI detection inference time. To tackle this problem, we propose UnionDet, a one-stage meta-architecture for HOI detection powered by a novel union-level detector that eliminates this additional inference stage by directly capturing the region of interaction. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time (4 × ∼ 14 ×) while outperforming state-of-the-art methods on two public datasets: V-COCO and HICO-DET.

Original language	English
Title of host publication	Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings
Editors	Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	498-514
Number of pages	17
ISBN (Print)	9783030585549
DOIs	https://doi.org/10.1007/978-3-030-58555-6_30
Publication status	Published - 2020
Event	16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom Duration: 2020 Aug 23 → 2020 Aug 28

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12360 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th European Conference on Computer Vision, ECCV 2020
Country/Territory	United Kingdom
City	Glasgow
Period	20/8/23 → 20/8/28

Bibliographical note

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.

Keywords

Human-object interaction detection
Object detection
Real-time detection
Visual relationships

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-58555-6_30

Cite this

Kim, B., Choi, T., Kang, J., & Kim, H. J. (2020). UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection. In A. Vedaldi, H. Bischof, T. Brox, & J.-M. Frahm (Eds.), Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings (pp. 498-514). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12360 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-58555-6_30

UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection. / Kim, Bumsoo; Choi, Taeho; Kang, Jaewoo et al.
Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. ed. / Andrea Vedaldi; Horst Bischof; Thomas Brox; Jan-Michael Frahm. Springer Science and Business Media Deutschland GmbH, 2020. p. 498-514 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12360 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kim, B, Choi, T, Kang, J & Kim, HJ 2020, UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection. in A Vedaldi, H Bischof, T Brox & J-M Frahm (eds), Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12360 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 498-514, 16th European Conference on Computer Vision, ECCV 2020, Glasgow, United Kingdom, 20/8/23. https://doi.org/10.1007/978-3-030-58555-6_30

Kim B, Choi T, Kang J , Kim HJ. UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection. In Vedaldi A, Bischof H, Brox T, Frahm JM, editors, Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 498-514. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-58555-6_30

Kim, Bumsoo ; Choi, Taeho ; Kang, Jaewoo et al. / UnionDet : Union-Level Detector Towards Real-Time Human-Object Interaction Detection. Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings. editor / Andrea Vedaldi ; Horst Bischof ; Thomas Brox ; Jan-Michael Frahm. Springer Science and Business Media Deutschland GmbH, 2020. pp. 498-514 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{633a66fd0ce442a2a17f2a070900d436,

title = "UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection",

abstract = "Recent advances in deep neural networks have achieved significant progress in detecting individual objects from an image. However, object detection is not sufficient to fully understand a visual scene. Towards a deeper visual understanding, the interactions between objects, especially humans and objects are essential. Most prior works have obtained this information with a bottom-up approach, where the objects are first detected and the interactions are predicted sequentially by pairing the objects. This is a major bottleneck in HOI detection inference time. To tackle this problem, we propose UnionDet, a one-stage meta-architecture for HOI detection powered by a novel union-level detector that eliminates this additional inference stage by directly capturing the region of interaction. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time (4 × ∼ 14 ×) while outperforming state-of-the-art methods on two public datasets: V-COCO and HICO-DET.",

keywords = "Human-object interaction detection, Object detection, Real-time detection, Visual relationships",

author = "Bumsoo Kim and Taeho Choi and Jaewoo Kang and Kim, {Hyunwoo J.}",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 16th European Conference on Computer Vision, ECCV 2020 ; Conference date: 23-08-2020 Through 28-08-2020",

year = "2020",

doi = "10.1007/978-3-030-58555-6_30",

language = "English",

isbn = "9783030585549",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "498--514",

editor = "Andrea Vedaldi and Horst Bischof and Thomas Brox and Jan-Michael Frahm",

booktitle = "Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings",

address = "Germany",

}

TY - GEN

T1 - UnionDet

T2 - 16th European Conference on Computer Vision, ECCV 2020

AU - Kim, Bumsoo

AU - Choi, Taeho

AU - Kang, Jaewoo

AU - Kim, Hyunwoo J.

PY - 2020

Y1 - 2020

N2 - Recent advances in deep neural networks have achieved significant progress in detecting individual objects from an image. However, object detection is not sufficient to fully understand a visual scene. Towards a deeper visual understanding, the interactions between objects, especially humans and objects are essential. Most prior works have obtained this information with a bottom-up approach, where the objects are first detected and the interactions are predicted sequentially by pairing the objects. This is a major bottleneck in HOI detection inference time. To tackle this problem, we propose UnionDet, a one-stage meta-architecture for HOI detection powered by a novel union-level detector that eliminates this additional inference stage by directly capturing the region of interaction. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time (4 × ∼ 14 ×) while outperforming state-of-the-art methods on two public datasets: V-COCO and HICO-DET.

AB - Recent advances in deep neural networks have achieved significant progress in detecting individual objects from an image. However, object detection is not sufficient to fully understand a visual scene. Towards a deeper visual understanding, the interactions between objects, especially humans and objects are essential. Most prior works have obtained this information with a bottom-up approach, where the objects are first detected and the interactions are predicted sequentially by pairing the objects. This is a major bottleneck in HOI detection inference time. To tackle this problem, we propose UnionDet, a one-stage meta-architecture for HOI detection powered by a novel union-level detector that eliminates this additional inference stage by directly capturing the region of interaction. Our one-stage detector for human-object interaction shows a significant reduction in interaction prediction time (4 × ∼ 14 ×) while outperforming state-of-the-art methods on two public datasets: V-COCO and HICO-DET.

KW - Human-object interaction detection

KW - Object detection

KW - Real-time detection

KW - Visual relationships

UR - http://www.scopus.com/inward/record.url?scp=85097420199&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-58555-6_30

DO - 10.1007/978-3-030-58555-6_30

M3 - Conference contribution

AN - SCOPUS:85097420199

SN - 9783030585549

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 498

EP - 514

BT - Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings

A2 - Vedaldi, Andrea

A2 - Bischof, Horst

A2 - Brox, Thomas

A2 - Frahm, Jan-Michael

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 August 2020 through 28 August 2020

ER -

UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this