GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data

Hongjae Lee; Changwoo Han; Jun Sang Yoo; Seung Won Jung

doi:10.1109/ICCVW60793.2023.00432

GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data

Hongjae Lee
, Changwoo Han
, Jun Sang Yoo
, Seung Won Jung^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

5 Citations (Scopus)

Abstract

Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets.

Original language	English
Title of host publication	Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	4003-4012
Number of pages	10
ISBN (Electronic)	9798350307443
DOIs	https://doi.org/10.1109/ICCVW60793.2023.00432
Publication status	Published - 2023
Event	19th IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 - Paris, France Duration: 2023 Oct 2 → 2023 Oct 6

Publication series

Name	Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023

Conference

Conference	19th IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023
Country/Territory	France
City	Paris
Period	23/10/2 → 23/10/6

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Deep learning
Domain adaptation
GPS
Optical flow
Semantic segmentation

ASJC Scopus subject areas

Artificial Intelligence
Computer Science Applications
Computer Vision and Pattern Recognition

Access to Document

10.1109/ICCVW60793.2023.00432

Cite this

Lee, H., Han, C., Yoo, J. S., & Jung, S. W. (2023). GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data. In Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 (pp. 4003-4012). (Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCVW60793.2023.00432

GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data. / Lee, Hongjae; Han, Changwoo; Yoo, Jun Sang et al.
Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. p. 4003-4012 (Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Lee, H, Han, C, Yoo, JS & Jung, SW 2023, GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data. in Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023. Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023, Institute of Electrical and Electronics Engineers Inc., pp. 4003-4012, 19th IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023, Paris, France, 23/10/2. https://doi.org/10.1109/ICCVW60793.2023.00432

Lee H, Han C, Yoo JS, Jung SW. GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data. In Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 4003-4012. (Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023). doi: 10.1109/ICCVW60793.2023.00432

Lee, Hongjae ; Han, Changwoo ; Yoo, Jun Sang et al. / GPS-GLASS : Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data. Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 4003-4012 (Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023).

@inproceedings{20c0af8e057d4a498527ec64b8c409dd,

title = "GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data",

abstract = "Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets.",

keywords = "Deep learning, Domain adaptation, GPS, Optical flow, Semantic segmentation",

author = "Hongjae Lee and Changwoo Han and Yoo, \{Jun Sang\} and Jung, \{Seung Won\}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 19th IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023 ; Conference date: 02-10-2023 Through 06-10-2023",

year = "2023",

doi = "10.1109/ICCVW60793.2023.00432",

language = "English",

series = "Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "4003--4012",

booktitle = "Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023",

}

TY - GEN

T1 - GPS-GLASS

T2 - 19th IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023

AU - Lee, Hongjae

AU - Han, Changwoo

AU - Yoo, Jun Sang

AU - Jung, Seung Won

PY - 2023

Y1 - 2023

N2 - Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets.

AB - Semantic segmentation for autonomous driving should be robust against various in-the-wild environments. Nighttime semantic segmentation is especially challenging due to a lack of annotated nighttime images and a large domain gap from daytime images with sufficient annotation. In this paper, we propose a novel GPS-based training framework for nighttime semantic segmentation. Given GPS-aligned pairs of daytime and nighttime images, we perform cross-domain correspondence matching to obtain pixel-level pseudo supervision. Moreover, we conduct flow estimation between daytime video frames and apply GPS-based scaling to acquire another pixel-level pseudo supervision. Using these pseudo supervisions with a confidence map, we train a nighttime semantic segmentation network without any annotation from nighttime images. Experimental results demonstrate the effectiveness of the proposed method on several nighttime semantic segmentation datasets.

KW - Deep learning

KW - Domain adaptation

KW - GPS

KW - Optical flow

KW - Semantic segmentation

UR - https://www.scopus.com/pages/publications/85182924375

U2 - 10.1109/ICCVW60793.2023.00432

DO - 10.1109/ICCVW60793.2023.00432

M3 - Conference contribution

AN - SCOPUS:85182924375

T3 - Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023

SP - 4003

EP - 4012

BT - Proceedings - 2023 IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2023

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 2 October 2023 through 6 October 2023

ER -

GPS-GLASS: Learning Nighttime Semantic Segmentation Using Daytime Video and GPS data

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this