Local Memory Read-and-Comparator for Video Object Segmentation

Yuk Heo, Yeong Jun Koh, Chang Su Kim

Research output: Contribution to journalArticlepeer-review

Abstract

Recently, the memory-based approach, which performs non-local matching between previously segmented frames and a query frame, has led to significant improvement in video object segmentation. However, the positional proximity of the target objects between the query and the local memory (previous frame), i.e. temporal smoothness, is often neglected. There are some attempts to solve the problem, but they are vulnerable and sensitive to large movements of target objects. In this paper, we propose local memory read-and-compare operations to address the problem. First, we propose local memory read and sequential local memory read modules to explore temporal smoothness between neighboring frames. Second, we propose the memory comparator to read the global memory and local memory adaptively by comparing the affinities of the global and local memories. Experimental results demonstrate that the proposed algorithm yields more strict segmentation results than the recent state-of-the-art algorithms. For example, the proposed algorithm improves the video object segmentation performance by 0.4% and 0.5% in terms of J\F on the most commonly used datasets, DAVIS2016 and DAVIS2017, respectively.

Original languageEnglish
Pages (from-to)90004-90016
Number of pages13
JournalIEEE Access
Volume10
DOIs
Publication statusPublished - 2022

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea (NRF) funded by the Korean Government (MSIT) under Grant NRF-2021R1A4A1031864, Grant NRF-2022R1A2B5B03002310, and Grant NRF-2022R1I1A3069113.

Publisher Copyright:
© 2013 IEEE.

Keywords

  • Memory network
  • semi-supervised video object segmentation
  • video object segmentation

ASJC Scopus subject areas

  • General Computer Science
  • General Materials Science
  • General Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Local Memory Read-and-Comparator for Video Object Segmentation'. Together they form a unique fingerprint.

Cite this