Sequential Clique Optimization for Unsupervised and Weakly Supervised Video Object Segmentation

Yeong Jun Koh; Yuk Heo; Chang Su Kim

doi:10.3390/electronics11182899

Sequential Clique Optimization for Unsupervised and Weakly Supervised Video Object Segmentation

Yeong Jun Koh, Yuk Heo, Chang Su Kim

Research output: Contribution to journal › Article › peer-review

1 Citation (Scopus)

Abstract

A novel video object segmentation algorithm, which segments out multiple objects in a video sequence in unsupervised or weakly supervised manners, is proposed in this work. First, we match visually important object instances to construct salient object tracks through a video sequence without any user supervision. We formulate this matching process as the problem to find maximal weight cliques in a complete k-partite graph and develop the sequential clique optimization algorithm to determine the cliques efficiently. Then, we convert the resultant salient object tracks into object segmentation results and refine them based on Markov random field optimization. Second, we adapt the sequential clique optimization algorithm to perform weakly supervised video object segmentation. To this end, we develop a sparse-to-dense network to convert the point cliques into segmentation results. The experimental results demonstrate that the proposed algorithm provides comparable or better performances than recent state-of-the-art VOS algorithms.

Original language	English
Article number	2899
Journal	Electronics (Switzerland)
Volume	11
Issue number	18
DOIs	https://doi.org/10.3390/electronics11182899
Publication status	Published - 2022 Sept

Bibliographical note

Funding Information:
This work was partly supported by the National Research Foundation of Korea (NRF) grants funded by the Korea government (MSIT) (NRF-2021R1A4A1031864), Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2022R1I1A3069113), Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. RS-2022-00155857, Artificial Intelligence Convergence Innovation Human Resources Development (Chungnam National University)), and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01441, Artificial Intelligence Convergence Research Center (Chungnam National University)).

Publisher Copyright:
© 2022 by the authors.

Keywords

convolutional neural networks
primary object segmentation
salient object detection
sequential clique optimization
video object segmentation

ASJC Scopus subject areas

Control and Systems Engineering
Signal Processing
Hardware and Architecture
Computer Networks and Communications
Electrical and Electronic Engineering

Access to Document

10.3390/electronics11182899

Cite this

@article{93ad6d56041b4d178cd7ab4f21076336,

title = "Sequential Clique Optimization for Unsupervised and Weakly Supervised Video Object Segmentation",

abstract = "A novel video object segmentation algorithm, which segments out multiple objects in a video sequence in unsupervised or weakly supervised manners, is proposed in this work. First, we match visually important object instances to construct salient object tracks through a video sequence without any user supervision. We formulate this matching process as the problem to find maximal weight cliques in a complete k-partite graph and develop the sequential clique optimization algorithm to determine the cliques efficiently. Then, we convert the resultant salient object tracks into object segmentation results and refine them based on Markov random field optimization. Second, we adapt the sequential clique optimization algorithm to perform weakly supervised video object segmentation. To this end, we develop a sparse-to-dense network to convert the point cliques into segmentation results. The experimental results demonstrate that the proposed algorithm provides comparable or better performances than recent state-of-the-art VOS algorithms.",

keywords = "convolutional neural networks, primary object segmentation, salient object detection, sequential clique optimization, video object segmentation",

author = "Koh, {Yeong Jun} and Yuk Heo and Kim, {Chang Su}",

note = "Funding Information: This work was partly supported by the National Research Foundation of Korea (NRF) grants funded by the Korea government (MSIT) (NRF-2021R1A4A1031864), Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2022R1I1A3069113), Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. RS-2022-00155857, Artificial Intelligence Convergence Innovation Human Resources Development (Chungnam National University)), and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01441, Artificial Intelligence Convergence Research Center (Chungnam National University)). Publisher Copyright: {\textcopyright} 2022 by the authors.",

year = "2022",

month = sep,

doi = "10.3390/electronics11182899",

language = "English",

volume = "11",

journal = "Electronics (Switzerland)",

issn = "2079-9292",

publisher = "Multidisciplinary Digital Publishing Institute (MDPI)",

number = "18",

}

TY - JOUR

T1 - Sequential Clique Optimization for Unsupervised and Weakly Supervised Video Object Segmentation

AU - Koh, Yeong Jun

AU - Heo, Yuk

AU - Kim, Chang Su

N1 - Funding Information: This work was partly supported by the National Research Foundation of Korea (NRF) grants funded by the Korea government (MSIT) (NRF-2021R1A4A1031864), Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2022R1I1A3069113), Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No. RS-2022-00155857, Artificial Intelligence Convergence Innovation Human Resources Development (Chungnam National University)), and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-01441, Artificial Intelligence Convergence Research Center (Chungnam National University)). Publisher Copyright: © 2022 by the authors.

PY - 2022/9

Y1 - 2022/9

N2 - A novel video object segmentation algorithm, which segments out multiple objects in a video sequence in unsupervised or weakly supervised manners, is proposed in this work. First, we match visually important object instances to construct salient object tracks through a video sequence without any user supervision. We formulate this matching process as the problem to find maximal weight cliques in a complete k-partite graph and develop the sequential clique optimization algorithm to determine the cliques efficiently. Then, we convert the resultant salient object tracks into object segmentation results and refine them based on Markov random field optimization. Second, we adapt the sequential clique optimization algorithm to perform weakly supervised video object segmentation. To this end, we develop a sparse-to-dense network to convert the point cliques into segmentation results. The experimental results demonstrate that the proposed algorithm provides comparable or better performances than recent state-of-the-art VOS algorithms.

AB - A novel video object segmentation algorithm, which segments out multiple objects in a video sequence in unsupervised or weakly supervised manners, is proposed in this work. First, we match visually important object instances to construct salient object tracks through a video sequence without any user supervision. We formulate this matching process as the problem to find maximal weight cliques in a complete k-partite graph and develop the sequential clique optimization algorithm to determine the cliques efficiently. Then, we convert the resultant salient object tracks into object segmentation results and refine them based on Markov random field optimization. Second, we adapt the sequential clique optimization algorithm to perform weakly supervised video object segmentation. To this end, we develop a sparse-to-dense network to convert the point cliques into segmentation results. The experimental results demonstrate that the proposed algorithm provides comparable or better performances than recent state-of-the-art VOS algorithms.

KW - convolutional neural networks

KW - primary object segmentation

KW - salient object detection

KW - sequential clique optimization

KW - video object segmentation

UR - http://www.scopus.com/inward/record.url?scp=85138691281&partnerID=8YFLogxK

U2 - 10.3390/electronics11182899

DO - 10.3390/electronics11182899

M3 - Article

AN - SCOPUS:85138691281

SN - 2079-9292

VL - 11

JO - Electronics (Switzerland)

JF - Electronics (Switzerland)

IS - 18

M1 - 2899

ER -

Sequential Clique Optimization for Unsupervised and Weakly Supervised Video Object Segmentation

Abstract

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this