TY - GEN
T1 - Streaming video segmentation via short-term hierarchical segmentation and frame-by-frame Markov random field optimization
AU - Jang, Won Dong
AU - Kim, Chang-Su
N1 - Funding Information:
This work was supported partly by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (No. NRF-2015R1A2A1A10055037), and partly by the MSIP, Korea, under the ITRC support program supervised by the Institute for Information & communications Technology Promotion (No. IITP-2016-R2720-16-0007).
Publisher Copyright:
© Springer International Publishing AG 2016.
PY - 2016
Y1 - 2016
N2 - An online video segmentation algorithm, based on shortterm hierarchical segmentation (STHS) and frame-by-frame Markov random field (MRF) optimization, is proposed in this work. We develop the STHS technique, which generates initial segments by sliding a short window of frames. In STHS, we apply spatial agglomerative clustering to each frame, and then adopt inter-frame bipartite graph matching to construct initial segments. Then, we partition each frame into final segments, by minimizing an MRF energy function composed of unary and pair wise costs. We compute the unary cost using the STHS initial segments and the segmentation result at the previous frame. We set the pair wise cost to encourage similar nodes to have the same segment label. Experimental results on a video segmentation benchmark dataset, VSB100, demonstrate that the proposed algorithm outperforms state-of-the-art online video segmentation techniques significantly.
AB - An online video segmentation algorithm, based on shortterm hierarchical segmentation (STHS) and frame-by-frame Markov random field (MRF) optimization, is proposed in this work. We develop the STHS technique, which generates initial segments by sliding a short window of frames. In STHS, we apply spatial agglomerative clustering to each frame, and then adopt inter-frame bipartite graph matching to construct initial segments. Then, we partition each frame into final segments, by minimizing an MRF energy function composed of unary and pair wise costs. We compute the unary cost using the STHS initial segments and the segmentation result at the previous frame. We set the pair wise cost to encourage similar nodes to have the same segment label. Experimental results on a video segmentation benchmark dataset, VSB100, demonstrate that the proposed algorithm outperforms state-of-the-art online video segmentation techniques significantly.
KW - Agglomerative clustering
KW - Graph matching
KW - Online segmentation
KW - Streaming segmentation
KW - Video segmentation
UR - http://www.scopus.com/inward/record.url?scp=84990062971&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-46466-4_36
DO - 10.1007/978-3-319-46466-4_36
M3 - Conference contribution
AN - SCOPUS:84990062971
SN - 9783319464657
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 599
EP - 615
BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings
A2 - Leibe, Bastian
A2 - Matas, Jiri
A2 - Sebe, Nicu
A2 - Welling, Max
PB - Springer Verlag
T2 - 14th European Conference on Computer Vision, ECCV 2016
Y2 - 8 October 2016 through 16 October 2016
ER -