Spatiotemporal saliency detection for video sequences based on random walk with restart

Hansang Kim, Youngbae Kim, Jae Young Sim, Chang-Su Kim

    Research output: Contribution to journalArticlepeer-review

    149 Citations (Scopus)

    Abstract

    A novel saliency detection algorithm for video sequences based on the random walk with restart (RWR) is proposed in this paper. We adopt RWR to detect spatially and temporally salient regions. More specifically, we first find a temporal saliency distribution using the features of motion distinctiveness, temporal consistency, and abrupt change. Among them, the motion distinctiveness is derived by comparing the motion profiles of image patches. Then, we employ the temporal saliency distribution as a restarting distribution of the random walker. In addition, we design the transition probability matrix for the walker using the spatial features of intensity, color, and compactness. Finally, we estimate the spatiotemporal saliency distribution by finding the steady-state distribution of the walker. The proposed algorithm detects foreground salient objects faithfully, while suppressing cluttered backgrounds effectively, by incorporating the spatial transition matrix and the temporal restarting distribution systematically. Experimental results on various video sequences demonstrate that the proposed algorithm outperforms conventional saliency detection algorithms qualitatively and quantitatively.

    Original languageEnglish
    Article number7091884
    Pages (from-to)2552-2564
    Number of pages13
    JournalIEEE Transactions on Image Processing
    Volume24
    Issue number8
    DOIs
    Publication statusPublished - 2015 Aug 1

    Bibliographical note

    Publisher Copyright:
    © 1992-2012 IEEE.

    Keywords

    • Saliency detection
    • motion profile
    • random walk with restart
    • spatiotemporal feature
    • video saliency

    ASJC Scopus subject areas

    • Software
    • Computer Graphics and Computer-Aided Design

    Fingerprint

    Dive into the research topics of 'Spatiotemporal saliency detection for video sequences based on random walk with restart'. Together they form a unique fingerprint.

    Cite this