Temporal action localization (TAL) is a task of identifying a set of actions in a video, which involves localizing the start and end frames and classifying each action instance. Existing methods have addressed this task by using predefined anchor windows or heuristic bottom-up boundary-matching strategies, which are major bottlenecks in inference time. Additionally, the main challenge is the inability to capture long-range actions due to a lack of global contextual information. In this paper, we present a novel anchor-free framework, referred to as HTNet, which predicts a set of langlestart time, end time, classrangle triplets from a video based on a Transformer architecture. After the prediction of coarse boundaries, we refine it through a background feature sampling (BFS) module and hierarchical Transformers, which enables our model to aggregate global contextual information and effectively exploit the inherent semantic relationships in a video. We demonstrate how our method localizes accurate action instances and achieves state-of-the-art performance on two TAL benchmark datasets: THUMOS14 and ActivityNet 1.3.
|Title of host publication||2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 - Proceedings|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||6|
|Publication status||Published - 2022|
|Event||2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 - Prague, Czech Republic|
Duration: 2022 Oct 9 → 2022 Oct 12
|Name||Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics|
|Conference||2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022|
|Period||22/10/9 → 22/10/12|
Bibliographical noteFunding Information:
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. B0101-15-0266, Development of High Performance Visual BigData Discovery Platform for Large-Scale Realtime Data Analysis, No. 2021-0-02068, Artificial Intelligence Innovation Hub).
© 2022 IEEE.
- Context Aggregation
- Temporal Action Detection
- Temporal Action Localization
ASJC Scopus subject areas
- Electrical and Electronic Engineering
- Control and Systems Engineering
- Human-Computer Interaction