Pose-Guided Graph Convolutional Networks for Skeleton-Based Action Recognition

Han Chen, Yifan Jiang, Hanseok Ko

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Graph convolutional networks (GCN), which can model the human body skeletons as spatial and temporal graphs, have shown remarkable potential in skeleton-based action recognition. However, in the existing GCN-based methods, graph-structured representation of the human skeleton makes it difficult to be fused with other modalities, especially in the early stages. This may limit their scalability and performance in action recognition tasks. In addition, the pose information, which naturally contains informative and discriminative clues for action recognition, is rarely explored together with skeleton data in existing methods. In this work, we proposed pose-guided GCN (PG-GCN), a multi-modal framework for high-performance human action recognition. In particular, a multi-stream network is constructed to simultaneously explore the robust features from both the pose and skeleton data, while a dynamic attention module is designed for early-stage feature fusion. The core idea of this module is to utilize a trainable graph to aggregate features from the skeleton stream with that of the pose stream, which leads to a network with more robust feature representation ability. Extensive experiments show that the proposed PG-GCN can achieve state-of-the-art performance on the NTU RGB+D 60 and NTU RGB+D 120 datasets.

Original languageEnglish
Pages (from-to)111725-111731
Number of pages7
JournalIEEE Access
Volume10
DOIs
Publication statusPublished - 2022

Bibliographical note

Publisher Copyright:
© 2013 IEEE.

Keywords

  • Action recognition
  • attention mechanism
  • feature fusion
  • graph convolutional networks
  • human skeleton
  • pose information

ASJC Scopus subject areas

  • General Engineering
  • General Materials Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Pose-Guided Graph Convolutional Networks for Skeleton-Based Action Recognition'. Together they form a unique fingerprint.

Cite this