Reconstructing 3D human body pose from stereo image sequences using hierarchical human body model learning

This paper presents a novel method for reconstructing a 3D human body pose using depth information based on top-down learning. The human body pose is represented by a linear combination of prototypes of 2D depth images and their corresponding 3D body models in terms of the position of a predetermined set of joints. In a 2D depth image, the optimal coefficients for a linear combination of prototypes of 2D depth images can be estimated using least square minimization. The 3D body model of the input depth image is obtained by applying the estimated coefficients to the corresponding 3D body model of prototypes. In the learning stage, the proposed method is hierarchically constructed by classifying the training data recursively into several clusters with silhouette images and depth images. In applying hierarchical human body model learning to estimate 3D human body pose, the similar pose in a silhouette image can be estimated as a different 3D human body pose. The proposed method has been tested with 20 persons' sequences. The proposed method achieved the average errors Of 12.3 degree for all human body components.