TY - GEN
T1 - Markerless 3d face tracking
AU - Walder, Christian
AU - Breidt, Martin
AU - Bülthoff, Heinrich
AU - Schölkopf, Bernhard
AU - Curio, Cristóbal
PY - 2009
Y1 - 2009
N2 - We present a novel algorithm for the markerless tracking of deforming surfaces such as faces. We acquire a sequence of 3D scans along with color images at 40Hz. The data is then represented by implicit surface and color functions, using a novel partition-of-unity type method of efficiently combining local regressors using nearest neighbor searches. Both these functions act on the 4D space of 3D plus time, and use temporal information to handle the noise in individual scans. After interactive registration of a template mesh to the first frame, it is then automatically deformed to track the scanned surface, using the variation of both shape and color as features in a dynamic energy minimization problem. Our prototype system yields high-quality animated 3D models in correspondence, at a rate of approximately twenty seconds per timestep. Tracking results for faces and other objects are presented.
AB - We present a novel algorithm for the markerless tracking of deforming surfaces such as faces. We acquire a sequence of 3D scans along with color images at 40Hz. The data is then represented by implicit surface and color functions, using a novel partition-of-unity type method of efficiently combining local regressors using nearest neighbor searches. Both these functions act on the 4D space of 3D plus time, and use temporal information to handle the noise in individual scans. After interactive registration of a template mesh to the first frame, it is then automatically deformed to track the scanned surface, using the variation of both shape and color as features in a dynamic energy minimization problem. Our prototype system yields high-quality animated 3D models in correspondence, at a rate of approximately twenty seconds per timestep. Tracking results for faces and other objects are presented.
UR - http://www.scopus.com/inward/record.url?scp=70350437838&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-03798-6_5
DO - 10.1007/978-3-642-03798-6_5
M3 - Conference contribution
AN - SCOPUS:70350437838
SN - 3642037976
SN - 9783642037979
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 41
EP - 50
BT - Pattern Recognition - 31st DAGM Symposium, Proceedings
T2 - 31st Annual Symposium of the Deutsche Arbeitsgemeinschaft fur Mustererkennung, DAGM 2009
Y2 - 9 September 2009 through 11 September 2009
ER -