Markerless 3d face tracking

Christian Walder, Martin Breidt, Heinrich Bülthoff, Bernhard Schölkopf, Cristóbal Curio

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)


We present a novel algorithm for the markerless tracking of deforming surfaces such as faces. We acquire a sequence of 3D scans along with color images at 40Hz. The data is then represented by implicit surface and color functions, using a novel partition-of-unity type method of efficiently combining local regressors using nearest neighbor searches. Both these functions act on the 4D space of 3D plus time, and use temporal information to handle the noise in individual scans. After interactive registration of a template mesh to the first frame, it is then automatically deformed to track the scanned surface, using the variation of both shape and color as features in a dynamic energy minimization problem. Our prototype system yields high-quality animated 3D models in correspondence, at a rate of approximately twenty seconds per timestep. Tracking results for faces and other objects are presented.

Original languageEnglish
Title of host publicationPattern Recognition - 31st DAGM Symposium, Proceedings
Number of pages10
Publication statusPublished - 2009
Event31st Annual Symposium of the Deutsche Arbeitsgemeinschaft fur Mustererkennung, DAGM 2009 - Jena, Germany
Duration: 2009 Sept 92009 Sept 11

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5748 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Other31st Annual Symposium of the Deutsche Arbeitsgemeinschaft fur Mustererkennung, DAGM 2009

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Markerless 3d face tracking'. Together they form a unique fingerprint.

Cite this