Touchless user interface based on marker detection and tracking for real-time mobile applications

A touchless user interaction system based on marker detection and tracking is proposed for real-time mobile applications. The proposed algorithm can robustly estimate users' control information with a camera on a mobile device, which often has limited hardware resources and is influenced by varying environmental conditions. First, we detect a pre-registered marker based on the normalized correlation coefficient. Then, we track the marker motion by employing the contrast invariant mean-shift algorithm. More specifically, the proposed contrast invariant mean-shift algorithm transforms a candidate frame in order to match its histogram into the histogram of the target frame. It then tracks feature points by performing the mean-shift on both original and transformed candidate frames adaptively. To evaluate the performance of the proposed interaction system, we implement 'Painting', 'Camera' and 'Virtual Keypad' applications. Experimental results demonstrate that the proposed algorithm provides better interaction performance than the conventional method, while demanding lower computational complexity and thus supporting real-time user interaction.