CAPTCHA-based image annotation

Although crowdsourcing-based image annotation services are accurate, they easily become too costly to assign proper labels on all the images on the Internet. In this paper, we propose a practical, efficient, and accurate CAPTCHA-based image annotation technique that requires no or little additional cost. If challenge sessions can be conducted such that bots are successfully defeated and only humans are likely to pass, inclusion of unlabeled images into the challenges and analysis of successful responses offer benefit equivalent to obtaining free and motivated crowdsourcing services. We conducted an experiment using 25 individuals in order to evaluate the effectiveness of the approach. Results are highly positive except that some participants occasionally made, as expected mistakes. In order to further improve accuracy of image labeling, crosschecking mechanism was introduced to effectively eliminate the risk of potential human errors.