Towards Text-based Human Search and Approach using a Robot Dog

In this paper, we propose a SOCratic model for Robots Approaching humans based on TExt System (SOCRATES) focusing on the human search and approach based on free-form textual description; the robot first searches for the target user, then the robot proceeds to approach in a human-friendly manner. We present a Socratic human search model that connects large pre-trained foundation models to solve the downstream task of searching for the target person based on textual descriptions. In particular, textual descriptions used for searching are composed of appearance (e.g., wearing white shirt with black hair) and location clues (e.g., a student that works with robots). Additionally, we propose a hybrid learning-based framework for generating human-friendly robotic motion to approach a person, consisting of a learning-from-demonstration module and a knowledge distillation module utilizing LLMs. We evaluate the search performance of the proposed method in both simulation and real-world environments using the Boston Dynamics Spot robot. Moreover, we evaluate the effectiveness of our proposed framework through analysis involving human participants and investigate the perceived warmth of our system.