Virtual humans are most natural and effective when it can act out and animate verbal/gestural actions. One popular method to realize this is to infer the actions from predefined phrases. This research aims to provide a more flexible method to activate various behaviors straight from natural conversations. Our approach uses BERT as the backbone for natural language understanding and, on top of it, a jointly learned sentence classifier (SC) and entity classifier (EC). The SC classifies the input into conversation or action, and EC extracts the entities for the action. The pilot study has shown promising results with high perceived naturalness and positive experiences.
|Title of host publication||Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||2|
|Publication status||Published - 2021 Mar|
|Event||2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021 - Virtual, Lisbon, Portugal|
Duration: 2021 Mar 27 → 2021 Apr 3
|Name||Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021|
|Conference||2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021|
|Period||21/3/27 → 21/4/3|
Bibliographical noteFunding Information:
H. Kim and G. Ali contributed equally to this paper. This work has supported by the National Research Council of Science & Technology grant by the Korea Government (MSIP CRC-20-02-KIST).
© 2021 IEEE.
- Computing methodologies
- Human-centered computing
- Interaction design process and methods
- Natural language processing
- Virtual reality
ASJC Scopus subject areas
- Human-Computer Interaction
- Media Technology
- Modelling and Simulation