Design and Analysis of Virtual Musical Environments (VME)

This paper proposes and demonstrates the concept of a “Virtual Musical Environment (VME)”, a generalization of a virtual music instrument, in which the virtual world itself acts as a source of multimodal feedback, and at the same time, a place of interaction. Naturally, the two important issues in designing a VME are: the display content and the control interface. These two issues are in fact inter-related as music performance is seen as a closed loop system composed of the user and the VME. The control interface, that changes the world, must be designed to be as natural and easy-to-use as possible for quick responses to the on-going music. The display must also be “vivid” in the sense that it must leave the user with a strong musical impression so that one remembers the “essence” of the musical content. We discuss some of our hypothesis on various important elements in designing an effective VME. Exploitation of these elements necessitates a highly controllable user interface with a simplified performance task to free the user from having to worry about the playing skills. We present our design of a VME, and report on the results of the usability experiments we carried out to test the controllability of the system.