Human Interface Engineering Laboratory, Osaka University
仮想道具によるインタフェース
|
Virtual Tools
|
[Major publications related to this topic]
A conventional glove-type hand gesture input device is used to measure
the motion of the user’s hand and fingers while handling the virtual chopsticks.
But if a one-to-one mapping between the corresponding joint angles is used,
the output motion is not always the same as the input motion because of
measurement errors, differences of skeletal structures, and so on. The
problem is even more serious when the motion data is used in an interactive
environment such as a virtual reality system. The motion that a user imagines
(mental model) is deviated from the reproduced motion (system behavior),
and this seriously degrades the intuitiveness, making the user interface
hard to use.
Therefore, we propose a technique to use a multiple regression formula
that establishes the relationship between multiple motion data sequences
measured by joint angle sensors of a hand gesture input device and the
status of the hand gesture while operating the chopsticks. Multiple regression
analysis is a method of representing the relationship between the predictor
variables and the criterion variable. Here, the joint angles measured by
a hand gesture input device are used as the predictor variables, and the
distance between the tips of the chopsticks is used to estimate the status
of the task of using chopsticks.
Once the multiple regression formula is established for each user, this
method should enable even users who cannot handle real chopsticks properly
to operate the virtual chopsticks in a way that matches his/her mental
image obtained from multiple joint angles, which are measured as a finger
motion. Therefore, this formula is used to represent the basic chopsticks.
With this method, exact calibration is not necessary, even for a group
of users who have a variety of skeletal structures.
Generating sign language by using the user’s intuitive hand action might
be a promising application. Application of the proposed method using multiple
regression analysis is not limited to operatingl chopsticks. It can also
be applied to the motion of other hand gestures and/or any body part that
can be represented by a few parameters.
|
The process while humans learn to use new tools can be analyzed by using the virtual chopsticks. Here, it is necessary to use a framework in which the degree of completeness of the user’s mental model can be easily controlled while the fundamentally invariant interface remains. For this purpose, virtual chopsticks can be used as a basic tool, and new tools having several levels of difficulty are developed based on the basic chopsticks. Although chopsticks are one of the most commonly used tools, handling chopsticks is sometimes considered to be a complicated and difficult task to do properly. Based on the basic chopsticks, three different types of new tools are generated by changing the coefficients in multiple regression formula, each with its own level of difficulty, using a standardized interface based on a glove-type hand gesture input device. |
 |
|
| basic chopsticks | reverse chopsticks | |
|
The process of humans learning to use new tools is analyzed with two different
approaches, i.e., a performance evaluation that directly observes the human
during interactions and an indirect observation of human brain activity
by fMRI. By comparing the results, we found that the observation of human
brain activity by fMRI is a useful method of evaluating the human interface
systems. Although there are many uncontrolled factors and differences between
experiments in an ordinary laboratory and in fMRI system, examination of
these differences and, if possible, comparison of their effect on brain
activities, remain interesting problems for both brain science and human-computer
interaction. |
 |
|
| velocity chopsticks | acceleration chopsticks | |
We also propose a model of representing a force feedback in manipulating virtual objects with virtual chopsticks. The model is established by using a leverage based on the correct chopsticks handling manner and the force is applied mainly to the index and middle finger.
|
Our virtual chopsticks project appeared on the front page of Mainichi newspaper |
|
|
|
|
We also propose a system of virtual chopsticks that enables a user to manipulate virtual soft objects intuitively. In this system, the object is defined by a linear FEM (finite element method) model. |
Other research projects in Osaka University Human Interface Engineering
Lab.
