Tiago Cardoso

A Developmental Coordination Disorder can be identified when children show motor skills either below the expected levels considered adequate to their physical age or the opportunities provided for their learning. This problem affects four to six percent of school-age children, meaning that, from a very early stage of their life, they have several difficulties to adapt to the daily needs. In order to reduce the impact caused by this disorder, a team of therapists from “ – collected a wide range of exercises that allow the stimulus of several motor areas, including both the Gross and Fine Motor Skills. However, the application of this therapeutics is restricted to regular appointments. Since the motor stimulus, in order to be effective, need continuous application, it was found to be necessary to have a tool that in a practical and affordable way, fulfill this need. Therefore, the proposal presented in this article describes the creation of a systematic collection of such exercises in a friendly user manner for the children to be able to exercise elsewhere.

Computers and technology can play a key role in supporting learning, namely for students with special educational needs. Nevertheless, despite the emphasis the society puts on the use of technology and its fast proliferation in the area of education, few initiatives focus on the specific needs of children with disabilities. Motivated by this gap, this research work proposes a computer assisted education application that targets to teach talking and reading through games. The work described herein was carried out in close cooperation with - , a child-care institution that works with children with distinct growth disorders, namely the Down syndrome.

The evolution of technology has risen new possibilities in the so called Natural User Interfaces research area. Among distinct initiatives, several researchers are working with the existing sensors towards improving the support to gesture languages. This article tackles the recognition of gestures, using the Kinect sensor, in order to create a gesture library and support the gesture recognition processes afterwards.

Virtual environments are an increasing trend in today’s society. In this scope, the avatar is the representation of the user in the virtual world. However, that relationship lacks empirical studies regarding the nature of the interaction between avatars and human beings. For that purpose it was studied how the avatar’s modeled morphology and dynamics affect its control by the user. An experiment was conducted to measure telepresence and ownership on participants who used a Kinect Natural User Interface (NUI). The body ownership of different avatars was assessed through a behavioral parameter, based on the concept of affordances, and a questionnaire of presence. The results show that the feelings of telepresence and ownership seem to be greater when the kinematics and the avatar’s proportions are closer to those of the user.

An increasing number of our interactions are mediated through e-technologies. In order to enhance the human’s feeling of presence into these virtual environments, also known as telepresence, the individual is usually embodied into an avatar. The natural adaptation capabilities, underlain by the plasticity of the body schema, of the human being make a body ownership of the avatar possible, in which the user feels more like his/her virtual alter ego than himself/herself. However, this phenomenon only occurs under specific conditions. Two experiments are designed to study the human’s feeling and performance according to a scale of natural relationship between the participant and the avatar. In both experiments, the human-avatar interaction is carried out by a Natural User Interface (NUI) and the individual’s performance is assessed through a behavioural index, based on the concept of affordances, and a questionnaire of presence The first experiment shows that the feeling of telepresence and ownership seem to be greater when the avatar’s kinematics and proportions are close to those of the user. However, the efficiency to complete the task is higher for a more mechanical and stereotypical avatar. The second experiment shows that the manipulation of the viewpoint induces a similar difference across the sessions. Results are discussed in terms of the neurobehavioral processes underlying performance in virtual worlds, which seem to be based on ownership when the virtual artefact ensures a preservation of sensorimotor contingencies, and simple geometrical mapping when the conditions become more artificial.

This book contains the outcome of the 9th IFIP WG 5.5 International Summer Workshop on Multimodal Interfaces, eNTERFACE 2013, held in Lisbon, Portugal, in July/August 2013. The 9 papers included in this book represent the results of a 4-week workshop, where senior and junior researchers worked together on projects tackling new trends in human-machine interaction (HMI). The papers are organized in two topical sections. The first one presents different proposals focused on some fundamental issues regarding multimodal interactions, i.e., telepresence, speech synthesis and interactive modeling. The second is a set of development examples in key areas of HMI applications, i.e., education, entertainment and assistive technologies.

Sign language is the hearing impaired form of communicating with other people, including listeners. Most cases, impaired people have learned sign language form childhood. The problem arises when a listener comes in contact with an impaired person. For instances, if a couple has a child which is impaired, the parents find a challenge to learn the sign language. In this article, a new playful approach to assist the listeners to learn sign language is proposed. This proposal is a serious game composed of two modes: School-mode and Competition-mode. The first offers a virtual school where the user learns to sign letters and the second offers an environment towards applying the learned letters. Behind the scenes, the proposal contains a sign language recognition system, based on three modules: 1 – the standardization of the Kinect depth camera data; 2 – a gesture library relying on the standardized data; and 3 – the real-time recognition of gestures. A prototype was developed – Kinect-Sign – and tested in a Portuguese Sign-Language school and on eNTERFACE’13 resulting in a joyful acceptance of the approach.

The sign language is widely used by deaf people around the globe. As the spoken languages, several sign languages do exist. The way sign language is learned by deaf people may have some details to be improved, but one can state that the existing learning mechanisms are effective when we talk about a deaf child, for example. The problem arises for the non-deaf persons that communicate with the deaf persons – the so-called listeners. If, for example, one couple has a new child that turns to be deaf, these two persons find a challenge to learn the sign language. In one hand, they cannot stop their working life, especially because of this sad news turns to be more costly, on the other hand, the existing mechanisms target the deaf-persons and are not prepared for the listeners. This paper proposes a new playful approach to help these listeners to learn the sign language. The proposal is a serious game composed of two modes: School-mode and Competition-mode. The first provides a school-like environment where the user learns the letter-signs and the second provides the user an environment used towards testing the learned skills. Behind the scenes, the proposal is based on two phases: 1 – the creation of a gestures library, relying on the Kinect depth camera; and 2 – the real-time recognition of gestures, by comparing what the depth camera information to the existing gestures previously stored in the library. A prototype system, supporting only the Portuguese sign language alphabet, was developed – the Kinect-Sign – and tested in a Portuguese Sign-Language school resulting in a joyful acceptance of the approach.

Serious games are usually applied to cognitive learning. However, the development of recent Natural User Interfaces (NUI) enables us to use serious games for a more holistic training. This project proposes to implement an educative game that teaches medical students how to carry out a neurological examination. The purpose is to improve not only the cognitive knowledge of the young physicians but also their sensorimotor skills. The game takes place in a virtual doctor office, which is modelled in 3D. Many possible clinical cases related to different neurologic diseases are simulated. The users are tested at two skill levels. First of all, they have to choose the right exam according to the patient’s symptoms. This initial stage is used to assess the theoretical expertise of the students. If the choice is correct, the physical examination begins. In order to evaluate the motor performance, a Kinect NUI is used. Such a device allows for the natural and intuitive control of the doctor’s avatar because there is a direct link between the individual’s and the avatar’s joints. Due to the fact that the real movements of the student are recorded with some accuracy, it is possible to gauge whether or not the medical gesture is properly applied to the virtual patient. In particular, the force and the precision of the movements are measured and compared to reference values. After each examination immediate feedback is given, so that the learners may rectify their choices and improve the quality of their motor skills at the next training session. Overall, the objective of this software is to enable the future physicians to practice a basic medical exam on a wide scope of neurologic disorders, without any risk for the patients and through an enjoyable approach.