Towards New Mappings between Emotion Representation Models

There are several models for representing emotions in affect-aware applications, and available emotion recognition solutions provide results using diverse emotion models. As multimodal fusion is beneficial in terms of both accuracy and reliability of emotion recognition, one of the challenges is mapping between the models of affect representation. This paper addresses this issue by: proposing a procedure to elaborate new mappings, recommending a set of metrics for evaluation of the mapping accuracy, and delivering new mapping matrices for estimating the dimensions of a Pleasure-Arousal-Dominance model from Ekman’s six basic emotions. The results are based on an analysis using three datasets that were constructed based on affect-annotated lexicons. The new mappings were obtained with linear regression learning methods. The proposed mappings showed better results on the datasets in comparison with the state-of-the-art matrix. The procedure, as well as the proposed metrics, might be used, not only in evaluation of the mappings between representation models, but also in comparison of emotion recognition and annotation results. Moreover, the datasets are published along with the paper and new mappings might be created and evaluated using the proposed methods. The study results might be interesting for both researchers and developers, who aim to extend their software solutions with affect recognition techniques.

Which digital games are appropriate for our children?

A. Landowska – 2018
Chapter contains advice on which digital games are accessible for children with diverse impairments. It providess a process for choosing the device, the game and adjusting it. It might be of use for therapists, techers and caregivers.

Automatic recognition of therapy progress among children with autism

A. Kołakowska , A. Landowska , A. Anzulewicz, K. Sobota – Scientific Reports – 2017
The article presents a research study on recognizing therapy progress among children with autism spectrum disorder. The progress is recognized on the basis of behavioural data gathered via five specially designed tablet games. Over 180 distinct parameters are calculated on the basis of raw data delivered via the game flow and tablet sensors - i.e. touch screen, accelerometer and gyroscope. The results obtained confirm the possibility of recognizing progress in particular areas of development. The recognition accuracy exceeds 80%. Moreover, the study identifies a subset of parameters which appear to be better predictors of therapy progress than others. The proposed method - consisting of data recording, parameter calculation formulas and prediction models - might be implemented in a tool to support both therapists and parents of autistic children. Such a tool might be used to monitor the course of the therapy, modify it and report its results.

Badania nad e-technologiami wspierającymi terapię autyzmu

A. Landowska Pismo PG – 2017
Artykuł opisuje badania i osiągnięcia we wspomaganiu automatycznego pomiaru postępów dzieci z autyzmem. Badania te zostały zrealizowane w ramach projektu AUTMON realizowanego na Politechnice Gdańskiej, WETI w latach 2015-2017

Behaviometrics of Digital Games for Children with Autism Spectrum Disorder

Te paper reports current stage of the project Automated Terapy Monitoring for Children with Developmental Disorders of Autism Spectrum (AUTMON), that aims at development of methods and tools to allow for the automatic evaluation of the therapy progress among children with autism. Finding objective measures suitable for evaluating therapy progress would let create a system supporting those who diagnose autism and the therapists working with the children. In future these measures could be also applied as optimization criteria in defning the optimal therapy path. Such tool could be helpful in preparing the therapy plan, choosing the type of tasks and their frequency. It might also follow the therapy course, predict its direction and indicate the points which probably require intervention and changing the plan. Obviously it would never replace a therapist, but make his work more effective. Moreover, it could also support the parents in their efforts to objectively report valuable observations to the therapists, which would be especially important in the case of a limited access to therapy centers.

Investigation of educational processes with affective computing methods

This paper concerns the monitoring of educational processes with the use of new technologies for the recognition of human emotions. This paper summarizes results from three experiments, aimed at the validation of applying emotion recognition to e-learning. An analysis of the experiments’ executions provides an evaluation of the emotion elicitation methods used to monitor learners. The comparison of affect recognition algorithms was based on the criteria of availability, accuracy, robustness to disturbance, and interference with the e-learning process. The lessons learned in these experiments might be of interest to teachers and e-learning tutors, as well as to those researchers who want to use affective computing methods in monitoring educational processes.

Limitations of Emotion Recognition from Facial Expressions in e-Learning Context

The paper concerns technology of automatic emotion recognition applied in e-learning environment. During a study of e-learning process the authors applied facial expressions observation via multiple video cameras. Preliminary analysis of the facial expressions using automatic emotion recognition tools revealed several unexpected results, including unavailability of recognition due to face coverage and significant inconsistency between the results obtained from two cameras. The paper presents the experiment on e-learning process and summarizes the observations that constitute limitations of emotion recognition from facial expressions applied in e-learning context. The paper might be of interest to researchers and practitioners who consider automatic emotion recognition as an option in monitoring e-learning processes.

Postrzeganie inwazyjności automatycznego rozpoznawania emocji w kontekście edukacyjnym

Artykuł dotyczy aktualnych trendów w konstrukcji inteligentnych systemów edukacyjnych, uwzględniających stan emocjonalny ucznia w doborze ścieżki nauczania. Artykuł przedstawia założenia i rezultaty badania ankietowego, którego celem było określenie, jak zastosowanie automatycznego rozpoznawania emocji jest postrzegane w kontekście edukacyjnym. Badanie przeprowadzono wśród studentów oraz pracowników uczelni wyższych w Polsce. Rezultaty mogą być przydatne dla badaczy informatyki afektywnej, zajmujących się zastosowaniami w edukacji, a także dla producentów aplikacji edukacyjnych, rozważających rozszerzenie systemów o aspekt rozpoznawania emocji.

Przyjazne aplikacje dla dzieci z autyzmem – e-edukacja na poziomie przedszkolnym

A. Landowska , M. Wróbel , I. Ruta-Sominka, A. Budzińska – EduAkcja. Magazyn Edukacji Elektronicznej – 2017
Artykuł dotyczy możliwości wykorzystania aplikacji na tablety do wspomagania terapii dzieci z zaburzeniami rozwojowymi ze spektrum autyzmu. Przedstawia studium przypadku skutecznego zastosowania e-technologii w kształceniu specjalnym na poziomie przedszkolnym. Artykuł opisuje zestaw aplikacji edukacyjnych o nazwie Przyjazne Aplikacje, których celem jest wspieranie terapii dzieci z autyzmem opartej o stosowaną analizę zachowania. Aplikacje te, konstruowane od kilku lat na Politechnice Gdańskiej we współpracy ze specjalistami z Instytutu Wspomagania Rozwoju Dziecka w Gdańsku (IWRD), od ponad roku skutecznie wspomagają nauczanie dzieci z autyzmem w przedszkolu specjalnym prowadzonym przez IWRD. Celem niniejszego opracowania jest przedstawienie założeń budowy Przyjaznych Aplikacji oraz ich ocena w praktyce.

Applications for investigating therapy progress of autistic children

The paper regards supporting behavioral therapy of autistic children with mobile applications, specifically applied for measuring the child’s progress. A family of five applications is presented, that was developed as an investigation tool within the project aimed at automation of therapy progress monitoring. The applications were already tested with children with autism spectrum disorder. Hereby we analyse children’ experience with the games, as a positive attitude towards the application is the key factor enabling practical application of the solutions in therapy. Two evaluation methods were applied: a behavioral study of video recordings of children interaction with the games and online behavioral tagging performed during measurement sessions. The paper also outlines the main challenges, encountered during sessions with autistic children. The study might be interesting for both researchers and practitioners applying e-technologies in autistics therapy