Single-Trial Recognition of Video Gamer’s Expertise from Brain Haemodynamic and Facial Emotion Responses
Metadatos
Afficher la notice complèteEditorial
Mdpi
Materia
Mind-controlled games Brain signals fNIRS Facial expressions
Date
2021-01-14Referencia bibliográfica
Andreu-Perez, A.R.; Kiani, M.; Andreu-Perez, J.; Reddy, P.; Andreu-Abela, J.; Pinto, M.; Izzetoglu, K. Single-Trial Recognition of Video Gamer’s Expertise from Brain Haemodynamic and Facial Emotion Responses. Brain Sci. 2021, 11, 106. [https://doi.org/10.3390/ brainsci11010106]
Résumé
With an increase in consumer demand of video gaming entertainment, the game industry
is exploring novel ways of game interaction such as providing direct interfaces between the game
and the gamers’ cognitive or affective responses. In this work, gamer’s brain activity has been
imaged using functional near infrared spectroscopy (fNIRS) whilst they watch video of a video
game (League of Legends) they play. A video of the face of the participants is also recorded for
each of a total of 15 trials where a trial is defined as watching a gameplay video. From the data
collected, i.e., gamer’s fNIRS data in combination with emotional state estimation from gamer’s
facial expressions, the expertise level of the gamers has been decoded per trial in a multi-modal
framework comprising of unsupervised deep feature learning and classification by state-of-the-art
models. The best tri-class classification accuracy is obtained using a cascade of random convolutional
kernel transform (ROCKET) feature extraction method and deep classifier at 91.44%. This is the first
work that aims at decoding expertise level of gamers using non-restrictive and portable technologies
for brain imaging, and emotional state recognition derived from gamers’ facial expressions. This work
has profound implications for novel designs of future human interactions with video games and
brain-controlled games.