Laszlo Hunyadi is professor at the university of Debreceniensis

An introduction to the multimodal corpus HuComTech and its annotation

The HuComTech corpus is aimed at improving human-machine interaction by describing and studying those aspects of human-human communication that are relevant for robotic applications. The corpus is based on 111 formal dialogs (job interviews), 111 informal dialogs (guided conversations) and 111 read out segments with a total length of about 60 hours. The language is Hungarian and the subjects are university students aged 19-28. The annotations include both video audio and encompass both physical descriptions and interpretative labeling. Special emphasis is put on unimodal and multimodal pragmatic annotation as well as the restricted annotation of spoken syntax. Annotation is mainly done manually, but additional automatic methods were also applied to video (fail expressions and emotions) and audio (prosody recognition). Based on the annotation results, a comparison has been made between the two approaches. Created partly using our own software, the metadata are now imported into Elan for queries and further annotation.

Our still ongoing annotation work yields an experience that may be useful both for researchers and developers.

Facebooktwittergoogle_pluslinkedin