Evaluation of Keyword Search in Affective Multimedia Databases

Multimedia documents such as pictures, videos, sounds and text provoke emotional responses of different intensity and polarity. These stimuli are stored in affective multimedia databases together with description of their semantics based on keywords from unsupervised glossaries, expected emotion elicitation potential and other important contextual information. Affective multimedia databases are important in many different areas of research, such as affective computing, human-computer interaction and cognitive sciences, where it is necessary to deliberately modulate emotional states of individuals. However, restrictions in the employed semantic data models impair retrieval performance measures thus severely limiting the databases’ overall usability. An experimental evaluation of multi-keyword search in affective multimedia databases, using lift charts as binomial classifiers optimized for retrieval precision or sensitivity, is presented. Suggestions for improving expressiveness and formality of data models are elaborated, as well as introduction of dedicated ontologies which could lead to better data interoperability.