Several libraries for affective input recognition have been incorporated into the toolkit, such as Real-time emotion recognition from speech.
Availability of the component: Reference contacts are Johannes Wagner and Elisabeth AndrÃ© of University of Augsburg
Notes: The component is developed under C++ and build as a static library. The core system including standard sensor devices, basic signal processing and machine learning pipeline. The source code is freely available under GNU Lesser General Public License (LGPL). The SSI/ModelUI (Graphical interface that assists a user to collect own training corpora and obtain personalized models to be used to train EmoVoice) is available for registered users prior to downloading the code. The component makes use of the following software: oscpack -- Open Sound Control packet manipulation library ; KISS FFT -- A mixed-radix Fast Fourier Transform Copyright (c) 2003-6 Mark Borgerding; WiiYourself! -- native C++ Wiimote library v1.14 BET (c) gl.tter 2007-8 - ; AVI utilities -- for creating avi files (c) 2002 Lucian Wischik. No restrictions on use.; Torch 3.1. -- A machine-learning library,Copyright (C) 2003-4 Ronan Collobert - ; TinyXML -- a simple, small, C++ XML parser copyright (c) 2000-2006 Lee Thomason. Documentation of the SSI framework includes a comprehensive tutorial with examples (update December 8, 2009).
A specific Callas Gesture Expressivity Experiment (CGEE) was performed in Greece, Germany and Italy, about the usage of SSI for synchronized recordings of gesture expressivity, emotional speech and mimics.
The component has been used for corpus acquisition and audio analysis in the ElektroEmotion Research Tool, Scientific Showcases (the CommonTouch) and in Proof-of-Concepts applications (EUCLIDE and the AV laughter machine).
Additional features of the component are described in the following papers, articles and poster:
- Smart Sensor Integration: A Framework for Multimodal Emotion Recognition in Real-Time: Abstract
- The Smart Sensor Integration Framework and its Application in EU Projects: Abstract
- SSI/Model UI - A Tool for the Acquisition and Annotation of Human Generated Signals: Abstract