A specific CALLAS Proof-of-Concept targeted affective edutainment by facilitating children approach (target is 6 to 13 years old) to the Opera by offering them the possibility to drive the behaviour of animated characters on a Virtual stage. This Proof-of-Concept is very relevant in the strategy of Fondazione Teatro Massimo, a CALLAS partner, for its educational program targeting schools in Italy: "La Scuola va al Massimo" [Sept/09 La Scuola va al Massimo].
The demonstrator, installed at Sala degli Stemmi of Teatro Massimo in Palermo by Digital Video, resulted very attractive since its 1st presentation during the Teatro Massimo 2008 Concert Season and at CALLAS event at CIMCIM 2009 in Rome, at Santa Cecilia. Watch a video of Interactive Opera here, see also poster.
The development is based on the CALLAS Framework it makes usage of several input components:
The audio-vide
o installation is based on a emotional interaction station, input touch screens, camera and microphones.
The demonstrator, installed at Sala degli Stemmi of Teatro Massimo in Palermo by Digital Video, resulted very attractive since its 1st presentation during the Teatro Massimo 2008 Concert Season and at CALLAS event at CIMCIM 2009 in Rome, at Santa Cecilia. Watch a video of Interactive Opera here, see also poster.
The development is based on the CALLAS Framework it makes usage of several input components:
- Facial Expression Recognition is used to interfering with the story's plot, depending on facial expressions of multiple people, each one standing for a different character;
- Gaze detection and Head Pose estimation is used to improve the stability of facial expression recognition by making it possible to function under head movements;
- Real-time emotion recognition from speech is used to map a discrete set of emotions compatible with the set of the characters' animations;
- Multi Keyword Spotting is used for selection of characters, activation and restarting, with emotional keywords weighted and combined with other input modalities into P-A-D values for emotion estimation
The audio-video installation is based on a emotional interaction station, input touch screens, camera and microphones.A Video Tutorial is explaining how the get Ekmanian emotion using system devices. Ekmanian emotions behaviour for characters is defined, with subtitles, to allow children focus on the plot of the action, music and sound library.