Augmented Reality for Art is subject of a dynamically-generated piece of digital art, e-Tree (Emotional Tree), developed in CALLAS as a virtual tree whose growth and evolution reflects the perceived affective response of the spectators. The leading digital artist Maurice Benayoun collaborated to develop the idea and artistic brief.
The e-Tree provide an interactive session where spectators' reactions are captured, making usage of Multi Keyword Spotting , Real-time emotion recognition from speech and Video Feature Extraction components, combined using a common affective model which then controls properties of the digital artwork.
Spectators interact and respond to the e-Tree, which aggregates affective reactions over a sustained period of interaction. The concept supports the exploration of "affective feedback loops" where a participant's response to dynamic artwork determines changes and development that occur within the artwork, to which the participant then also responds. The responses of the viewers of the installation are captured using cameras and microphones, which feed CALLAS components that produce affective interpretations of those responses. The affective input is combined into a dimensional affective model (according to the PAD temperament model) which then drives the growth and branching of the tree.
- Small, sustained positive input causes the tree to start to "twitch" and come "alive".
- Larger, sustained positive inputs cause it to branch and grow progressively more quickly.
- Negative inputs cause growth to shrink back, and larger negative inputs cause the tree to lose branches (which re-appear as affective values return to neutral).
Initial CALLAS e-Tree was based on an experimental version of ARToolkit, described in following papers presented at ISMAR2007:
- A High-level Event System for Augmented Reality: Abstract
- An Emotionally Responsive AR Art Installation: Abstract
- E-Tree: Emotionally Driven Augmented Reality Art: Abstract
- An affective model of user experience for interactive art: Abstract