Creative Technology

Multisensory Experiences/Technology

These are some of the project awards we have in this area:

SenseX: Sensory Experiences for Interactive Technologies

[Obrist PI, ERC Starting Grant, EUR 1.5Mio, April 2015 - March 2020] 

The senses we call upon to interact with technology are still very limited relying mostly on visual and auditory senses. The grand challenge and vision of this project is to gain a rich and integrated understanding on touch, taste, and smell experiences for interactive technologies. We aim to achieve this ambitious grand vision by 1) creating a ‘sensory interaction framework’ on the bases of a systematic empirical investigation of touch, taste, and smell experiences, 2) integrating the generated understanding on the three senses into meaningful and efficient experiential cross-sensory gamuts and interaction principles, and 3) demonstrating the added value of the created experiential understanding on touch, taste, and smell – aka the experiential gamuts – through their integration into the development of multi-sensory systems verifying the short-, mid- and long-term societal and scientific impact (short-term: multi-sensory media experiences; mid-term: interaction concepts for partially sensory impaired people; long-term: multi-sensory interaction approach for life beyond Earth). This research will pioneer novel interaction concepts for interactive technologies in relation to essential components of multi-sensory experiences. This project will transform existing interaction paradigm in Human-Computer Interaction (HCI) and likewise impact other disciplines such as sensory and cognitive sciences by delivering ground-breaking new insights on the experiential dimensions underlying neurological processes and human perception. Project website: http://multi-sensory.info/

OWidgets: Design of a new non-existing layer of information delivery through smell

[Obrist PI, ERC Proof-of-Concept, EUR 149,903, Febr 2017 - Jan 2018]

The idea for this proposal is to demonstrate the design of olfactory widgets (OWidgets) as a proof-of-concept implementation based on the semantic integration of smell to convey specific information to the user. We will showcase the meaningful design of OWidgets in two distinct markets. Widgets are well established as simple application extensions, integrated in a larger application and usually pre-installed on a device. Widgets allow quick access to information without requiring a user to open the application that manages this information. OWidgets will expand the existing range of widgets and provide users with a new non-existing layer of information delivery through smell, and consequently allow the creation of novel experiences in various application contexts.
 
Seeing the World through Sound: Developing Multi-Sensory Technology for Young Blind People

[Obrist CI, RM Phillips Endownment Fund, £58,955, Dec 2014 - Nov 2016]

Sensory Substitution Devices (SSDs) convert visual information into another sensory channel (e.g. sound) to improve the everyday functioning of blind and visually impaired persons (BVIP). However, the range of possible functions and options for translating vision into sound is largely open-ended. To provide constraints on the design of this technology, we interviewed ten BVIPs who were briefly trained in the use of three novel devices that, collectively, showcase a large range of design permutations. The SSDs include the ‘Depth-vOICe,’ ‘Synaestheatre’ and ‘Creole’ that offer high spatial, temporal, and colour resolutions respectively via a variety of sound outputs (electronic tones, instruments, vocals). The participants identified a range of practical concerns in relation to the devices (e.g. curb detection, recognition, mental effort) but also highlighted experiential aspects. This included both curiosity about the visual world (e.g. understanding shades of colour, the shape of cars, seeing the night sky) and the desire for the substituting sound to be responsive to movement of the device and aesthetically engaging.

Multisensory Technology for Space Exploration

[Obrist PI, HEIF Kick Start Network Award, £3,000, Nov 2014 - July 2015]

With this network a dialogue between different organisations and key players concerned with the future prospect of humanity in space was initiated. New contacts were established and a first step was made towards the creation of a research agenda for designing and developing multisensory experiences and technologies in the context of space explorations.