Nuno N. Correia

Associate Professor in Digital Transformation at Tallinn University

Nuno N. Correia is an Associate Professor in Digital Transformation at Tallinn University. Since 2000, he has been teaching and conducting research in media art and interaction design, in several universities internationally (such as Aalto U., Goldsmiths U. London, U. Greenwich, U. Madeira). Nuno is also an associated researcher at the Interactive Technologies Institute (ITI/LARSyS, Portugal).

He has led several research projects combining sound, image and interaction design, namely: Moving Digits: Augmented Dance for Engaged Audience (EU Creative Europe, 2018-20), Enabling Audiovisual User Interfaces for Multisensorial Interaction (EU Marie Skłodowska-Curie Individual Fellowship, 2014-16) and Audiovisual Vector Exchange (Aalto Media Factory Seed Funding, 2013-2014).
Nuno holds a PhD in New Media from Aalto University (Finland), with the thesis Interactive Audiovisual Objects (2013). Previously, he was Senior Interaction Designer at the design consultancy Fjord, including work for Nokia, Microsoft and HP. Nuno has published nearly 50 peer-reviewed articles, in HCI conferences such as CHI and TEI, and in audio conferences such as NIME and Audio Mostly. He founded the Sound, Image and Interaction Design Symposium (SIIDS).

Audiovisual User Interfaces for Multisensory Design

The concept of Audiovisual User Interfaces (AVUI) links interaction, sound and image, building upon the concept of Graphical User Interface (GUI) by adding interconnected audio and visuals. By integrating interrelated sonic and visual feedback, reacting to user interactions, AVUIs can lead to more usable, playful, informative and captivating UIs, compared to a traditional GUI – particularly in use cases where accessibility and/or engagement are determinant.

Sound Design for Embodied Interaction

November 23, 2021 at 10:00
Tallinn University, Narva mnt 25, Tallinn, Estonia

Embodied Sonic Interaction Prototyping

Sound has a rich history as a modality for enhancing interaction design. The field of sonic interaction design “explores ways in which sound can be used to convey information, meaning and aesthetic and emotional qualities in interactive contexts”, where “in order to foster an embodied experience, both the interface and its sonic behaviour must be carefully designed” (Franinovic and Salter, 2013). In the book “Where the Action Is”, Dourish (2004) set the foundation for embodied interaction in HCI – as interaction design for and with the human body. There is a great potential for combining embodied interaction and sonic interaction design, for engagement and usability. This workshop aims to explore this combination, by adopting rapid prototyping, similarly to related workshop approaches (Franinovic et al., 2013; Caramiaux et al., 2015), but it is also informed by experience prototyping (Buchenau and Suri, 2000) and embodied design improvisation (Sirkin and Ju, 2014).

About the prototyping techniques

Experience prototyping can be defined as “a form of prototyping that enables design team members, users and clients to gain a first-hand appreciation of existing or future conditions through active engagement with prototypes” (Buchenau and Suri, 2000). The main aim of Experience Prototyping in evaluating design ideas is to enable the exploration of possible solutions and to direct designers towards a more informed development of the user experience and its components (Buchenau and Suri, 2000). Embodied design improvisation (Sirkin and Ju, 2014) is “a generative and evaluative technique to elicit tacit knowledge about the embodied experience”. It included “storyboarding, Wizard of Oz prototyping, domain expert improvisation, video prototyping and crowdsourced experimentation” (Sirkin and Ju, 2014).

Learning outcomes

Development of rapid embodied experience prototypes using video.
Exploration of sound design for experience prototypes using Ocenaudio and Freesound.org.

Structure

Morning

Introduction to embodied sonic experience design
Introduction to embodied prototyping techniques
Introduction to sound design with Ocenaudio
Presentation of workshop challenges and suggestions for approaches
Creation of participant groups

Afternoon

Prototyping in groups
Presentation of prototypes

Required software and hardware

Ocenaudio (free: https://www.ocenaudio.com)
Any video editing software, for example, Apple iMovie or DaVinci Resolve (free: https://www.blackmagicdesign.com/uk/products/davinciresolve/)
As additional options, Zoom and OBS might be useful and may open video recording possibilities, even for co-located participants (https://zoom.us, https://obsproject.com)
A laptop
A video camera or smartphone for recording videos (depending on the approach, a laptop webcam might be enough)
Headphones

References

Buchenau, M. and Suri, J. F. (2000) Experience Prototyping, In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS ’00, New York, NY, USA, ACM, pp. 424–433, [online] Available at: http://doi.acm.org/10.1145/347642.347802

Caramiaux, B., Altavilla, A., Pobiner, S. and Tanaka, A. (2015) Form Follows Sound: Designing Interactions from Sonic Memories, In Proc. CHI’15, Seoul.

Dourish, P. (2004) Where the Action Is: The Foundations of Embodied Interaction, New Ed edition, Cambridge, Mass., The MIT Press.

Franinovic, K. and Salter, C. (2013) The Experience of Sonic Interaction, In Franinovic, K. and Serafin, S. (eds.), Sonic Interaction Design, Cambridge, MA, The MIT Press, pp. 39–76.

Franinovic, K., Hug, D. and Visell, Y. (2007) Sound Embodied: Explorations of Sonic Interaction Design for everyday objects in a workshop setting, In Proceedings of the 13th int. conference on Auditory Display, Montreal.

Sirkin, D. and Ju, W. (2014) Using Embodied Design Improvisation as a Design Research Tool, In Proc. of Intl Conf on Human Behavior in Design.

The workshop duration is 7 hours

280 €€

336€ with VAT

Buy now