Nuno N. CorreiaAssociate Professor in Digital Transformation at Tallinn University
Nuno N. Correia is an Associate Professor in Digital Transformation at Tallinn University. Since 2000, he has been teaching and conducting research in media art and interaction design, in several universities internationally (such as Aalto U., Goldsmiths U. London, U. Greenwich, U. Madeira). Nuno is also an associated researcher at the Interactive Technologies Institute (ITI/LARSyS, Portugal).
He has led several research projects combining sound, image and interaction design, namely: Moving Digits: Augmented Dance for Engaged Audience (EU Creative Europe, 2018-20), Enabling Audiovisual User Interfaces for Multisensorial Interaction (EU Marie Skłodowska-Curie Individual Fellowship, 2014-16) and Audiovisual Vector Exchange (Aalto Media Factory Seed Funding, 2013-2014).
Nuno holds a PhD in New Media from Aalto University (Finland), with the thesis Interactive Audiovisual Objects (2013). Previously, he was Senior Interaction Designer at the design consultancy Fjord, including work for Nokia, Microsoft and HP. Nuno has published nearly 50 peer-reviewed articles, in HCI conferences such as CHI and TEI, and in audio conferences such as NIME and Audio Mostly. He founded the Sound, Image and Interaction Design Symposium (SIIDS).
Audiovisual User Interfaces for Multisensory Design
The concept of Audiovisual User Interfaces (AVUI) links interaction, sound and image, building upon the concept of Graphical User Interface (GUI) by adding interconnected audio and visuals. By integrating interrelated sonic and visual feedback, reacting to user interactions, AVUIs can lead to more usable, playful, informative and captivating UIs, compared to a traditional GUI – particularly in use cases where accessibility and/or engagement are determinant.
Sound Design for Embodied Interaction
- November 23, 2021 at 10:00
- Tallinn University, Narva mnt 25, Tallinn, Estonia
Embodied Sonic Interaction Prototyping
Sound has a rich history as a modality for enhancing interaction design. The field of sonic interaction design “explores ways in which sound can be used to convey information, meaning and aesthetic and emotional qualities in interactive contexts”, where “in order to foster an embodied experience, both the interface and its sonic behaviour must be carefully designed” (Franinovic and Salter, 2013). In the book “Where the Action Is”, Dourish (2004) set the foundation for embodied interaction in HCI – as interaction design for and with the human body. There is a great potential for combining embodied interaction and sonic interaction design, for engagement and usability. This workshop aims to explore this combination, by adopting rapid prototyping, similarly to related workshop approaches (Franinovic et al., 2013; Caramiaux et al., 2015), but it is also informed by experience prototyping (Buchenau and Suri, 2000) and embodied design improvisation (Sirkin and Ju, 2014).
About the prototyping techniques
Experience prototyping can be defined as “a form of prototyping that enables design team members, users and clients to gain a first-hand appreciation of existing or future conditions through active engagement with prototypes” (Buchenau and Suri, 2000). The main aim of Experience Prototyping in evaluating design ideas is to enable the exploration of possible solutions and to direct designers towards a more informed development of the user experience and its components (Buchenau and Suri, 2000). Embodied design improvisation (Sirkin and Ju, 2014) is “a generative and evaluative technique to elicit tacit knowledge about the embodied experience”. It included “storyboarding, Wizard of Oz prototyping, domain expert improvisation, video prototyping and crowdsourced experimentation” (Sirkin and Ju, 2014).
- Development of rapid embodied experience prototypes using video.
- Exploration of sound design for experience prototypes using Ocenaudio and Freesound.org.
- Introduction to embodied sonic experience design
- Introduction to embodied prototyping techniques
- Introduction to sound design with Ocenaudio
- Presentation of workshop challenges and suggestions for approaches
- Creation of participant groups
- Prototyping in groups
- Presentation of prototypes
Required software and hardware
- Ocenaudio (free: https://www.ocenaudio.com)
- Any video editing software, for example, Apple iMovie or DaVinci Resolve (free: https://www.blackmagicdesign.com/uk/products/davinciresolve/)
- As additional options, Zoom and OBS might be useful and may open video recording possibilities, even for co-located participants (https://zoom.us, https://obsproject.com)
- A laptop
- A video camera or smartphone for recording videos (depending on the approach, a laptop webcam might be enough)
Buchenau, M. and Suri, J. F. (2000) Experience Prototyping, In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS ’00, New York, NY, USA, ACM, pp. 424–433, [online] Available at: http://doi.acm.org/10.1145/347642.347802
Caramiaux, B., Altavilla, A., Pobiner, S. and Tanaka, A. (2015) Form Follows Sound: Designing Interactions from Sonic Memories, In Proc. CHI’15, Seoul.
Dourish, P. (2004) Where the Action Is: The Foundations of Embodied Interaction, New Ed edition, Cambridge, Mass., The MIT Press.
Franinovic, K. and Salter, C. (2013) The Experience of Sonic Interaction, In Franinovic, K. and Serafin, S. (eds.), Sonic Interaction Design, Cambridge, MA, The MIT Press, pp. 39–76.
Franinovic, K., Hug, D. and Visell, Y. (2007) Sound Embodied: Explorations of Sonic Interaction Design for everyday objects in a workshop setting, In Proceedings of the 13th int. conference on Auditory Display, Montreal.
Sirkin, D. and Ju, W. (2014) Using Embodied Design Improvisation as a Design Research Tool, In Proc. of Intl Conf on Human Behavior in Design.
The workshop duration is 7 hours
336€ with VATBuy now