interaction magic

Life beyond the screen

Since 2020, I've been teaching courses in Interaction Technologies and Prototyping. The course length and location vary, but the core idea is always the same.

This is about exploring life beyond the screen.

Grid of interaction technologies images

Design masters programmes typically place their emphasis on either brand and visual form language (industrial design) or the design of digital experiences and services (interaction design).

This course is different, living in the forgotten middle ground of emotional, physical experiences.

Nobody wakes up excited to stroke a pane of glass. Emotion is physical, and it is interaction with the full range of human senses that defines the feeling and emotion of a product or service.

A key foundation of the course is to prototype everything. To truly understand your ideas, you must experience them for real. This Experience Prototyping approach drives the design of the whole programme, which is split 50:50 between design exploration and hands-on prototyping time. Students join with only a rudimentary background in Arduino, and their electronics and software skills are developed in tandem with their design thinking.

The final assignment from the class of 2020 & 2021: Emotional buttons! Click for sound.

Designing hardware interactions

Stand at a petrol station and plug the nozzle into your car. The smell of gasoline. The vibration of the cable. The audible hum of the machine behind you. All of this familiar sensory feedback comes for free, thanks to the natural physics of that wet, heavy liquid flowing through the pipe.

Charging an electric vehicle affords no such natural feedback. As designers our job is to invent this feedback ourselves. What is the metaphor we can use to explain how that energy enters your car? What does electricity smell like? What sort of personality does it have?

The basic interaction toolkit has six parts: switches, haptics, gesture, light, sound, and motion. We explore each topic through a mix of group research, case studies, design challenges, Arduino prototyping and practical experiments. Along the way the students re-design all manner of hardware: toilet flushes, LED bike navigation, gestural interactions with headphone cables and robotic movements for a socially distanced wishing well.

Since 2021 I've also invited in two expert collaborators to share their experience in the fields of haptics and sound design. Daniel Shor (WOW Tech) and Tom Ridley (formerly Dyson, now Pirate.com) add immensely to the quality of learning in these areas - I couldn't do it without you guys!

Cardboard prototyping of EV chargers, before applying the interactive electronics on top.
Cardboard prototyping of EV chargers, before applying the interactive electronics on top.

Emotional buttons

Towards the end of the course, students are challenged to embody what they've learned into an emotional button.

From a starting point like anger or patience, they build references from films, memes and cartoons to extract the core of an emotion and recreate it as an interactive experience. Projects are often assembled with whatever's to hand: shoeboxes, old soap dishes, a mouse trap or dismantled dehumidifier.

The creative variety is a joy to see. The fearful button hides in the dark, recoiling from an advancing kitchen knife. The sassy button waves away the advancing finger. The jealous button grows increasingly irritable when its neighbour is pressed.

Development of Henry the Angry Button, by Anna.

To encourage succinct communication and presentation, students submit just five slides and a one minute point-of-view video to communicate the overall experience. They are also banned from submitting any source code: concepts are assessed on the output and how that experience makes you feel. Saving time and focusing the effort in the right places with Wizard-of-Oz techniques is highly recommended. In the words of one student "I imported library Housemate.h to help achieve the final movement".

With permission, I've embedded below some examples from the classes of 2020 and 2021 Human Computer Interaction (MSc) at FH Salzburg:

Angry Button, by Anna Neumann
Fearful button, by Robin Baudisch
Anxious Button, by Michael Giang
Sassy button, by Philipp Wagner
Ashamed Button, by Irina Mittermayr
Hyperactive Button, by Fredi Hirschmann
Startled button, by Jakob Bechinie
Enthusiastic Button, by Ece Dinçer

Remote prototyping

One novel aspect of the course has been adapting it for remote teaching. In fact, since the course began in 2020 it has never been delivered in person.

Teaching an online course in physical interface design is... interesting. The original intent for the first lab was a group trip to the flea market to purchase and dismantle curious buttons, switches and knobs. Instead, we made extensive use of Miro and a home Arduino kit, supplemented by some custom parts which I hand delivered to the students across Salzburg one spring morning.

From Spring 2022, we plan to be back in the workshop together again. Fingers crossed!

Student research in Miro from Lab 5
Student research in Miro from Lab 5

Courses to date

I've taught variants of this course from a one-day crash course to a 150-hour semester programme. Each year I teach a major semester course for the Human Computer Interaction (MSc) and Design & Product Management (MA) at FH Salzburg.

Grid of interaction technologies images

I'm also a visiting lecturer at TU Delft in the Netherlands and the Institute of Chemical Biology Centre for Doctoral Training at Imperial College.

If you run a design degree programme anywhere in the world, and like the sound of building and playing with emotional buttons, let's speak!