How Attention Shapes Interaction Design

Wednesday August 23rd, 2006 in , , ,

This is a follow up to my recent post about Resonating Technology, so again I’ll use a mobile phone as a case study. This time around the focus is on how attention affects interaction design and the overall user experience.

To start of lets set a few goals that will support a well designed user interface:

  • Don’t present the user with choices that make no sense.
  • The manual should be no longer than five lines of text - in fact, there shouldn’t even be a need for a manual.

Ok, let’s deal with the problem of choice. To this date our toolbox for handling this problem has primarily consisted of two things: menus and buttons. But what if we were to do something completely different? What I’m proposing instead is something I’ll call the Intention-based User Interface which is based on gesture-recognition and the use of attention data.

Let’s take a look at the definition for gestures:

  1. A motion of the limbs or body made to express or help express thought or to emphasize speech.
  2. The act of moving the limbs or body as an expression of thought or emphasis.
  3. An act or a remark made as a formality or as a sign of intention or attitude.

In short, when a user makes a specific gesture it’s a sign of intention and thought. The job of the phone and the user interface is to interpret the gesture and respond accordingly. To make this possible the camera has to see the user and feel how the user is holding it - including rotation, yaw, pitch, pressure etc. Let me give you a few example of how this could work:

  • Taking a picture - we use our eyes to take pictures, and point if we want someone to see what we see. So to take a picture with the phone, I aim it, hold it steady and apply pressure on the top right of it’s frame. If I need to zoom I just slide one of my fingers across the left side of the phone. In contrast if the phone is stationary on the table, the camera covered, this indicates that the user is not currently interested in taking a picture.
  • Answering the phone - we all know the gesture for this - hold it close to one of your ears.
  • Calling someone - Other than the touch-sensitive frame of the phone it has a full-sized touch-screen on each side. Underneath each semi-transparent screen there’s a wide-angle camera that’s used for taking pictures and performing gesture recognition. As you write on a blank area of the screen with your finger or the pen, the phone will interpret your input and present you with the options that match. For example, if I write “Martin” or “555 21″, it will display contacts that match, find notes containing that word and so forth. I circle the contact item with martins name, and hold the phone to my ear to make the call.
  • Writing notes and text messages - Now if I wanted to write a note or text message I would simply continue to do so in the blank area of the screen. As the text gets longer the logical choice is to present options for saving the note, or sending it as a text message to a contact.

What we’ve done is instead of always presenting the user with a multitude of buttons and menus, we let the users do what comes natural, and then update the user interface to present choices that make most sense in that given situation.

Where does this leave us with the second goal of no need for a manual? Pretty good actually. My guess is five minutes to get a demonstration in the store, and you’re ready to go. For example, the salesclerk could tell you the following:

  1. Just aim it at something you want to take a picture of and press down.
  2. It responds to what you write - names, phone numbers, notes, you name it - when you’re done select what you want to do
  3. Hold it to your ear to answer a call

There you have it - interaction design really does benefit from attention. We’ve successfully removed choices that don’t make sense to the user and made better use of basic human behavior and skill sets.

PS: Someone please make this phone.


2 Responses to “How Attention Shapes Interaction Design”

  1. mm Says:

    “Answering the phone - we all know the gesture for this - hold it close to one of your ears.”

    Yes, it’s very intuitive to put a loudly ringing phone right up to your ear, LOL :P

  2. Jim Meyer Says:

    It could just vibrate or stop ringing once it recognized that your are holding it or taking it out of your pocket etc. The point is that it should be aware that you are close by and about to answer the phone, so it shouldn’t need to ring loudly in the first place.

Leave a Reply