Multi-touch on the desktop

Now that the iPhone has given us all a taste of a multi-touch user interface, I have been hearing many people say how cool it would be to have touch-based input on a new line of desktop displays from Apple.

If you’re one of the people who think that a multi-touch monitor is a good idea, try this little experiment: touch the top and bottom of your display repeatedly for five minutes. Unless you’re able to beat the governor of California in an arm wrestling match, you’ll give up well before that time limit. Now can you imagine using an interface like this for an eight hour work day?

One of the things that people don’t realize about the iPhone is that it works at a low angle (as opposed to the high angle of your desktop or laptop display.) Our bodies are more comfortable and adept at handling repetitive physical tasks when they are performed at these low angles. What works well for the eyes does not work well for the hands.

If you’re old enough to remember a time before CAD systems, you’ll likely remember drafting tables. These tables were adjustable from completely flat (a low angle) to somewhere around 40° (a medium angle.) A drafting table is an environment where it is easy to work with your hands and associated tools for hours on end.

The iPhone’s multi-touch UI works similarly: if you watch people use it, I think you’ll see a lot more people working at waist level than at chest level. The only time you need the interface close to your head is when you’re enjoying those 3 pt fonts in MobileSafari :-)

Of course, Apple could come up with some kind of ergonomic multi-touch desk. Or we could all go out and buy a Microsoft Surface real soon now. However, I’m pretty happy with the recent demise of the glass-based CRT and not looking forward to the added weight that a touch based interface would add to a 30″ LCD monitor.

But even if there was a solution to the ergonomic issues, there would be problems mixing mouse-based applications (with small hit areas) with touch-based inputs (and large hit areas). Touch-based UI is not something you just bolt onto existing applications—it’s something that has to be designed in from the start.

You can already see this mismatch between the mouse-based and touch-based environments. All you need to do is view a web application that is targeted at the iPhone browser. In a desktop environment the controls seem large, but on the phone they are comfortably sized.

Resolution independent interfaces may solve some of the problems with control sizes, but the fact remains that a desktop interface has a much higher information density than a mobile one. A desktop is a multi-tasking environment while a mobile device is typically oriented towards a single task (making a call, finding a restaurant, getting directions, etc.) Don’t assume that the multi-touch you are using in a single task environment, with its lower information density and more focused interface, will be equally successful in a high density, multi-tasking desktop. Take another look at Jeff Han’s amazing demo and realize that he’s only working in one application at a time—what happens when you add a browser, an e-mail client, and some of your other favorite applications to that desktop?

I also find it difficult to believe that any kind of touch-based UI will replace the keyboard anytime soon. For people who are touch typists, you can’t beat the feedback of a key press for common applications like word processing and e-mail. Eventually we’ll have haptic interfaces with simulated feedback which will obviate the need for a separate keyboard.

The bottom line is that we’ve only just begun a journey that will fundamentally change the way we interact with machines. A major part of this change will be evaluating new and better ways to use computers—what has worked well in the past may not work so well in the future. And because of the magnitude of this change, I think there will be an extended period where touch-based, mouse-based and keyboard-based interfaces will need to coexist. If we’re not careful about developing these new interfaces, we’ll end up with something like Victor Frankenstein’s creation: pieced together and frightening.

Update: Tog agrees with me. Make sure to check out the Starfire video for ideas on how horizontal and vertical work surfaces can be integrated. Even though the cultural and technological elements are a bit dated, the human-centered design is still relevant.