Touching the 'Infinite Canvas'

This morning, Microsoft CEO Bill Gates demoed "Touch Wall," a concept technology that is headed for Windows and quite possibly Office.

Bill's Touch Wall demo is available for download in Windows Media Video or MPEG 2. It's an amazing multi-touch wall—the surface can be almost anything—where content lays out on what the company describes as an "infinite canvas."

The infinite canvas, where live content spreads out across the surface, is a compelling user interface that works because of multi-touch. One way to conceptualize the canvas: The New York Times Web site viewed on iPhone. Content—text, videos and links—spreads around in non-linear fashion. The content can be accessed and resized using finger gestures. The YouTube video below (posted by TechCrunch) puts the concept into practice.

Microsoft has got something here, and Apple should really, really, really take notes. With iPhone, Apple brought multi-touch to the masses. But Microsoft is looking to go lots further, and the user interface concept is more sensible.

That said, Microsoft probably isn't sensible enough to exploit the concepts that it has got. Microsoft isn't in the business of releasing radically, new UIs, because it is hung up maintaining backward compatibility for its zillions of business customers. Apple has shown more willingness to pioneer new ways, even when it sometimes hurts customers (that's no criticism).

The graphical UI motif is exhausted. The mouse and keyboard are inadequate for interacting with computers and other devices. Anthropologically speaking, used together, the mouse and keyboard are an unnatural user interface. Human beings are tool users that experience and manipulate the world through five senses. There is little in human biological or cultural experience that equates to either device.

The best tools are really extensions of the hands; the mouse and keyboard UI is neither. The keyboard is a particularly unnatural construct, by the way fingers are used or by keys' alphabetical organization, which is based on the number of times letters are likely to be tapped.

For the human tool user, hands, fingers and touch are especially important for experiencing and manipulating objects or surroundings. The shopping mall is a laboratory for understanding this concept. First buyers look, and then they touch. People examine as much with their hands as their eyes.

But people need something to manipulate, to touch. The iPhone UI is deficient in this regard. People just touch buttons. Big deal. But how iPhone users interact with Web pages and photos is much more exciting. People use their fingers to lithely navigate and manipulate content. The size of iPhone's screen makes it a very finite canvas, which is one reason for the button approach. Apple has limited options.

By contrast, Microsoft worked from a 6' by 4' screen, which is a very large area for laying out content. There's no reason why a computer screen wouldn't be big enough, when the content is correctly presented—and that brings up something else about human tool users. Human beings interact with the world spatially, in three dimensions. My wife makes jewelry. I watch how she works with the beads, which she lays out on floor or table in front of her. Her spatial bead orientation strongly reminds me of Microsoft's Touch Wall canvas.

Apple must think about how best to extend the computing UI and better align it with concepts of touch, finger-and-hand manipulation and spatial orientation. The iPhone UI is a great starting place. But what about Mac OS? That's where a digital, infinite canvas makes lots of sense. If Apple doesn't act, Microsoft will. If Apple fails to extend the UI around touch and spatial concepts, Microsoft will reap the sales benefits with Windows.

Posted by Joe Wilcox on May 14, 2008 4:51 PM