John Underkoffler, inventor of the data interface used in the Minority Report, talks about the g-speak Spatial Operating Environment and the future of user interfaces in a recent TED talk. Watching this makes it hard to ignore the fact that the best has yet to come. And it’s a great example of why TED matters.
Topic : Gestural Interfaces
28
Nov 09
Nov 09
Beyond Mice and Keyboards
The ubiquitous keyboard and mouse that have dominated computing for the last 30 years are getting some company and competition as gesture interfaces become a reality outside the test lab.
Microsoft’s Project Natal for Xbox 360 promises an immersive user experience in which the interface becomes more invisible than ever before. With Natal the user is the interface. Looking to take the user experience far beyond Nintendo’s Wii, Natal uses a 3-D depth camera and microphone for motion, gesture, and audio input. Xbox claims Natal will let people steer an on-screen race car by moving their arms in steering motions and use gestures like actual kicks to move a soccer ball on screen. In one demo, Natal recognizes a person’s face and automatically logs them into their Xbox profile. Think Wii without the controller. Wikipedia has a brief article on Natal’s background and technology.
And if you think this is a just going to be a high-tech gamer toy, look at the opportunities for communication and commerce in this post on Engadget. Imagine manipulating your TV’s menu system with the same gestures you’d use on an iPhone. No convoluted controller or touch screen required. It’s like Minority Report in your media room.
Motion-detecting interfaces aren’t limited to efforts as ambitious as Natal. Here’s a look at Pek Pongpaet using the accelerometer in the WiiMote to control an on-screen X-Wing fighter. Many areas of education, from aeronautics to architecture, could be revolutionized with touchable and movable experiences. Pek also did a recent demo at DePaul University in Chicago where he used the Wii Balance Board to connect to a website through WiiFlash Server to steer a car on screen by leaning in the direction he wanted to steer the car.
It’s clear new ways of human-computer interaction are coming thanks to multi-touch UIs and gestural interfaces. Aching gamers’ thumbs everywhere will be rejoicing.
31
Oct 09
Oct 09
Thought Provoking Look at Multi-touch Interfaces
I recently saw a thought-provoking video from 10/GUI on the potential for multi-touch user interfaces in desktop computing. The video suggests a radical change in desktop UIs that could bring the interactivity of the iPhone, and more, to a desktop O/S.
This is exciting stuff. While I see plenty of issues with the concept as presented by 10/GUI, there is no doubt multi-touch technology gain an increasingly important presence in desktop computing. It will be up to UX professionals to make sure it isn’t just a technology in search of a problem to solve. In fact, the new challenges that will need to be addressed in the realm of human-computer interaction promise a very interesting future.
For example, in the 10/GUI video the multi-touch pad is placed in front of the keyboard. This could potentially force the keyboard to be farther away from the user than they like. I also see problems with unintended signals getting sent to the touch surface by accident when the user is actually typing on the keyboard (this happens all the time now with laptop touch pads set in front of the keyboard). Sure, the touch surface could be smart enough to know when someone is typing and not be engaged, much like how the iPhone dims as you bring it to your ear to conserve the battery when you are using it as a phone, but what would this do for applications in which the user has to move quickly between the two input modes.
On the other hand, the 10/GUI concept solves the problem with systems like HP TouchSmart that force the user to place their hand in front of the screen and obscure their view (not to mention the smudge marks on the screen).
Maybe it will be better to place the touch surface to the left or right of the keyboard in a user-selected location, much like Wacom tablets are used today. Wacom’s Bamboo is in fact moving us a step closer to the future imagined by 10/GUI. Beyond the ergonomic challenges, there is the learning curve for people who are not computer power users and the challenge of getting people to think about information spaces as linear, as 10/GUI’s con10uum proposes.
Yet another question is what applications exist that could really benefit from this kind of desktop input. The slider example offered by 10/GUI is not an accurate reflection of how soundboards are used. Usually a sound engineer will be manipulating just one or two inputs at the same time. Of course the iPhone has shown that once you build a technology infrastructure to support new means of interaction, the creative power of the development community will find new and exciting ways to use it. Virtual piano anyone? Exciting stuff indeed.
Ever since Microsoft Surface gave us a glimpse of multi-touch interfaces beyond the smartphone, we’ve wondered what future interactive experiences might be like. While far from perfect, the 10/GUI con10uum concept is another opportunity to get us all thinking about how we might design a very different future.