Expanding User Interaction Beyond Two Dimensions

Among the winners of the 2007 PopSci Invention Awards were the developers of a motion tracking ring used to control a cursor called the Magic Mouse. With Nintendo's Wii game console users can use the controller to interact with three dimensional virtual environments through intuitive hand motions. Some users have written software enabling them to use the Wii controller as a mouse. But, this software functions far from seamlessly, and though the Magic Mouse may make a useful replacement for the traditional mouse, it still falls far short of the fully immersive interface that can be made possible by applying motion-capture to HCI.

Although the evolution of the operating system has been in the direction of maximizing the amount of information a user can effectively manipulate on the desktop at once, current operating systems have, for the most part, been confined to operating in a layered two dimensional environment.

The Windows Vista operating system creates the aesthetic of managing windows in a three dimensional desktop. However, this ostensible third dimension only serves to add a slight aesthetic flare to the traditional layered windows environment, and only a very small usability increase. In order to take full advantage of computing in a three dimensional environment, users must be given greater freedom of motion, such as the ability to rotate objects.

The IoImmersive working group advocates going beyond this using mapping algorithms that enable the developer and client to decide how their virtual world will be designed. Users must be able to map images or windows onto three dimensional surfaces and manipulate polyhedra whose spaces are composed of windows and files.