Two Fronts of Immersion: Software

Continuing our discussion of immersive interaction environments let's move on to the software front. The iPhone's software contains modifications designed to enhance and enable the functionality of hardware changes, like the touch screen. The most significant of these are gestures which convert mouse or finger actions into computational actions. These gestures include swiping left to indicate deletion, swiping down to insert a line break, and others. Additionally, the multi-touch features enhance interactions, such as zooming and rotating, that were cumbersome with only one touch (mouse) point.

Still, these changes are all within a 2D environment. The step into a true 3D interface has not yet been taken in any significant way. A product that makes more of a shift into a 3D environment is CoolIris's Firefox plug-in for navigating images, video, web searches, and other content. The plug-in provides an intuitive 3D interface but is limited in its usefulness because it operates on set content and the 3D navigation is restrained. For example, one can't move 360 degrees in three dimensions, only 120 degrees or so in one dimension and transverse along another.

In CoolIris's defense, enabling this sort of intuitive navigation is a hard problem in itself and the more degrees of freedom the harder the problem becomes. There is no reason why the efficiency gains made through altering the presentation environment that CoolIris uses cannot be used in other contexts and with other products. We can also benefit from extensions that allow more degrees of freedom to be used intuitively, perhaps combining multi-touch with 3D environments.

In conclusion, we must move beyond the mouse and keyboard hardware paradigm and we must also move beyond the 2D user interface environment. Steps are being made to do just this and these should be encouraged. Only through experimenting with a multiplicity of hardware and UI environments will we know what works best in what context.