For the longest time, people around the world have been trying to build better software interfaces to make computers stop sucking, from touch screens to voice control to motion sensors and various forms of haptic feedback. On the backend of things we have the mass exodus to the cloud, granting the average user redundant backups, realtime collaboration/communication, and less hassle by managing the mundane for them. We see that more and more people are taking their work and play with them on netbooks and smartphones, but those devices still can’t do what desktops can.
So how do we get the power of a desktop system into our mobile systems? Streaming. All we need is a screen and a fast connection (4G is probably the tipping point) to bring the computing power and storage capacity of servers into our hands, wherever we are. But streaming isn’t enough–we still need the interactive capacity of a full keyboard and mouse, along with the screenspace of multiple monitors in order to fully realize the power of computing in portable gadgets. What kind of device can make all of this mobile?
Enter augmented reality. With a new kind of head-mounted display that’s essentially special eyewear with built-in microcameras and projectors or transparent LED screens, we can turn our entire field of vision into a fully interactive computing environment. (Here’s some promising tech in that arena–pictured right.) We will be able to take reality and enhance it with digital information pulled from the servers that are processing our input in realtime. And because this augmented environment is completely digital, we can interact with it in any way imaginable. We could use hand gestures (think Kinect, but from the first person perspective), voice recognition, virtual keyboards and other buttons either “floating” or projected onto any surface, and completely new ways we have yet to think of.
The possibilities are endless because we don’t need to manufacture new hardware–it’s all in the software. It would be a lot like the Iron Man interface, except everything will only be displayed through our glasses so no one else can see what we’re doing–unless we want them to. Check it out:
We can bring all the benefits of digital collaboration together with the benefits of physical collaboration with shared virtual spaces. We could choose which information others see and what they can’t. Using advanced image recognition software we can get rid of clunky special objects (like AR markers/tags) and place virtual reality over real objects. There’s been a lot of progress in this area and huge steps are being made every year. Indeed, we may see the augrecon sooner than you may expect:
So this is the essence of the Augmented Reality Controller concept, or augrecon for short. I will be detailing this idea further, providing others with ideas for how, exactly, we can build this with today’s technology. The augrecon is a disruptive technology on the scale of the internet itself, ie a MASSIVE shift in direction. It’s going to be interesting.
Update 2/22/2012: I’ve been thinking about this concept for almost a decade now, and I honestly thought it would take at least 5 more years to really see it come through. It seems it’ll be here much sooner. As in, this year. Apparently, Google will be the first to deploy it’s Android-based glasses. What’s really interesting is that game developers like Valve and id Software are already creating content for these devices, which only makes sense considering games are a primary driver of platform adoption. The specific server-to-HUD design I outlined in this article is fast approaching as well I think, but not as quickly as the HUD itself.
