Recent rumors make it seem likely that Intel will be entering the Augmented Reality (AR) market, using its RealSense technology as a key component. Given Intel’s heavy-and-continuing investment in RealSense across nearly every other area of computing, this makes complete sense. Intel is already promoting RealSense for laptop use, desktop use, mobile phone use, and as part of Virtual Reality (VR) headsets. So far it has gotten limited traction in any of those areas, but each new generation brings new promise.
RealSense basics
At core, RealSense modules consist of two Infrared (IR) sensors that receive information created by projecting a structured light field using an IR laser illuminator. This allows the module to create a VGA-resolution depth map of objects within its field of view. Production modules then also include one or more RGB cameras that can be used to create a combined RGB+Depth rendering. Expanded versions like the ZS300 also include an accelerometer and gyroscope. I’ve demoed several different iterations of RealSense and have been very pleased with its ability to track gestures, even in tough lighting conditions. However, there are a couple issues that have stood in the way of its broader adoption.RealSense issues that need to be addressed
First, OEMs need to decide whether to use the Front-Facing (F) or the Rear-Facing (R) version of RealSense. The former is optimized for close range use, and designed to (appropriately enough) be embedded on the front side of a laptop or mobile device. It is great for recognizing a user’s face or allowing control of a screen or application, but doesn’t allow for anything beyond a desktop in operating area. Conversely, the R unit isn’t really able to track details close up, but can measure depth out to about 12 feet — still not enough for large rooms, but useful for many VR and AR applications. For Intel to get ahead of the increasingly crowded AR headset space, it’ll need to make sure it can provide a RealSense module that works well all the way from close-up gestures to at least as far as the average living room wall.
Second, while the IR projection system in RealSense is fairly robust, it doesn’t work well outdoors, or in environments with a lot of other direct light. That puts it at a disadvantage compared with systems that rely on either stereo cameras or cameras and projected light. For VR, that isn’t a huge deal, as most current consumer applications are indoors, and in situations where lighting can be controlled. For AR, this will be more of a stumbling block, as many use cases for AR involve annotating the world at large. Researchers I’ve spoken with who have used RealSense are also not satisfied with the quality of depth maps the devices provide.
Finally, PC-powered RealSense headsets are limited by requiring a USB 3.0 tether. For Intel to be competitive with un-tethered products like Microsoft’s HoloLens, it will need to have enough horsepower in the headset for it to run standalone. Fortunately, the IR-based depth mapping in RealSense doesn’t require as much processing power as the vision-based approaches in other systems. But mapping virtual objects onto the real world in a mobile device does require plenty of processing power — more than current mobile CPUs and GPUs can provide. That’s one reason Microsoft developed a proprietary Holographic Processing Unit and Google has partnered with chip startup Movidius to provide custom vision processing silicon. Look for Intel to do something similar as part of its AR offering.
In addition to addressing these issues, Intel will need to find a solid partner for the AR glasses themselves. Fortunately there are some excellent products under development in this area, including from companies including Lumus and ODG. Intel would need to integrate the technology from one of them with its own chipset and with RealSense, but that would be much more straightforward than trying to invent its own AR headset from scratch — especially given the large apparent lead of competitive platforms like Microsoft’s HoloLens and Magic Leap.
Intel’s RealSense is already part of the impressive Daqri AR Smart Helmet, so it is clearly possible to use it in a special-purpose, high-end device. But it remains to be seen whether Intel can make it the right approach for high-volume consumer AR devices
No comments:
Post a Comment