- Matt Weinberger/Business Insider
The coolest announcement of CES 2016, to me, was the news that Lenovo and Google are working together to put the search giant’s nifty Project Tango sensors into a forthcoming consumer smartphone.
Project Tango, like similar efforts from Microsoft, Apple, and Intel, is trying to give smartphones and other devices the ability to perceive depth and space the way humans do.
That sounds nerdy, but the potential effects of the shift towards depth perception are profound: Get ready for apps that help you measure furniture or guide you straight to your friends in a crowded club.
It’s the future of tech, as our devices learn to make more sense of the world around us. And it’s just starting to sneak into all the tech we use.
SoftKinetic, a startup that Sony bought in late 2015, has been doing this for a while now. In fact, it’s the technology platform behind the new BMW 7 Series‘ very high-tech dashboard gesture controls, where a wave or a swipe answers the phone or turns up the music.
Check it out in action:
This is just the first wave of what this technology can do, says SoftKinetic CMO Eric Krzselo.
At first, Krzselo says, this technology’s most natural on-ramp will be via the smartphone, since depth perception software can help tremendously with camera features like autofocus.
But once the sensor is in more phones, developers can give their apps the ability to “read” where the human hand is in three-dimensional space, as demonstrated in the BMW 7 Series, it means you can also have people interact with digital objects in virtual or augmented reality, with their own hands.
In plainer terms: I got to use an Oculus Rift virtual reality headset, souped up with a SoftKinetic sensor, that let me wave my hands in front of my face and see them reflected in the screen. I didn’t need a controller or a keyboard to grab and throw boxes around in virtual reality. All I needed to do was wave.
- Matt Weinberger
As a Sony subsidiary, SoftKinetic obviously has strong relationships there, and this could well make its way into a future version of the Sony PlayStation 4 virtual reality headset, Krzselo says.
But just as Sony sells its image sensor components to Apple and other sometimes-rivals, there’s nothing stopping SoftKinetic from offering these sensors to car manufacturers, phone manufacturers, or Facebook’s Oculus Rift VR headset, for them to build into their products as they see fit.
In another virtual reality demo, I used SoftKinetic’s gesture technology to control holographically-projected controls on the dashboard of a car, pushing virtual buttons.
Microsoft and Intel have strong investments in this area too, as similar sensors are making their way to the HoloLens holographic computer and Intel-powered systems like the HTC Vive. If you want to see the future, check out Tiltbrush, a VR paint program that lets you step inside a painting, using the depth effect to grand effect.
In short, the future of smartphones is deep, as we get more hardware that understands the world around us like never before.