There's an inevitable moment when you're in a virtual reality (VR) experience and look down. Maybe you wish you hadn't, because it's at that moment you don't see your hands.
It's then that you remember you're not really in outer space, or driving a Formula 1 race car. It's often unexpected and usually disappointing. You can sense your hands holding the controller in your lap, yet when you look down or hold them up, they don't manifest before your eyes. You're taken out of the experience, even if just a little bit.
Manus VR hopes to solve this problem, and do so in a more immersive way than other virtual reality controller solutions for headsets like Oculus Rift, PlayStation VR and HTC Vive.
Instead of holding a controller, Manus uses gloves to not only bring your hands but also your fingers into the VR world.
"What we often see is that as soon as somebody puts on the [VR] headset, the first thing they try is to grab stuff, even though their hands aren't there," Jules Blok, lead programmer for Manus, told techradar at last week's GDC 2016. "Even with the motion controllers, we still see people putting them away to try to grab something with their hands. So, clearly, there was a need to track this."
The Netherlands-based team decided to create a solution that uses existing positional tracking of VR controllers mixed with sensors in its gloves to measure hand pose and finger movements. Dual sensors in the fingers help track grips, bends and points, letting users turn their motions into functions.
Finger sensors may sound like the obvious way to bring your digits into the digital world, but Blok says it's far from it.
"A lot of what we've heard from other companies who've tried to use flex sensors is they've had trouble calibrating them," he says. "We've put a lot of research into having a good calibration profile that will work for a wide range of different hands."
The result, as I'll describe below, is wondrous. This was just a small demo at a convention, and there are improvements to be had, but Manus is on the path to perhaps one day solving one of VR's most persistent problems.
All hands
The gloves Manus let the public try at GDC are its engineering sample. The development kit, shipping in Q3 2016 for $250, look more like bike riding gloves, but those were housed behind a glass case.
When I sat down to try Manus' demo, the first thing Blok did was strap two HTC Vive Pre controllers to my wrists. I'd certainly noticed them on other users before, and Blok explained this is a temporary solution for positional tracking. Eventually, the team wants to build custom bracelets for each headset that will integrate with the specific systems.
Next Blok helped me slip on the gloves, which have a neoprene feel. I could tell there were sensors sewn in, and a small rumble pack offered some weight, but they were soft and offered decent dexterity.
Finally came the headset and headphones, and suddenly I was in a VR dollhouse where I had to help an orphan find her way home by solving a series of puzzles.
It wasn't immediately obvious what I was supposed to do (there are no instructions or visual cues), so for a minute or so I simply played around with my hands for the very first time in virtual reality.
I have to say, it was delightful. The hands I saw - a pair of bluish geometric mitts that are see-through to the skeleton, for some reason - responded to whatever I was doing in the real world. I could make a fist, point with one finger, and move my hands above and around me, and the virtual hands responded.
The response time was generally good, though there were occasional lags and glitches. The hands also disappeared if the glove slipped too far from the Vive controller, so I had to pull them on a little tighter or risk my hands vanishing before my eyes.
When I "touched" an object - there were knobs, levers and piano keys I could press down on - a slight rumble and the corresponding movement of the object wasn't quite enough to make me feel like I was interacting with a tangible object, but it was enough for a game that's clearly set in a fantasy realm and still a demo. Just the fact I could use my hands and fingers to interact with the game at all was impressive enough for the moment.
Eventually, Blok gently pulled up one of my headphones and told what I needed to do, so it was time to put my hands - all of them - to work. First, I had to grab floating fireflies, push down on a lever (to open a corresponding lantern), and put the fireflies one-by-one into the wire frames to light a path for the orphan.
To perform the movements with my real hands and watch the virtual objects respond as if I was actually touching and moving them was supremely satisfying.
Next, I had to clear away and assemble a series of broken pieces into a staircase for the orphan to climb. This was more buggy than the first puzzle as the rocky pieces had to be put in the right position to hold, and if I moved them too quickly or my grip didn't lock on at the right spot, the pieces reverted back to their original positions.
But it was challenging in a good way to grab a rock and move it by clenching my fingers and extending my arm to where it needed to go. I could even subtly manipulate the pieces with small finger movements, as if I was performing a precise task in the real world.
Then came a simple errand of pushing down plugs with my index finger to clear a way for the orphan. My final challenge was to imitate three-key tunes played on a virtual piano. I won't say it felt like I was playing an instrument, but again there was that satisfaction in pointing and pressing in real life and watching an action in the game result from it.
After a cheery goodbye, the orphan skipped her way home and my demo was over.
What struck me most about the experience was how much a part of the Pillow's Willow world I felt because of my hands. Instead of being a passive observer to the action, I was the action. Whatever movement I performed with my hands had a direct impact in the orphan's virtual world, and that's something I've never experienced so acutely before.
While you can pick up objects using the Oculus Touch controllers for Rift, you're not really opening and closing your fingers to do so. Instead, you press a trigger button. So to make a fist around a rock and see it move in real-time was a huge departure from what I've experienced in VR previously.
It's not all fun and games
While my hands helped suck me into the purple fantasy realm of Pillow's Willow, Manus has a long way to go before we're all wearing gloves and picking up objects in VR.
In order for Manus to achieve the device agnosticism it wants and build custom bracelets that work for each headset, VR companies have to open up its positional tracking systems. Oculus hasn't done that yet, though Manus is working with Razer's OSVR. While its dev kit will ship with HTC Vive Pre support, Manus is still in talks with the firm to gain access to its hardware.
It's an obstacle, but Manus co-founder Bob Vlemmix is optimistic headset makers will come around based on the positive response the gloves garnered.
"We even had [Oculus founder] Palmer Luckey try [the gloves] out," Vlemmix said. "He said it was great, and really cool."
It doesn't end with hardware support, however. In order for users to have the best experience, Blok said games will have to be built specifically for hand tracking. This, again, involves open SDKs, which aren't available across the board yet.
What's more, with developers just beginning to crack the surface of visual VR games, will they really want to focus on developing titles that use hands, too?
"That's one of the things I'm always thinking about," Vlemmix said. "It does add to the level of immersion, so it depends on the developer. We're getting a lot of requests from developers. They really want it, so I don't think that's an issue."
Manus wants to bring its gloves to consumers in 2017. It plans to add improvements, like thumb and finger spread tracking, for an even more accurate read of hand movements. It's also working on improving its physics so that you can get a sense of weight and shape from the virtual objects you interact with.
There are clearly challenges to turning Manus gloves into a viable solution that will satisfy the platform's hand needs, though from my short time with them on, I can say that going back to an eyes-only experience will be incredibly hard.
Are the gloves perfect? Not by a long shot. But did they give me the most interactive, immersive experience in virtual reality I've had to date? I'll high-five to that.
- Our updated hands on review of PlayStation VR
from Techradar - All the latest technology news