Oculus says they could have shipped Touch alongside the launch of the Rift back in April, but explains why they’ve been biding their time.
Despite delaying their Touch VR motion controller to the latter half of 2016, Oculus says the hardware could have gone out the door on day one.
“It was possible for us to launch the first Touch device when we showed it, and everyone said ‘this is awesome,’ we could have shipped it,” said Jason Rubin, Head of Content at Oculus, speaking with Road to VR at Gamescom 2016 earlier this month.
Although a strong hardware foundation was there, the company didn’t want to put the hardware out before they felt there was an equally strong software base to support it, Rubin explains.
“[If we launched Touch with the Rift] there would have been a bunch of demos and a few good titles (like Job Simulator and Fantastic Contraption) […] we wanted to give our developers enough time to really create a launch line up, a good slate of titles that would last hours as opposed to minutes of enjoyment, and we think that that takes time. So more than tweaking the hardware, we wanted to give the software some time.”
Given that decision, Oculus opted to continue tweaking the Touch design to improve both ergonomics and performance as developers continued to toil away on the software side.
“Developers are really happy with what they’ve got, and we didn’t want to release it until we were happy with the ergonomics, every button was in the right place, everything was perfect,” Rubin said.
While Oculus has been drumming up its soon to launch Touch VR controller for many months now, the company doesn’t expect motion input to supersede the gamepad gameplay that Rift users are using today.
The introduction of an Xbox One gamepad as the default input device for the Oculus Rift at launch came as a surprise to many, especially given prior comments from the company’s founder singling out the gamepad as a poor choice for VR input. Meanwhile, the HTC Vive, which launched with motion controls in the box, has garnered praise for its immersive input. Combined with Oculus’ announcement of Touch all the way back in 2015, Rift users have been very eager to get their hands into virtual reality. But that doesn’t mean VR titles designed for gamepads will go the wayside, says Oculus.
Speaking with Road to VR at Gamescom 2016 last month, Jason Rubin, Head of Content at Oculus said that the company doesn’t expect gamepads to disappear from VR once Touch controllers hits the market.
“…we’re really strong proponents of the gamepad. We think there are some types of games that play incredibly well with gamepads, and we don’t believe gamepads are going away,” Rubin said. “There are developers that want us to continue shipping a gamepad [in the box with the Oculus Rift headset]. Whether or not we continue forever or just a small amount of time is undecided and unannounced, but gamepads have a valid place in the ecosystem.”
Rubin pointed to the apparent success of gamepad-only titles found on the Oculus Home platform, and says he remains skeptical of locking into a sort of standardized VR input at this stage in the ecosystem.
“We don’t share the belief that ‘if it’s a gamepad, it’s an atrocity and not a VR title’, and the proof of that is the ratings consumers are giving the games that we have out right now. In the long run it’s hard to say what the final control spec will be, or if there will be a final control spec,” he said.
He elaborated on what he called “VR purists,” and suggested that some were applying a double-standard to VR games designed for the gamepad, saying that many of the people hating on games lacking motion controls also wanted to play them themselves.
“There are VR purists that believe ‘if it doesn’t have motion controls, it isn’t VR’. I happen to not agree with that, and I don’t think Oculus agrees with that. Interestingly, many of the same people are dying to play our titles brought out on gamepad. They’re extremely vocal about [them],” Rubin said. “So you really can’t have it both ways. Titles that we brought out are some of the best reviewed, most full titles out there, and at the same time there’s a small population that believes VR has to be a certain amount of purity. VR is much broader, and we think the audience is broader, and we think what VR represents is much broader than some other people.”
Rubin went on to point to the company’s work with Samsung to launch Gear VR—which is a relatively affordable VR headset compared to the likes of high-priced units like the Rift and Vive—as an exemplar of the company’s “broader” approach to VR.
Oculus says Touch will launch in Q4 of this year, but has yet to announce a price of specific date. We’re expecting to hear more on Touch at the Oculus Connect developer conference in October.
I had a chance to try out the Manus VR hand-tracked controller on the expo floor of GDC this year and saw that there a couple of really strong use cases for having your hands and fingers tracked in VR. You can be a lot more expressive within social VR, and in mixed reality experiences where passive haptic feedback is available, having your hands tracked can actually increase the level of embodied presence.
I had a chance to catch up with the lead designer of Manus VR, Stijn Stumpel, at GDC where we compared Manus VR to Leap Motion, talked about how the flex sensors work, the use cases where having tracked hands makes sense, their extremely polished demo called Pillow’s Willow, and where they’re going in the future.
LISTEN TO THE VOICES OF VR PODCAST
At GDC, Manus VR strapped an HTC Vive controller to the back of my wrist, and it gave a lot more consistent tracking of the location of my hands as a result; I didn’t have to worry about keeping my hands within my field of view like I do with optically tracked solutions like Leap Motion. There was some uncanniness in not being able to actually physically grab objects, which can break presence. And I also experienced a lot more than 20ms of latency in my finger movements, which was a presence breaker. But I was told that they are able to achieve much better latency performance in their lab environment.
Manus VR just announced in a press release that their “gloves are being used in experiments to train NASA astronauts in mixed reality to prepare them for the International Space Station.” Here’s some footage of some of that training that they’ve released.
They also announced that Manus VR is joining the first SteamVR Tracking class being taught by Synapse on September 12th in order to create a version of their glove that has the SteamVR Tracking sensors built in. So I expect to see the next iteration remove the stopgap solution of attaching a SteamVR controller onto the back of your arm. With the increased amount of tracking on the arm, then they might also start to be able to do a lot more accurate inverse kinematic tracking of your body and be able to have a powerful invocation of the virtual body ownership illusion.
Swedish robotics specialists are working on a system to allow game developers to deliver realistic, realtime and dynamic interaction animations, so that your VR hands can finally grasp in-game objects convincingly.
By the end of this year, every major virtual reality platform will have its own motion control solution, from the Vive’s SteamVR devices to Oculus’ Touch, but whilst these controller give developers the ability to deliver near 1:1 mapped input in VR, the virtual results of those actions, particularly with our hands, can look, well, pretty ‘shonky’. Objects we pick up often just stick incongruously to the in-game controller models and, should the developer have included hand models, snap into a predefined multi-purpose positions which is more often than not pretty unconvincing.
Hand presence isn’t just about accuracy of their position in space then, but also about representing the myriad subtleties our endlessly adaptable digits are capable of. For example, observe yourself picking up a tumbler glass, your fingers will wrap the cylinder securely, picking a wine glass though and you may clasp more daintily by the stem. Not only that, if your grip will change depending where on that object you choose to hold it. The sorts of subtleties we take for granted in every day life are generally deemed expendable in games and ‘brute forcing’ a real solution – say a canned animation for every world object – is clearly out of the question.
Now, Swedish company Gleechi claim to be well on their way to resolving these issues. The company claims that the animation systems they’re building are based on 8 years of robotic research conducted by the dedicated research group at KTH, Sweden’s leading technical university.
Gleechi’s says that its VirtualGrasp product resolves the need for labour intensive manual animations for the hands by using a “predictive and adaptive algorithm” which analyses the ‘physical’ properties of a virtual object, deciphering the most appropriate and realistic grip formation for the in-game hand model and snapping to that position. The software is still in an early state, but as you can see in the video embedded at the top of the page, it really does seem to work and seeing it in action you realise just how poor most in-game interactions look.
Clearly VirtualGrasp is a technology with general purpose benefits for many games, but in VR that extra piece of the realism puzzle, one that further ties your physical self to your virtual self, may have more significant implication for immersion and presence.
Today at the Steam Dev Days conference in Seattle, Valve is showing off brand new prototypes of its Steam VR controllers.
Today’s HTC Vive controllers are bulky compared to what’s coming
While the original VR controllers that ship with the HTC Vive are undoubtedly functional, it’s widely agreed that Oculus’ forthcoming ‘Touch’ controllers are a big step forward in ergonomics. That gap may soon be a thing of the past, as Valve is showcasing new VR controller prototypes at Steam Dev Days which offer a much different take on the design.
The new SteamVR controller prototypes have a much smaller footprint handprint than what’s in the hands of HTC Vive users today. The prototypes are not so much held as they are (optionally) gripped; a band hooking over the side of the user’s palm connects the core of the controller to a sort of backhand gripper which appears to keep the controller attached to the hand even while it isn’t being held.
The controllers can be seen dotted in uncovered SteamVR Tracking sensors, just like prototypes of the original controller. One source from the event says that each controller has 21 sensors.
Developers at the conference today who have tried the controller say that users can ‘let go’ of the controller while in use, and it stays attached to the hand. This allows virtual objects to be thrown with the aid of the natural muscle-memory of opening one’s hand as they throw, an instinct that must be subdued with other controllers to save from throwing the controller clear across the room (always to hit a TV, somehow). Always wear your wrist straps, folks.
"Being actually able to throw things and let go…it's huge" @saniul on the new #Vive controllers #SteamDevDays
It appears that the controller may also support variable states between ‘open hand’ and ‘gripped hand’, reflecting a more natural connection between the user’s real and virtual hand positions.
The Valve VR controller prototypes appear to be 3D printed and feature a trigger and trackpad with three face buttons surrounding it. Some photos appear to show an array of LEDs across the front of the controller though the component’s function remains unknown.
With Valve not inviting any press to Steam Dev Days, further details surrounding the controller are thin; it’s not currently known what the company’s plans are for the controller going forward, but we’ll keep you in the know as we learn more.
Microsoft Research has devised two novel methods for more realistic haptic feedback on virtual reality controllers. They call it NormalTouch and TextureTouch.
Haptic feedback in general-purpose controllers has been limited to vibration feedback since the introduction of the Rumble Pak for the Nintendo 64 in 1997. Vibration motors come in all shapes and sizes, the most popular being the Eccentric Rotating Mass (ERM) motor, found most modern gamepads. Mobile phones often use very small ERM motors, or. more recently, linear actuators. Linear actuators tend to offer more haptic ‘detail’ and responsiveness, as can be found in Apple’s ‘Taptic Engine’, the HTC Vive controllers, an the Oculus Touch controllers. While vibrations as haptic feedback is the current state of the art in the consumer realm, limitations remain.
Tactile feedback has proven to be effective across a wide variety of applications, but if you’re looking for significant force or resistance in your haptics, you need kinesthetic feedback. This is commonly available through force-feedback controllers, which tend to be designed for a specific task, such as joysticks for flight/space simulators, and wheels for driving simulators. The wealth of powerful haptic hardware on the market is one of the main reasons why flight and driving simulations are already so effective in VR. The closest product to a general-purpose kinesthetic controller is probably still the Novint Falcon, first shown in 2006, but this is also fairly limited, as it needs to be attached to a desk.
Microsoft Research’s new experimental controllers bring kinesthetics into the VR space, offering two types of force-feedback applied to fully-tracked motion controllers. NormalTouch uses three servo motors to operate a small disc with tilt and extrusion movements, and TextureTouch uses a bank of 16 servos to operate a 4×4 pixel array of small blocks that move up and down to correspond to virtual shapes and structures. The result is a feeling of physical resistance as you drag your finger across a virtual shape, with enough fidelity to actually convey a sense of touch and an understanding of an object’s form and texture.
In both controllers, the feedback surface acts on a single finger or thumb, which may limit the practical use cases. But the key point is that this type of feedback is normally the domain of dedicated devices, elaborate gloves, or exoskeletons, whereas Microsoft Research’s designs are based on a normal handheld controller, which Michael Abrash, Oculus’ Chief Scientist, recently suggested could remain the standard input for VR for decades to come.
Texture is one thing, but offering real resistance (where the virtual world can push back on you) is still a pipe dream however, as there is nothing preventing the user from clipping through objects with today’s VR controllers. But with more realistic haptics, the desire to clip through something is reduced, in the same way that more realistic VR visuals often prevents people from trying to walk through virtual objects.
In their testing, the Microsoft Research team developed a ‘penetration compensation’ technique, that made it appear that the user’s hand was not clipping, by decoupling them from the real tracking location. The finger is the most sensitive part of the hand to kinesthetic feedback, so this is effective, although it remains to be seen how this haptic-visual mismatch could work in a less controlled environment.
The research group’s findings are promising even at this early stage. Three tests were run—targeting accuracy, tracing accuracy and fidelity assessment—and both controllers were used, comparing them to vibration-only feedback and visual-only feedback. Both new haptic feedback techniques demonstrated advantages over vibration and visual only tests, despite some of the limitations of the prototypes causing issues.
The designs can no doubt be improved dramatically in terms of ergonomics, range of movement, responsiveness and detail, but already testers reacted positively to the heightened sense of touch. It was noted that the fact it’s already effective demonstrates the overriding power of the visual system, and that perhaps fully detailed or accurate feedback isn’t too critical, as the visual system automatically makes the corrections.
Perhaps the toughest challenge of this project is in improving the physical design. Any device with a large number of mechanical parts always comes at a cost, usually in the form of weight and noise, and that’s certainly the case here. If this technology was utilized for a consumer product it would need to get smaller while staying quiet and reliable. It’s an area of research that is worth pursuing further, but it’s unclear at this stage how likely these prototype haptic technologies are to find their way into a real product.
Tactical Haptics, a company pioneering a novel form of haptic feedback which can create compelling sensations that go far beyond rumble, announced today it has raised $2.2 million to create a development kit of a haptic VR controller as a stepping stone to an eventual consumer product.
Tactical Haptics is one of the OGs of the of the new VR landscape. The company was among just five or so companies exhibiting anything related to VR back at GDC 2013 (the first year Oculus attended the show). At the time they were showing their ‘Reactive Grip’ haptic technology attached to a hacked up Razer Hydra (a popular VR motion controller in those early days, long before the likes of Vive controllers and Oculus Touch). This (old) video, explains how it works:
Reactive Grip is a novel method of haptic feedback which uses sliding segments in the handle of a controller to create ‘shear’ forces in your hand which mimic an object moving against your palm, like the handle of a gun when it shoots, or the handle of a sword when it comes in contact with an enemy. The effect is unique and impressively convincing for certain interactions, and in many cases feels more authentic than mere rumble.
Tactical Haptics tried to jumpstart their path to creating a Reactive Grip developer kit in an unsuccessful 2013 Kickstarter that raised nearly $90,000, but was unfortunately well short of its $175,000 goal. But that didn’t mean defeat for the company, which has been refining its tech ever since; we’ve seen multiple prototype iterations since that time, including demonstrations tapping into the tracking of the Vive’s controllers and Oculus Touch.
Now the company has announced that they’ve raised $2.2 million to create a development kit of a Reactive Grip VR controller. $749,000 of the funds come from a National Science Foundation grant, while the other $1.47 million comes as venture captial in a round led by SV Tech Ventures and the Youku Global Media Fund, with participation by SIG Asia Investment Fund, Sand Hill Angels, and the Stanford-StartX Fund. The company says the funds will be used to “[create] a developer kit, including mini-games, for VR game developers so they can integrate the company’s advanced haptic controller with their VR game content.”
The company plans for the development kit to have cross-platform support between the Rift and Vive, and include its own tracking to function as a replacement for the official VR controllers on those platforms.
Exactly what form that tracking will take remains unclear, especially given the competing tracking technologies employed by the Rift and Vive. When we reached out to Tactical Haptics CEO, Will Provancher, for comment he shared the following:
…our plan for tracking won’t be to mount an entire Vive or Touch controller on our haptic controller. So we plan to have a more integrated solution than what we currently do (which is mount a Vive or Touch controller on our controller).
I can say that we are attending the SteamVR tracking partner program training, which will create several options for us to implement tracking.
However, the exact implementation of what we do with respect to motion tracking integration will depend on a lot of factors so it’s hard to say more than this at this point.
While Oculus has said previously that they planned to open their tracking API up to third parties, Valve is the only one of the two to have done that so far, meaning its more likely that we’d see a development kit that’s compatible with SteamVR Tracking sooner than Rift’s ‘Constellation’ tracking.
Whatever form it comes in, we look forward to haptic tech in VR that goes beyond common rumble, and hope that Tactical Haptics’ approach is just the first of many third-party VR controller choices for users.
The new Leap Motion Mobile Platform consists of hardware and software optimised for VR and AR hand tracking on mobile devices. Building on the success of the original Leap Motion device, the brand new hardware aims to be tightly integrated into future mobile VR headsets.
Designed as a natural motion interface for PC and Mac, the original Leap Motion Controller began shipping in volume in July 2013, for $80. This attractive price was largely achieved by a breakthrough in software; the hardware itself was fairly simple, containing two cameras and three infrared LEDs. Early hand tracking applications which interfaced with traditional displays seemed somewhat abstract, but the compact dimensions and light weight meant that Leap Motion Controllers soon found themselves attached to the front of Oculus Rift development kits. This allowed users to finally ‘see’ their hands in VR, with fully-tracked fingers, marking the beginning of a fruitful relationship between Leap Motion and VR.
Since then, Leap Motion has improved their VR support significantly, and the importance of software was further illustrated by the huge jump in technology delivered by ‘Project Orion’ which began as a major software update in February designed specifically for VR. While still using the original hardware, Orion delivered massive improvements to tracking speed and accuracy. However, the Leap Motion Controller’s hardware was finalised in 2012, and we’ve been waiting patiently for the second phase of Orion, a brand new tracking system designed for VR and built in to headsets.
That day is almost here, with Leap Motion’s chief technology officer David Holz revealing the Leap Motion Mobile Platform on the company’s blog. The improvements address the field of view, the ‘biggest request from the VR community’; the brand new sensor, which is designed for low-power mobile devices, delivers a 180×180 degree field of view—up from 140×120 degrees on the original Leap Motion Controller—and is said to run at 10 times the speed of the original hardware while using “much lower power”. The increased field of view means the user’s hands can continue to be tracked even when held in a more natural location rather than directly front of them, a major pain point for VR use with the original device.
As clarified in an answer in the blog’s comments, this is intended to be embedded technology, and the company has no plans for a new standalone peripheral. No further information was provided in terms of timeframe or pricing, but the company says we can expect to see Leap Motion technology in multiple mobile headsets in the near future.
Holz said in the announcement that the company has created a reference platform using the new Leap Motion Mobile Platform that’s built on the Gear VR headset. The plans to demo the reference device at upcoming events starting this month.
Ilium VR are working on a new VR-centric gun peripheral which plans to use Valve’s SteamVR ‘Lighthouse’ tracking to bring realistic weapon control to virtual reality games.
Shooting things in games is cool. Shooting things in VR can be infinitely cooler. Add in a dedicated controller that gives you the physical feel of a weapon and add tracking and peripheral functionality to it and you have something that has the potential to take shooting things to the next level.
Ilium certainly think so as they already begun a crowdfunding campaign to kick off their plans to build a virtual reality-centric gun peripheral to be powered by Valve’s Lighthouse tracking system. Athena is the peripheral’s name and it will come in the form of a futuristic looking assault rifle.
For clarity, outside of the design renders seen in the above promotional video, the rifle peripheral being demonstrated is not Athena but Ilium VR’s earlier VR gun dev kit “The Persuader”, modeled on the M1A1 Thompson ‘Tommy Gun’ and using a rudimentary mount for a standard SteamVR controller in order to leverage its tracking.
The Athena is an altogether more ambitious beast and promises recoil action, the ability to reload via a magazine pull-out / pull-in action, whilst also integrating standard joystick and button inputs. What makes the device interesting however is the team’s plans to integrate Valve’s ‘open’ SteamVR ‘Lighthouse’ tracking functionality into the device. Basically, the final peripheral would be clustered with sensors to enable the gun to detect the sweeps from SteamVR’s laser base stations, and as with the packed-in SteamVR controllers, beaming that data wirelessly for a compatible game to map to in game actions.
We reported recently that Valve’s long awaited plans to open up the Lighthouse platform to form a tracking standard of sorts for developers and manufacturers had finally begun in earnest. Valve announced just prior to the company’s Steam Dev Days event that it’s already licensed Lighthouse tracking to some 300 companies, royalty free since it’s original announcement back in august. According to Valve, the company are gearing up to show off “new VR peripherals” which will be made available for “demonstration and design collaboration with attendees,” at some point in the future.
The Athena is one of the first gaming specific product we’ve seen to take advantage of Valve’s new initiative and it’s a brave move. Traditionally, peripherals for gaming have rarely fared well. Gaming systems (including the SteamVR powered HTC Vive of course) ship with standard controllers allowing developers to target them and hone their software knowing that everyone who buys their game or application will own those devices. 3rd party peripherals on the other hand require the manufacturer to acquire interest and support from those same developers to add in legacy support for the proprietary device. The Catch 22 situation of course is that, developers don’t want to commit resources to build around a device few customers own and customers won’t buy hardware without software support.
Illium are of course aware of this and have tried to mitigate the issue by entered into a partnership with Invrse Studios (The Nest) and Vertigo Games (Arizona Sunshine) who will both include updates to add native support for Athena to their games. A good start indeed.
Athena is currently being offered via an Indiegogo campaign with a lofty $100,000 funding target. Backers can notionally get their hands on a single gun along with The Nest and Arizona Sunshine for $300. The device itself has a timeline for delivery of December 2017. The campaign seems troubled currently however, with just over $4000 of that goal currently met. It’s not clear how key the funding is to progress of the product.
It’s an interesting project and one we hope to see more of in the future, especially if the Ilium team manage to square the peripheral support issue mentioned above. However it does join Stryker, another VR-centric peripheral featuring impressive recoil, in the same space. We’ll watch with interest to see how much traction both projects gain in the future.
At GDC 2017, developers Impulse Gear confirmed that their VR shooter Farpoint has Co-op, and will launch in a bundle with the PS VR Aim Controller on May 16th. An ‘unnerving space adventure set on a hostile alien world’, Farpoint is a free-movement FPS exclusive to PlayStation VR.
Build from the group up for PlayStation VR by independent studio Impulse Gear, Farpoint is a free-movement FPS designed to be played with the PS VR Aim Controller. While the game can be played on a standard PS4 gamepad, the new peripheral was developed by Sony with input from Impulse Gear, and will be launching as a bundle on May 16th.
During a developer session at this week’s GDC about the Aim controller, which is expected to receive support for several future PS VR titles, Impulse Gear confirmed Farpoint will have a co-op mode.
The PSVR Aim Controller has a friendlier appearance than the gun-like Sharp Shooter PS3 accessory, and benefits from having PlayStation Move-style features such as a tracking sphere, trigger and thumbstick integrated into the unit, meaning no separate Move controllers are required. The integrated motion sensors mean that the new controller is more accurate than the Sharp Shooter, able to deliver 1:1 tracking of the in-game weapon.
Farpoint is most notable for its free movement (often referred to as ‘full locomotion’) – something that is typically avoided in VR FPS in favour of a teleport mechanic due to its tendency to cause nausea. The 1:1 weapon tracking, combined with careful attention to movement speed and animation, assisted by IKinema’s real-time inverse kinematics, means that it is able to deliver a comfortable experience, as described in our hands-on.
Tactical Haptics, developers of the Reactive Grip controller, are showing their latest prototypes now with attachments for the Vive Tracker, Oculus Touch, and a custom-built SteamVR Tracking solution. The controller employs a unique solution to haptic feedback which aims to recreate the feeling of friction against objects in your hands rather than just rumble. The company is moving toward bringing a development kit of the device to developers.
Tactical Haptics has been in development of their Reactive Grip haptic technology for several years now. Having shown off some of their earliest prototypes at GDC 2013—years before HTC and Oculus even began talking about VR motion controllers—tracking has remained a hurdle in getting the product ready for consumers. This (old) video shows the foundation of the haptic technology which we’ve said ‘proves VR needs more than rumble‘.
Vive Tracker and Oculus Touch Tracking
Now that both Oculus’ Constellation and Valve’s SteamVR Tracking systems are deployed in users’ homes, the door is open to using those systems as add-ons to track the Reactive Grip controller for use in VR. That means users who already own Touch or a Vive Tracker can attach those peripherals without the need to bear the cost of additional tracking hardware built into the controller.
While the company had shown off a similar approach previously by attaching the Vive controllers to their haptic controller, Tactical Haptics founder William Provancher says that between the Vive Tracker and Touch controllers, the lighter weight and more compact profiles make the overall device lighter, more balanced, and more comfortable to use. Though that’s not to say that Vive controller adapter might not be offered when the Reactive Grip controller becomes available.
Custom SteamVR Tracking
Thanks to Valve opening up their tracking solution to third-parties over the seven months, Tactical Haptics is also experimenting with a custom SteamVR Tracking solution which could be offered for those who want to buy an all-in-one controller. Provancher says the company attended the SteamVR Tracking development course and had created a working SteamVR Tracking integration for the controller in just a few weeks. Though the company is still refining the integration, Provancher says early tests reveal that it tracks just as well as the Vive Tracker.
At GDC 2017 this week, the company was showing off the new controller prototypes with new mini-games made to show what its like to develop for the controller and what sorts of applications the unique haptic feedback can be applied to.
Using the Reactive Grip controller, I played a game that was something like ‘VR Asteroids’ where I used my hand to fly a little ship around to avoid asteroids and incoming fire from enemy ships. Using the orientation of the controller and the trigger, I could fire the ship’s weapons to destroy asteroids and enemy ships. The controller’s haptics gave me a sense of the ships momentum in my hand and feedback as my ship took damage and fired its weapons.
The other game, Cyber Golf, was like a futuristic version of disc golf where the goal was to throw the disk into a goal which was blocked by obstacles. In the game I held a wand-like tool which could be used to grab the disk. Grabbing on the disc’s edge let me throw it like a frisbee, while grabbing the core extended a laser-rope from the wand-tool that let me whirl the disc over my head like a lasso and then throw it for extra distance. While spinning the disk over my head, the controller gave me a sense of the disc’s weight as its momentum pulled the tool in a circular motion in my hand.
Both mini-games were fun and functional, but not the most compelling demos I’ve seen (and felt) from these controllers. Prior demos that I’ve tried using the controller—like gun shooting, sword wielding, and using a ‘Gravity Gun’-like tool to swing boxes around—gave me a more immersive sense of connection between what I was doing and how the haptics felt on my hand. But, importantly, the new mini-games on display at GDC show how the tech can be applied in a more abstract way, which opens the doors to more gameplay possibilities that would make use of the controller’s unique haptics.
In November, Tactical Haptics announced that they’d raised $2.2 million to finalize a development kit of the Reactive Grip controller, and now the company has begun soliciting developer interest for dev kits. The company suggests reaching out by email to info@tacticalhaptics.com for more details about development kits.
Qualcomm has debuted an updated version of their VR Headset Reference Design now with Leap Motion’s new 180-degree hand-tracking to bring gesture control to mobile VR headsets. The new headset and Leap Motion tracking module was shown off during last week’s GDC 2017.
Qualcomm’s VR Headset Reference Design has been upgraded to the company’s new Snapdragon 835 mobile platform. The purpose of the headset, which the company calls the VRDK (Virtual Reality Development Kit), is to act as a foundation for Qualcomm’s device partners to make their own VR headsets based on Qualcomm’s mobile computing hardware.
And now, Qualcomm’s latest VRDK brings hand-tracking into the mix thanks to Leap Motion. Leap Motion has been working on hand-tracking technology since 2010, and in recent years has pivoted their focus toward use as an input technology for VR. And while the company’s initial hand tracking device—originally built for use as a desktop peripheral—has seen some use in VR by strapping the device to the front of a headset, the limited field of view meant that users had to told their hands up in front of their face for the device to be able to track their hands in VR. Ultimately VR controllers have thus far become the defacto standard for motion input on tethered VR headsets.
But when it comes to mobile VR, where the goal is to have a single, self-contained unit that doesn’t rely on external tracking sensors or beacons, Leap Motion may have found a perfect fit; hand-tracking is more immersive than the limited rotation-only controllers that we see with Daydream and others (like the newly announced Gear VR). Having the tracking be totally on-board also means one less piece of equipment to tote around, helping to keep mobile VR portable and easy to use.
Leap Motion identified this sweet spot a while back and has been teasing a new mobile solution that would address the field-of-view limitation that came from strapping the company’s pre-VR device onto VR headsets. The company formally announced the mobile made-for-VR module in late 2016, and now we’re seeing the first glimpses of integration into Qualcomm’s newest VRDK, which I got to try out at GDC 2017 last week.
Though the new Leap Motion mobile module is technically still an attachment to Qualcomm’s VRDK, Leap Motion says that it will be directly integrated into mobile VR headsets built on the VRDK that opt for the hand-tracking tech.
The new mobile module as seen at GDC 2017 hugs closely to the Snapdragon 835 VRDK and was clearly made to fit the device specifically. With two wide-angle lenses, Leap Motion says the module provides a 180 degree field of view for hand-tracking. Indeed, I could feel a significnat difference between the new module and the old one. With the headset on and my hands out in front of me, I could grab objects and let them out of my own field of view through the headset, and when I looked down I could see that I was still holding the object.
The increased tracking field of view is bolstered by smart tweaks to the hand-tracking software; such that if I was holding an object and then turned my head (causing the object to truly leave the tracking module’s field of view) the software would remember that I was holding that object (and in which hand) once it came back into view, and often identify my hand holding the object before it came back into the headset’s own field of view, making a big improvement from the compelling-but-frustrating experience of the original desktop module.
The mobile module doesn’t just have a wider field of view, it’s also built for high power efficiency so that it makes sense to add to mobile VR headsets. Leap Motion says that the module runs at 10x the speed of the original device at significantly lower power. It’s also very tiny.
VRgluv has blown past its $100,000 funding goal in 56 hours on their Kickstarter page, with 27 days still to go. The product is described as the “first affordable force feedback gloves” that feature “total hand tracking, full force feedback, and pressure sensitivity”.
Update (4/28/17, 12:39PM PT): VRGluv has now exceeded its $100,000 Kickstarter goal, presently just over $113,000. The company has announced in an update a $250,000 stretch goal which will add a replaceable battery to the glove. The company has also opened a new $370 pricing tier limited to 400 backers.
Original Article (4/27/17): Going live on March 31st, VRgluv’s website revealed their haptic gloves, compatible with both HTC Vive and Oculus Rift hardware, said to be comfortable, functional, and affordable (despite looking rather clunky). An appealing $300 price point (relatively speaking) for the Super Early Bird is the likely reason for the rapid influx of early backers and, being limited to 100 backers, is already sold out. The next tier at $350 is limited to 200 sets, followed by a ‘Kickstarter special’ price at $400, with the final retail price expected to be $580.
As shown in the Kickstarter video, different adapters allow for Oculus Touch controllers, HTC Vive controllers or Vive Trackers to clip to the sides of the gloves to perform the spatial tracking duties, with the gloves containing proprietary technology to determine finger positions and grip strength. Ideally, Vive Trackers would be used—being the least-bulky attachment to what is already a chunky pair of gloves—although VRgluv describes the units as ‘lightweight’, and the adapters are said to be carefully designed to hold each tracking solution in the most balanced position.
VRgluv is one of several devices in development that provide a haptic feedback solution for hand interaction in VR. Others include the EXOS, the Dexmo exoskeleton, the temperature-changing Senso and PowerClaw, the Gloveone and Avatar VR from NeuroDigital Technologies, and more. As the haptics challenge is being approached from so many different angles, it’s difficult to predict if one product will rise to the top; this area of VR development is likely to remain experimental and niche, although VRgluv’s price is impressive considering the low volume, wireless technology, rechargeable batteries, and the likely high number of mechanical components involved.
VRgluv is aiming to deliver the first sets to customers in December 2017.
Today Microsoft has unveiled their Windows Mixed Reality motion controllers. The controllers fill in a major gap in Microsoft’s strategy to become the “most complete platform across the broadest range of mixed reality devices and experiences.”
While we expect to hear a lot more about the company’s VR controllers at the Build conference today, Microsoft has so far confirmed the following:
The controllers will come bundled with the Acer VR headset for $400 and be available on retail shelves this holiday season
The controllers offer 6 DOF tracking, and are tracked within your field of view using the sensors in the headset and in the controllers
Additional sensor technology coming soon could extend the controller’s tracking abilities outside of the headset’s field of view
When asked if the controllers were only available via a bundle from a given headset partner, Microsoft clarified that their hardware partners will be able to decide on an individual basis if they would like to bundle a pair of controllers or not. Microsoft confirmed that just as there are specifications for Windows VR headsets, there are similar specifications that must be met for any given OEMs implementation of the Windows VR motion controllers. This could mean that we’ll see many flavors of the Windows motion controllers, just as there are already many companies (Acer, ASUS, Dell, HP, Lenovo, and 3Glasses) planning to release a Windows MR headset.
Photo courtesy Microsoft via CNET
There are however a different set of these specifications for developer hardware and the hardware that will end up being shipped to consumers. This means that the development kit headsets we’ve seen so far from companies like Acer could be somewhat under the specifications we’ll see at the consumer launch. However, one member of the Microsoft MR team said that there would be “basically no difference” between the current Acer dev kit and the version that will ship to end users.
Acer’s Windows-compatible mixed reality headset will ship with bundled VR controllers | Image courtesy Windows Central
When asked to provide more information on the new motion controllers, a member of the Windows MR team said that the controllers were in fact true six degrees of freedom controllers that take advantage of Microsoft’s inside-out tracking. The team member said that for now the controllers need to be within a given sightline of the headset, but that there may be additional sensor technology coming soon that could extend the controller’s tracking to fully behind a user’s head or back.
Having done the core R&D for what would become the Vive controllers, Valve is continuing to iterate on VR input. The company’s new ‘Knuckles’ controllers are not so much ‘held’ as they are ‘worn’, offering users a natural grab and release motion without dropping the controller. Since their debut toward the end of 2016 at the company’s Steam Dev Days event, a Knuckles development kit is now shipping to select developers and brings ergonomic improvements over earlier prototypes.
Though there’s no open application to get your hands on the SteamVR Knuckles dev kit, the new controllers are now shipping to select developers. Among them is Cloudhead Games, one of the world’s most senior VR game studios, and an early VR collaborator with Valve. The studio posted pictures of their Knuckles dev kit delivery this week.
Valve introduced the Knuckles controllers toward the end of 2016. From that prototype version to the latest dev kit Valve has tweaked the controller quite significantly for improved ergonomics, Cloudhead confirmed via Twitter, calling the latest version “considerably more comfortable over all.”
From a comparison photo between the original prototype and the new Knuckles dev kit we can see that the main grip has been increased in size while the mounting mechanism has been simplified from a proto-glove with velcro to a piece of fabric with a cinch. The center button has changed from a sunken oval to a circle.
Image courtesy Cloudhead Games
From the photo we can see that the latest Knuckles dev kit still has exposed photodiodes, the sensors that cover the controller and allow it to be tracked through space; like the production Vive controllers today, we’d expect to see those encased in IR-transparent plastic by the time the consumer model is nailed down to protect the sensors from damage.
Another photo from Cloudhead shows the evolution from some of the earliest SteamVR controllers to the Knuckles dev kit.
Image courtesy Cloudhead Games
Cloudhead games also confirmed that the Knuckles controller is backward compatible with existing SteamVR Tracking basestations (ie: those that ship today with the Vive).
Valve hasn’t yet announced when the controller will make its way to consumers or what they might cost. Though Valve plans to sell SteamVR Tracking basestations directly, our guess is that the company will leave it up to SteamVR partners like HTC to manufacture the Knuckles controllers en mass; with LG also working on its own SteamVR headset, we presume the controller design will be open for either company to draw from.
New footage of Valve’s new ‘Knuckles’ controllers has emerged over the past few days, as developers begin to test out the new device. The ‘next-gen’ SteamVR controller prototypes represent a major advancement over the current Vive motion controllers, with five finger tracking and an ‘open hand’ grip.
The new controllers have been shipping to select developers, with prominent VR studio Cloudhead Games being one of the first the receive packages, which they’ve been keen to share on social media. A short video (heading this article) was released to their YouTube channel, presented by Cloudhead’s CEO and Creative Director Denny Unger, showing the simple package and intuitive hardware design.
Unger points out that you can grab and use them just like the Vive controllers, but the “magic happens” when you pull the cord to tighten the cinch that allows the user to fully release their grip. This short clip from Cloudhead shows the sort of advantages that can bring:
Playing around within the default SteamVR Home environment using a ‘five finger’ hand model option for the avatar, Unger highlights the capacitive sensors on the grip that allow for individual finger detection, and the trackpad that offers a more granular control of the virtual thumb position. The studio has been closely associated with Valve’s new design from an early stage, having been asked to create a demo for the first prototype hardware at its announcement at Steam Dev Days last year.
Other short clips of the controllers in action show developers rolling their fingers across the grip and ‘flipping the bird’, along with reactions and ‘unboxings’ from other notable VR studios such as Owlchemy Labs, Radial Games and Vertigo Games; see a collection of these Twitter ‘Moments’ here. Valve have not revealed their plans for a consumer version of the Knuckles controllers at this time.
Microsoft debuted the lengthily-named “Windows Mixed Reality motion controllers” back in May, but until now we haven’t had a chance to actually try them out. During a recent meeting with Microsoft in San Francisco, I got to try the VR controllers for the first time paired with the Acer Windows VR headset.
Microsoft’s VR controllers are designed to let you reach into VR and interact naturally with the virtual world. With both a trackpad and a thumbstick, they look like a crossbreed of the Oculus Touch controllers and the Vive controllers.
In addition to the trackpad and thumbstick, there’s also a menu button and a Start button, as well as a grip button along the handle. The big circular parts on the front contain an array of LEDs which provide bright markers for the headset’s on-board cameras to detect and track. Microsoft tells us that the shipping version of the controllers will indeed use visible-light, just like we’ve seen in renderings and promo videos. (Microsoft didn’t allow any pictures of the controllers during my hands-on time).
Buttons and Inputs
Image courtesy Microsoft
Grabbing the controllers for the first time, they didn’t feel quite as elegant as either Touch or the Vive controllers. The odd side-by-side trackpad & thumbstick arrangement is useable, but seems to effectively put neither of the two in an ideal position for your thumb. The grip button is indeed a binary button (rather than being pressure sensitive), and doesn’t feel so much like a “grab” as it does a clicky button press with your palm.
Ergonomics
Though they resemble Touch with their ring-shaped tracking appendages, the Windows motion controllers are actually noticeably larger and clunkier thanks to the placement of the tracking rings, which don’t encompass your hand like Touch, making the controllers easier to bump together, especially when their physical outline is hidden in VR.
The shape of the rings is necessary though, as they need to present a substantial surface area from which the headset’s on-board cameras can track their movement. Though I was using the Acer dev kit headset, our understanding is that these controllers will work with any of the soon to be released Windows VR headsets (all of which feature on-board cameras).
Pros and Cons of Inside-out Controller Tracking
Image courtesy Microsoft
This method of controller tracking differs from both the Rift and the Vive in that it’s the cameras on the headset which are watching the controllers to track their movement (which is called ‘Inside Out’), whereas the Rift and Vive both use external sensors to track their controllers (called ‘Outside In’).
The upside to this approach is that you don’t need to set up any external trackers, but the downside is that the controllers must always be in view of the headset’s front-facing cameras to be properly tracked. Thankfully, the size of the tracking volume felt reasonable; for basic use (like reaching out in front of me to grab virtual objects), I didn’t feel like my reach was artificially limited by the camera’s field of view.
Outside of the Box Tracking
And for times when your hands will go out of the camera’s field of view, Microsoft is doing its best to compensate. When that happens, the system relies purely on the controller’s on-board IMU to estimate positional movement until it reappears in the camera’s view. This works well enough for quick jumps in and out of the camera’s view, but after a second or two, the IMU-only tracking estimation is too unreliable, and it appears that the system will eventually freeze the location of the controllers in the air and only feed them the rotation data from the IMU, though they snap quickly back into their proper place as soon as they’re brought back into view. It remains to be seen how much this limitation (the need to be seen by the front-facing cameras) will impact different VR games and apps, and how effectively it can be designed around.
Tracking Accuracy
As for the tracking accuracy when they are in sight of the camera, I did see some jumpiness here and there—especially if I was rotating my body while moving the controllers—but on the whole they seem entirely usable, and (in my short time with them thus far) to be more accurate than the PlayStation Move controllers.
As part of my testing, I played Arizona Sunshine (2016), and found that guns were steady when I held them out in front of me and aimed down their sites; I didn’t have any trouble landing zombie headshots. Granted, for inside-out controller tracking, holding a gun up in front of me to aim is pretty much the best case scenario—I’m curious to see how other common input modalities hold up (like shooting a bow and arrow or swinging a sword).
The occasional jumps didn’t present much issue in a shooting scenarios, but for more precise uses, like VR drawing, painting, and animating, it remains to be seen if those jumps will cause any usability issues.
– – — – –
Microsoft says that the Mixed Reality motion controllers will be bundled with Windows VR headsets starting at $400 this holiday.
Got questions about the Mixed Reality controllers? I’ll be taking questions in our comment section below.
Earlier this week we shared our first hands-on with Microsoft’s VR controllers, but at the time the company didn’t allow us to document the session with photos or videos. Now, a developer with the controllers has produced a handy overview that shows how they work and gives a glimpse of them in action.
The Windows “Mixed Reality” (Microsoft’s term for AR and VR) controllers are unique among similar VR controllers (like those of the Rift and Vive) because they don’t require external sensor for motion tracking. Instead they are tracked by cameras on the VR headset (as far as we know, all of the Windows VR headsets will support the controllers). That means that setup is simplified, but also comes with a crucial downside which is that the controllers lose their positional tracking when outside of the camera’s view for more than a second or two.
Developer Sean Ong demonstrates this, while overviewing the controllers, by placing a bag over the tracking markers to block them from the camera’s view. When that happens the positional tracking is lost but the controller continues to reflect proper rotation thanks to internal rotation sensors. When the controller comes back into view of the camera, they pop back into place and regain positional tracking.
For many VR games and applications this limitation not even be noticeable, especially because Microsoft is doing a bit of prediction to compensate for brief moments of positional tracking loss, though it could impact the experience for some apps where players frequently have their hands outside of the camera’s field of view.
Otherwise, Ong’s experience matches my own thoughts after a hands-on with the controllers: the placement of the trackpad and thumbstick is a little strange, and while the tracking may have some limitations, it’s definitely functional.
In a study lead by Eisuke Fujinawa at the University of Tokyo, a team of students created a procedure for designing compact VR controllers that feel physically larger. Exploring the concept of ‘haptic shape illusion’, the controllers have data-driven, precise mass properties, aiming to simulate the same feeling in the hand as the larger objects on which they are based.
Simulating the feel of real objects is a fundamental haptics challenge in VR. Today’s general-purpose motion controllers for VR work best when the virtual object is reasonably similar in size and weight; very large or heavy virtual objects immediately seem unrealistic when picked up.
One solution is to use specific controllers for a given application—for instance attaching a tracker to a real baseball bat; in a hands-on with one such solution, Road to VR’s Ben Lang described the significance of gripping a real bat and how that influenced his swing compared to a lightweight controller. But swinging a controller the size and weight of a baseball bat around your living room probably isn’t the best idea.
As shown in the video below, researchers from the University of Tokyo attempted to create much smaller objects that retain the same perceived size. The team designed an automated system which takes the original weight and size of an object and then creates a more compact but similar feeling output through precise mass arrangement.
The paper refers to several ecological psychology studies into how humans perceive the size of an object through touch alone, supporting the idea that perceived length and width is strongly related to the moment of inertia about the hand position.
The team concentrated its efforts on this haptic shape perception, collecting data from participants wielding different sample controllers in VR to determine their perceived sizes, having never seen the controllers in reality. This data allowed the creation of a ‘shape perception model’, which optimises the design of a large object within smaller size constraints, outputting CAD data for fabrication.
The object is deformed to fit the size constraints, holes are cut out, and weights are placed at specific points to maintain the original moment of inertia.
Image courtesy Fujinawa et al.
The team had VR developers in mind, as this approach could offer a potential benefit in demonstrating a product with a more realistic controller. The CAD data output means that smaller, safer prototype controllers that give the impression of wielding larger objects can be created quickly with a laser cutter or 3D printer.
Tactical Haptics’ newly developed haptic controller prototype uses mechanical sockets that allow them to be mated in different configurations on-the-fly, in order to match a particular virtual interaction more closely than standard VR motion controllers. The controllers incorporate the company’s ‘Reactive Grip’ technology, a unique form of haptic feedback.
Image courtesy Tactical Haptics
San Francisco-based Tactical Haptics is debuting its reconfigurable haptic controllers together with new demo content at CES this week. The hardware is based on the haptic controllers used for Justice League: An IMAX VR Exclusive that has operated at the Los Angeles IMAX VR Centre since November 2017, but with the ability to be mated together in common interaction configurations, such as ‘gamepad’, ‘steering wheel’, or ‘machine gun’ poses.
Image courtesy Tactical Haptics
As described in the press release provided to Road to VR, the mechanical sockets (which appear to be fitted with magnets) “provide a mate-point … to form a semi-rigid coupling between the controllers that allows the users to effortlessly maintain the mated poses.” The images are shown with Oculus Touch controllers for tracking purposes, but they also have mounts for Vive Trackers.
Image courtesy Tactical Haptics
Colony Defense, a new game developed by Tactical Haptics to demonstrate the hardware, is a first-person experience with building and combat elements. The player is asked to join the two controllers to create a ‘physics gun’, then separate the controllers to operate a jet pack and weapon each hand, and the option to combine the controllers in the ‘machine gun’ configuration to operate a heavier blaster. The company says that “significant effort” was put into optimising the placement of the sockets to result in ergonomic poses and to aid on-the-fly reconfiguration while wearing a headset.
Image courtesy Tactical Haptics
A new “brick breaker” style game called Cyber Smash is also at the show, which the company says demonstrates “feeling the inertia of throwing smash-balls and settling of the ball after it rebounds and is caught by the player.” As highlighted by the IMAX VR Centre partnership, the company is currently focused on location-based entertainment, and is working on multiplayer versions of both games for this purpose. It is seeking partner opportunities with high-profile LBEs while at CES this week.
Both games make use of Tactical Haptics’ core innovation: an advanced haptic feedback technology called Reactive Grip, showcased in various prototype controllers since 2013. Actuated plates in the controller’s handle apply friction and shear forces in the hand, creating various tactile illusions such as inertia and elasticity.
Stay tuned to Road to VR for further coverage of CES 2018, including a hands-on with the new Tactical Haptics controller prototype.
I had a chance to try out the 
