Varjo offers a new type of high-def VR/AR display tuned to the user’s eye motions

The folks at Varjo think they’re made a breakthrough in how goggles for virtual reality and augmented reality work. They are onto something.

Most VR/AR goggles have two displays, one for each eye, and they strive to drive those displays at the highest resolution possible. Their hardware and software takes into account that as the goggles move, the viewpoint has to move in a seamless way, without delay. If there’s delay, the “willing suspension of disbelief” required to make VR work fails, and in some cases, the user experiences nausea and disorientation. Not good.

The challenge come from making the display sufficiently high resolution to allow the user to make objects look photorealistic. That lets user manipulate virtual machine controls, operate flight simulators, read virtual text, and so-on. Most AR/VR systems try to make the display uniformly high resolution, so that no matter where the user looks, the resolution is there.

Varjo, based in Finland, has a different approach. They take advantage of the fact that the rods and cones in the human eye sees in high resolution in the spot that the eye’s fovea is pointing at – and much lower elsewhere. So while the whole display is capable of high resolution, Varjo uses fovea detectors to do “gaze tracking” to see what the user is looking at, and makes that area super high resolution. When the fovea moves to another spot, that area is almost instantly bumped up to super high resolution, while the original area is downgraded to a reduced resolution.

Sound simple? It’s not, and that’s why the initial Varjo technology will be targeted at professional applications, like doctors, computer-aided design workers, or remote instrument operators. Prototypes of the goggles will be available this year to software developers, and the first products should ship to customers at the end of 2018. The price of the goggles is said to be “thousands, not tens of thousands” of dollars, according to Urho Konttori, the company’s founder. We talked by phone; he was in the U.S. doing demos in San Francisco and New York, but unfortunately, I wasn’t able to attend one of them.

Now, Varjo isn’t the first to use gaze tracking technology to try to optimize the image. According to Konttori, other vendors use medium resolution where the eye is pointing, and low resolution elsewhere, just enough to establish context. By contrast, he says that Varjo uses super high resolution where the user looks, and high resolution elsewhere. Because each eye’s motion is tracked separately, the system can also tell when the user is looking at objects close to user (because the eyes are at a more converged angle) or farther away (the eyes are at a more parallel angle).

“In our prototype, wherever you are looking, that’s the center of the high resolution display,” he said. “The whole image looks to be in focus, no matter where you look. Even in our prototype, we can move the display projection ten times faster than the human eye.”

Konttori says that the effective resolution of the product, called 20/20, is 70 megapixels, updated in real time based on head motion and gaze tracking. That compares to fewer than 2 megapixels for Oculus, Vive, HoloLens and Magic Leap. (This graphic from Varjo compared their display to an unnamed competitor.) What’s more, he said the CPU/GPU power needed to drive this display isn’t huge. “The total pixel count is less than in a single 4K monitor. you need roughly 2x the GPU compared to a conventional VR set for the same scene.”

The current prototypes use two video connectors and two USB connectors. Konttori says that this will drop to one video connector and one USB connector shortly, so that the device can be driven by smaller professional-grade computers, such as a gaming laptop, though he expects most will be connected to workstations.

Konttori will be back in the U.S. later this year. I’m looking forward to getting my hands (and eyes) on a Varjo prototype. Will report back when I’ve actually seen it.