Virtual perfection: why is 8K resolution insufficient for realism?

Palmer Lucky, founder and creator of Oculus Rift, becomes a perfectionist when it comes to creating the best virtual reality experience. Not so long ago, we presented a small overview of everything the guys had achieved in developing the best of virtual reality re-creation devices, and now we found out that Lucky actually believes that we have no idea what pixel density should be.

"Let's just say," say the creators of Oculus Rift. - “What is meant by Retina Display is bullshit.”

"There is a point beyond which you can no longer distinguish individual pixels, but this does not mean that you cannot distinguish more details," says Lucky. “You can still see smoothed lines on Retina displays.” You cannot select pixels, but you see anti-aliasing. Let's say you want to take a real snapshot of your hair. You cannot make them in natural size, otherwise they will look scary and jagged. There is a definite difference between when you cannot see the pixels and the fact that you cannot improve the details. ”

Chicking a full-size hair image may seem pompous, but it is Lucky who thinks about this level of detail when he thinks about the future of virtual reality.

“To get to the point where you can't see the pixels, I think for the Rift field of view, you need a resolution of about 8K in one eye,” says the developer. - “And in order to reach the point when you have nothing to improve, you need to overcome this barrier several times. It sounds funny, but for ten years TVs have been on the user market, and now we have tablets and smartphones with the same resolution. Count off ten years from now, and a helmet mounted display with 8K resolution will not seem like a funny invention. ”

More pixels, less trouble

Before we get to this level of image in front of your face, Lucky says that now all the efforts of Rift developers are focused precisely on the current resolution.

“The visual part of the Rift is one of the most important. Every time we manage to increase the number of pixels, everything changes drastically. When working with the phone, a high density of pixels per inch is particularly invisible, and after a certain threshold it becomes all the same; but in the case of virtual reality, doubling the pixels still brings a noticeable improvement. ”

As for the current situation of the Oculus Rift HD, even with HD-quality, you can see the small borders between pixels, if you really try, but the additional resolution provides a more convenient opportunity to read in the fine print, to notice distant enemies or other small details. Increasing resolution also reduces the level of nausea, sometimes occurring at lower resolution.

Oculus Vice President Neith Mitchell says that the final version of the Rift will be at least as good as the HD prototype, which the company can already boast of (although it does not disclose the details or the price).

“We try not to use this trick, when something amazing is shown, like a concept car, and then something else is released, and people say: bullshit,” adds Lucky. - “It would be too evil an act on our part. There are a lot of flaws, both in optics and in screens, all because it is a prototype. The consumer version will be significantly better in several key areas, and we want to focus on the user side, which means there will never be too many improvements. ”

Delay and position

From left to right: Palmer Lucky, Neith Mitchell, and CEO Brendin Ayribe

Another important issue in delivering truly realistic virtual reality is latency. The time between the movement of your head and the display of it on the screen is the key to create a believable projection. But Lucky and Mitchell say that this problem is as close to its resolution as the Rift device itself is complete.

"We can make the hardware quality and below the threshold of human perception without astronomical prices," says Lucky. “Our hardware will eventually break all records for the lowest latency, but not everything depends only on us.”

This is because equipment is only one part of the delay equation. Even if the delay in the Rift image is no more than one or two milliseconds, the game engine must keep pace with this level of performance.

“If they launch their game at 30 frames per second, huge lags will begin in virtual reality,” says Mitchell. “That's why when you look at a game like Team Fortress 2, which for the most part eats up as many frames as you give it, you can start the game with an extremely high frame rate and it will seem that there are no delays at all.”

Even if you cannot score 500 frames per second in software, there are some predictable tricks with which the equipment can “fool” the user by reducing the delay.

“It turns out that people are pretty predictable machines, we know how they work. For example, if a person starts to turn his head, we can say: yes, we know that he is starting to turn his head, which means we know the approximate curve. He cannot instantly stop the movement, which is also quite predictable. ”

This kind of prediction has developed a bad reputation over a long time, because you can predict only 50 milliseconds of the future, and this is not enough. But when the delay of the system falls to 20 milliseconds, the extra 10 seconds of predictions of future movements can be left without any problems. Such a prediction will reduce the level of delay so that it will seem to you that it is zero.

Reducing the delay will also be key in refining the positional tracking - tracking the user's position, as well as his head from side to side or back and forth, and not just the body rotations in place. Mitchell considers the most important thing to be added to the current Rift prototype to enhance the virtual reality experience. The technology of such tracking is easily accessible, but for the consumer version of the Oculus Rift, you need to consider the cost of technology, as well as tracking area. According to the developers, it will be difficult to predict whether you will rush around the room or sit quietly in place.

“Positional tracking is not just going to buy a specific module,” adds Lucky. “We can buy hardware at a fairly low price, but software is hard to make. It is like Kinect. PrimeSense did a lot of hardware, but PrimeSense did not do Kinect. Microsoft had to spend a lot of money to develop software around this hardware. ”

The article is based on materials https://hi-news.ru/computers/virtualnoe-sovershenstvo-pochemu-razresheniya-8k-nedostatochno-dlya-realistichnosti.html.