Asynchronous Reprojection aka stuttering, ghosting, etc

[TechLad76 0]

There’s a huge issue with VR in today’s flight sims that barely gets a mention, and caused me to waste AU$1200 on a Reverb. Asynchronous Reprojection (hereafter referred to as AR), also known as synchronous timewarp, asynchronous spacewarp and SteamVR Motion Smoothing, is a technology used to “mimic” extra frames when the user’s computer can not match the refresh rate of the headset. Many VR users refer to the issues caused by it as ghosting, judder, stuttering, or smoothness. I’m sure there are some other names I’m not familiar with, but the vague way in which it is described, not to mention the huge variety of “fixes” to deal with it, are part of the reason I’m writing this. I thought I’d make a post about it to help clear up some confusion, and potentially save some simmers from spending a lot of money chasing a VR experience that currently does not exist. Note that I’m not a VR professional, but I was a technology journalist focusing on enthusiast PC gaming hardware for 20 years, with a huge passion for VR. I apologise if I make any inaccuracies in this post, please feel free to clear them up with your feedback.

Also, I’m not sledging you if you are a VR user and enjoy it – lucky you! I wish I didn’t notice this problem, as VR is an incredible experience; however, I think if you’ve used a lot of other VR titles where AR is not used at all, it’s very hard not to notice the side-effects. I also have an unproven theory that being a gamer for over 30 years means my eyes are more susceptible to image quality issues.

AR is generally not visible on “through-the-lens” videos due to way cameras differ to the human eye, and it’s more of an issue with high resolution HMDs like the Reverb. AR automatically kicks in when the PC supplying the frames can not match a framerate equal to the refresh rate of the HMD. If you’re running a lower resolution HMD than the Reverb, such as the Rift or Vive, and can actually match your PC’s outputted framerate to the HMD’s refresh rate, AR is not used, so won’t be an issue at all. AR was conceived to be used in rare instances when the PC “hiccups”, and can’t match the refresh rate – it absolutely was not designed to be used 100% of the time, as is often seen in flight sims. This is why every other VR game is designed to ensure that the frame rate matches the refresh rate.

Note that my PC is comprised of basically the fastest components currently available – an i7 9700K overclocked to 4.9GHz on all 8 cores, 32GB of DDR4-3866MHz, an RTX 2080 Ti overclocked by 10% in both memory and core, and twin 2TB M.2 Samsung Evo Plus SSDs. I’ve been building, overclocking and writing about enthusiast level PCs since 1999, so if you’re about to offer feedback along the lines of “you obviously haven’t configured your PC correctly”, trust me when I say my PC is purring like a kitten.)And I found it impossible to hit the 60fps required to remove AR entirely in any DCS missions more complex than a single-aircraft circuit in an area with simple terrain. I’m hoping the upcoming RTX 3090 will allow me to hit 60fps, thus removing AR, in which case I will revisit the VR simming experience. But until then AR is a disappointing deal-breaker that has seen my Reverb Pro gathering dust for six months.

So, what actually is AR? According to Wikipedia, it is “a class of motion interpolation technologies aimed ensuring a virtual reality headset's responsiveness to user motion even when the GPU isn't able to keep up with the headset's target framerate, and to reduce perceived input lag at all times regardless of framerate. Reprojection involves the headset's driver taking one or multiple previously rendered frames and using newer motion information from the headset's sensors to "reproject" or "warp" the previous frame into a prediction of what a normally rendered frame would look like. "Asynchronous" refers to this process being continuously performed in parallel with rendering, allowing reprojected frames to be displayed without delay in case a regular frame isn't rendered in time, and is used in all frames by default to reduce perceived latency.”

Basically speaking, if your PC is outputting 45fps to a 90Hz HMD, the HMD needs to display 90 frames regardless. Therefore, it uses a clever algorithm to match every frame from your PC with a fake second frame. This means that rotational image display remains smooth, even if the objects within the scene do not. Unfortunately, artificially creating 50% of the frames is not a miracle software hack that works without any issues. If it did, we’d see it in every game engine on the planet, 2D or 3D! Imagine being able to double your performance without needing the hardware to do so!

So then, what do AR’s side effects, or artefacts, actually look like? For me, the thing most noticeable was the ground “jumping” or “stuttering” the closer I got to the ground. Worst of all though was in dogfights or formation flying; other aircraft have a very weird “shadow” image overlayed over the true 3D model which lags behind them by about a meter, yet also hovers over the top of them. It’s rather difficult to describe, and unfortunately I can’t find any illustrative diagrams. Please post some if you can find them!

As mentioned, some people don’t appear to notice AR’s artefacts. Also, it seems to be minimised if you’re looking at objects that aren’t moving across the scene quickly – such as the ground from high altitude, or in slower-moving aircraft. The best way to remove its issues is to not use it – but to do this, your PC must output frames that match the HMD’s refresh rate. This can sometimes be achieved by lowering the graphical detail or resolution, but often it cannot. You can also forcefully disable AR by config tweaks in most platforms, but this often results in the overall experience feeling less smooth when you rotate your head.

I hope this description helps those who are struggling to get a smooth VR experience. Also, if you’re looking to make the splash into VR, this is a rather large issue that you should be aware of. I implore you to road-test your chosen HMD in person before spending big dollars, I certainly wish I had! Note that watching a through the lens video or hearing other people swear that they don’t notice it doesn’t mean you won’t (luck you if you don’t!). I can’t speak of which version of AR is superior, be it Oculus’ ASW, or SteamVR’s Motion Smoothing, as I’ve only seen it on the Reverb Pro, but would love to hear the opinions of those who can.

I’m sure I’ve made some erroneous statements within, please feel free to correct me. And don’t take it personally if you love the VR experience even with AR on; I’m not attacking you at all, I actually envy you!

 

 

Edited August 19, 2020 by TechLad76
updated for clarity

[severniae 78]

Are you aware that you can turn it off completely?

I use a Oculus Rift S - in steam VR I set the reprojection mode to 'legacy' (which enables the older, oculus driver based 'always on' mode) which I can then disable by opening the oculus OVR tool and completely disable reprojection.

Even when the sim is running at 30 FPS, I still don't see any ghosting or strangeness. It does feel 'smoother' with the steam motion smoothing, but as yourself have noted (at length) the artifacts make using the sim unbearable.

Try it out and let me know.

Of course in XPlane you could also install the OVR script that allows you to manipulate the reprojection, and multisampling directly from within the simulator.

[Flyfox 114]

On 8/19/2020 at 9:44 AM, TechLad76 said:

I’m sure I’ve made some erroneous statements within, please feel free to correct me. And don’t take it personally if you love the VR experience even with AR on; I’m not attacking you at all, I actually envy you!

 

 

I am using the Rift S with the Oculus software for my P3D installation and I never experienced anything like what you describe above. My personal experience with this setup is far more realistic and pleasant than anything I had before by just using flat screens. Ok, when I put on the headset, the first few seconds feel a bit strange due to the very limited viewing angle. But as soon as I start operating the aircraft, I don‘t notice such limitations any more and the flight experience is just awesome. 

My personal tip: Try to concentrate on your aircraft and the procedures and you will soon realize how incredibly immersive the VR flying can be! Enjoy your Reverb!

[TechLad76 0]

11 hours ago, severniae said:

Are you aware that you can turn it off completely?

 

 

Yep, as I mentioned there are tweaks that are buried away in options screens or config files that allow AR to be disabled completely. I've only got the Reverb at the moment (have also used the Rift-S, HTC Vive Pro and Vive Cosmos, but no longer have them), and found that when disabling AR it caused major jerkiness when moving my head. Which is exactly what AR is designed to stop... but it comes at the cost of perceived ghosting. It's definitely not as noticeable in GA simming like XP11, until you get closer to the ground that is. But when it comes to dogfighting in DCS, man, it's horrible. As I enter the merge in a 1v1 WVR knife fight, the other guy is ghosting like crazy for me. To me it looks horrible, so until I can run at 60fps, I'm using a big 55" OLED 4K 120Hz LG C9 display with TrackIR. IMHO it's a much superior experience.

[TechLad76 0]

10 hours ago, Flyfox said:

I am using the Rift S with the Oculus software for my P3D installation and I never experienced anything like what you describe above. My personal experience with this setup is far more realistic and pleasant than anything I had before by just using flat screens. Ok, when I put on the headset, the first few seconds feel a bit strange due to the very limited viewing angle. But as soon as I start operating the aircraft, I don‘t notice such limitations any more and the flight experience is just awesome. 

My personal tip: Try to concentrate on your aircraft and the procedures and you will soon realize how incredibly immersive the VR flying can be! Enjoy your Reverb!

Heh, I've spent over 100 hours trying to enjoy the Reverb in DCS, IL2 and XP11, but found I spent more time flying the options screen, trying to get a nice experience, than I did flying the planes! I'm envious that you don't notice the issue; can I ask if you've tried many non-simming VR experiences for a lengthy period of time?

[Nedo68 533]

i preffer to drive/fly without additional smoothing, but its only possible now with a 2080ti,

but thats just my personal experience i made with different HMD's the last years (first was 2012 with the DK1), as an example racing without smoothing feel much more realistic for me, but i can drive with smoothing too,

actually you need it if your FPS dropping.

 

 

Edited August 21, 2020 by Nedo68

[VeryBumpy 272]

On 8/21/2020 at 2:48 PM, Nedo68 said:

but its only possible now with a 2080ti,

 

 

 

Yup, this is the key. Truly enjoyable VR is at high res with high frame rates. Only obtainable by spending a bunch of $$$$$$ on uber fast hardware.

Or

you drop the graphics to pants levels.

[cannow 61]

ASW for rift s  has been broken on P3d V5 since day 1, V4 works fine. MS2020 also broken when working properly makes a huge difference . High graphics cards help and allow no ASW to be an option but should work way better with less processing power if ASW was implemented correctly . Benn banging on about this for last 2 years . Hoping with wider gamer community on MS2020 will bring pressure to fix on 2020 . P3D posted first day of release on V5 it was not working. ASW2.0 requires information embedded in the sim to work I believe and they do not want the work to implement. Much easier to sat buy better hardware. 

See link for explantion and depth data required

Introducing ASW 2.0: Better Accuracy, Lower Latency | Oculus