A Frame-Time Analysis of P3D v3 -- Clouds

[AviatorMoser 353]

Hello,

 

I've been meaning to do an analysis on P3D with clouds for a while now, and also compare it to results from FSX:SE.

 

This is a simple experiment, and the motivation for this experiment arrives from my dissatisfaction with P3D's performance when it comes to heavy cloud conditions. From just a visual evaluation, it had appeared P3D ran much more sluggishly in overcast conditions than it did in light weather, but I had never completed a quantitative analysis until now to confirm what my eyes were telling me.

 

So this is the objective of this little experiment: determine what is the performance hit of overcast conditions in P3D versus FSX:SE using the same addons, the same ASN settings, and the same benchmark.

 

My P3D display settings. They are identical to my previous frametime analysis post here.

 

 

 

 

 

Identical settings were applied to FSX:SE as much as possible.

 

In the case of P3D, I have no modified nVidia Inspector settings from the stock profile. That means no SGSAA is applied whatsoever in P3D. However, in the FSX:SE case, 4X SGSAA is applied, along with 8X CSAA (equivalent to approximately 4X MSAA in performance). This should actually put FSX:SE at a disadvantage since it has additional antialiasing to perform, but we'll have to confirm that in the benchmark results.

 

The Benchmark:

 

• Left traffic pattern at PAGY in an Alabeo DA40 model
• FTX Southern Alaska
• A REX 3 cloud texture set
• Volumetric cloud effect is disabled in P3D.
• Two cloud layers in Active Sky Next
• Historical weather at 15:45 UTC December 26, 2015 -- results in overcast and precipitation

Control experiment:

 

I will monitor the frame times in the above benchmark for both simulations without Active Sky Next weather applied to see what their performance is in clear conditions.

 

Prepar3D 3.1:  Clear Weather, FTFF = 0.1, 4X MSAA, 30 FPS Lock

 

 

Here we see pretty good performance. The average frame rate is about 28 FPS. The frametimes are stable. 99% of all frames are rendered at a rate > 21 FPS. There is variation from 33.3 ms to 40 ms between frames, but that is not discernable in the naked eye. This is good performance that is comparable to the final results found in my previous frame-time analysis thread.

 

FSX:SE:  Clear Weather, FTFF = 0.1, 8X CSAA, 4X SGSAA, 30 FPS Lock

 

 

Despite more aggressive AA and SGSAA, FSX:SE performs slightly better and smoother than P3D in the clear weather case. Avg frame rate is 29 FPS, but the variation in frame times are even lower. 99% of all frames are rendered at a rate > 23 FPS (21 FPS in P3D), and 99.9% of all frame rates are rendered > 21 FPS (18 FPS in P3D). A surprising result considering how much smoother and better P3D is touted to run better.

 

Wintry weather conditions benchmark:

 

Now, Active Sky Next is applied. ASN settings are identical for both simulators.

 

Prepar3D 3.1:  Wintry Weather, FTFF = 0.1, 4X MSAA, 30 FPS Lock

 

 

The results are a bit unsatisfactory. Avg frame rates have dropped to 23 FPS. That is less than a typical feature film. There is much more variation in the frame times with 99% of all frames being rendered > 17 FPS, which leads to roughness in the rendering discernable by the naked eye. Also, there are some spikes in the time series. Some frames when I am taking off and landing lead to long frame times in excess of 120 ms. These are stutters and that is no good. There is definitely a performance hit caused by only two cloud layers which leads to slow frame rates that can be agonizing over time.

 

FSX:SE:  Wintry Weather, FTFF = 0.1, 8X CSAA, 4X SGSAA, 30 FPS Lock

 

 

FSX is barely bothered by clouds, even with sharper AA and less shimmering with the SGSAA. A surprising result again. With SGSAA, I would have expected there to be a major performance hit when overcast is enabled. It seems FSX shrugs it all off.

 

The average frame rate was maintained at 29 FPS. 99% of all frames were rendered > 22 FPS (17 FPS in P3D), and 99.9% > 20 FPS (12 FPS in P3D). So the variance in frame times has decreased ever so slightly, but one can argue this decrease is so small that it would be undetectable to the naked eye. For all practical purposes, one can say there is no change between FSX performance in clear or wintry conditions.

 

Conclusions:

 

Something is not right with P3D and its cloud rendering, at least on my system. It struggles to render two overcast layers even without SGSAA. FSX on the other hand doesn't even to notice the clouds. I can actually go far and say FSX performs better with 8X CSAA and SGSAA in heavy overcast conditions than P3D in clear conditions with less AA and SGSAA. It should be the other way around, and I wish it was considering how much development has been put into the rendering side of P3D. However, some optimization in the cloud rendering needs to be done.

[Kimo 14]

Interesting post thanks for sharing.

[ppo 67]

From a quick analysis, I'd say your video card is weak particularly for P3D which uses more the video card than FSX.

No doubt P3D is having a hard time with clouds BUT your system is possibly unbalanced...

But good tests anyways. Tests are always useful.

The conclusion is therefore wrong. P3Dv3.1 is quite a different overall engine than FSX.

The right conclusion could have been: My system performs better in FSX in clouds than P3Dv3.1.

My 2 cents

[AviatorMoser 353]

From a quick analysis, I'd say your video card is weak particularly for P3D which uses more the video card than FSX.

 

I would agree with that assessment, but my video card is only truly stressed by P3D in cloudy conditions where the usage spikes to 99%, and performance begins to dip. Outside of heavy cloud conditions, the GPU usage sits around 70-80%, and the video memory usage is about 75% regardless of clouds.

 

So the card is more than capable of pushing a locked 30 FPS in non-cloudy conditions, but you can throw that right out the window when overcast conditions come into play.

 

Now seeing that the clouds look identical between FSX and P3D (after disabling the volumetric effect in the shaders), I think the rendering changes to the ESP engine by P3D may need some optimizations because this performance hit is a bit too much. Has there been a major change in how clouds are rendered? Is this supposed to translate into a large visual improvement?

 

These clouds after all are only rotating 2D sprites, and you shouldn't need a GTX 970 to render them considering they are based off 2004-2006 technology and methods.

[AnkH 2,233]

These clouds after all are only rotating 2D sprites, and you shouldn't need a GTX 970 to render them considering they are based off 2004-2006 technology and methods.

 

Reality and theory are sometimes far apart, no? Fact is, your GTX-660 is not a rocket anymore. Just as an example: my 980Ti is about 300% faster than your 660 and guess what? I have almost no drop in FPS no matter how overcast my clouds are. Only thing dumping my FPS in P3Dv3 is still the AI traffic....

 

And your calculations are really illustrative: while you lock your FPS at 30, you are basically running into a CPU limit while flying around with no clouds, no wonder your GPU then only uses 70-80%. As soon as you load your GPU with the drawing of clouds, you switch from a CPU Limit to a GPU Limit and then your FPS start to drop below 30, as the GPU simply can not handle the load anymore. Video memory usage is a very bad indicator of GPU load by the way...

 

Still, you are of course perfectly right about the comparison between FSX and P3D, I would also have guessed that P3D has also improved the rendering of the clouds and obviously they did not. But to be fair, you can only properly compare FSX and P3D when you also chose the right and corresponding settings. In above screenshots, you have HDR active, something FSX does not offer at all...

[Nuno Pinto 1,394]

Yeah you really need to get rid of frame locks when benchmarking anything.

[AviatorMoser 353]

Reality and theory are sometimes far apart, no? Fact is, your GTX-660 is not a rocket anymore. Just as an example: my 980Ti is about 300% faster than your 660 and guess what? I have almost no drop in FPS no matter how overcast my clouds are. Only thing dumping my FPS in P3Dv3 is still the AI traffic....

 

And your calculations are really illustrative: while you lock your FPS at 30, you are basically running into a CPU limit while flying around with no clouds, no wonder your GPU then only uses 70-80%. As soon as you load your GPU with the drawing of clouds, you switch from a CPU Limit to a GPU Limit and then your FPS start to drop below 30, as the GPU simply can not handle the load anymore. Video memory usage is a very bad indicator of GPU load by the way...

 

Still, you are of course perfectly right about the comparison between FSX and P3D, I would also have guessed that P3D has also improved the rendering of the clouds and obviously they did not. But to be fair, you can only properly compare FSX and P3D when you also chose the right and corresponding settings. In above screenshots, you have HDR active, something FSX does not offer at all...

 

Ha! My GTX 660 was never a rocket

 

I expect a $600 card to not be overwhelmed by two cloud layers. Overall I just wanted to see if the performance hit for clouds is similar between the two programs. I use a frame lock exactly for the purpose of leaving a GPU 'reserve' for the clouds in either simulation, so that neither are already overwhelmed before adding clouds. I only mention VRAM as a check on whether all of my video memory is consumed in P3D, hence the lower performance, but it is not.

 

In reality, I could probably add 10 cloud layers with 4X SGSAA in FSX with a minimal performance hit, but I expect P3D to become a slideshow due to the inefficiencies in the cloud rendering. I don't think HDR is the culprit.

 

Maybe you're right in that this is an apple and oranges comparison, but I would have really liked if the development team left the cloud rendering as an apple, because this performance hit is a regression, and I shouldn't need at least a $500 GPU to render cloud sprites, especially when it was done easily enough before for much cheaper.

 

But oh well. That's the way it goes I guess. Two steps forward in the rendering, one step back.

 

 

Yeah you really need to get rid of frame locks when benchmarking anything.

 

Why? I just need to see if it hits the my frame rate goal or not. Without frame locks, I get tremendous microstuttering. I show that in the previous frame time threads. My goal isn't a maximum frame rate. It's a smooth, 30 FPS.

[be77solo 125]

Definitely must be GPU related... I don't see a drop in FPS regardless of cloud settings (tried up to 5 ASN layers with max coverage) in either P3Dv2.5, v3.0, v3.1 or FSXSE.... I am running a NVIDIA 970m.

 

Only things that make much difference in FPS that I can tell is heavy AI traffic,  PMDG stuff and full fat Orbx regions. Weather luckily makes no difference, even on my mobile CPU.

 

By comparison, XPlane 10 brings things to a stuttering stand still when cranking up weather on every machine I have tried, so I'm happy with P3D/FSXSE approach ha

[armchair_pilot 54]

I know this thread is old, but clouds still perform awful in v3.2. I did a test with only 3 cloud layers in ASN and instrument conditions. Despite using REX soft clouds, flying through them or even by only looking at them in the air dropped my fps from stable 30 down to 10. It doesn't even matter if I'm using soft clouds or 2048 textures, the performance hit is the same. My GPU is a GTX770 4GB, although an old card it is still a decent performer and those fps hits can't be normal. Clearly a rendering issue that LM still has to fix...

 

EDIT: And this is without SGSS, only 4x MSAA!

[GSalden 4,656]

I have no such problems with clouds...

ASN has 4 cloud layers and I use 32 bit clouds..

 

With an ovc I loose around 20% performance ...that's it.

[TheBoom 31]

A known issue. Though I've to agree it's worse in 3.2. They improved the aa so I guess that will cost more overhead at the same settings.

[Afterburner 374]

If you fly during sunset or sunrise and have your cloud shadows activated, it can drag down your FPS drastically.

 

Check this video, in which I have documented the impact of dense clouds on FPS in Prepar3D v3.2 during dawn/dusk:

 

[tf51d 284]

I think the biggest difference between performance of FSX with Clouds and P3D with clouds, are that P3D clouds are volumetric which will perform worse since they are 3D objects versus the 2D sprites in FSX. So any comparison to FSX would be comparing apples to oranges.in this case.

[Gregg_Seipp 565]

Has P3D fixed their overcast layers in 3.x?  In 2.5 I've tried to create a 4000' deep overcast to do some IMC training but I never could get it solid enough to do the work no matter how much I tried.  When I flew into the layer there were very large gaps all over the place.  Tried it with P3D weather as well as Active Sky.  Anybody willing to do an experiment for me?

 

Gregg

[NZ255 309]

I think the biggest difference between performance of FSX with Clouds and P3D with clouds, are that P3D clouds are volumetric which will perform worse since they are 3D objects versus the 2D sprites in FSX. So any comparison to FSX would be comparing apples to oranges.in this case.

No I don't think that's the case yet. There's volumetric fog, but the clouds are still rotating sprites.