SledDriver

P3D multicore usage anomoly

Recommended Posts

Posted (edited)

"Texture loading is I/O, which is trivial to multi-thread since those threads aren't doing much except waiting for said I/O to finish. That's why there was such a huge win with FSX SP1."

Indeed--a huge win by taking a task that was accomplished for years in the main thread when someone realized it could be ported off the main thread.  I read a while back that XP sends its AI into other cores.  And besides, I disagree w/ your premise that AI traffic must be a dependent part of the whole integrated world--it may take reimagining how that works, but I don't see it as necessary.  Same same for ground traffic on roadways.

My point is that it will take some thinking outside the box to access all of that horsepower in waiting, that elephant in your PC case, and my guess is that even though Luke you seem to have already come to a final conclusion that it's too expensive and the yield won't offset the development cost that way of thinking never spawns breakthroughs.  But yes, it appears it will take some serious outside of the box thinking to do more with parallel processing in a flight sim--but that is where the most potential processing power remains it appears.

Edited by Noel

Share this post


Link to post
Help AVSIM continue to serve you!
Please donate today!

15 hours ago, SledDriver said:

Got any pointers to useful config threads? More than happy to read up. I have years of experience tweaking FSX, and about 1 week on P3D, so am all ears for learning right now.

There are quite a few discussions of the 30Hz VSync technique here.  P3D does not need anywhere near the kind of tweaking that FSX did...I leave HT off and leave the affinity mask alone.  My best results have been to limit the CPU and GPU workloads so that they are not running all-out all of the time.  Maintaining some processor headroom allows the absorption of workload spikes without making the sim miss frames. 

For the CPU, I let the hardware VSync throttle the CPU load back by limiting frame production.  That's done by setting frame rate to unlimited and turning on VSync in the P3D settings, and no messing with nVidia inspector or other external utilities.  Done that way, with a 30 Hz hardware refresh rate set on the display, the CPU is constrained to produce frames at the hardware refresh rate, and no more.  When the CPU is trying to get to 60 or unlimited fps, then it will run the main thread (usually on core 0) at 100% all of the time.  It's pedaling as hard as it can all the time, and when workload spikes occur, such as loading AI, autogen, crossing tile boundaries, airport scenery loading etc, the best rate the CPU can hold will fluctuate significantly, and that fluctuation produces stutters.  In the interest of smoothness, a steady flow of frames at 30 fps is far preferable to 45 fps with excursions all over the map--and it leaves room for the CPU to take up additional load fluctuations with much less chance of dropping below the 30fps threshold.  Use of GSync to deal with more frame rate fluctuations might also be a good way to go if you have the hardware, as the monitor can roll with the flow of varying frame rates (up to a point) without producing stuttering or tearing.

For the GPU, I keep steady-state GPU load to no more than 80% with the heaviest workload I intend to throw at it.  That means managing AA, dynamic lighting, cloud layers/texture sizes etc so that I don't hit the 95-100% wall on GPU load, because that'll produce the same kind of frame rate fluctuations as a maxxed-out CPU.  I typically run my 4K 30Hz display with 4xSSAA except at night with DL on in heavy weather, where I change to 4xMSAA on the fly to keep the GPU load under control.  I monitor CPU and GPU load using MSI Afterburner, with the stats displayed on the LCD screen of a Logitech G13 game controller that also serves as a view controller.

Regards

Share this post


Link to post
Posted (edited)
1 hour ago, Noel said:

Indeed--a huge win by taking a task that was accomplished for years in the main thread when someone realized it could be ported off the main thread.  I read a while back that XP sends its AI into other cores.  And besides, I disagree w/ your premise that AI traffic must be a dependent part of the whole integrated world--it may take reimagining how that works, but I don't see it as necessary.  Same same for ground traffic on roadways.

You use words like "realize" and "reimagine" like the people at LM are stupid and lack any ability to consider significant changes to their product and its code. Given what they are doing, I am extremely confident that is not the case. But if you feel you are so much more imaginative than their engineers, perhaps you should apply. Everyone is looking for skilled software engineers these days.

Adding threading support to FSX SP1 wasn't a question of 'realization'. If you had suggested doing so at the time FS9 was released or even FSX, you would have been quite correctly shut down. Single-thread performance was continuing to ramp up every year, and 99.9% of consumers had just a single-core processor. Multi-threading on a single-core CPU actually slows things down a bit if multiple threads are CPU bound since you have to do expensive context switching.

Don't forget as well that C++ doesn't have anywhere near the same amount of threading constructs as a modern language such as C# or Java. Multi-threading code in those languages is something that an engineering manager or architect won't take lightly, since it adds cost not just in terms of development but also ongoing maintenance. To rewrite something multi-threaded in C++ in an old code base? I'm impressed SP1 was as stable as it was, but that too is likely because they were conservative.

The assertion that X-Plane sends AI to other cores needs qualification. First, it's worth asking what exactly XP is doing (modeling, render, etc) and it's a different engine that P3D. Have LM engineers had conversations about shifting the work around? Probably extensive conversations, but it again goes back to what is the cost and effort, likelihood of success and risk to the stability of the platform, plus the elephant in the room - what do L-M's "other" customers want from the sim?

I've come to my conclusions because I've been a software engineer and now an engineering manager for over a quarter century. I don't claim specific insight into LM, but their actions look exactly like a team that is slowly, conservatively modifying a legacy codebase. To suggest it merely needs a little "imagination" is not fair to LM at all.

Cheers!

Edited by Luke
  • Like 2
  • Upvote 3

Share this post


Link to post
Posted (edited)
26 minutes ago, Luke said:

I've come to my conclusions because I've been a software engineer and now an engineering manager for over a quarter century. I don't claim specific insight into LM, but their actions look exactly like a team that is slowly, conservatively modifying a legacy codebase. To suggest it merely needs a little "imagination" is not fair to LM at all.

Cheers!

Bingo. I always get a chuckle when people talk about tossing the "old engine" lock, stock, and barrel. I see what you see. They are modernizing it from the "outside in". Meaning that in order to get the entire rendering engine overhauled, they need to start by adding all of the supporting elements to the existing code. Things like HDR, PBR, and DL have been added, but the old lighting and texture routines are also still in place running side by side. Eventually, the legacy bits will be removed when LM stops using them. at some point down the road.

In the end, they will toss the legacy engine, but it will be done by removing it after the new code is in place. Piece by piece.

Edited by MDFlier

Share this post


Link to post
11 hours ago, Luke said:

You use words like "realize" and "reimagine" like the people at LM are stupid and lack any ability to consider significant changes to their product and its code. Given what they are doing, I am extremely confident that is not the case. But if you feel you are so much more imaginative than their engineers, perhaps you should apply. Everyone is looking for skilled software engineers these days.

Adding threading support to FSX SP1 wasn't a question of 'realization'. If you had suggested doing so at the time FS9 was released or even FSX, you would have been quite correctly shut down. Single-thread performance was continuing to ramp up every year, and 99.9% of consumers had just a single-core processor. Multi-threading on a single-core CPU actually slows things down a bit if multiple threads are CPU bound since you have to do expensive context switching.

Don't forget as well that C++ doesn't have anywhere near the same amount of threading constructs as a modern language such as C# or Java. Multi-threading code in those languages is something that an engineering manager or architect won't take lightly, since it adds cost not just in terms of development but also ongoing maintenance. To rewrite something multi-threaded in C++ in an old code base? I'm impressed SP1 was as stable as it was, but that too is likely because they were conservative.

The assertion that X-Plane sends AI to other cores needs qualification. First, it's worth asking what exactly XP is doing (modeling, render, etc) and it's a different engine that P3D. Have LM engineers had conversations about shifting the work around? Probably extensive conversations, but it again goes back to what is the cost and effort, likelihood of success and risk to the stability of the platform, plus the elephant in the room - what do L-M's "other" customers want from the sim?

I've come to my conclusions because I've been a software engineer and now an engineering manager for over a quarter century. I don't claim specific insight into LM, but their actions look exactly like a team that is slowly, conservatively modifying a legacy codebase. To suggest it merely needs a little "imagination" is not fair to LM at all.

Cheers!

I'm not talking about LM or LR in particular so I'm not talking about rewriting code base, I'm talking about where the untapped processing power lives, and if that means off to another programming language that is more suited to do more w/ parallel processing that's where the future lies.  My assertion was simply my passing on what I read many years ago I believe on their website re XP and AI.  And further, I never said all LM needs was a little imagination--that is how you reacted to what I wrote.  I said "But yes, it appears it will take some serious outside of the box thinking to do more with parallel processing in a flight sim--but that is where the most potential processing power remains it appears."  Said nothing about LM except they are the developer with massive deep pockets so yes it would be great to see an entity w/ those potential resources develop a modern FS maximizing multithreading in whatever languages facilitate that the most because, once again, that is where the processing power lies.

Share this post


Link to post

It seems everyone is missing the key facts about this core miss-usage.

Affinity masks don't sort it. All they do is move the 100% core usage to another core permanently.

I've been tweaking MS FSX and now P3D for 30 years, and what I am about to write is giving me the smoothest animation I have ever seen in these flight sims.

The measures I take are as below:

1) Turn off Hyperthreading.

2) Turn off SpeedStep in the BIOS

3) Turn on any Turbo mode permanently.

The previous two points are setting the CPU to work at a constant speed - this is often a good thing to do for real-time apps such as sims or music/video production). For instance, my 8700K runs at 5GHz all the time on all cores, including when AVX instructions are in use. This requires careful overclocking/cooling. Even if you can't reach a high overclock, it's better to have the CPU running constant for the sim rather than all these modern power saving features speeding up and slowing down the cpu - they can't predict future processing demand, so just cause problems in this type of application - fine for word processing/photoshop etc.

4) Install everything on SSD drives.

5) Add the FIBER_FRAME setting to the P3D cfg file and lower the value a bit - it defaults to 0.33. 0.2 is working well for me, but machines vary.

6) Don't bother with an affinity mask settings, in fact delete any you have made.

7) Start P3D.

8) Open TaskManager. Go to details and fine Prepare3D.exe. Right click it and disable core0. Click OK. Now right click it again and re-enable core0. Click OK. Now the load is spread evenly over all cores. I've read many of the posts saying that it's OK for core0 to be hammered at 100%, but this is not my experience in many years. With a properly tuned and configured machine, I get smoother performance this way.

9) Set your P3D frame rate to limited 30fps, Triple buffering ON. Yes I've read all the theories on which is the best way to set this, but I am simply relating what has worked very well for me.

10) Now get some decent hardware monitor tools to watch you CPU usage and GPU usage and spend some time tweaking all the other settings inside P3D so that your CPU is running at around 50% and your GPU at around 75% during normal flight with all your add-ons doing their thing. Do not max out the CPU or GPU as you need to leave headroom for when you are flying near very busy scenery, like big airports.

Summary.

So what you may detect reading all the above, everything is aimed at getting persistent, consistent performance throughout a sim session. Glassy smooth animation is the most important thing in simming. Any microstutters and the illusion of immersion is gone instantly.

P3D still has a way to go, but the above methods and settings have resulted in a totally enjoyable experience with smooth animation even in busy areas.

Hope this helps someone.

Cheers

 

With all the above in place

Share this post


Link to post

Happy you've got it all worked out - for you..

Quote:

"Now the load is spread evenly over all cores."

Interesting observation, that I have never witnessed  :cool:

 

 

Share this post


Link to post

You wont get that with an Affinity Mask. You have to disable/enable core 0 manually then it should work.

Share this post


Link to post
Posted (edited)
51 minutes ago, SledDriver said:

Open TaskManager. Go to details and fine Prepare3D.exe. Right click it and disable core0. Click OK. Now right click it again and re-enable core0. Click OK. Now the load is spread evenly over all cores.

I've observed the same on my end (i7-9700k). Now I'm trying to see if I notice improvements in performance or not. But it is nice to see it not 100% all the time.

Edited by pvupilot

Share this post


Link to post
23 minutes ago, SledDriver said:

You wont get that with an Affinity Mask. You have to disable/enable core 0 manually then it should work.

The load just moves to another core..

  • Upvote 1

Share this post


Link to post
41 minutes ago, Bert Pieke said:

The load just moves to another core..

Yep.  When it comes to computer resources, little is to be gained by robbing Peter to pay Paul.

Greg

Share this post


Link to post
Posted (edited)
1 hour ago, Bert Pieke said:

Happy you've got it all worked out - for you..

Quote:

"Now the load is spread evenly over all cores."

Interesting observation, that I have never witnessed  :cool:

I haven't either--but is that because we haven't set up the sim in this fashion, w/ the little trick to get "the load spread evenly over all cores?"

Simple enough to try but it seems to contradict how others have described P3D's built-in work sorting schema.

Edited by Noel

Share this post


Link to post
14 minutes ago, Noel said:

I haven't either--but is that because we haven't set up the sim in this fashion, w/ the little trick to get "the load spread evenly over all cores?"

Simple enough to try but it seems to contradict how others have described P3D's built-in work sorting schema.

I did try it... :wink:

Share this post


Link to post
4 hours ago, Bert Pieke said:

The load just moves to another core..

Do you guys deliberately bury your heads in the sand. I took the time and trouble to write that report because it works.

And you just quip that the load moves to another core. NO IT DOESN'T. It moves to another core if you just set a static affinity mask.

If you do the core0 disable/enable routine each time just after starting the sim, the core0 hit is OBVIOUSLY successfully shared across all the cores.

Why can't you guys see this? I despair at the reams of forum topics here and elsewhere with (usually nontechnical) people continuously droning on about affinity masks. Affinity masks do not sort the problem.

It's a very strange world where you just ignore solid evidence gained by extensive controlled iterative testing to get where I am today with a smooth sim.

And the other theory that core0 should be allowed to run at 100% as it is doing 'special things', is utter technical BS. Computer cores aren't magic. They have finite resources. If the sim loads a core up to 100%, something else has to wait. Hence problems. This 100% thing goes back well into the history of FSX, and it was never a good thing. 

Still each to his own. Me to my smooth lovely sim. You to your continued search for nirvana.

Share this post


Link to post
Posted (edited)
3 hours ago, Bert Pieke said:

I did try it... :wink:

Another quip. Reporting your results would be more productive, even if they go against my suggestions.

There is more to getting a smooth sim than "trying the test". did you notice I wrote many points which are all important to get smoothness.

No one quick trick will sort a sim out.

 

Edited by SledDriver

Share this post


Link to post
3 hours ago, Noel said:

Simple enough to try but it seems to contradict how others have described P3D's built-in work sorting schema.

There were 10 points to my list of things - they are all important. Doing just one thing on the list is not going to get you far.

Perhaps I should say that I have in my time been a professional software developer and system designer, have programmed 3D graphics applications, and have done that in assembler/machine code, so I have some grasp as to how a CPU works at the low level, and I cannot support the idea that one core running a 100% while the others are relatively lightly loaded is ever a good thing.

There seems to be a whole 'religion' of thought about how this sim and it's predecessor might work - it's taken on its own life online - much of which is written by well-meaning, frustrated, non-computer-developer/technical people.

I've tweaked FSX for umpteen years ever since it was Sublogic.

I've owned P3D for just 2 weeks, and in that time have done extensive iterative performance tests on my hardware and the sim software, and in just two weeks I now have a sim performing excellently. It's not perfect - that's why it's still in development and LM are rewriting whole chunks of it as time goes by, but it is a delight to use and runs very well.

It seems many are just going round in circles trying to achieve the same thing, and have been for years, so I wrote what I thought might be a useful guide to how I've achieved good smoothness in the sim.

Do what you want with it, including ignoring it. But quipping that you tried one test on a critical list of 10 things, proves nothing.

 

Share this post


Link to post

I'm glad you have a smooth running sim, SledDriver.

Cheers,

Mark

  • Like 1
  • Upvote 1

Share this post


Link to post
Posted (edited)
1 hour ago, SledDriver said:

I've tweaked FSX for umpteen years ever since it was Sublogic.

I've owned P3D for just 2 weeks, ..

Kind of what I suspected..  :unsure:

I also came from an FSX background, a year ago...

P3D actually is different, and you may also discover some more things as you get to use it more.

I am certainly happy that you like it.. and maybe the best thing is to just leave it at that..

Edited by Bert Pieke
  • Like 1

Share this post


Link to post
Posted (edited)
2 hours ago, SledDriver said:

Another quip. Reporting your results would be more productive, even if they go against my suggestions.

 

OK, I accept your challenge..

Here is my "recipe" for a 4770K@4.2 / GTX1070 system,  just tested with P3DV4.5.

1. Set an affinity mask of 246 for an eight core (quad+HT) processor.

    // 246=11 11 01 10

2. Start up P3D with fps set to unlimited, Vsync off.

3.  Start up RTSS and select Scanline Sync / 2 with 60 Hz for a 60 Hz monitor

4.  After loading P3D, use Task Manager to enable the third core, leaving the first core unselected.

Go fly!  :cool:

 

Edited by Bert Pieke

Share this post


Link to post
3 minutes ago, Bert Pieke said:

Start up RTSS and select Sync rate / 2 with 60 Hz for a 60 Hz monitor

This is the key...

Share this post


Link to post
Posted (edited)
11 hours ago, Bert Pieke said:

4.  After loading P3D, use Task Manager to enable the third core, leaving the first core unselected.

 

Sorry, sloppy writing...set affinity to 11 11 11 10 in step 4.

(enabling the fourth core)

If you have HT turned off, you can probably enable all cores.

Edited by Bert Pieke

Share this post


Link to post
13 hours ago, Bert Pieke said:

2. Start up P3D with fps set to unlimited, Vsync off.

3.  Start up RTSS and select Scanline Sync / 2 with 60 Hz for a 60 Hz monitor

So you are using RTSS to control frame rates externally to the sim controls?

>Sorry, sloppy writing...set affinity to 11 11 11 10 in step 4.

Where are you setting this? TaskMan or P3D.cfg? I'm not aware that P3D.cfg is read after initial sim startup?

Share this post


Link to post
Posted (edited)
1 hour ago, SledDriver said:

So you are using RTSS to control frame rates externally to the sim controls?

>Sorry, sloppy writing...set affinity to 11 11 11 10 in step 4.

Where are you setting this? TaskMan or P3D.cfg? I'm not aware that P3D.cfg is read after initial sim startup?

RTSS, not via its frame rate limiter, but by using its Vsync options.

Step 4 is via Task Manager.

Edited by Bert Pieke

Share this post


Link to post

Struggling a bit here.

I have RTSS installed. Followed your instructions as to the startup order.

Got the sim running. Turned off Vsync and made framelimit unlimited in sim.

Opened RTSS.

Added a new profile for P3D. Selected it.

Clicked the Scanline Sync until it said x/2.

Entered 60 into the space beside Scanline Sync.

Is that it?

Is there anything else I need to do in RTSS? Cos the framerate in the sim according to the in sim FPS (Shift-Z) is jumping around between 36 and 39. I was expecting it to be locked on 30.

What have I done wrong?

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now