P3D 5.3 Hotfix 2 released

[Greggy_D 936]

TOGA just released a patch for Envshade which fixes the horizon issue:

This update fixes the broken horizon haze in default P3D v5.3 HF2 and fixes the invisible ground markings at night on UK2000 airports.

[SteveW 1,792]

1 hour ago, Seat* said:

I am trying to understand this with the use of the Job Scheduler in P3D. When I think I understand at least some of it, I read @SteveW  reply to @shermank who has a processor ( Intel Core i7-10700) with the same number of cores (8) and HT enabled as I have in my Intel Core i9-9900K. P3D gives me an AffinityMask=65535 and not 65493. Why isn´t the AM the same from P3D? Here is what I use P3D.cfg
[JobScheduler]        
AffinityMask=65535        
P3DCoreAffinityMask=1364        
MainThreadScheduler=0        
RenderThreadScheduler=2        
FrameWorkerThreadScheduler=3
There's something I haven't understood right?        

/Thomas
 

Firstly, the AM values presented by P3D v5.3 are a very basic set that can be improved on with HT Enabled. The P3D AffinityMask will represent a 'one' in each Logical Processor location. 65535=1111111111111111

The best thing to do to make things nice and clear is lay out an affinity map for the CPU and use the nomenclature that separates HT Logical Processors on a per core basis with commas.

So with HT Enabled we can write: 65535=11,11,11,11,11,11,11,11 showing there are 8 cores emulating 16 Logical Processors. We can see that every Logical Processor is enabled with 65535. This would be the default setting from P3D v5.3

Taking my example and having a look at the Logical Processor Map. Remember that core zero (LP0 and LP1) are the rightmost pair in the binary:

[JobScheduler]
AffinityMask=65493
P3DCoreAffinityMask=16341
MainThreadScheduler=0
RenderThreadScheduler=1
FrameWorkerThreadScheduler=2

HT Enabled 8 core / 16 LP
07,06,05,04,03,02,01,00=core number
11,11,11,11,11,01,01,01=AffinityMask = 65493
00,11,11,11,11,01,01,01=P3DCoreAffinityMask = 16341
00,00,00,00,00,00,00,01=MainThreadScheduler = 0 = LP 0
00,00,00,00,00,00,01,00=RenderThreadScheduler = 1 = LP 2
00,00,00,00,00,01,00,00=FrameWorkerThreadScheduler = 2 = LP 4
 

As can be seen in the map, the Main, Render and FrameWorker are the first three 'ones' encountered in the P3DCoreAffinityMask mask from right to left, each have a core to themselves (01,01,01).

In this case, on the right, the Affinitymask is the same as the P3DCoreAffinityMask. This way we are not enabling Add-ons launched by the sim to occupy the first three cores. The most important task is the MainThreadScheduler and interrupting this activity by sharing the core will reduce the fps capability.

On the Left we can allocate as many Logical Processors as we like. In the example I have allowed the AffinityMask to include the LPs of core 7, but the P3DCoreAffinityMask is not assigned those two LPs, this leaves those two LPs available to Add-ons launched by the sim and not interrupted by activity in the P3DCoreAffinityMask. When add-ons are launched by the sim they only 'see' cores (LPs) in the AffinityMask for use.

[Michael Moe 495]

It can be overwhelming with AM and sometimes i need the helicopter view.

So to cut it simple  my I74770k 4 cores 8LP HT ON.

AM 253,253,0,1,2 in p3d.cfg. or 253,253,0,2,4? (Whats the diff)

Any other P3D related stuff with Process Lasso on 3,5,7 splittet between these? (Away from main cores right?)

Thanks Michael Moe

 

[shermank 170]

Steve, just a quick thank you for your advice...and from the looks of this thread, I am not the only simmer curious about the Job Scheduler configuration.

 

Sherm

[Cruachan 705]

Hi Steve,

I wonder whether some of the confusion arises from stating ‘Addons are launched by the sim’ which might imply that such Addons are being called by P3D. Consequently such Addons might be being considered as having the right to use any of the same LPs hidden by the mask and intended to be reserved for P3D which, of course, is not the intention.

Strictly speaking, Addons are not actually launched by the sim. Instead each is launched outside the running sim by the user or via a 3rd Party facilitator like SimStarter or FSUIPC in a predefined order. They will then link/connect to the sim while running on any available unmasked core or LP/LPs, as determined by the Windows Job Scheduler which is not ideal or, preferably, will occupy only those specific cores/LPs as assigned by a bitmask described in a bat file or, more easily in my case, via the GUI of Process Lasso. These latter solutions should ensure that any Addon is prevented from running on physical cores 0, 1 and 2.

Currently, I’m experimenting by keeping everything P3D related away from core 0 (LPs 0 and 1 with HT=ON). Leave that core for Windows related tasks. Early results are suggesting that this is having the desired effect by preventing the P3D main thread on LP2 from saturating at 100%. This was not achievable on my somewhat conservative setup while the main thread was running on LP0 which meant I had little or no spare overhead available.

Best regards,

Mike

Edited January 24, 2022 by Cruachan

[jlohrenz 133]

I have an 8700k and thought the old rule of thumb was to run with HT turned off.  Has that logic changed?  I read discussion of the potential benefits of HT but that seemed to be with chips that have more cores than what I'm running.  Right now, with HT off, I have my system running a nice smooth 5Ghz on all cores with no heating problems so maybe I should leave it alone?

Would appreciate any suggestions. Since I need to stretch out a bit more from this machine considering the cost of new hardware.

[Ray Proudfoot 8,958]

@jlohrenz, my i7-8086K is very similar to your CPU and I recently turned off HT as I wanted to push the overclocking and HT runs hotter.

I was running 3 cores at 5.2 with the other 3 at 4.9 but freezes for a few seconds with a buzzing from the speakers turned out to be overclocking too much.

Back at 5.1 for 3 cores now and the freezes are pretty rare so I may drop them back to 5.0.

Leave well alone. You could post your [JobScheduler] section from Prepar3D.cfg so we can see how that side Is configured.

[SteveW 1,792]

15 hours ago, Cruachan said:

Hi Steve,

I wonder whether some of the confusion arises from stating ‘Addons are launched by the sim’ which might imply that such Addons are being called by P3D. Consequently such Addons might be being considered as having the right to use any of the same LPs hidden by the mask and intended to be reserved for P3D which, of course, is not the intention.

Strictly speaking, Addons are not actually launched by the sim. Instead each is launched outside the running sim by the user or via a 3rd Party facilitator like SimStarter or FSUIPC in a predefined order. They will then link/connect to the sim while running on any available unmasked core or LP/LPs, as determined by the Windows Job Scheduler which is not ideal or, preferably, will occupy only those specific cores/LPs as assigned by a bitmask described in a bat file or, more easily in my case, via the GUI of Process Lasso. These latter solutions should ensure that any Addon is prevented from running on physical cores 0, 1 and 2.

Currently, I’m experimenting by keeping everything P3D related away from core 0 (LPs 0 and 1 with HT=ON). Leave that core for Windows related tasks. Early results are suggesting that this is having the desired effect by preventing the P3D main thread on LP2 from saturating at 100%. This was not achievable on my somewhat conservative setup while the main thread was running on LP0 which meant I had little or no spare overhead available.

Best regards,

Mike

That's right Mike, I am implying "being called by P3D". Generally add-ons are launched externally or by another program these days, basically they see all cores (or LPs) allocated to the launching process, that includes their SimConnect client. Those launched from within the sim get access to cores allocated to the sim in AffinityMask, that is why you now have AffinityMask and P3DCoreAffinityMask, it is designed that way so that you can have more cores beyond those allocated in P3DCoreAffinityMask. Even so, it is best to not recommend those sister LPs of the main task cores be made available even if one does not have sim launched stuff, it may happen. Generally add-ons do not set up an affinity so other tools are used to do that with them launched externally or by a batch file. Keep an eye on Task Manager performance graphs which shows fairly well if there's anything going on to disturb the main tasks. An efficient backend is helpful to allow the main task to remain consistent.

Regards

Steve

[SteveW 1,792]

11 hours ago, jlohrenz said:

I have an 8700k and thought the old rule of thumb was to run with HT turned off.  Has that logic changed?  I read discussion of the potential benefits of HT but that seemed to be with chips that have more cores than what I'm running.  Right now, with HT off, I have my system running a nice smooth 5Ghz on all cores with no heating problems so maybe I should leave it alone?

Would appreciate any suggestions. Since I need to stretch out a bit more from this machine considering the cost of new hardware.

HT enabled systems load files quicker, run networking quicker, all that is highly threaded. Because the CPU does more work with HT enabled, it comes with more heat (heat = work done). Systems that want to be overclocked to the max so that the main task can run faster might experience excess heat so they run with HT disabled even though that limits the overall performance of the system. The file loading performance can be increased by15% simply by turning on HT, that might be better than increasing the clock speed by 5% to gain extra fps when the fps is capped.

[SteveW 1,792]

For the technically minded Hyperthreading (HT) cores contain extra hardware to do task switching more efficiently. In a simplified explanation, we can look at a non HT core with two identical threads: The CPU runs one thread for a time, sends and receives data and instructions for that thread, processes those instructions saves away results and so on. But at the end of that time the CPU prepares to run the other thread for a time by saving away the state of the first thread so that it can be resumed later. Each thread is run for a time and they swap over so that from the outside the two asynchronous threads appear to run synchronously. With HT enabled we have two Logical Processors (LP) emulated by the core with the extra HT hardware. The JobScheduler assigns one thread to each LP and the process of swapping each thread for a time is the same because it is only one core. However, with HT the state of each thread is held so that it can resume without fuss this saves half the switching time across all threads if they are distributed among HT LPs equally. Also, even though one thread is stopped to allow the other processing time, the data and instructions for those threads continue synchronously which also saves time when the threads resume.

[virtuali 4,421]

17 hours ago, Cruachan said:

Strictly speaking, Addons are not actually launched by the sim. Instead each is launched outside the running sim by the user or via a 3rd Party facilitator like SimStarter or FSUIPC in a predefined order.

That's not entirely accurate.

The sim can launch add-ons without any user intervention or without using any utilities to start things, and this is possible either from the add-on.xml file ( the EXE Category ), or from an EXE.XML file, which P3D still supports, even if it's considered to be a legacy practice.

In relationship to HT/Affinity, there's no functional difference between being started by the sim, or being started in another way. As EXEs, they have their own HT/Affinity settings, which are independent from the sim and not controlled by it. But depending how many of these .EXE add-ons you have, this might impact your decision to set the simulator own Affinity, because running just the sim and nothing else, is completely different than running dozen of other .EXE in parallel with it.

If one want to be **very** accurate, there is a difference between "add-ons started by the sim" and "add-on started in other ways", and it's a permission issue, since a process started by another process will usually inherit the launcher permissions so, for example, if you started P3D with admin rights, the add-ons started by the sim will also start with admin rights. Instead, add-ons started in different way will start using their own permission or, if they don't have any, the permissions of the user that started them. But permissions won't affect anything related to HF/Affinity, they might (or might not) affect compatibility, file access, etc, depending what the add-on is supposed to do.

Instead, if add-ons ( or even if *parts* of your add-ons ) use a .DLL, that's a completely different matter, since a .DLL is running IN-PROCESS with the sim, which means they will be affected by HT/Affinity settings of the sim

Edited January 25, 2022 by virtuali

[d.tsakiris 418]

1 hour ago, virtuali said:

In relationship to HT/Affinity, there's no functional difference between being started by the sim, or being started in another way. As EXEs, they have their own HT/Affinity settings, which are independent from the sim and not controlled by it.

I'm not sure that part is true. I have come across processes whose affinity I couldn't change with the Task Manager. Those were programs that were started by P3D, IIRC, via exe and add-on.xml, and they had the same affinity as P3D itself. I usually keep other processes away from the cores P3D is running on, but in those cases I couldn't. That's probably what the new affinity mask settings are for.

[SteveW 1,792]

Permissions, Privileges and Run as Admin have been a major source of unnecessary problems for a decade.

If we want to be very accurate; a process started by a parent inherits the core affinity of the parent and inherits the permissions of the parent.

The permissions situation has been a total mess since the "User" lost Modify permission to the \Program Files folder which became read only after Windows XP.

When we install Windows we are the first user and by definition an Admin otherwise we could not administer the PC. However for normal everyday use of the system our permissions are restricted, but we can Run as Admin to elevate those permissions and privileges, for example when we run an installing program.

When a program installs, for example Prepar3D, it elevates permissions to Admin and puts programs in \Program Files and creates a folder for our normal everyday user permissible files in \AppData. We can run P3D as an everyday user and there is no need to Run as Admin because our user settings are in a folder with "everyday user permissible files". The program files are protected from tampering, being stored in a read-only folder.

Let's look at an imaginary add-on that installs its program in \Program Files and also installs its everyday user permissible files into the same folder. We run the sim as a User and then when we operate the add-on it has User privileges so when we go to save settings it can't because we cannot write to a read-only location with ordinary everyday permissions.

That add-on could do either of two things but there is a third option:

When the installer runs it has Admin permissions and during that time it can enable Modify permission (read/write) for the Users group on its file location so that settings can be written out even though it installed in read-only \Program Files because its own folder has the Modify permission for Users.

OR, the installer can make a folder for the "everyday user permissible files" outside of \Program Files perhaps as Prepar3D does in \Users\you\AppData.

Both of those solutions are correct. But in the third option the add-on does not set up permission for itself or install its user files in a read/write location and instead expects the user to launch the Simulator with Run as Admin privileges, or they might recommend to install the simulator in a private folder that will ordinarily have Modify permission.

That need not happen because the proper way out of this hole is that we can very simply give the Modify Allow permission to the Users Group on the Add-on folder rather than be forced to Run as Admin every day or re-install the simulator.

Network shares are affected by permissions so another way of handling access to network shares is to Run as Admin which is again the wrong way to do things. Instead the network share should have been set to enable users in the authenticated Users Group. Authenticated means we logged onto the Windows PC (or Network) with our sign-on and password.

In an industrial situation a PC might be locked down to prevent unwanted access or tampering, the regular User cannot Run as Admin unless a password is applied. There's no room for poorly set permissions in these cases and to Run as Admin requires a real life Administrator to come over to the PC and type in a password to gain Admin privileges for that user. Setting up P3D at an industrial site, they would mostly require it to run without Admin privileges.

[Cruachan 705]

2 hours ago, SteveW said:

Keep an eye on Task Manager performance graphs which shows fairly well if there's anything going on to disturb the main tasks. An efficient backend is helpful to allow the main task to remain consistent.

Hi Steve,

So, keeping P3D away entirely from core zero, which is favoured by the Windows O/S, would seem to be a good idea?

The down side is that it reduces the core count available for rendering based operations and/or running Addons. However, in multi-cored CPUs (8+) with HT=ON, unused LPs on those rendering cores could be brought into play by adjusting the Affinity Mask to compensate for this deficit.

Regards,

Mike

Edited January 25, 2022 by Cruachan

[SteveW 1,792]

3 minutes ago, Cruachan said:

Hi Steve,

So, keeping P3D away entirely from core zero, which is favoured by the Windows O/S, would seem to be a good idea?

The down side is that it reduces the core count available for rendering based operations and/or running Addons. However, in multi-cored CPUs (6+) with HT=ON, unused LPs on those rendering cores could be brought into play by adjusting the Affinity Mask to compensate for this deficit.

Regards,

Mike

Core zero is not favoured by the O/S. Also, I don't see much happening on core zero when the sim is operating, very few Windows processes are actually running hard enough to impact any core they are on. Add-ons on the other hand can consume perhaps 20% of a core at times and we don't want that happening on the core with the screen view updating.

With older versions of P3D we would normally see the first core used by the screen view updating process.

With the P3D v5.3 affinity extensions we can modify on which core that resides with the MainThreadScheduler item.

On an HT Enabled 8 core / 16 LP system I would do the following:

07,06,05,04,03,02,01,00=core number
11,11,11,11,11,01,01,01=AffinityMask=65493
11,11,11,11,11,01,01,01=P3DCoreAffinityMask=65493 OR
00,11,11,11,11,01,01,01=P3DCoreAffinityMask=16341 - uses slightly less bandwidth for scenery gathering
00,00,00,00,00,00,00,01=MainThreadScheduler=0
00,00,00,00,00,00,01,00=RenderThreadScheduler=1
00,00,00,00,00,01,00,00=FrameWorkerThreadScheduler=2
 

Keep the ABC order of Main, Render and FrameWorker and things work very well, other orders may not work as well or even cause CTDs, although that should not happen, there are still things to be completed in P3D around these extensions..