Profiling FSX via Process Monitor

[Guest]

It would be nice to have memory usage data displayed next to the FPS (Shift-Z) data ... makes it a lot easier to understand what condition triggers OOM or if one is getting very close to the edge.  I understand FSUIPC can do the low memory chimes but no definition of threshold (when to start chiming).

 

Really happy about my scenery discovery (maybe some folks already knew about this issue, but I never came across this in all the many threads I've read).  I've suspected this issue for some time but never got around to actually testing out my theory. 

 

I guess Flight planners should now start working with scenery files (enabled/disable scenery based on one's flight plan).  Question I need to discover now is what's the scenery enable threshold for my setup.

[Guest]

Here are some screenshots ...

 

MSE Virginia 2.0 and West Virginia Enabled, FlightBeam KIAD enabled, FSDT KSFO enabled, Traffic 360 Airport Facilities enabled, ES-3D-Library enabled (wanted to keep this always enabled).

 

http://robainscough.com/images/c0ae65c4fd7e98ddffd97aa57dd58b7e.jpg  - 2560 x 1600 (JPG that was compressed for web deployment so not as good as original but close)

http://robainscough.com/images/0b795f16568f2601923c7c6793b744a1.jpg

 

The ONLY change, enabled ALL my scenery

 

http://robainscough.com/images/50cf02d6437f8d8de79c1f79deb8ac42.jpg

http://robainscough.com/images/52b7e239155d2e957eeaa4f91a3de888.jpg

 

As you can see, with ALL my scenery enabled, the same area is pretty darn ugly.  But again, want to be clear, the ONLY change made was Enable/Disable of scenery areas, NO FSX.CFG settings were modified.

 

I still haven't figured out what the threshold is yet (how many scenery areas I can have enabled at the same time before I run into the same texture tile loading problems).  Another possibility I might need to consider is that perhaps it's just a few specific malformed scenery files are causing this problem -- that will be hard to discover given ALL the scenery files I have.  It seems unlikely, but who knows.

 

Would love to hear from others that have A LOT of photo scenery and see if they've run into this problem?

 

Here is my long long list of scenery areas I have (and what I enabled/disabled - I'm using Addit! Pro )

http://robainscough.com/images/84c0cbe56f639374fcd8efea6a875fb1.jpg

http://robainscough.com/images/b5e84050a1c21715d594b0819e751fe2.jpg

http://robainscough.com/images/32ccc4424617bfb499b1190d10fabf1b.jpg

http://robainscough.com/images/f327d279446b85743bc88fd9d715fdf4.jpg

http://robainscough.com/images/891b95bb1e70adf97c7a5dd749f8546a.jpg

http://robainscough.com/images/0933769c4ec84e51536ae5ec0238e13f.jpg

http://robainscough.com/images/6e2a7b28e508a1a8764fcba6b9a687ec.jpg

http://robainscough.com/images/5e0b63a577170360a88ead8b0799cef2.jpg

[GHarrall 494]

I'm not surprised you are having issues with the massive size of that scenery library!

[Guest]

 

 

I'm not surprised you are having issues with the massive size of that scenery library!

 

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries.

 

I wonder if this is something P3D developers discovered?  I have not tested this problem to see if it exists in P3D (a lot of work to load that much scenery).  But on the plus side, if LM have found this problem and addressed it for V2.0, then I can certainly see why they didn't go down the path of 64bit processing ... fixing this problem alone could free up a large chunk of 32bit addressable space.

[Guest]

Very interesting! Does this also apply to default FSX regions? If I fly in Europe all the time, could I disable all America, Africa, Asia and Oceania regions in the Scenery Library...? I also use FTXG. I wonder if disabling certain regions won't have a negative effect on the active regions...? Missing textures or autogen...? I also noticed FGS comes with a few cities (Rio, Las Vegas): I suppose they can be turned off too? All base packs should be kept on, I suppose.

 

If all your findings only apply to addon scenery then the above can be disregarded, of course. ^_^

 

EDIT

Just had a look at your screenshots of all the sceneries. Wow, that's a lot! But I also noticed you have all default scenery enabled so I guess my question has been answered and all this is only about addon scenery...

[tf51d 284]

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries.

 

I wonder if this is something P3D developers discovered?  I have not tested this problem to see if it exists in P3D (a lot of work to load that much scenery).  But on the plus side, if LM have found this problem and addressed it for V2.0, then I can certainly see why they didn't go down the path of 64bit processing ... fixing this problem alone could free up a large chunk of 32bit addressable space.

I don't think it's actually loading the foreign scenery, but I would think it would have to read part of the files, to identify, if it needs to be loaded. How else would FSX know whether a scenery file was aplicable to the area you are located in? It would have to read some sort of header to identify if the scenery matches the coordinate radius you are placed at.

[JRBarrett 3,070]

It would be nice to have memory usage data displayed next to the FPS (Shift-Z) data ... makes it a lot easier to understand what condition triggers OOM or if one is getting very close to the edge. I understand FSUIPC can do the low memory chimes but no definition of threshold (when to start chiming).

FSUIPC can indeed give an in-game, continuously updated display of free VAS.

 

To enable it, go to the. FSUIPC "logging" tab.

 

In the first box marked "specific value checks", enter the value "024C"

 

Change the "type" associated with that value from "S8" to "S32"

 

You may then select the checkbox to have the value at offset 024C displayed either within the FS main window, or in the upper title bar.

 

Save these changes and exit the FSUIPC menu.

 

You will now have an in-game display of the amount of free VAS memory remaining. I believe it updates every 10 seconds.

 

On my (Win 7) system, I will typically start out with about 3.1 GB of free VAS when first loading FSX with the Trike. You can easily see at a glance how the VAS pool is impacted by scenery, aircraft complexity, weather etc.

 

I believe the FSUIPC will beging sounding the warning chime when free VAS gets below about 250 MB.

 

 

[Pete Dowson 558]

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries

 

I am pretty sure FS only reads the header section of each BGL, to check whether its area affects the "reality bubble" in which you are flying. I don't think it reads and retains the files themselves. It would of course be better if each scenery layer could have an overall "area of effectiveness" definition so that layers could be skipped when not applicable.

 

What FSX is doing is similar to what all versions of FS have done for a long long time. In FS4 days, running under DOS, I wrote a "TSR" (Terminate and Stay Resident) program which did a scenery scan before FS was loaded and compiled an in-memory table of all of the header sections of all the scenery files. It placed an intercept on the FS Open and Read calls into DOS, and if the file was one of those it had dealt with, and the area being read was only the header, it supplied the data back and didn't pass the commands on to DOS. The program was called "QuckScene" and was very popular as it smoothed FS4's operation considerably.

 

But those were the days of slow disks with no caching. These days with super-fast disks, and large caches both in the disk hardware and in Windows, I don't think my "QuickScene" method would have much effect on performance. You'd need a way of automatically skipping complete layers -- in fact an automatic way of doing what some folks are doing, eliminating those layers which aren't being used.

 

Regards

Pete

[Guest]

 

 

I am pretty sure FS only reads the header section of each BGL, to check whether its area affects the "reality bubble" in which you are flying. I don't think it reads and retains the files themselves. It would of course be better if each scenery layer could have an overall "area of effectiveness" definition so that layers could be skipped when not applicable.

 

I'm no scenery expert but I was under the assumption FSX builds a scenery database with indexes ... if not, what exactly is FSX doing any time I change my scenery and FSX shows me a message "Building database/indexes" at startup?

 

Great input Pete (as always) - I'm pretty sure you are 100% correct as that would be why lots of scenery is causing a HiRes tile load problem.  If FSX has to read every single BGL header to know if it needs to load scenery then that's clearly going to be a HUGE bottleneck and very inefficient (also explains why enable/disable of scenery is so effective at reducing the problem) - even the fastest of systems and IO devices isn't going to be able to manage this process fast enough.  For example, I have 73,058 BGL files for FSX ... if FSX has to process that many headers in real time then it's no wonder I see this problem.

 

I can't imagine why Aces would code this process knowing it's going to be a big problem as people add more scenery?  My only guess is to retain compatibility?  Seems like a harsh penalty to me and very limiting.

 

I do remember the days of TSR ... hmmm ... in todays OS I think one would need some kernel driver that could communicate to a user-mode app (aka FSUIPC) or maybe a Windows Service to "try" to solve this problem.  Or we can hope LM have seen the problem and will fix in V2.0 of P3D?  Certainly do NOT think this is good news for all those scenery devs and one better stick to the "enabled" flight plan.

 

 

 

FSUIPC can indeed give an in-game, continuously updated display of free VAS.

 

Excellent, thank you, great info.

[FSMP 95]

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries.

 

I wonder if this is something P3D developers discovered?  I have not tested this problem to see if it exists in P3D (a lot of work to load that much scenery).  But on the plus side, if LM have found this problem and addressed it for V2.0, then I can certainly see why they didn't go down the path of 64bit processing ... fixing this problem alone could free up a large chunk of 32bit addressable space.

 

The typical memory issue with FSX is not the Lack of free memory, but rather, the lack of a big enough CHUNK of contiguous memory, when FSX call for some additional Memory allocation.

 

After a time running, FSX has called for numerous Chunks of memory, of differencing sizes, and has also released  chunks of memory.

 

However, as the memory becomes more and more Fragmented, there eventually reaches a time when FSX calls for  CHUNK of memory, and there is no where , amongst all the free memory, where there is a Contiguous Block of memory, big enough to meet that need.      That is not to say that thee is not a considerable amount of available free memory, but it is now all in smaller, broken up lengths,  none of which is long enough to meet the allocation need, so an memory allocation error occurs. FSX is programmed to deal with this error, by displaying a Useless Error Message, and once you acknowledge that popup error message, FSX is then programed to close FSX.

 

So, while there can be plenty of FREE memory available, none is any longer in a large enough contiguous block, to meet FSX;s needs.

 

This is why FSX runs for some time, before "running out" of memory.

 

Note: Most tools to display available memory, will display the Total of all these small memory blocks,  so while it may appear that you still have plenty of free memory available,  it is effectively unusable by FSX.

 

I do not know what Pete's FSUIP looks for when determining available memory, but I suspect he is being smart, and looking for how much memory, is available in the typically Largest Block sizes that FSX calls for, to determine  is one is close to running out of "available" memory. ??

 

What FSX really "needed" was a better "internal" memory management system, that prevented Fragmentation,  and maybe actively did something to defrag in real time, while FSX was running.  -- too late now for FSX,  unless someone can devise some form of defragmenter  that can be "linked" into FSX.

 

I only know of one person that might be able to achieve that  ...and incorporate it into FSUIPC maybe.  ? 

[Guest]

 

 

However, as the memory becomes more and more Fragmented, there eventually reaches a time when FSX calls for CHUNK of memory, and there is no where , amongst all the free memory, where there is a Contiguous Block of memory, big enough to meet that need

 

I can't obviously say for sure, but as I understand it FSX (C++) works with VAS so fragmentation shouldn't be a big problem.  Physical address space will not be contiguous under VAS and doesn't need to be (not a problem).  To get accurate VAS to Physical mapping "I think" that would have to be in KERNEL mode (aka not in FSX) and then communicate to FSX/FSUIPC (user mode) ... or maybe use memory descriptor list?  But again, there is some assembler code in FSX that it appears has been around for a long long long time and nobody and seems to be interested in touching it ... "feared" by Aces and LM.   So who knows what's going on there.

[tf51d 284]

I'm no scenery expert but I was under the assumption FSX builds a scenery database with indexes ... if not, what exactly is FSX doing any time I change my scenery and FSX shows me a message "Building database/indexes" at startup?

 

You can see the files and indexes it creates in the ProgramData\Microsoft\FSX\SceneryIndexes folder. I think what it's actually indexing is the airports, runways, parking, taxiways, etc, and radio NAVAIDS, both for User and ATC/AI use.

[Guest]

Ah, ok, that's it, thank you.

[FSMP 95]

I can't obviously say for sure, but as I understand it FSX (C++) works with VAS so fragmentation shouldn't be a big problem.  Physical address space will not be contiguous under VAS and doesn't need to be (not a problem).  To get accurate VAS to Physical mapping "I think" that would have to be in KERNEL mode (aka not in FSX) and then communicate to FSX/FSUIPC (user mode) ... or maybe use memory descriptor list?  But again, there is some assembler code in FSX that it appears has been around for a long long long time and nobody and seems to be interested in touching it ... "feared" by Aces and LM.   So who knows what's going on there.

 

  "Who knows what's going on there"  --   GREAT QUESTION !

 

Try Googling:

 

c++ Virtual address space fragmentation

 

Microsoft can explain it far clearer than I can  in a form post .          :help:

[tf51d 284]

 

 

I can't obviously say for sure, but as I understand it FSX (C++) works with VAS so fragmentation shouldn't be a big problem. Physical address space will not be contiguous under VAS and doesn't need to be (not a problem).

 

FSX also include legacy assembler code (The reason why P3D can't go to 64 bit so easily!) So unless those routines do proper cleanup/housekeeping of the memory it uses, then you can have fragmentation of the VAS.