Jump to content

Archived

This topic is now archived and is closed to further replies.

Guest

Profiling FSX via Process Monitor

Recommended Posts

Guest

I got some free time and I'm trying to determine what exactly FSX is doing while it's executing.  I started Process Monitor (64bit v3.05) and executed FSX.exe and applied filters (so that I only see what FSX.EXE is doing).

 

So the stream of data starts to get logged in Process Monitor (16,000,000+ events by the time I stopped monitoring) ... FSX is at the Free Flight screen.  I start looking at the event data and find a few interesting bits of info:

 

1.  You can copy any specific version of DX9 or DX10 (files/dlls) into the root of FSX and FSX will use that specific version rather than what's installed on your OS.

 

Examples: you can see where FSX.EXE doesn't find the DX file in the root of FSX so proceeds to get the currently installed version from the OS

http://robainscough.com/images/8bc8b037b366b4af600e2426c63ace4d.jpg

http://robainscough.com/images/8c36014d18f989fbade7ff21bcf438d3.jpg

http://robainscough.com/images/bcb244f81648771eb33f5fd8caa8e779.jpg

 

2.  Some well know Aircraft developers and Microsoft appear to be referencing fonts that may not exist on the destination computer (not a big deal, just a little sloppy).

 

http://robainscough.com/images/f1f7ca7fccc49854e361a8cfc0365a8b.jpg

 

3.  But I don't understand why FSX would be looking at my drive's root from where FSX is installed -- i.e. FSX is installed in E:\Microsoft Flight Simulator X, but I'm seeing "PATH NOT FOUND" references to E:\Scenery\Global\Texture?  This just seems like a bug - perhaps a known issue?  It should be referencing E:\Microsoft Flight Simulator X\Scenery\Global\Texture, so what is it doing and why?

 

http://robainscough.com/images/81603aa35548df3bf601aaca62289fb3.jpg

 

Anyway, lots of interesting stuff ... my goal is to try to equate FSX.CFG changes and scenery changes with real world performance.  I also noted that every single scenery add-on I have installed gets accessed regardless of display need - I think this is a know issue - but why it's important is it confirms that the more scenery you add, the slower FSX is going to operate - disabling scenery does seem help as those disabled scenery areas are no longer being looked at by FSX. 

 

I'm not specifically interested in FPS as that's less important than overall system throughput.  I'm specifically looking at FSX.EXE event durations (ms) during the course of a flight ... I've not really seen anyone or any group able to identify performance bottlenecks with a specific setting (or combination of settings) in a specific flight environment.  What I see a lot of is, person A tested this on a system and it was faster, person B tries it and no difference, person C tries it and it's slower ... in other words there is no concrete "cause and effect" being identified.

 

For example, lets say loading 4096 textures introduces stutters ... so what's actually the bottle neck/limitation, is it the IO device (SSD/HD), is it a saturated Bus, is the CPU just too busy and can't process the request, is the GPU taking to long to get the texture loaded and applied??  Process monitor alone can't do this, at least I haven't found one yet that can and log everything to a file over time (CPU,GPU,Audio,I/O,Bus, etc.).  

 

My goal is to find a way to hit the magic settings, but do it in a way that is NOT trial and error ... data flow is a balancing act ... but if you have good scales, one should be able to find the perfect balance ... now what to test on a system and one should have a repeatable formula for knowing what to set in all cases.

 

What got me interested in this process is the Affinity setting ... what I noticed is that some Affinity settings will actually saturate the bus so much (in other words the threading IS working but it's killing my bus) that HD Textures don't always load with increased blank/corrupted tiles ... it's not a problem with the performance of the SSD it's a problem with bus saturation (too fast is actually bad in this case).  So this starts my quest ... I hope to report back with more than just trial and error.

 

Rob.

 

 

 

 

Share this post


Link to post
Share on other sites

Very interesting stuff....and beyond me but I get the general intent.

 

I'll stick to designing models for fsx.

Share this post


Link to post
Share on other sites
Guest

Theoretically it should be just a matter of gathering the data and doing the math (it's a finite solution with repeatable results) ... similar to how many more recent game products profile one's computer to set the appropriate graphics settings, the primary difference will be end user objects vs. preset objective.

 

Keep designing models, it's all good.

Share this post


Link to post
Share on other sites

I also noted that every single scenery add-on I have installed gets accessed regardless of display need - I think this is a know issue - but why it's important is it confirms that the more scenery you add, the slower FSX is going to operate - disabling scenery does seem help as those disabled scenery areas are no longer being looked at by FSX. 

 

I wish I downloaded some scenery manager suggested by users in another thread, that quickly disables unused sceneries. Unfortunately I can't find the discussion now. Very interesting observations, robains.

 

Dirk.

Share this post


Link to post
Share on other sites

Can't you just untick the scenery you don't use in the scenery library manager? Or use the scenery config editor app?

Share this post


Link to post
Share on other sites

Can't you just untick the scenery you don't use in the scenery library manager? Or use the scenery config editor app?

 

I don't want to enter FSX to mess with the library.

 

So was it scenery config editor? If it is in FSX SDK, then it was some different custom manager.

 

Dirk.

Share this post


Link to post
Share on other sites

I wish I downloaded some scenery manager suggested by users in another thread, that quickly disables unused sceneries. Unfortunately I can't find the discussion now. Very interesting observations, robains.

 

Dirk.

 

scenery config editor http://sourceforge.net/projects/fs-sceditor/

Share this post


Link to post
Share on other sites

In perfmon you can get some information, which could log to a file.  Go to Process, then look for ID Process, then in the box below look for  FSX.  I don't see anything for video right at this moment.  Maybe there's an nvidia utility for that?  Here's nvidia utility  https://developer.nvidia.com/nvidia-perfkit

Share this post


Link to post
Share on other sites
Guest

Ok, making some progress ... there does indeed appear to be a relationship between the quantity of scenery areas enabled causing delays in the loading of hires textures (Photo sceneries) over the current flight area.

 

I'm finding these entries have little to no affect at all in terms of loading HiRes textures (I'm running DX10 Preview):

 

AffinityMask

TextureMaxLoad

FIBER_FRAME_TIME_FRACTION

SWAP_WAIT_TIMEOUT

TEXTURE_BANDWIDTH_MULT

LOD_RADIUS  (yes, 9.5 or 4.5 doesn't impact hires texture loading)

TEXTURE_MAX_LOAD (4096, 2048, 1024 makes no difference in texture loading either)

 

These setting may (and do) affect FPS and/or remove/add stutters, but they appear to do nothing in terms of HiRes textures loading of photoscenery.

 

I my case, I'm using FB KIAD and MSE Virginia (and West Virginia) enabled, I disabled all other 3rd party photoscenery and airports (I have a lot 250+).  While looking at Process Monitor with and without all scenery enabled I can see FSX making the FILE IO calls to "enabled" scenery that has nothing to do with the current area I'm flying (like California - opposite side of the country).  IN contrast, when ONLY Virginia is enabled, I don't see any file IO events for California (which is disabled).

 

I'm guessing there is some threshold of "enabled" areas that will put my system over the top - need to find out.

 

I need to dig more and come up with some real world video and screen shot samples to present in a more formal manner and make sure I've got my variables restricted.

 

Rob.

 

EDIT: Just want to be clear, my goal is not FPS nor avoiding OOM (but on a side note, with most scenery disabled I'm seeing less memory used).

Share this post


Link to post
Share on other sites

The path not found error could be caused by a misformatted FSX registry path, usually a missing backslash. Try running flight1's registry repair tool.

Share this post


Link to post
Share on other sites

No,  you have just opened a "whole can of worms" by looking at FSX in Process Explorer !!

 

Why one would look for Fonts  in   C:/windows/font/fonts   is beyond me.

 

C:Windows    +  /fonts    YES

 

but not

 

C:/Windows/font  + /fonts   !!!

 

---

You will also see the dreaded  uiautomationcore.dll being called   :)

 

--

 

I sometimes get an error with FSX.. a popup  that says  "Cannot read from File",   -- STUPID error messages does not say WHAT file  ...  but so far, have failed to see that file not being able to be read in Process Explorer.

 

  The more you watch Process Explorer, the more you realize why FSX runs so slowly and inefficiently  !!

Share this post


Link to post
Share on other sites

I my case, I'm using FB KIAD and MSE Virginia (and West Virginia) enabled, I disabled all other 3rd party photoscenery and airports (I have a lot 250+).  While looking at Process Monitor with and without all scenery enabled I can see FSX making the FILE IO calls to "enabled" scenery that has nothing to do with the current area I'm flying (like California - opposite side of the country).  IN contrast, when ONLY Virginia is enabled, I don't see any file IO events for California (which is disabled).

 

 

Curious if you had any AI running? I would expect that the game would need to know if California was there to route a flight from KSFO(?) to KIAD. But I am not well versed in the finer points of AI to know if it might affect flightplans that are in use.

Share this post


Link to post
Share on other sites
Guest

Running Traffic 360 at 20% - enabled in my testing.

 

With MSE California disabled FSX just loads default ... Which is what I'm seeing. But the feature that allows one to jump into AI aircraft could be reason FSX access the scenery (default or otherwise). May also explain why AI traffic introduce such a high performance impact.

 

But to be clear on the errors process monitor is showing - these are Handled Exceptions, meaning FSX deals with error. Unhandled exceptions are the ones a user would see as FSX doesn't deal with the error (like an OOM error).

Share this post


Link to post
Share on other sites

Actually,  FSX's  typical OOM  error message is the message from a Handled Exception !

ie

On memory allocation error, throw up useless error message, and then really upset the user, by telling them,  to" just" Restart FSX , (and start their 4 hour flight all over again).

 

At the time FSX was coded,  there were not the memory intensive addons, and it seems that little consideration was given to what, at the time, was a very rare occurrence, that memory would be exhausted, (or fragmented to such a state,  that it became unusable)

 

In hindsight, this particular exception handing could probably have been code to correct the  low memory condition, and the simulator  could have continued to run.

Share this post


Link to post
Share on other sites
Guest

It would be nice to have memory usage data displayed next to the FPS (Shift-Z) data ... makes it a lot easier to understand what condition triggers OOM or if one is getting very close to the edge.  I understand FSUIPC can do the low memory chimes but no definition of threshold (when to start chiming).

 

Really happy about my scenery discovery (maybe some folks already knew about this issue, but I never came across this in all the many threads I've read).  I've suspected this issue for some time but never got around to actually testing out my theory. 

 

I guess Flight planners should now start working with scenery files (enabled/disable scenery based on one's flight plan).  Question I need to discover now is what's the scenery enable threshold for my setup.

Share this post


Link to post
Share on other sites
Guest

Here are some screenshots ...

 

MSE Virginia 2.0 and West Virginia Enabled, FlightBeam KIAD enabled, FSDT KSFO enabled, Traffic 360 Airport Facilities enabled, ES-3D-Library enabled (wanted to keep this always enabled).

 

http://robainscough.com/images/c0ae65c4fd7e98ddffd97aa57dd58b7e.jpg  - 2560 x 1600 (JPG that was compressed for web deployment so not as good as original but close)

http://robainscough.com/images/0b795f16568f2601923c7c6793b744a1.jpg

 

The ONLY change, enabled ALL my scenery

 

http://robainscough.com/images/50cf02d6437f8d8de79c1f79deb8ac42.jpg

http://robainscough.com/images/52b7e239155d2e957eeaa4f91a3de888.jpg

 

As you can see, with ALL my scenery enabled, the same area is pretty darn ugly.  But again, want to be clear, the ONLY change made was Enable/Disable of scenery areas, NO FSX.CFG settings were modified.

 

I still haven't figured out what the threshold is yet (how many scenery areas I can have enabled at the same time before I run into the same texture tile loading problems).  Another possibility I might need to consider is that perhaps it's just a few specific malformed scenery files are causing this problem -- that will be hard to discover given ALL the scenery files I have.  It seems unlikely, but who knows.

 

Would love to hear from others that have A LOT of photo scenery and see if they've run into this problem?

 

Here is my long long list of scenery areas I have (and what I enabled/disabled - I'm using Addit! Pro )

http://robainscough.com/images/84c0cbe56f639374fcd8efea6a875fb1.jpg

http://robainscough.com/images/b5e84050a1c21715d594b0819e751fe2.jpg

http://robainscough.com/images/32ccc4424617bfb499b1190d10fabf1b.jpg

http://robainscough.com/images/f327d279446b85743bc88fd9d715fdf4.jpg

http://robainscough.com/images/891b95bb1e70adf97c7a5dd749f8546a.jpg

http://robainscough.com/images/0933769c4ec84e51536ae5ec0238e13f.jpg

http://robainscough.com/images/6e2a7b28e508a1a8764fcba6b9a687ec.jpg

http://robainscough.com/images/5e0b63a577170360a88ead8b0799cef2.jpg

Share this post


Link to post
Share on other sites

I'm not surprised you are having issues with the massive size of that scenery library!

Share this post


Link to post
Share on other sites
Guest

 

 


I'm not surprised you are having issues with the massive size of that scenery library!

 

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries.

 

I wonder if this is something P3D developers discovered?  I have not tested this problem to see if it exists in P3D (a lot of work to load that much scenery).  But on the plus side, if LM have found this problem and addressed it for V2.0, then I can certainly see why they didn't go down the path of 64bit processing ... fixing this problem alone could free up a large chunk of 32bit addressable space.

Share this post


Link to post
Share on other sites
Guest

Very interesting! Does this also apply to default FSX regions? If I fly in Europe all the time, could I disable all America, Africa, Asia and Oceania regions in the Scenery Library...? I also use FTXG. I wonder if disabling certain regions won't have a negative effect on the active regions...? Missing textures or autogen...? I also noticed FGS comes with a few cities (Rio, Las Vegas): I suppose they can be turned off too? All base packs should be kept on, I suppose.

 

If all your findings only apply to addon scenery then the above can be disregarded, of course. ^_^

 

EDIT

Just had a look at your screenshots of all the sceneries. Wow, that's a lot! But I also noticed you have all default scenery enabled so I guess my question has been answered and all this is only about addon scenery...

Share this post


Link to post
Share on other sites

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries.

 

I wonder if this is something P3D developers discovered?  I have not tested this problem to see if it exists in P3D (a lot of work to load that much scenery).  But on the plus side, if LM have found this problem and addressed it for V2.0, then I can certainly see why they didn't go down the path of 64bit processing ... fixing this problem alone could free up a large chunk of 32bit addressable space.

I don't think it's actually loading the foreign scenery, but I would think it would have to read part of the files, to identify, if it needs to be loaded. How else would FSX know whether a scenery file was aplicable to the area you are located in? It would have to read some sort of header to identify if the scenery matches the coordinate radius you are placed at.

Share this post


Link to post
Share on other sites

It would be nice to have memory usage data displayed next to the FPS (Shift-Z) data ... makes it a lot easier to understand what condition triggers OOM or if one is getting very close to the edge. I understand FSUIPC can do the low memory chimes but no definition of threshold (when to start chiming).

FSUIPC can indeed give an in-game, continuously updated display of free VAS.

 

To enable it, go to the. FSUIPC "logging" tab.

 

In the first box marked "specific value checks", enter the value "024C"

 

Change the "type" associated with that value from "S8" to "S32"

 

You may then select the checkbox to have the value at offset 024C displayed either within the FS main window, or in the upper title bar.

 

Save these changes and exit the FSUIPC menu.

 

You will now have an in-game display of the amount of free VAS memory remaining. I believe it updates every 10 seconds.

 

On my (Win 7) system, I will typically start out with about 3.1 GB of free VAS when first loading FSX with the Trike. You can easily see at a glance how the VAS pool is impacted by scenery, aircraft complexity, weather etc.

 

I believe the FSUIPC will beging sounding the warning chime when free VAS gets below about 250 MB.

 

 

Share this post


Link to post
Share on other sites

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries

 

I am pretty sure FS only reads the header section of each BGL, to check whether its area affects the "reality bubble" in which you are flying. I don't think it reads and retains the files themselves. It would of course be better if each scenery layer could have an overall "area of effectiveness" definition so that layers could be skipped when not applicable.

 

What FSX is doing is similar to what all versions of FS have done for a long long time. In FS4 days, running under DOS, I wrote a "TSR" (Terminate and Stay Resident) program which did a scenery scan before FS was loaded and compiled an in-memory table of all of the header sections of all the scenery files. It placed an intercept on the FS Open and Read calls into DOS, and if the file was one of those it had dealt with, and the area being read was only the header, it supplied the data back and didn't pass the commands on to DOS. The program was called "QuckScene" and was very popular as it smoothed FS4's operation considerably.

 

But those were the days of slow disks with no caching. These days with super-fast disks, and large caches both in the disk hardware and in Windows, I don't think my "QuickScene" method would have much effect on performance. You'd need a way of automatically skipping complete layers -- in fact an automatic way of doing what some folks are doing, eliminating those layers which aren't being used.

 

Regards

Pete

Share this post


Link to post
Share on other sites
Guest

 

 


I am pretty sure FS only reads the header section of each BGL, to check whether its area affects the "reality bubble" in which you are flying. I don't think it reads and retains the files themselves. It would of course be better if each scenery layer could have an overall "area of effectiveness" definition so that layers could be skipped when not applicable.

 

I'm no scenery expert but I was under the assumption FSX builds a scenery database with indexes ... if not, what exactly is FSX doing any time I change my scenery and FSX shows me a message "Building database/indexes" at startup?

 

Great input Pete (as always) - I'm pretty sure you are 100% correct as that would be why lots of scenery is causing a HiRes tile load problem.  If FSX has to read every single BGL header to know if it needs to load scenery then that's clearly going to be a HUGE bottleneck and very inefficient (also explains why enable/disable of scenery is so effective at reducing the problem) - even the fastest of systems and IO devices isn't going to be able to manage this process fast enough.  For example, I have 73,058 BGL files for FSX ... if FSX has to process that many headers in real time then it's no wonder I see this problem.

 

I can't imagine why Aces would code this process knowing it's going to be a big problem as people add more scenery?  My only guess is to retain compatibility?  Seems like a harsh penalty to me and very limiting.

 

I do remember the days of TSR ... hmmm ... in todays OS I think one would need some kernel driver that could communicate to a user-mode app (aka FSUIPC) or maybe a Windows Service to "try" to solve this problem.  Or we can hope LM have seen the problem and will fix in V2.0 of P3D?  Certainly do NOT think this is good news for all those scenery devs and one better stick to the "enabled" flight plan.

 

 

 


FSUIPC can indeed give an in-game, continuously updated display of free VAS.

 

Excellent, thank you, great info.

Share this post


Link to post
Share on other sites

I was, apparently scenery that isn't visible gets loaded ... I made a bad assumption that FSX was smarter about scenery loading.  In fact, FSX will load UK scenery (my VFR GenX UK/Scotland) even when I'm sitting on the runway at KSFO (about 8000+ miles away) ... I can certainly see why FSX hits OOM pretty easily and why so many have a case of the blurries.

 

I wonder if this is something P3D developers discovered?  I have not tested this problem to see if it exists in P3D (a lot of work to load that much scenery).  But on the plus side, if LM have found this problem and addressed it for V2.0, then I can certainly see why they didn't go down the path of 64bit processing ... fixing this problem alone could free up a large chunk of 32bit addressable space.

 

The typical memory issue with FSX is not the Lack of free memory, but rather, the lack of a big enough CHUNK of contiguous memory, when FSX call for some additional Memory allocation.

 

After a time running, FSX has called for numerous Chunks of memory, of differencing sizes, and has also released  chunks of memory.

 

However, as the memory becomes more and more Fragmented, there eventually reaches a time when FSX calls for  CHUNK of memory, and there is no where , amongst all the free memory, where there is a Contiguous Block of memory, big enough to meet that need.      That is not to say that thee is not a considerable amount of available free memory, but it is now all in smaller, broken up lengths,  none of which is long enough to meet the allocation need, so an memory allocation error occurs. FSX is programmed to deal with this error, by displaying a Useless Error Message, and once you acknowledge that popup error message, FSX is then programed to close FSX.

 

So, while there can be plenty of FREE memory available, none is any longer in a large enough contiguous block, to meet FSX;s needs.

 

This is why FSX runs for some time, before "running out" of memory.

 

Note: Most tools to display available memory, will display the Total of all these small memory blocks,  so while it may appear that you still have plenty of free memory available,  it is effectively unusable by FSX.

 

I do not know what Pete's FSUIP looks for when determining available memory, but I suspect he is being smart, and looking for how much memory, is available in the typically Largest Block sizes that FSX calls for, to determine  is one is close to running out of "available" memory. ??

 

What FSX really "needed" was a better "internal" memory management system, that prevented Fragmentation,  and maybe actively did something to defrag in real time, while FSX was running.  -- too late now for FSX,  unless someone can devise some form of defragmenter  that can be "linked" into FSX.

 

I only know of one person that might be able to achieve that  ...and incorporate it into FSUIPC maybe.  ? 

Share this post


Link to post
Share on other sites

×
×
  • Create New...