Texture Lookup Goose Chase

[cklink 0]

Hi All,I was playing around with filemonitor to see if I could find out why FSX frame rates where dropping for me after about an hour of flight. Memstatus would start off at about 800K and end up around 1700k an hour later in the same region. Anyways in searching I found out something else that interested me. (BTW I have no mem leaks I can see)Understanding that I do not have any programming knowledge, I was wondering HOW MUCH the following would contribute to the reason framerates are such a hit at larger airports, or why studders are so promminant near them? Or if this sort of thing is normal. I noticed that most of the land textures, buildings etc and related items are always found on the first query. It it most promenent with the aircraft, boats and ground vehicles.A: Only the main texture folder of each aircraft has the pilot, bump, specular & reflection - textures in them. All other variants (ie: texture.2) do not. On average I see that about 10-14 unsucessful queries are made until these textures are located and read.B: Repeat the above searching for the Specular,Bump & LM textures for the Ships and ground vehicles.B: Most of the VEH_air ***** ground vehicles, the calls for textures end up on a goose chase - again making about 10 or so queries (including ones like D:/Program Files/Textures ??) Some of them are within thier respective folder, but others are in the main Scenery/Global/Texture folder (even though they are only for that model)C: Every time GLOBALENV_AC_CHROME.DDS is called for (on every airplane), it again takes an average of about 14 unsucessful queries until the texture is finally found. This includes it going back to read the scenery index to find the location (which for me is on a different drive), and then it still takes about another 6 queries to finally find it in the root texture folder.D: You can repeat this whole "query chase" thing for quite a bit of "NON TERRAIN" textures, including queries for gauges (eg: 737_400.dll).You should download filemonitor and try it to see exactly what I mean. You can get it here: http://www.microsoft.com/technet/sysintern...sk/Filemon.mspxI configured mine to "include" FSX, and HIGHLIGHT "NOT FOUND".Attached is a screenie of an example:http://forums.avsim.net/user_files/162525.jpgTo my basics of understanding, I don't know why you would want so much "looking" going on - as wouldn't is speed things up if you knew where to look on the first go at it. The reason I question this as to frame impacts, as if all this looking is going on every time a texture needs to reload - doesn't that slow things up ? Times that by all the ground vehicles and aircraft at an airport ....Or am I on another planet with my "basic thinking" .... Again ..;)Regards'Garett

[cklink 0]

Update:Compared against FS9, and FS9 has way less errors than FSX. I mean ALOT less.There are similar ones regarding the gauges, but almost every texture is always found on the first query.For now I'm going to copy the bump,specular,pilots,reflections etc from each aircraft type's main Texture folder over to the variants folders. (ie texture.2, .3 etc)Question, does having the same texture in two locations screw anything up? I'd like to clean up the texture folders on the rest of the simobjects (boats/ground vehicles) by putting the textures in the textures folders, instead of having half of them in the objects texture folder and the other half in the main root texture folder. (ie specular/bump/alpha etc.) I might be wasting my time by this, but I am curious to see if there is any performance gain.Regards'Garett

[LuisFelizTirado 33]

Hello Garett,I had done the same thing and was as surprised as you by the results. Quite puzzling and food for thought.In the previous version of FS, if something (texture, gauge) was not found on the first try, then it would not be displayed at all. To ensure that these elements could be found, the FS Team seems to have created more possibilities for fallback folders - the relevant cfg files can be found in there somewhere.It would be interesting to see if modifying or deleting these cfg files reduces the number of file queries (and increases performance, but probably not, as this must be a very fast operation.)You can copy the specular and bump maps to the variants folders without any problem.Please let us know what you come up with - this is a very interesting and neglected area of study.Best regards.Luis

[Ramasurinen 0]

Hey Garret. Your post got me thinking about the consistently worsening stutters I've been getting with fsx, especially during long flights. So taking your suggestion (thanks!) I loaded up filemon, ran a complete session for an hour, jumping all over the place, saved out the log and sat down to take a good long look at it. All I can say is, what a mess!It's not just aircraft textures that FSX is having issues locating, it's just about everything. Aircraft textures are mostly load once and forget about it, so I was more curious to see what insanity was going on with the scenery, which is constantly being loaded and dumped.1. It looks in the fsx/texture folder for textures that are in fsx/scenery/global/texture or fsx/scenery/world/texture folders about half the time. The other half it does exactly the opposite, and generally looks two to four times until it gets the right one.2. Half of the autogen textures are in the wrong format. It'll be looking for a DDS file and there's only a BMP, or vice versa. On top of that about half the bmp files are inefficient palettized 8 bit images, when they'd be better off as dxt1's. Yuk.3. It's constantly looking for a series of missing fs9 style AGN autogen description bitmaps that simply don't exist. And by constantly I mean every 10th sim event at least, no matter where you are.Ugh!So I set aside the better part of the day to sort out this mess, starting with autogen. I moved all the textures that were named correctly to the places FSX wanted to find them first. Then I loaded in all the bmps that were supposed to be dds files and converted them to that, and vice versa with dds's that were supposed to bmps. I tried creating dummy agn files to replace the missing ones it was looking for, but that just resulted in losing rural autogen for the most part, forests etc. Really not sure what's up with that one, but then I'm no scenery design guru.Anyway, the upshot of spending a day fixing some of this dog's breakfast?Scenery load times are considerably faster, microstutters are down about 90%, and framerate is up at least 20%. All my track-ir quick panning stutters are all but gone. Seattle, full autogen, both seatac and boeing field in view, as well as downtown, silky smooth 25 fps locked. Prior to that I'd be lucky to get 20 fps with normal autogen, and turns were a stutter fest. And yeah, the drive is in perfect shape fragmentation wise, and dedicated to FSX.Umm...optimization? I don't think they had time. Ouch.I'll post a full list of what I did soon, but the results so far just with fixes to non-region specific autogen are amazing. I haven't even looked at aircraft yet.Cheers, and thanks for the tip.

[Guest dswo]

Wow! I hope ACES is reading this.

[Guest Vilk]

thanks for the report. looking forward to a quick tutorial:-)

[cklink 0]

Agreed on the the perfomance gain, and the mess ....I did most of the simobjects, building & support textures and aircraft. Agree with you on the boost, as I have to date reduced my studders at CYVR & KSEA (my test APTS) and also gained a few fps. I was around a steady 21-25, and now getting 27-28. Flying from CYVR to downtown is a constant 28-30. I also find textures in flight are better at keeping up as well. I did complete PC shut downs to test, making sure it just wasn't faster due to stuff left in cache or memory. Saved flights, no changes."1. It looks in the fsx/texture folder for textures that are in fsx/scenery/global/texture or fsx/scenery/world/texture folders about half the time. The other half it does exactly the opposite, and generally looks two to four times until it gets the right one."Agreed - It looks like common default "look location" is "Scenery/Global/Texture" but the files are in the root FSX texture folder. Building & support textures included .... Airportparkingspace, chainlink, blastfence, BLDB*, aptsign etc etc etc."2. Half of the autogen textures are in the wrong format. It'll be looking for a DDS file and there's only a BMP, or vice versa. On top of that about half the bmp files are inefficient palettized 8 bit images, when they'd be better off as dxt1's. Yuk."The bmp/dds alternating goes with some of the aircraft and building textures. Example, BLDB2 is bmp, but BLD3 is dds. But it first looks for them in the Scenery/Global/Texture folder ... which they are not there. (PS - I only moved mine, not fixed them as to dds/bmp etc.)"3. It's constantly looking for a series of missing fs9 style AGN autogen description bitmaps that simply don't exist. And by constantly I mean every 10th sim event at least, no matter where you are."I left the autogen references for textures that have "hard coded" autogen (as I've heard) to the texture call. This may expain why you lost your autogen with dummy files. Don't know if there's much can do with that one ..What really bugs me is a continual reference/query to Drive/Program Files/Scenery/Texture for every simobject texture.cfg read ... (less aircaft)I wonder if the "fallback" in the cfg is correctly working. (Luis I tried deleting a cfg, but it only made for more scouring)Maybe if we cleary identified the "directory location" instead of ...... etc we could save all this thrashing around.The only errors I have left are references to agn files, and sound files for the boats. Just about everything else is cleaned up (for my test areas). One more thing - I renamed, moved and used the GLOBALENV_AC_CHROME_128 as GLOBALENV_AC_CHROME. Smaller file by 1200K.Sorry guys, its going to take me a bit to write all this out, and I only did the objects that were giving errors. I didn't do the ones that were not called (euro bus etc). The only thing that scares me - is that I'm gonna be screwed for a patch .. With all these files moved all over the place. Maybe I can make a batch file & restorer for this, I'll see. Needs more testing first.RegardsGarett

[barryward12345 0]

I wonder if this is the "Silver Bullet"??Barry

[Guest kimdahl]

Hi guysEarlier in this thread David hoped that someone from ACES would see this. Well, I have just made a comment in Phil Taylor's blog with a link to this thread. I also think this issue is far to important not to be seen by ACES.Best regardsKim DahlDenmark

[Guest koorby]

Quick! Someone seize this opportunity and code a $10 FSX tuner program and make a fortune! To say I am hardly surprised at the lack of query optimization in the code is an understatement. Thank god we have an FS community who are persistent in their reverse engineering and debugging of the MS code... great find guys, I look forward to your findings in more detail!

[Guest cwright]

Luis,I've also found interesting things with FileMon. These are related to the installation of the Englend & Wales VFR (photo-real, 1 meter) scenery. We had recently established that when FSX is loading the first flight, it spends several minutes solidly accessing *all* the VFR bgl files - even if the flight is in Japan! During this time the progress bar is stuck on 5%I have now found that actually during the flight FSX also accesses the VFR files, in this case the flight was in Africa. Worse, during these VFR accesses the frame rates fell sharply, from above 16 FPS to 8 FPS, and sometimes to 4 or 6 FPS. I simply can't see why FSX should insist on accessing bgl files from half way around the world.It looks like there are major problems in FSX that are almost guaranteed to produce lower frame rates and stutters. It may be that the installation of many large VFR bgl files makes this situation worse. Adam Szofran was taking part in the thread and came up with a very useful cfg mod that stops FSX from preloading the default flight. But since we posted these findings Adam hasn't replied. I'm just going to add a post to that thread about my latest finding. I really hope Adam will respond, as I'm sure he could throw a lot of light on this puzzle.FileMon is emerging as a very powerful tool. But from Microsoft's perspective it has opened a whole new can of worms!Best regards, Chris

[LuisFelizTirado 33]

These inconsistencies are just fascinating, Chris.I found that FS X, for reasons of optimization, will always pre-load the default flight. If it is on the other side of the world matters not a bit.So, I would spend 2 or 3 minutes just waiting while this flight loaded and this, before even getting to the user interface.In addition, memory usage would go through the roof, and I would always saturate and exhaust RAM before even getting to the user interface!FileMon showed that FS X was accessing the default Friday Harbor flight, a dense and complex location.By changing my default flight to somewhere in the area where I fly (low scenery complexity overall, few objects), I now get much quicker loading times and am at the user interface almost right away. Plus I never exhaust my RAM any more.Since this scenery area is now pre-loaded, loading the actual flight is also much quicker, since much of it is already loaded.So, in order to optimize performance, we should always pay attention to these small things, like the default flight. If only the FS Team would have warned us of this, and indicated all the other changes they made, then it would be easier for us to improve performance.It seems that moving wayward textures (objects) to the Global texture folder will also improve access times, but this will probably not turn out to be the solution that will double frame rates. There is just too much resource-intensive stuff going on in FS X. Still, it is a very interesting fix and one that we should all probably apply.Best regards.Luis

[Guest baksteen33]

Would somebody like to host a couple of files? :-) Or add to the same? Or whatever... :-) I would have ca 37MB (packed) thus far. Pls note I haven't touched this project in weeks (& after handing it to an Aces rep). The SimObjects done: Boats (100%) and groundvehicles ca 50%. These two folders become roughly fivefold their original sizes. As far as I would dare to judge, it 'only' adds to smoothness. If you're interested, pls PM me. Maybe we can co-ordinate our efforts? Cheers and kind regards Jaap

[Guest Ozzie]

Put a referrence on the hacker sites - it'll be fixed in 3 days :)

[GaryGB 10]

Excellent research guys!Looking forward to learning more about this, and what to do about it!:) GaryGB