Long loading times with VFR scenery

[Guest cwright]

Adam, I have made another discovery. We have already established that when the first flight is being loaded, FSX insists on spending several minutes solidly accessing probably all the VFR bgl files - even if the flight is half way around the world from the VFR area.I have now found that FSX accesses the VFR files during the flight, again when far from the VFR area, in this case Africa. In one occurrence FSX solidly accessed probably all the VFR files over several minutes. During bursts of VFR access the indicated frame rate fell sharply, from 16 FPS plus, to typically 8 FPS and often 4 or 6 FPS. I had noticed this behaviour before, but now I know why.It's bad enough that time is wasted accessing VFR files when loading the flight, but it is a major problem if it causes low frame rates and stutters due to excessive and unnecessary disk accesses.Of course, there may be a reason why FSX must access scenery files from half way around the world but I certainly can't think what it could be. Would it be possible to throw some light on this mystery?Many thanks.Best regards, Chris

[Adam Szofran 0]

Scott,I expect that splitting files along QMID 15 (1 km) or even QMID 13 (4 km) boundaries won't make things better. You'll end up with billions of tiny little files that will only increase load time.-Adam

[Adam Szofran 0]

Chris,The FSX photo scenery file format contains tiles at a range of LODs. This was supposed to improve load times because the lower LOD tiles are prebuilt instead of having to be constructed at load time like is required for FS9 photo scenery. It's possible that the VFR scenery contains tiles at some of the very lowest LODs. If so, then the low LOD data from the VFR files would be loaded when assembling the textures that cover large parts of the globe. That shouldn't be a big deal because the amount of data read from each file should be fairly small. However, this could be a problem if there are a large number of spatially overlapping files with data at the very lowest LODs. How many files come with the VFR scenery? I haven't gotten my hands on the VFR scenery yet.You can see for yourself which LODs are included with the VFR scenery by opening one of the VFR files in the TMFViewer tool that comes with the FSX SDK. Select "Level of Detail" from the View menu and note the range of LODs that are present in the file. Menu items for LODs that are present will be enabled; missing LODs are greyed.-Adam

[Guest cbuchner1]

Less files per directory might improve the filesystem's performance. Don't know how well NTFS handles a gazillion of .BMP files in one texture folder ;)

[Cruachan 705]

Hi Chris,I've been following this interesting thread and would like to make one comment.Adam may have answered your question and I don't pretend to understand the content of his response...yet, but will keep trying :)However, I too have noticed this behaviour in FS9. Not so much with the VFR scenery, but during loads and while in-flight, Filemon seems to indicate that the sim is accessing or at least querying files that, on the face of it, have no relevance to the flight currently taking place.In particular it's frankly astonishing to see how many files are being queried during the initial load of the sim. It's as if every scenery and aircraft component (which is a lot when you have a >45GB install like mine) is being looked at.It soon became obvious that the larger the installation the slower FS9 became when, for example, accessing menus. This I accepted as the price to be paid under these circumstances. However, I do wonder whether all this repetitive background activity contributes to the overall throttling of performance of the sim and might also explain the periodic stutters. I am perhaps one of the fortunate ones in that the sim (FS9) is generally smooth in flight, but I do experience the odd stutter from time to time, usually during turns. I just put this down to refreshing the scenery loads from the HD. You know, when you do a 360 degree pan all is fine for a while and this, presumably, is because this action has forced the loading of all the local ground textures around you.If your reasoning holds any substance, it must be bordering on the miraculous that the sim (FS9) is smooth in flight at all. Somewhat similar behaviour exists in FSX so there may well be common factors here, the most likely being that the engine core remains basically unchanged at this point in time.This is all speculation by someone who has never programmed anything worthwhile beyond Microsoft QuickBasic. What I do know is that I once wrote an accounts suite that I was quite proud of at the time and, indeed, it became the financial backbone of our partnership (General Medical Practitioners) for many years before my retirement. This program was developed labouriously over several years with gradually increasing and useful functionality. At first the performance of the compiled program was okay, albeit very slow in places as it was running on a C64. Then it was ported over to an Amiga 500, then an Apricot Xi and finally to a 486DX. For me, what was gratifying was to see the performance enhancements as it was moved to run on the faster machines. In fact, it actually became a problem on the 486DX as I had to lengthen the time delay loops embedded in the program as, in places, it was running too darn fast, and this was despite any inefficient code!! I suspect the same scenario will, in the end, apply to FSX (as it happened to FS9 with all the add-ons) unless something can be done about it now to help it perform decently with some headroom on the majority of current hardware out there.Regards,MikeP.S. Does anyone know if it is possible to send output to an inkjet printer from a program written in QuickBasic?

[GHD 21]

During the initial loading of FS, it needs to access EVERY folder referred to in the scenery.cfg to see whether any files have been added or modified. If it finds any changes it must re-build the scenery indexes.George

[Cruachan 705]

Hi George"During the initial loading of FS, it needs to access EVERY folder referred to in the scenery.cfg to see whether any files have been added or modified. If it finds any changes it must re-build the scenery indexes."Yes, and that's perfectly logical, but this behaviour only occurs once after a scenery component is added/modified. What of all the querying activity going on during subsequent loads and also during flights? This is what I, and others, find difficulty understanding. That's where the logic seems to break down.Again, perhaps some further clarification from Adam would prove helpful.MikeEdit: I do note that FS always seems to 'know' when a change has been made. Try switching the order of a couple of lines in one of your add-on scenery areas in Scenery.cfg (FS9) and you will see what I mean. So, I have to assume that FS9 rescans the whole kit and caboodle routinely at each load to detect any changes made. That way the scenery.dat files are kept up to date. That's understood and necessary. This, however, doesn't explain all the apparently irrelevant and unnecessary background scanning which is taking place during a flight, nor why all the aircraft (default and addons) have to be rescanned each time the sim is loaded. I am assuming that this behaviour is similar in FSX.

[GHD 21]

>> but this behaviour only occurs once after a scenery component is added/modifiedBut how does FS know that scenery has been added/modified if it doesn't check every time it is loaded?George

[Guest cwright]

George, I wondered about this. It would be pretty inefficient if FSX had to chase every single scenery file on startup. It may well be that Windows does the job. Certainly it stores the modification/creation date for every file. Quite possibly it does the same for every directory and sub-directory as well. If so, it would be a simple and very quick task to detect any changes. FSX would simply have to check all the top-level scenery directories (e.g. addon scenery) for modification dates that changed.Of course, if this were so there's no guarantee that FSX actually does that. Maybe it doggedly checks every single scenery file when all it has to do is check a few directories....Best regards, Chris

[GHD 21]

I have 261 'top-level' directories.George

[nudata 10]

Another two "pence" - In the past three days I have done a lot of work with high resolution 0.25m/pixel photo scenery. Today I looked at the drive where I have Windows, FSX and the BGLs. It was totally fragmented. I had defragmented about a week prior. I had seen this before but paid no attention. After defrag the fragments are gone but the files are distributed all over the disk. Rather than speculate on the cause I just toss this out as info.Regards,Dick Boley

[GHD 21]

That's the problem with putting applications and data on the same drive as the OS.George

[basys 4]

Hi FolksAFAIK / At a guess -FSX should only read the scenery folder's checksums versus the scenery indexes held at -C:Documents and SettingsAll UsersApplication DataMicrosoftFSXSceneryIndexesReading all files in scenery folders and reindexing,should only occur after either the scenery config editor is run,or when a scenery has been added by manually editing the scenery.cfg.HTHATBPaulhttp://www.basys.f9.co.uk/resources/images..._sig_500x94.jpg

[Guest player1]

>Another two "pence" - In the past three days I have done a>lot of work with high resolution 0.25m/pixel photo scenery.>Today I looked at the drive where I have Windows, FSX and the>BGLs. It was totally fragmented. I had defragmented about a>week prior. I had seen this before but paid no attention.>After defrag the fragments are gone but the files are>distributed all over the disk. Rather than speculate on the>cause I just toss this out as info.>Regards,>Dick BoleyBesides installing on a seperate patition or HD, get a good defrager like O&O and have it defrage by Name and you will see a very nice improvement.My Photoscenery is now up to 60 gigs for just 1000 square miles including Water & Blend mask and my loading and access times are very surprisingly quick, much quicker than my old FS9 scenery of the same area when it was less than a few hundred MB.Now I'm starting to work on all four seasons, I have Spring and summer and will just proccess some custon scripts in PS to do the others....Terabite drives couldn't have come at a better time!

[Guest player1]

>Just to confirm that my computer has much the same results as>Chris.>>I had done a full defrag on my 160 GB hard disk before>installing the scenery.My hard drive's primary and secondary>are using the DMA setting.>>I had already previously tried changing the default flight to>a default terrain area, with no luck. The new long initial FSX>to GUI loading times were exactly the same (photoscenery or>default terrain areas). >>I tried the DisablePreload = 1 which certainly solves the>initial FSX GUI loading time (now down to 50 seconds).>>However as Chris says, when you click Fly, it now stays on 5%>for a very long time (2 minutes), and eventually after 4>minutes 50 seconds you can fly. So no real gain, just the load>time shifted to the Fly side.>>Fly now for a Default scenery area takes 3 minutes 30 seconds>(and it stays on 5% for well over 1 minute). It used to load>up quickly in a minute or so previously, and never sat at 5%>before. So somehow this Photoscenery is affecting FSX even on>a default terrain flight.>>Lastly, my FS9 Horizon San Francisco scenery worked perfectly>in FSX, and I had used it for the past month, with no long>initial loading times (just 2 minutes 30 seconds on the Fly>side)and no problems.>>>Regards>>DavidYes but what kind of defrag? Date created, date last accessed, Name, type?A good defrager that allows you to sort by name will give you the best option if you are accessing many bgls as in the case of bmp/photoscenery/fs9.In the case of FSX photoscenery the textures are part of the bgl file so there is a much smaller number of files to be accessed.The standard Windows defrag works but it really is not enough when you are talking about the amount of files in a decent FS install, you really need a good defragger, makes it seem as though you just got a nice shiny new very fast HD...:)