Long loading times with VFR scenery

[GHD 21]

After applying the cfg mod, the total time was reduced by 1min 50secs to 6min 28secs or about 22%.George

[Guest cwright]

Adam,I have no other VFR scenery installed, just the Horizon England and Wales. I've done some more tests. I wanted to see if the installation of VFR scenery causes longer load times for non-VFR scenery.The cfg mod is useful because the FSX startup is now consistent and doesn't change with VFR enabled or disabled. All the delays occur when loading the first flight.I loaded a non-VFR flight (in Japan).With VFR enabled the flight load times were 283, 293 and 289 seconds (I restarted Windows each time before the measurements).With VFR disabled the times were 225 and 228 seconds. The average times are therefore:VFR enabled: 288 seconds.VFR disabled: 226 seconds (about 21% less). Most likely times for loading actual VFR scenery would be a little larger due to the extra textures.It seems strange that the loading of non-VFR scenery should be influenced by the presence of VFR scenery. I suspect this is strongly linked with long loading times. Remember, I see relatively small increases in loading times, but others are reporting loading times of twenty minutes or more! Can you think of a mechanism to explain this?Best regards, Chris

[Adam Szofran 0]

Chris,Well, if there are corrupt scenery files on your disk, then all bets are off. That's the only other idea I've got, other than the three possibilities I listed in my previous post. Sounds like none of them apply to you, though.-Adam

[Guest cbuchner1]

The following applies to FS2002/2004 style photo scenery consisting of one BGL file and thousands of BMP tiles.During my Google Earth interface programming project I noticed the photoscenery tile loading engine of FSX is quite inefficient. It will always load the entire BMP file even when only a low resolution mipmap (e.g. from a far-away tile) is required. This puts a lot of I/O stress on the system. Also the simulator keeps accessing the same time multiple times for no apparent reason. ;-(

[Adam Szofran 0]

Nope. FSX reads only the desired mip level from each file; it does not read the whole file at once because that would be wasteful. If it later needs a different mip, it goes back and reads just that portion of the file that it needs. That's why you may see the same file accessed more than once, but FSX is reading different parts of the file each time. There's no doubt that FS9-style photo scenery is very poorly structured for performance due to all the billions of separate texture files. It takes even longer to load FS9-style photo scenery in FSX because FSX has a much larger visible scenery radius than FS9. That's why we adopted a new photo scenery file format for FSX that eliminates the need for separate texture files and needs very few file I/O operations to load. I would strongly discourage anyone from making new scenery for FSX that uses the old FS9 data format. It's simply not going to perform very well compared to the newer stuff.-Adam

[Manny 1,293]

>Nope. FSX reads only the desired mip level from each file;>it does not read the whole file at once because that would be>wasteful. If it later needs a different mip, it goes back and>reads just that portion of the file that it needs. That's why>you may see the same file accessed more than once, but FSX is>reading different parts of the file each time. There's no>doubt that FS9-style photo scenery is very poorly structured>for performance due to all the billions of separate texture>files. It takes even longer to load FS9-style photo scenery>in FSX because FSX has a much larger visible scenery radius>than FS9. That's why we adopted a new photo scenery file>format for FSX that eliminates the need for separate texture>files and needs very few file I/O operations to load. I would>strongly discourage anyone from making new scenery for FSX>that uses the old FS9 data format. It's simply not going to>perform very well compared to the newer stuff.>>-AdamThats a real good news. When I run Megascenery (FS9 version) in FSX, Because of a good machine and a great video card 8800 with 764MB video memory.... I can fly at 600kts with no blurries. I was concerned what would happen when they publish better resolution mega scenery for FSX. It's good to hear that it would be compensated by the new photo texture sceneries load strategy.Manny

[Guest cwright]

Adam,I've removed the corrupt files detected by TmfViewer, though some may possibly remain. Horizon are aware of this and say there will be a patch to fix it next week.I'm quite sure the loading times are nothing to do with corrupt files. My measurements strongly support other users' observations that the installation of VFR causes longer loading times for non-VFR scenery. Obviously, the presence of corrupt VFR files shouldn't effect non-VFR scenery. I have a suspicion that this effect is directly linked to the long loading times.Can you think of an explanation for this observation: that VFR can cause longer loading times for non-VFR scenery?This question is important because it eliminates the possibly of bad VFR files. It also eliminates the fact that loading VFR scenery would in general take longer due to the extra textures.Best regards, Chris

[Guest allcott]

>Adam,>I have no other VFR scenery installed, just the Horizon>England and Wales.>> I've done some more tests. I wanted to see if the>installation of VFR scenery causes longer load times for>non-VFR scenery.>>The cfg mod is useful because the FSX startup is now>consistent and doesn't change with VFR enabled or disabled.>All the delays occur when loading the first flight.>>I loaded a non-VFR flight (in Japan).>>With VFR enabled the flight load times were 283, 293 and 289>seconds (I restarted Windows each time before the>measurements).>>With VFR disabled the times were 225 and 228 seconds. The>average times are therefore:>>VFR enabled: 288 seconds.>>VFR disabled: 226 seconds (about 21% less).>> Most likely times for loading actual VFR scenery would be a>little larger due to the extra textures.>>It seems strange that the loading of non-VFR scenery should be>influenced by the presence of VFR scenery. I suspect this is>strongly linked with long loading times. Remember, I see>relatively small increases in loading times, but others are>reporting loading times of twenty minutes or more!>> Can you think of a mechanism to explain this?>>Best regards,> Chris>>Simple, really. The `mechanism` is the hard drive seek head. Looking at the file sizes of VFR scenery they `inhabit` the HD NOT as large numbers of small files, but as small number of large files. This is not good for seek times on a hard drive as this actively prevents proper defragmentation because the huge file intrudes on the capacity of the defragmentation to actually place files in the correct location. This I know from working with large audio and video files, where correct location of the files is vitally important to speed of access. A temporary solution might be to relocate the files to another hard drive, defrag the drive, return the VFR files, then use the manual placement facility of Ultimate Defrag to position the VFR files on the fastest part fo the platter. Then defrag again.Alternatively, one could actually run the VFR files from the remote location. Tto experiment it's only a matter of copying the whole scenery package (I appreciate you need the HD space to be able to do it!) then simply copy and amend the scenery .cfg to point to the new location and try a fresh start of FSX to measure the boot and load times. The repeat with the other scenery .cfg. and keep the one that gives you faster access. In effect, you want to distance the files from the app running them. That is the neatest solution in audio/visual work, where you may only be loading one or two of these large files at a time, but working on both once accessed for merge/edit purposes using the app.This does not explain why the files are being accessed when not being called, but are you sure that is the case? If the scenario above is what is happening, then the VFR files are simply pushing the `wanted` files to a point on the HD platter where the seek/read time is much longer - 20-25% would be typical as the difference between a fragmented drives' file access time and one that has been defragged.Longer term, I suspect a solution for the user might be NOT to co-locate the VFR scenery files on the same HD. Or for the files to be increased in number but reduced in size, by the developer.Also, check the file structure. I can't believe anyone is running a FAT32 drive these days, when NTFS is so superior, but this alone would account for the difference in file access times.Allcott

[Guest cbuchner1]

Samba shows pread() calls on the virtual file system covering the entire file, regardless of which mip level is really loaded.It might be worthwhile checking with FileMon if this also applies to read access to the local file system. It could be an issue with the Windows networking client on Windows pre-caching the entire file for speedy access.

[Guest cwright]

>Simple, really. The `mechanism` is the hard drive seek head.>Looking at the file sizes of VFR scenery they `inhabit` the HD>NOT as large numbers of small files, but as small number of>large files. This is not good for seek times on a hard drive>as this actively prevents proper defragmentation because the>huge file intrudes on the capacity of the defragmentation to>actually place files in the correct location. This I know from>working with large audio and video files, where correct>location of the files is vitally important to speed of access.I don't think the long load times are caused by the presence of those big files. The delays only happen if the VFR scenery is enabled. If I disable the VFR then things are back to normal.I have found something very significant in the last hour or so. See my reply to Adam!Best regards, Chris

[Guest david W.]

Just to confirm that my computer has much the same results as Chris.I had done a full defrag on my 160 GB hard disk before installing the scenery.My hard drive's primary and secondary are using the DMA setting.I had already previously tried changing the default flight to a default terrain area, with no luck. The new long initial FSX to GUI loading times were exactly the same (photoscenery or default terrain areas). I tried the DisablePreload = 1 which certainly solves the initial FSX GUI loading time (now down to 50 seconds).However as Chris says, when you click Fly, it now stays on 5% for a very long time (2 minutes), and eventually after 4 minutes 50 seconds you can fly. So no real gain, just the load time shifted to the Fly side.Fly now for a Default scenery area takes 3 minutes 30 seconds (and it stays on 5% for well over 1 minute). It used to load up quickly in a minute or so previously, and never sat at 5% before. So somehow this Photoscenery is affecting FSX even on a default terrain flight.Lastly, my FS9 Horizon San Francisco scenery worked perfectly in FSX, and I had used it for the past month, with no long initial loading times (just 2 minutes 30 seconds on the Fly side)and no problems.RegardsDavidPentium 4 3.0gHz 478 socket HT (3 yrs old)2 GB RamGeforce AGP 7600gs 256mb (New)160 GB HD Maxdor

[Guest cwright]

Adam, I've found something very significant. I downloaded FileMon and watched the activity when loading FSX and loading the flight. It shows some curious things when loading FSX, but that's another story.After loading FSX I loaded a flight in Japan while monitoring with FileMon. It confirms my measurements almost perfectly. For about two minutes FSX is solidly accessing the VFR bgl files. This is the period when the progress indicator is stuck on 5%.This is extremely surprising, particularly as Japan is quite a long way from England! As I understand it, whenever new scenery is installed, FSX updates the database so that, presumably, for any geographical location it knows where to start loading files.I then re-loaded the same Japan flight. This is very quick as it's already in memory. I didn't see any access to the VFR.I then loaded a different non-VFR flight. FileMon showed some access to the VFR, but quite small, maybe ten seconds.So, in my case, loading a non-VFR flight the first time increases the load time by two minutes due to accesses to VFR scenery. It's not so bad for me, but others are reporting load times between ten and twenty minutes! Imagine being forced to wait twenty minutes when you don't even intend to use the VFR scenery....Best regards, Chris

[GHD 21]

I created a flight in the antipodes and watched the disk accesses via Perfmon. The red line is the drive containing FSX and the blue line is the drive containing the UK photographic scenery (nothing whatsoever to do with Australia). XP Pro and swapfile are on another drive. Three separate drives are used.Note. When the progress bar stops at 5%, all of the disk activity is on the UK scenery drive. After 5% the activity switches to FSX and the default scenery. If the scenery indexes are correct, there should have been no accesses to the UK scenery drive.George

[Guest cwright]

George, those graphs are really interesting. I like the way the blue line drops immediately at the end of the 5% There can be little doubt as to what is happening. The question is, why? This behaviour is quite surprising. To be honest, I can't think of a reasonable explanation. After all, the whole point of the database must be to link scenery areas with the actual files that will be required. Of course, this is updated every time a scenery file is changed. Hopefully Adam will come up with an explanation, or maybe even a way to stop this.FileMon lists every file accessed. It spent about two minutes solidly spewing out VFR paths when the progress bar was stuck on 5%. It would take quite a while to check the whole log, but most likely FSX accessed every single VFR bgl file.By the way, it shows that FSX is accessing the mesh files as well as the photo scenery.Best regards, Chris

[scott967 62]

If Adam is still following this, I am curious what the trade-off is of using the SplitFileLOD inf directive for photo-scenery. For example QMID 15 or maybe 13.scott s..