Side by Side video of photo scenery texture loading using HDD Vs SSD

[dtmicro 1]

To be clear, Yes --- reboot the computer before a set of tests, but don't reboot between each test. When doing performance testing in FSX, you do NOT even want to restart FSX, much less reboot the computer between tests.

You need to be in the same "reboot instance" of FSX and Windows, so just use tiles that you know were never loaded before (hence renaming or a fresh compile)...

 

Also, although I compile the tiles myself (this ensures they are not cached, because they were never even read before), you can probably avoid recompiling just by renaming the BGL files (tiles) and restart FSX. This way you can be sure the "covert stuff" running in the background is in a similar state, and that the swap file is in a similar state, and all the background garbage that runs in Windows whether we want it to or not is in the same state. After each reboot, there are other things that can be problematic as well, because Windows can be in a different background state between reboots.

 

All this said, I believe the reason your tests may be showing faster SSD times is partly due to your "fresh environment" that you keep creating by rebooting and restarting FSX. If you let Windows sit there for 10 minutes or so after a reboot, and then play FSX for 5-10 minutes, then do your fresh tile testing, it will perform differently. This is how we actually play the game anyhow.
 

Yes, some FSX stuff will get cached, but NOT fresh tiles, there is no possible mechanism for FSX to cache a tile that has never even been read before,. Also, when we play FSX for a few minutes, some stuff is cached already when flying in a real environment (hence some object libraries), so my testing methodology is almost opposite of yours.

 

All this rebooting is actually giving the SSD an advantage that won't exist in regular FSX usage.

[SAAB340 28]

Lets just agree to disagree.

[SAAB340 28]

Not sure were actually talking about the same thing here.

Let me summarise.

 

Years ago I measures if an SSD improved texture loading vs HDD. I came up with the answer that it didn't as long as you had a decent HDD. I posted this here on Avsim.

 

As I aquired more and more photo scenery and used it more I felt like that wasn't the case as I flew around in FSX. I felt texture loading was indeed slower using the Velociraptor despite my previous testing.

 

I tried a new testing scenario more in line with the way I was using my photo scenery and ended up with an result that reverersed the results from my previous testing. It was showing that the SSD is indeed quicker at texture loading at this use case. I posted this again.

 

I record a video showing the difference hoping that a side by side video can be a bit easier to grasp than just numbers on a spread sheet.

 

I get told that the way I do this test that confirms my in game experience is wrong because I reboot between tests. My test is very repeatable giving different but consistent results from different drives. I'm getting told that the right way to test would be not to reboot between tests even though I can with my own ears hear that there's no data getting read of the HDD if I do that. I can through software confirm that the data isn't getting read of the disk if I do that. And my results also agree. They are very consistently the same regardless of fitted drive if I dont reboot.

 

Let's just agree to disagree.

[SAAB340 28]

It is true that different processes can start running in Windows from one boot to the next that can affect the results. That's why I make several measurements for each result in my spreadsheets. As the test is very repeatable that will show up as a slower measurement and when that happens a quick look in performance monitor often shows this is the case and the measurement is than discarded.

 

Its also true that having a lot of objects around can give you variable results. When I redesigned the test I did initially try it over mainland Scotland using all the Horizon photo scenery as well as treeacapes and scotflight active as that is how I use it in game. That test scenario did produce a lot more variability between repeted tests on the same disc still showing HDDs slower than SSDs. Just harder to pin down a reliable result in how much slower. By using process explorer, process monitor and disc mon I found out that there is one FSX thread that isn't assigned affinity by the affinitymask in the cfg that is the tread that reads most data of the disks. When we include a lot of objects to read the CPU load of this thread increases and as its prefered affinity changes every time FSX is restarted it introduced a lot more variability between measurements. In particular with Hyperthreading active and with certain affinitymask. That's why I moved the test to where it is now where I get concistant results with all afinitymasks.

 

Affinitymask 4089 on a hyperthreaded hexa core is one that introduces a lot of variability together with a lot of objects. 4081 is a lot better in those cases. The same is true for affinitymask 249 on a hyperthread quad core where 241 is better or default 15 on a non hyperthreaded quad core where 14 is better.

[SAAB340 28]

I'm thankful that youre trying to validate my results with a different test. Its always good when that is done.

 

This way that I started measuring texture loading was initially designed get an actual measurement to show that FSX can make use of multicore and Hyperthreading. It used to be like swearing in church if you said FSX could make use of Hyperthreading as it (in most cases) didn't improve FPS and if your FPS doesn't go up it can't be doing anything right... A few years later i feel this general misconception has finally changed and you don't get told FSX can't use Hyperthreading anymore.

 

It just so happened that the same testing method could be used to test storage as well.

 

I'm sure there will be other ways to conduct a test that can be repeated with consistent results. Possibly even one where rebooting isn't needed.

 

Worth to remember is that using the same testing method as in the video created for this thread in another location that puts less stress on the storage system I can get a result that says there's not much difference at all between SSD and the Velociraptor. That doesn't make the results in the Video wrong. It clearly shows a difference in texture loading time at that test cenario. The results in the video also reflects my in game experience.

[dtmicro 1]

I am not trying to give you a hard time here, however as an FSX developer for about a year now, but I've been a programmer for more than 20 years, I am absolutely certain that your test results have an issue...

 

If you look at the common sense numbers, it can be derived why very simply...

 

How long does it take to load 100 tiles in FSX fully resolved, vs the raw hard drive speed that is possible?

 

Well it can take a VERY long time to load 100 tiles in FULL detail if you jumped to each one, I mean we are talking many many minutes. Yet the HARD DRIVE load time is only seconds.

It is something like 100:1 or more on how much CPU bottleneck the compressed 1.2m NAIP tiles are loading vs. the drive, and absolutely most definitely not more than 20:1 with all variables accounted for (GPU, CPU, HD, Memory, etc...), so the max improvement with a tile load on an SSD drive is only 5%.

 

You cannot ignore simple ratios, all computers are based off ratios, and ratios do not lie. This is the basis of computer science is ratios, that is where you start when analyzing this stuff. We can derive ratios outside of other variables, because we know the raw tile loading time if you COPY the tile outside of FSX from the drive vs. the CPU time de-compiling it. Nothing else matters, there is no argument to supplement this fact, I am sorry.

 

The Proof is in the numbers

 

With the MSE state of Utah as a perfect example, there are about 5000 tiles in the state of Utah total (and this is about 40GB of data if I recall (thereabouts, just install Utah and then check the total size of all MSE installed files to verify). This 40GB of data is already compressed by the compiler.

 

I know this ALL to be true because I have downloaded the whole state in RAW Format myself and compiled it myself from the NAIP imagery site (www.thenationalmapviewer.gov). I know at LOD = Auto, 15 (1.2m) is the size they are using with probably 85% compression setting (it is NOT a coincidence that when I use these settings in the FSX compiler, that my tile sizes come out about the same size as MSE). I am talking about tiles, not files (two different things here).

 

Even 100 tiles already compressed is generally less than 1.5 GB (I know this because I can compile a tile myself individually and see the size), so this can be read in 20 seconds or less on an average 7k RPM SATA even when broken into smaller files (before CPU processing). The hard drive is of course reading the compressed data, then the CPU is decompressing it. The ratios are unfavorable for the hard drive to increase performance.

 

Yet the HD could load this entire set in seconds, the reason is because ALL the load is on the CPU, not the drive. And when I say all the load, I mean greater than 95%, so there is no way to increase the tile loads more than 1-5% with an SSD drive, it is absolutely impossible with our current CPU's.

 

The only thing that COULD be possible is putting the game fully on the SSD drive could be compensating something that is causing a faster load (I will test this later, but I am doubtful).

 

This isn't theoretical, any developer that has ever dealt with and understands FSX image and compression knows this to be true just by looking at the tile sizes and general load times.

 

This isn't anything new I'm coming up with, in other forums this issue has been beat to death and we already know because of tile sizes vs. load times that it's all on the CPU, and you can speed up the hard-drive thousands of times and see no real benefit to tile load times, that is until you start using uncompressed tiles (no-one does this as they just become too gigantic to manage, a state could end up being over 300GB of data or worse even for 1.2m).
 

FSX developers already know this, go to the Orbx forum and ask a developer if you doubt it, they'll tell you the same thing. There is a bunch of SSD propaganda, and I love SSD drives myself, but it just doesn't help FSX as much as people want to believe it does. It does help some things, and initial load times, but other than that it won't help tile load times (not unless something out of the ordinary going on with a config).

[dtmicro 1]

I know down to the specific coding algorithms why FSX loads these things slow, it is because FSX relies on the CPU to decompress the tiles instead of caching Gaussian Blurred images and lower-res distant Mip Maps, and in modern compression schemes using MIP MAPs (like GTA V for instance), the pressure is on the video card because modern cards and API's (like DX 11+) have built-in optimized compression options and MIP MAP options to GREATLY improve this. That is what FSX coders did not take advantage of, that is why the tiles load so slow, and yes it is possible to make them load instantly by making better use of Gaussian distance blurring with MIP MAPS in the distance. FSX is loading a specific LOD or resolution level of several tiles simultaneously instead of using many many MIP map levels that are graduated like GTA 5 does.

 

This is why if you fly in the game Grand Theft Auto 5 on the PC the tiles load instantly with ZERO delay at 30fps on a modern system, but as you get farther from the mountains in GTA 5 the mountains actually get blurrier (not clearer like in FSX). FSX does it backwards, instead of MIP MAP'n and using very small amounts of compression on the photo-real tiles, FSX only has this capability on objects that have MIP MAP's pre-compiled, so FSX is basically just loading a piece of the image like a JPEG used to do when it was interlaced. So this is why FSX is so slow at loading tiles, but yet with a high LOD setting can remain sharp looking (sharper than GTA 5's mountains for instance).

 

That is also why ORBX stuff can OOM very easily while Photoreal does not OOM easily (even though some claim the contrary, test it yourself), but if you DISABLE all ORBX (set to default in FSX) and use Photoreal, your chances of OOM are much lower. I know this to be true theoretically, but I also know it in my own testing. People that get an OOM in a Photoreal session get it for a different reason (texture of plane, objects in distance, whatever). Photoreal is unloaded from memory more easily than some objects, whereas there is a base memory assignment that is required for auto-gen and similar things that are loading fast on the screen.

 

I noticed that on a vanilla install of FSX while running PHOTO-REAL that an OOM is EXTREMELY rare even at a LOD of 8.5 in VERY HIGH RES photo-real, but run that in Vancouver and Slew at 4x around, and BAM OOM quite frequently with ORBX.

 

Even on a vanilla FSX install flying at normal speeds, run Orbx with a LOD of 9.0 or more and you're going to get an OOM in some of their areas too often (Northern Cal for example). It's the object libraries and Auto-gen causing it, and the textures on top of that. However, Photoreal is different because in MSE's case, no auto-gen and few objects.

 

That's how we know, I don't know how you can debate this really, I would suggest going to the FS-Dev area and getting input, and any capable FSX developer will re-iterate what I said here.

[SAAB340 28]

 

 

You need to be in the same "reboot instance" of FSX and Windows, so just use tiles that you know were never loaded before (hence renaming or a fresh compile)...

Well, I managed to squeeze in a brief testing session today.

 

I copied the CA, OR, WA and LA+SD ultra res scenery so I had the bgls on my 840EVO SSD, my 850PRO SSD and my Velociraptor HDD at the same time. I made sure the HDD was well defragged. In total this scenery is 200GB and 3007bgls.

 

I re-booted the computer and used process explorer to monitor the disk activity.

 

I fired up FSX. Made sure the scenery library pointed to the 840EVO SSD.

Ran the test. Clocked 24.7s to load the scenery. Software told me loads of files were read of the disk. No strange non FSX related disc activity was observed.

 

I kept FSX running. Went in to scenery library and pointed to use the scenery off the Velociraptor HDD.

Ran the test. Clocked 30.0s to load the scenery. Software and my ears told me loads of files were read of the disk. No strange non FSX related disc activity was observed. The HDD was very notably slower even though i was still in the same windows and FSX reboot instance.

 

I kept FSX running. Made sure the scenery library pointed to the 850PRO SSD.

Ran the test. Clocked 24.3s to load the scenery. Software told me loads of files were read of the disk. No strange non FSX related disc activity was observed.

 

I kept FSX running. Made sure the scenery library once again pointed to the Velociraptor HDD.

Ran the test. Clocked 23.8s to load the scenery.  Software and my ears told me barely any files were actually read of the disk. No strange non FSX related disc activity was observed. That data was obviously read from somewhere. But not off the disk. My uneducated guess is cached in RAM.

 

Still kept FSX running. Made sure the scenery library pointed to the 840 EVO once again.

Ran the test. Clocked 24.3s to load the scenery. Software told me barely any files were actually read of the disk. No strange non FSX related disc activity was observed.

 

So to summarize. I've once again been able to recreate a SSD being notably faster to load photo scenery compared to a Velociraptor HDD as long as the scenery is read of the disk. This time using a method that didn't involve any re-booting of either the computer or restarting of FSX itself.  This method was however very time consuming compared to re-booting the computer. I only acquired one 'valid' result for the 850 PRO and one for the Velociraptor. The results in my spreadsheets are averages from at least 3 valid results. In no way does it however indicate that the results using the method where I reboot the computer are invalid.

 

 

 

Well it can take a VERY long time to load 100 tiles in FULL detail if you jumped to each one, I mean we are talking many many minutes. Yet the HARD DRIVE load time is only seconds.

It has mainly to do with access times. HDDs keep the CPU waiting for each packet of data it reads of the disk. Its mainly 4k reads and they are not sequential. The SSD has fast enough access time to fully put the bottleneck on the CPU again. The HDD is not there. There's no measurable difference between SSDs even though one SSD on paper is a lot faster than the other. 

 

 

 

 

The Proof is in the numbers

 

I know. That's why I prefer to produce them through tests and post what I find. Yet another set of numbers above showing a SSD being faster to load photoreal over a HDD. Just as I feel when flying in the game, but you cant produce a number to a feeling. That's why we need appropriate tests to produce them. 

 

 

 

I noticed that on a vanilla install of FSX while running PHOTO-REAL that an OOM is EXTREMELY rare even at a LOD of 8.5 in VERY HIGH RES photo-real, but run that in Vancouver and Slew at 4x around, and BAM OOM quite frequently with ORBX.
 
Even on a vanilla FSX install flying at normal speeds, run Orbx with a LOD of 9.0 or more and you're going to get an OOM in some of their areas too often (Northern Cal for example). It's the object libraries and Auto-gen causing it, and the textures on top of that. However, Photoreal is different because in MSE's case, no auto-gen and few objects.

I totally agree with you. Photoreal can be used with high LOD without OOMs. As soon as you venture outside and on to an area with a lot of various autogen the OOMs happens quite quickly. Even with default scenery. 

 

 

 

That's how we know, I don't know how you can debate this really, I would suggest going to the FS-Dev area and getting input, and any capable FSX developer will re-iterate what I said here.

I'm such a stubborn ###### that questions things and speak up when I don't agree. Born that way. Sorry  :wink: It's also very important to be humble and be able to change your mind when proven wrong but don't feel that has happened in this instance.

[Christopher Low 5,325]

As I understand it, you can get OOMs relatively easily with photoreal scenery if you have lots of these areas enabled in the Scenery Library, because FSX/P3D loads all of the photoreal scenery areas that are active (rather than just the ones within a certain radius of your aircraft). In fact, I can easily induce OOMs even when I have unused photoscenery areas disabled. That's because I use ES TreeScapes autogen at maximum complexity, scenery at maximum complexity, and UT2 AI @ 100%!!

[SAAB340 28]

As I understand it, you can get OOMs relatively easily with photoreal scenery if you have lots of these areas enabled in the Scenery Library, because FSX/P3D loads all of the photoreal scenery areas that are active (rather than just the ones within a certain radius of your aircraft). In fact, I can easily induce OOMs even when I have unused photoscenery areas disabled. That's because I use ES TreeScapes autogen at maximum complexity, scenery at maximum complexity, and UT2 AI @ 100%!!

As far as I can recall from logging disk reads last year, FSX reads the first 4k of every active scenery bgl shortly after initial startup or when you load your first flight if you have disablepreload=1 in the cfg. That's why the initial load time takes forever if you have a lot of active scenery and also the reason the SSD is so much faster compared to HDD when it comes to that. SSDs are great at short 'random' reads.

 

During a flight from SanDiego to LA with LOD9.0 in the cfg i saw photoreal bgls in western NC getting loaded from disk so it does load photoreal far from the plane as you fly along. Way, way further than the horizon. Obviously at lower LOD the further away from the plane you get. Never saw it load any of my active New England states or the active UK photoreal though. It did however touch mesh as far away as Turkey whilst flying on the US west coast. That's almost half way across the globe. Guess it would have touched the NewEngland photoreal bgls should I have selected top down view and zoomed out until half the globe was visible.

[dtmicro 1]

All I can say to simplify what I was trying to convey, is below...

 

-------------------------------------------------------------------------------------------------------------------------------------

For instance this one tile takes 45 seconds to load, do you know how big it is on the drive?

12.5MB

 

Now you keep posting test results that attempt to tell me that a 12.5 MB file that takes 45 seconds to load is sped up by a faster drive?
 

The drive loads the file in less than 1/2 second vs. 45 seconds for the CPU, that's almost a 100:1 ratio.

 

The tiles are compressed 3x in size or more, sometimes down to 5x smaller.

-------------------------------------------------------------------------------------------------------------------------------------

 

To give you an idea of what level of experience I am coming from, I wrote an add-on extension to the compiler by myself to allow the compiler to be multi-PC aware so I can compile tiles across multiple PC's at the same time. I even wrote an estimator that measures the actual CPU usage of compiler compression times.

 

Here is a screenshot of internal software I wrote myself that I use to compile tiles... Now I cannot continue this argument on the basis of you keep posting test result numbers, because I already know from having hard-coded some of this stuff myself. I suggest to take the discussion to a developer thread, as I have already stated everything I know to convey in order to properly inform you.

 

https://drive.google.com/file/d/0B5O8U7-RCNZaUEd3a1dNUkt0MWc/view?usp=sharing

 

Random access times and seek metrics are not an issue here, each tile size that loads in FSX from the NAIP imagery is a different size, but they range from 6000x4000 upwards to 17,000 x 15,000 or so. They are huge!

 

These are not tiny files or images all loading all broken up across the drive, a single BGL file generally contains many of these large tiles, and I can copy the tiles myself in Windows and see how long it takes the tiles to COPY from the hard drive (because I am compiling them myself). I can copy an entire visible area (say from AGL 2500) in 10 or so seconds (100+ tiles) even with a normal drive.

 

Now I am just starting to wonder if you are just testing something entirely different or if these numbers are somehow being recorded incorrectly.

As noted I have the raw files right in front of me and I can see the compression ratios of individual tiles myself and can watch the compiler times from compiling them myself. I can also see the CPU load in trying to read the compression, and the CPU load is ridiculously high both on the COMPILE TIMES and the In-Game Tile Load, there is no drive load. Sure if you slew across the scenery super-fast that causes random access and seek issues, but who flies at 2000+ mph LOW over high-res photo-real scenery, because nothing has loaded more than the first level of detail, so it's just a blurry mess. We don't play FSX like that, and yes in that cause SSD will make it load much faster.

 

The drive load in the game is not from the tiles loading, it is from other stuff. I am not talking about FSX loading things off the drive, I am talking ONLY about the tiles (Photo-real scenery). I am not sure how you are pulling the data, but I have multiple ways of seeing the compression and de-compression time of the CPU at a lower level than you are viewing it at.

 

FSX does do some weird file scanning thing to the BGL files even while in the game, that is when you put the scenery on a separate drive, though it's still faster. It also keeps open read-only locks on the files.

[SAAB340 28]

 

 

Now I am just starting to wonder if you are just testing something entirely different or if these numbers are somehow being recorded incorrectly.

I'm not picking a fight and I'm not doubting your competence in your profession. You really sound like you know what you are talking about. All I've done is record videos and put them side by side showing how the ground textures load slower when the photo scenery bgls are stored on my Velociraptor vs stored on my SSD. The only difference between the two sides of the video is what storage the photo scenery is stored on. That's the only difference. There's no tricks. Just a visible difference in load times. The times i quoted are the time in seconds between the aircraft stopping and the ground textures you can see are fully sharpened.

[dtmicro 1]

If that is true, then something is messing with FSX loading when you have side-by-side scenery loading off different drives right next to each other while slewing too quickly. I am guessing it has to do with the internal indexing code process in FSX while slewing across the tiles too fast. I know the FSX code here is very imperfect because it forces us to reload the scenery if you slew too fast. We don't slew fast when flying in FSX, we load the scenery and are over 1 single tile (at least when flying low enough) for many seconds.

 

You are asking me to believe that a file that is VERY highly compressed, and the tile inside the file which is generally 10-20MB will be greatly sped up by using an SSD drive while when I do the test myself, I am not seeing any speed increases (and if I did I might scratch my head, since these file sizes are small). Try it with one tile, don't slew but fly normally, and then watch the difference in the load times.

[dtmicro 1]

As I understand it, you can get OOMs relatively easily with photoreal scenery if you have lots of these areas enabled in the Scenery Library, because FSX/P3D loads all of the photoreal scenery areas that are active (rather than just the ones within a certain radius of your aircraft). In fact, I can easily induce OOMs even when I have unused photoscenery areas disabled. That's because I use ES TreeScapes autogen at maximum complexity, scenery at maximum complexity, and UT2 AI @ 100%!!

 

It doesn't load the tiles in high detail out of the LOD zone, it cannot and will not do that. It may load a very simplified low-res mesh and a shade of the color of the area so at very high altitudes you can still see some remnant look of the elevation data, but when using PR without mesh, it already does this in FSX vanilla anyhow.

 

There are however a lot of variables going on here with CTD's from OOM's, so it's hard to know all of them.

 

Custom airports and custom plane models are OOM magnets. I've actually never had an OOM in a PR scenery before, but I haven't added auto-gen yet.

 

 

[SAAB340 28]

 

 

If that is true, then something is messing with FSX loading when you have side-by-side scenery loading off different drives right next to each other while slewing too quickly.

The 'side-by-side' video is obviously separate videos that have been cut and combined next to each other.

 

 

 

I am guessing it has to do with the internal indexing code process in FSX while slewing across the tiles too fast. I know the FSX code here is messed up because it forces us to reload the scenery if you slew too fast.

Sounds interesting. Tell me more.