FSX-SE MS FSX vs DT FSX SE: The Conclusive Performance Benchmark

[Bofajking 0]

So the only question is, with the programming/code improvements to Terrain Cache Flushing, Vertex and Index Buffers, VS2013 compiler, Texture Composition, and other changes that aren't in the change log but that Pete Dawson says are "significant" and the numerous reports from users of better frames and smoother performance, is it just placebo or is it real? 

 

A little about me: I have been a flight simmer for the better part of 15 years. I have belonged to about 3 different VA's where I have accumulated about 3000 hours and have about 180 real world hours in various aircraft such as the 172, 150, and Cherokee. I am an avid computer builder, overclocker, and a watercooling freak. In real life, I am the VP of the Global Technology and Operations group of a large financial institution. I own products from FSDT, Flightbeam, ORBX, Pacsim, PMDG, FS Global, Cloud 9, Aerosim, blah blah blah. Basically I spend too much money on this. :-)

So enough about me, on to the benchmarks.

 

First, let me share with you the computer I used for these tests: 

• Intel I7 3960k (overclocked to 4.7ghz)
  
• 2xGTX 680 4gb models on 344.75 drivers (I need the Vmem)
  
• 2x Samsung 840 SSD's running in raid (1tb of storage running at 980mb/s)
  
• 16 gb of 2000 mhz corsair dominator gt DDR3
  
• Asus Rampage 4 extreme
  
• 2 gallons of water :-)
  

 

Both the FSX and FSX SE were done on clean build with all files, registries, and folders cleaned as well as all TEMP data. They are like being on a brand new hard drive and in the case of FSX:SE it was on a newly formatted drive because ORBX installers still have a little work to do for compatibility when switching sims.

 

The tests were conducted in So Cal with the following addons:

• Textures: ORBX Global
  
• Mesh: FS Global Mesh
  
• Vector Data: FTX Vector (frozen water and golf courses turned off due to graphic anomolies and ran the auto airport elevation tool)
  
• Airports: FSDT KLAX, KONT and KLGB by Shez Ansari
  
• Aircraft: Aerosim 787
  

Thats it, no REX textures, no weather addons, just these terrain addons. Weather used was Cold Fronts. Needless to say this is a OOM/VAS crash waiting to happen as these are all high poly mega airports and a ton of Vector and texture data across So. Cal. 

 

My flight plan took me from KLAX 24R, direct to KLGB, to the PDZ approach to KONT 26R. Cruise altitude was 11000 and speed was 250kn with a manually managed VNAV fixed climb  of 1800 fpm after gear up @ 500 ft asl.

 

The settings are follows: 

Graphics Tab -

• Target Frame Rate: Unlimited
  
• Resolution: 2560x1440x32
  
• Filtering: Anisotropic/Anti-alising on
  
• Global Texture Res: Very High
  
• DX9
  
• Lens Flare
  
• Advanced animations
  

Aircraft Tab -

• Global Settings Ultra High
  
• Scenery Tab -
  
• All sliders to the right except for mesh resolution @ 5m
  
• Land Detail Textures On and Ground Scenery Shadows Off
  

Weather Tab -

• Cloud Draw Distance: 90m
  
• Thermal Visualization: Natural
  
• No Weather changes
  
• Detailed clouds: maximum
  

Traffic Tab - 

• Airline traffic: 40
  
• Gen Av: 20
  
• Airport Vehicle Density: 20
  
• Road Vehicles: 20
  
• Ships, ferrys, and leisure boats: 40
  

 

**In FSX, I applied the Max Texture Size to 4096 and High Mem Fix. I did not tweak or adjust any other FSX.CFG settings**

 

One more note is that I took these strictly with screen shots in sim and desktop shots. I was worried that FS Recorder would create too much debate so these are raw, unadulterated, straight from sim results. They were tough to reliably get but after a few hours I got consistent data. 

 

Now before we get into the results, I can tell you that I already knew how FSX was going to perform, I have used this benchmark for years as my "stress test" to test out new planes, tweaks, programs, and anything else that I wanted to test a quantifiable performance increase. I can tell you that the only way to get FSX to complete this flight is to turn down Scenery Complexity to Very Dense, Autogen Density to Dense, turn off cars, and decrease Airport Vehicle Density to High, this will not be reliable on DX9 and will usually require you to run DX10 as it manages VAS better. I have done it countless times and these are generally the only settings that will allow FSX to complete this pattern. But can FSX: SE do it?

 

First lets look at departure. I took some idle shots from the ground. You can see that FPS is pretty much exactly the same give or take a few frames which fluctuated anyway. 

 

FSX 20.9 FPS:

 

 

 

FSX SE 21.3 FPS (note, that for some reason colors look to have more "punch" in FSX SE):

 

 

 

FSX 17.4 FPS:

 

 

 

FSX SE 14.7 FPS:

 

 

 

Now, lets get in the air, these shots are taken after making a 180 to the left to intercept the course to KLGB. 

 

FSX 13.3 FPS:

 

 

 

FSX SE 11.5 FPS:

 

 

 

Ok, so now we are just beginning our turn over KLGB and you can start to feel the strain on the sim. We have now fully loaded 2 high quality airports with a ridiculous amount of autogen and the VAS is building. 

 

FSX 22.4 FPS and 2.78 GB of Memory used:

 

 

 

FSX SE 18.1 FPS and 2.79 GB of Memory used (note the color difference once again):

 

 

 

Now we are 12 NM from PDZ and the KONT scenery is about to be loaded we also have a number of other airports populating such as KRIV, KCNO, KPOC. This is usually where the FSX "pings" start. I was looking downward and both sims performed very well. 

 

FSX 40.6 FPS and 2.78 GB of Memory:

 

 

 

FSX SE 40.7 and 2.64 GB of Memory (this is where things begin to get interesting because I have never seen Memory fall while flying this route even in DX10 and it appears that the Fix to flush all levels of detail in terrain cache is possibly starting to kick in. 

 

 

 

Sadly, this is where the story ends for FSX, about 4 miles up the pinging began and as soon as the PDZ procedure turn started the sim died. 

 

This is also where the FSX SE story just begins... 

 

I didnt hear a single ping, I made it through the turn and said "to hell with it, lets land". The PDZ turn to 27R is a wide one and the plane did another 180 through to make it to the runway. This was while loading even more autogen, descending, and making the approach. It was absolutely shocking that with the sim COMPLETELY maxed out it was able to make this approach successfully. My mind is blown. Here is one last shot for the end. As you can see, the Memory barely climbed and it still stayed well within limits and FPS did very well. 

 

FSX SE 28.4 FPS and 3.10 GB of Memory:

 

 

I will conclude by saying that I am yet to convert to P3D, or X-Plane because FSX has always served my needs well, but as more and more addons have come into play over the years, I have found myself running into OOM problems at mega airports and have had to turn down eye candy and invest a good sum of money into the DX10 fixer to manage the VAS of modern day simming.

 

We know that without a significant re-write of the FSX code, it is very hard to increase FPS significantly, but with the stability and VAS improvements in FSX SE, simply put, FSX SE manages my flying much better. 

 

Any questions, please let me know. I put hours into the installs, setup, and validation of this test so I hope you enjoyed. 

 

One last shot, here is my computer "Penelope" who made this all possible

 

Edit: I have changed the verbiage to "Memory" for clarity. Private Working Set Memory as indicated in the screen shots will yield the same general indications of a looming OOM as VAS monitoring will. Its data that cant be shared anyway and when it reached 3.7gb or so FSX dies. 

[smakar 0]

 

 

Any questions, please let me know. I put hours into the installs, setup, and validation of this test so I hope you enjoyed.

No questions - just a heart-felt Thank You for taking the time to do this and share the results, in a very easy to understand way. It is greatly appreciated!

[Parallel 0]

Grest work on the benchmark, thank you.

[Airline2Sim 489]

Great work. Disappointing to note however that the frames actually appear to be lower in SE, which is the opposite of most people's perception of this newer version. A couple of frames doesn't seem much but the difference between 30 and 27 is visible to the human eye. The better management of VAS is a big plus though.

[Bryn 72]

Are you sure that's VAS? I didn't think you could measure VAS in the task manager.

[JSkorna 137]

 

 

Are you sure that's VAS?

 

No, it's not.

[Bofajking 0]

Are you sure that's VAS? I didn't think you could measure VAS in the task manager.

 

 

No, it's not.

 

I have changed the verbiage to "Memory" for you both. At the end of the day, that Private Working Set Memory will yield the same general indications of a looming OOM as VAS monitoring will. Its data that cant be shared anyway and when it reaches 3.7gb or so we all know FSX dies.

[ryanbatc 12,820]

Errr.... working set is physical memory, ie RAM (simple answer).  You wanted to show virtual address space, you need Processor Explorer, or VMMAP, then select "virtual size"

 

RAM has ZERO to do with OOM's in FSX.  

 

Anyway, thanks for the fps comparisons.

[alepro21 154]

Looks to me that the zoom factor differs between the two sims which might explain the difference in frame rate.

[Bofajking 0]

Great work. Disappointing to note however that the frames actually appear to be lower in SE, which is the opposite of most people's perception of this newer version. A couple of frames doesn't seem much but the difference between 30 and 27 is visible to the human eye. The better management of VAS is a big plus though.

 

Yup! The FPS went back and forth between the two but they were always generally +/- 4 FPS of each other. 

 

 

Looks to me that the zoom factor differs between the two sims which might explain the difference in frame rate.

 

That may have a little to do with it but mostly the frame rates were exactly the same until FSX started hitting its memory cap. 

Errr.... working set is physical memory, ie RAM (simple answer).  You wanted to show virtual address space, you need Processor Explorer, or VMMAP, then select "virtual size"

 

RAM has ZERO to do with OOM's in FSX.  

 

Anyway, thanks for the fps comparisons.

 

In my experience, memory leaks or intense scenery will allow FSX to peak out @ around 3.7 gb of memory before it crashes and gives me the OOM error. Thats what was tested and thats what was proven. 

[KingGhidorah 209]

Have you confirmed that MS FSX died due to an OOM?

[Bofajking 0]

Have you confirmed that MS FSX died due to an OOM?

Yes. The pinging and the message that pops up saying your computer has run out of available memory is a dead give away. Lol.

[DetCord 24]

Mate of mine did the exact same thing using MSI Afterburner. His results were pretty much the same as yours with regards to the frames, that being that the differences were minimal or that the Steam version only minutely underperformed the non-Steam version. Certainly seems like a placebo effect to me.

 

I'll be picking it up once they work out the bugs and kinks.

 

As for the colors popping more, I don't see a difference. Except maybe that you took them at differing distances which would account for altered ambient lighting and other facets.

 

Side by side - http://screenshotcomparison.com/comparison/105500

[KingGhidorah 209]

duh, I missed that, Bofajking!

[Anxu00 176]

Thanks for the comparative works.  I have never ventured to set my settings that high, plus with a beastly machine spec like yours and had frame rate like 11 and 12 FPS, what's it like when you lowered your settings? And does people actually "fly" with 11-12 FPS?