CPUs compared clock for clock. FSXmarkCPU

[SAAB340 28]

I have created a benchmark that compares how different CPU architectures and cache sizes affects performance in FSX.

These are the results:

 

This benchmark is fully CPU limited as long as you have a fairly modern graphics card. That means that if you increase the CPU clockspeed by 25%, you’ll get a 25% higher result.

 

So what do these 4 initial results tell us? It shows that Haswell offers a very healthy 11% IPC improvement over IvyBridge in FSX. Haswell is also 17% faster than SandyBridge-E and 25% faster than Lynnfield at the same clockspeed.

You could also say that when it comes to FPS in FSX, Haswell @ 4.30GHz = IvyBridge @ 4.77GHz = SandyBridge-E @ 5.02GHz.

 

As you can see, there are many CPUs missing. I hope you can all help filling those gaps.

 

The requirements are:

CPU @ 4.00GHz

RAM @ 1600Mhz 9-9-9-28 1T

nVidia GPU, minimum GTX460

(I decided to restrict the benchmark to nVidia only just to be sure we always compare apples to apples. I have analyzed results from several systems with nVidia GPUs and the FPS always follows the same pattern. I’ve only analyzed one result with AMD GPU and the FPS pattern during the benchmark was slightly different to the others. Thanks dazz for providing results with a SandyBridge+AMD GPU. Unfortunatly I can’t treat them as valid)

 

A clean install is preferred. If you don’t have a clean install you have to make sure that you don’t use any add-on textures from for example REX or ActiveSky. You also have to make sure you don’t have any other add-on program running at the same time as this might impact the benchmarks.

This is a step by step guide of how to run it:

1. You need FSXmark07. You are not going to run that benchmark but you are going to use its flight path. Credit to Gary Dunne for creating FSXmark07. It can be downloaded from the Avsim Library if you don’t have it installed already. Just search for: fsxmark07.zip
   It is called "FSX Dynamic Benchmark Package" and is located in Category: Flight Simulator X – Utilities. You need to copy these 4 files in to your Flight Simulator X directory located in My Documents:
   FSXMark07.FLT
   FSXMark07.FSSAVE
   FSXMark07.PLN
   FSXMark07.WX
2. Set your computer to run at 4.00GHz CPU, 1600MHz 9-9-9-28 1T RAM. Make sure any Turbo mode is disabled.
3. Disable HyperThreading if you have it.
4. Rename your FSX cfg. This will be a backup of your current settings that you’ll restore later.
5. Start FSX and let it rebuild the cfg
6. Set Realism to Hard under Settings, Realism...
7. Untick all add-on scenery you have under Settings, Scenery library... (Yes, I know this is a tedious bit if you have plenty of add-ons)
8. Set the settings as per these settings (You should always set resolution to 1680x1050x32 even if your screen is capable of higher settings):
9. Close FSX and open the cfg. Add these lines to the top:
   [bufferPools]
   UsePools=0
   [JOBSCHEDULER]
   AffinityMask=14
10. Make sure you disable any additional AA in nVidia inspector. (One way of doing this is to perform a clean install of the driver)
11. Start FRAPS and set it up with Benchmarking Hotkey "P", tick to include these 3 benchmark settings; FPS, Frametimes and MinMaxAvg, tick and set benchmark to stop after 240 seconds.
12. Start FSX and load the FSXmark07 flight. Await the mouse pointer to disappear and start the benchmark by pressing "P". Wait until FRAPS has finished benchmarking after 4 minutes when the FPS counter reappears in the corner.
13. Exit FSX fully by pressing CTRL+C
14. Restart FSX and carry out another 5 benchmarks, awaiting the mouse pointer to disappear before starting, and be careful to not nock the mouse to make it appear during the benchmarks. Exit FSX fully with CTRL+C between each benchmark.
15. You should now have 18 csv result files in the FRAPS saved benchmarks directory. Keep these as I might want you to send them to me for further analysis. Open the minmaxavg.csv for each benchmark with Notepad to see the results. Post the results in this thread. Remember to tell what CPU and GPU you’ve been using.
    Example:
    CPU: i7 3770K
    GPU: GTX660 3GB
    Frames, Time (ms), Min, Max, Avg
      9693,    240000,  26,  58, 40.388
      9852,    240000,  26,  59, 41.050
      9716,    240000,  26,  59, 40.483
      9833,    240000,  26,  59, 40.971
      9828,    240000,  26,  60, 40.950
      9903,    240000,  27,  60, 41.263
     
16. Replace the cfg with your previously backed up cfg.
17. Re-enable your add-on sceneries in Settings, Scenery library...
18. Reset your previous additional AA in nVidia inspector
19. Reset your computer to your previous CPU and RAM speeds

Done
Thanks for contributing

 

Thanks to HLJames for providing the Haswell data and thanks to Tom for providing the IvyBridge data.

[flying_w 58]

Thank you for performing this analysis, it makes for interesting reading. Up to now I'd ruled out an upgrade from Sandybridge to Haswell, but it may be worth reconsidering. Two questions:

 

1) Are there any differences in CPU cache size for the processors tested so far?

2) Is everyone using the same AI package? I imagine both the workload of model rendering and schedules can affect CPU performance. Perhaps there are no models in sight in the test flight, but FSX still needs to drive the AI through their workflow.

 

Simon

[westman 257]

Thanks Lars

Have a I5-760 shall a do a test?

Its the same conclusion a made when testing Haswell the IPC % gain in Fsx

Is huge compared to IB and SB , retired My 3770k the Haswell outperfomed

The heavy overclocked subzero cooled 3770k cooled with a H100!!!!!!

[JimmiG 244]

You could also say that when it comes to FPS in FSX, Haswell @ 4.30GHz = IvyBridge @ 4.77GHz = SandyBridge-E @ 5.02GHz.

 

 

This means Haswell is a tiny improvement after all, but certainly not by much. Getting 4.3 GHz from Haswell is extremely effortless, where as getting 5 GHz out of SB or 4.8 GHz out of IB requires some effort. Also keep in mind that SB-E has more cache and more memory channels than the regular IB and Haswell.

 

How many data points do you have for each CPU and how much do they vary? I guess I could install FSX and give it a shot on my system if you need more data for the 4770K.

[SAAB340 28]

Thanks Lars

Have a I5-760 shall a do a test?

 

Feel free to do one. That way we can add Lynnfield i5 to the chart

[SAAB340 28]

How many data points do you have for each CPU and how much do they vary? I guess I could install FSX and give it a shot on my system if you need more data for the 4770K.

Each result uses the average out of 5 flights. You can see the numbers from the IvyBridge above. I only have results from one different computer for each CPU at the moment. They are all from a fresh install of FSX. The more data points the better.

[SAAB340 28]

Thank you for performing this analysis, it makes for interesting reading. Up to now I'd ruled out an upgrade from Sandybridge to Haswell, but it may be worth reconsidering. Two questions:

 

1) Are there any differences in CPU cache size for the processors tested so far?

2) Is everyone using the same AI package? I imagine both the workload of model rendering and schedules can affect CPU performance. Perhaps there are no models in sight in the test flight, but FSX still needs to drive the AI through their workflow.

 

Simon

1. The i7- Haswell, IvyBridge and Lynnnfield all have 8MB cache. The SandyBridge-E -K model (3930K) has 12MB cache.

 

2. Yes. You need to get FSX back to stock with no add-ons for this comparison. (That's why a fresh install is preferred.) The testflight is flying above Seattle. It starts north of downtown that it flies past. It passes Boeing Field and SETAC.

[HLJames 2,568]

One test is worth a thousand expert opinions!

 

HLJAMES

[larshall 4]

CPU i5 3570K

GPU GTX 670 OC 2GB

 

Frames, Time (ms), Min, Max, Avg

8973, 240000, 24, 53, 37.388

9021, 240000, 24, 55, 37.588

9012, 240000, 24, 55, 37.550

9071, 240000, 24, 56, 37.796

8989, 240000, 24, 54, 37.454

8973, 240000, 24, 53, 37.388

 

Losing my SSD (after less than one year!) at least gives me the opportunity to do a little testing on a clean machine.

So for what it's worth these are the results for my 3570K.

[SAAB340 28]

The result image have now been updated with IvyBridge i5 (3750K). Thanks to larshall for providing the data.

 

It is quite an interesting result I have to say. It says that the i5 3570K only delivers 92% of the performance of the i7 3770K at the same clockspeed. It's only the cache size that differs between those CPUs (as HyperThreading is disabled on the i7 in this benchmark). It looks like FSX makes use of the 8MB L3 cache on the i7 vs the 6MB on the i5. Both CPUs have been tested on a fresh install of Windows7 and FSX+Acceleration. I've had a look at the .csv files as well and everything looks normal there.

[Jettix 1]

Thanks for test. I'm about to build new rig and for me It was a difficult choice - 3570k or 3770k. Now it seems to be solved. Would be very interesting to see at Hasw i5.

 

Please make one point clear for me. Is it true i7 die a bit more physical size than i5 due to bigger L3 cache, what  provides a bit better heat spreading respectively? 

 

I would gladly took a part in testing but I cant see a way to make my RAM work at required settings with Q9550 and my sim is far from being called clean.

[SAAB340 28]

Please make one point clear for me. Is it true i7 die a bit more physical size than i5 due to bigger L3 cache, what  provides a bit better heat spreading respectively? 

 

The die on the i5 and i7 are exactly the same. The i5 just has HyperThreading and 2 of the full 8 MB L3 cache deactivated.

[GHarrall 494]

The result image have now been updated with IvyBridge i5 (3750K). Thanks to larshall for providing the data.

 

It is quite an interesting result I have to say. It says that the i5 3570K only delivers 92% of the performance of the i7 3770K at the same clockspeed. It's only the cache size that differs between those CPUs (as HyperThreading is disabled on the i7 in this benchmark). It looks like FSX makes use of the 8MB L3 cache on the i7 vs the 6MB on the i5. Both CPUs have been tested on a fresh install of Windows7 and FSX+Acceleration. I've had a look at the .csv files as well and everything looks normal there.

to be fair, NickN has been saying that higher amounts of L3 cache are important in fsx for some time, hence why he recommends i7 over i5

[SAAB340 28]

to be fair, NickN has been saying that higher amounts of L3 cache are important in fsx for some time, hence why he recommends i7 over i5

I find it important to be able to put a number to it so we can see how much it actually differs.

So i7 3770K @4.4GHz = i5 3570K @ 4.8GHz.

[cvearl 60]

Can someone please tell me the correct ZOOM for this test? I am doing it but since a newly created FSX.CFG defaults the "wideviewaspect=" line to false, what is the right zoom? It appears when loading the flight it is .70 Is this right? That's pretty "zoomed in" when using false. But is the default FSX zoom. Just curious if I am doing this right.

 

This is important because different zoom levels with and without that setting will result in drastically different performance numbers.

 

Thanks!

 

Charles.