Touchy touchy subject eh. From what I can see, having an opinion on FSXmark11 is the equivalent of kicking a mans donkey.

Hi Westman,

 

The skewed results comes from not comparing resolutions ... the FSXMark11 document suggests a very specific resolution.  For example:

 

GPU         Resolution          FPS

GTX680      1920x1080           30

980Ti       1920x1080           31

GTX680      2560x1440           23

980Ti       2560x1440           29

GTX680      3840x2160           19

980Ti       3840x2160           27

 

 

As you can see, by NOT running FSXMark11 at higher resolutions you are NOT taking into account the difference in GPU's performance capabilities as resolution increases ... so at 1920 x 1080 a 980Ti will provide similar FPS as a GTX680, but as the resolution starts to increase on up to 3840 x 2160 a 980Ti would show a considerable benefit over a GTX680.  The FSXMark11 documentation says:

 

 

Also make sure to use the 1680x1050x32 screen resolution regardless of your monitor. The 1680x1050x32 setting should be compatible with most monitors including 1980x1020. This will help to ensure even comparison between systems.

 

This is controlling performance potential and skewing results to be more CPU dependent by eliminating higher resolutions.  That's why most testing you see on the internet and testing benchmark products (like Fire Strike in 3DMark) include at least 3 "base" graphics settings (Low, Medium, and High).

 

Other issue with FSXMark11 is that it doesn't address GPU intensive graphics features found in P3D V2.x and V3.x ... those highly demanding GPU graphics features are:

 

1.  Tessellation (DX11 exclusive for hardware acceleration)

2.  Volumetric Fog (3 pass render process)

3.  Shadow processing (Clouds and more)

4.  3D wave animations (Water = Ultra) which uses CUDA libraries

5.  HDR

6.  Reflection options

7.  VC mipmapping

8.  FSX max Load radius is 4.5, P3D it is 6.5

 

Although FSXMark11 suggest it can be used for ESP/P3D it really can't without including those 8 items above ... perhaps it can be used for P3D v1.x but not for P3D 2.x or 3.x as you aren't testing key GPU stressing options.

 

Finally, FSXMark11 specifically excludes any add-ons ... I'll go out on a limb and suggest that 95% of the Flight Simmers (regardless of platform) don't operate their flight sim with NO 3rd party add-ons.  As you know, 3rd party add-ons can reduces one's performance results by 50% or more.  I understand that it would be close to impossible to come up with a "benchmark" process that could take into account the many 1000's of 3rd party products ... and this is why I'm suggesting NOT to even try ... go with a generic benchmark tool like 3DMark Fire Strike ... it'll provide enough of a hint of how hardware CPU and GPU will work relative to others across different resolutions.

 

Cheers, Rob.

What's clear to me is that a procedure like FSXmark11 could really be of benefit for P3D v3 for hardware guidance.

 

The most obvious name for this benchmark would be to name it P3Dmark15 if it was created now.

 

It could use the same flight path as FSXmark11. (Wouldn't have to)

 

It could use the same testing procesure as FSXmark11. (Wouldn't have to)

 

It would have its own unique, specified settings for the benchmark. (Including aircraft type)

 

It could have a set of different optional resolutions specified on top of a standard 1920x1080 that I believe almost everyone's rig is capable of today. (My suggestions would be 2560x1440, 5760x1080 and 3840x2160)

 

It wouldn't take very long to define this benchmark if the same flight path and test procedure as FSXmark11 was used.

 

Would this be the most optimal benchmark one could concieve for compareing how different hardware perform in P3D v3? No.

 

Would this be more relevant than 3Dmark scores for how different hardware performs in P3D v3? Yes!!!

 

 

 

It's probably not a bad idea to upgrade FSXmark11 to FSXmark15 either. With a few optional resolutions specified and maybe a few changes to the settings etc? (Any benefit from harmonising with the settings for a P3Dmark15 as far as possible?)

 

FSXmark11's resolution was specified to what it is because it was the highest resolution almost all monitors of the day were capable of in order to be able to facilitate as many people as possible to do the benchmark. That could obviously be changed today.

Hmmmm ... I honestly don't know how useful a P3DMark15 would be based on the FSXMark process.  

 

Issues I see:

 

1.  There are NO 2048 or 4096 textures in a base P3D V3 or a base FSX so the 2048 and 4096 settings would never be "tested" ... you'd have to load some Add-on that uses 2048 and/or 4096 textures ... 3DMark fire strike tests do indeed load 2048 and 4096 textures.

 

2.  Who uses FSX and P3D without add-ons?

 

Neither in my opinion would be a good "absolute" test ... perhaps some combination of both 3DMark and P3DMark15 will give a "relative" idea.

 

Cheers, Rob.

FSXMArk11 was intentionally conceived to be CPU bound. The idea was to have a test, very much like it's predecessor, FSXMark07, as simple as possible, without addons so that anyone could run it.

It's not meant to be a perfect metric for FSX performance, but since it's a well known fact that FSX is severely CPU bound, and addons just make matters worse, having a simple CPU test helps compare CPUs in FSX.

 

I don't have any experience with P3D, and of course, if it's better balanced with regards to CPU-GPU performance, this test may not be as meaningful as in FSX, but getting the GPU performance out of the picture will also give you a meaningful metric for CPU performance to compare CPUs in P3D.

 

I would say that's a lot better that any other synthetic benchmark, but that's just me. Those 3Dmark benchmark results can be found in the web anyway.

Is P3D not CPU bound around large airports with complex addons at 1080p with a mid range GTX960?

Because if it is, then adding massive resolutions and powerful GPUs will at best, get you the same performance than with that GTX960 @ 1080p  

Rob, 

 

Maybe you could put the bug in LM ear to make some sort of short flight demo to test FPS on different machines. With set graphics and flight plans.

 

Maybe you could put the bug in LM ear to make some sort of short flight demo to test FPS on different machines. With set graphics and flight plans.

 

Myself and others have been asking for performance measurement tools for some time now ... still hoping it happens in some future version.

 

Cheers, Rob.

Myself and others have been asking for performance measurement tools for some time now ... still hoping it happens in some future version.

 

Cheers, Rob.

Keep pusing them Rob.

 

3dMark Firestreke (std Extreme Ultra) have some drawbacks for example HT on Vs HT off.

And the wellknown win 8-8.1 8-10% boost over w7-W10