Automating Prepar3D FPS Testing to Compare GTX 970 vs 1080 Performance

[slantalpha 0]

The title says it all - is there a way to automate Prepar3D FPS testing with different graphics settings?

X-Plane 11 does - you basically record a flight and have it replay, and the system will log FPS statistics for you so that it can be consistent every time.

With testing Prepar3D, I can try and take off the same way every time from the same airport, but there would be a subjective component to it. And there's no way to change the size of the FPS counter either, so it makes it hard to see.

I'm running an i7-4790k on a stock 4.4 GHz turbo with a GTX 970 on triple 1440p monitors. Just borrowed a friend's GTX 1080 to run a few performance tests.

Edited June 12, 2018 by jeff0000

[downscc 1,327]

You presume frame rate is the primary metric for performance?  Would that be peak, mean or other statistic?

[SteveW 1,792]

16 minutes ago, downscc said:

You presume frame rate is the primary metric for performance?

He's not saying that is he, although he might think that. I would assume from what he says he simply wanted to compare fps when he makes changes to GPU and other settings to see what they do. But you're correct, simple fps comparison is not an indication of performance

[fghdgdfdfgfgf 111]

I don't think you need a laboratory to pit the 970 vs 1080 running 3 1440p monitors ...the difference should be pretty damn obvious.

 

Now you could use the FRAPS benchmark utility with a saved flight.

edit: there's also a FRAPS framerate analyzer utility (don't have linlk) that you could use with fraps for a more detailed view of whats happening.

Edit 2: the other utility is called Frafs Bench Viewer. You need Fraps to use it.

Edited June 12, 2018 by FunknNasty

[DylanC 181]

He may be on to something.  In my mind an algorithm could be run to pit your system and settings against set simulation situations and automatically fine tune your system based on FPS, GPU/CPU usage and memory usage.   Easy to say, I'm sure hard to do across multiple simulation platforms, let alone one.

[SteveW 1,792]

I use IFPro to ensure the flight is repeated exactly the same each time with AI traffic weather and so on. Using real time weather is no good it's got to be the same each time. I save a flight on AP about to make a turn onto final, something like that. I can make a config change and run the flight again and again. I run two to four minutes of flight and make graphs related to performance, fps and other data (30+ P3D items) recorded with IFPro during the flight. I usually make sure to repeat the same test three times each.

[Shomron 251]

18 hours ago, jeff0000 said:

The title says it all - is there a way to automate Prepar3D FPS testing with different graphics settings?

X-Plane 11 does - you basically record a flight and have it replay, and the system will log FPS statistics for you so that it can be consistent every time.

With testing Prepar3D, I can try and take off the same way every time from the same airport, but there would be a subjective component to it. And there's no way to change the size of the FPS counter either, so it makes it hard to see.

I'm running an i7-4790k on a stock 4.4 GHz turbo with a GTX 970 on triple 1440p monitors. Just borrowed a friend's GTX 1080 to run a few performance tests.

Happy if you could report the results back here since I have the same CPU & GPU as you so and planning to upgrade the GPU.

I am using a single monitor though...

[SteveW 1,792]

Instead of looking at fps (frames per second) we can study the time it takes to load the sim and load a scenario. We can use a stopwatch. Might seem a bit over simplified but in fact the rapidity with which the sim can load is a major factor on how quickly new resources can be utilised by the sim and hence affects the performance directly across the board. It's interesting to note that we can add cores and see the load time reduce up to a point whereby obviously there's nothing more to gain. What we can watch is, that adding ever more cores uses up more shared cache in the CPU reducing the performance without gain to the loading time.

Edited June 13, 2018 by SteveW

[ayarov 7]

On 6/13/2018 at 9:03 AM, SteveW said:

I use IFPro to ensure the flight is repeated exactly the same each time with AI traffic weather and so on. Using real time weather is no good it's got to be the same each time. I save a flight on AP about to make a turn onto final, something like that. I can make a config change and run the flight again and again. I run two to four minutes of flight and make graphs related to performance, fps and other data (30+ P3D items) recorded with IFPro during the flight. I usually make sure to repeat the same test three times each.

Hi Steve,

 

does IFPro support recording and replaying flights. 

My understanding is that you do your flight once manually, for example a 5 min approach into a heavy scenery with some static weather. Then you use IFPro to repeat this whole exact scenario as you flew it without you touching any controls under different graphics or tweak settings. I know of no utility that can do that in p3d  yet.

[SteveW 1,792]

9 hours ago, ayarov said:

My understanding is that you do your flight once manually, for example a 5 min approach into a heavy scenery with some static weather. Then you use IFPro to repeat this whole exact scenario as you flew it without you touching any controls under different graphics or tweak settings. I know of no utility that can do that in p3d  yet.

No I wouldn't bother to do it that way:

Instead I use IFPro to ensure I get the exact same repeat of what are otherwise variables in the sim

What I do is set a flight onto AP with IFPro and save the flight just before a turn. Making sure the flight can start un-paused and then it is paused after a couple of minutes and a graph produced - then repeated exactly the same each time.

Another good test of the setup is simply to measure the sim with the plane parked at the gate for a few minutes and don't touch any controls. Adjust a single item and repeat to see the exact influence.

Without proper tools it's all just guessing, but with IFP used as a repeatable test harness is great, there's nothing else to match it.

Edited July 10, 2018 by SteveW