Archived

This topic is now archived and is closed to further replies.

Petraeus

Standardised Fps Measures For Aircraft

Recommended Posts

If you've been following this thread http://forums1.avsim.net/index.php?showtopic=239078, you'll see there's quite a bit of interest in the standardised measurement of the FPS achieved by different aircraft models. It would be very useful if airplane reviewers could also provide such measurements in their reviews. If they follow the procedure below, they can produce a figure which should be relatively independent of their own hardware and software configuration.1. Set up the stock Microsoft CRJ700 at the end of any runway where the overall FPS is high (so that the absolute number is large and the results will be more 'granular'. Not La Guardia or Kennedy where the figure may be only 2 or 3!) Go for somewhere well into double figures. Sit in the VC looking forward, switch the avionics on and set the engine running. Set Traffic to zero and Weather to constant, to remove possible sources of variability. Leave for five minutes, then measure the average FPS.2. Ditto, but with the aircraft under review.Its standardised performance will be Av. FPS (2) / Av. FPS (1). This value can then be compared against the table that we'll be building up in the FSX Forum.

Share this post


Link to post
Help AVSIM continue to serve you!
Please donate today!

If you've been following this thread http://forums1.avsim.net/index.php?showtopic=239078, you'll see there's quite a bit of interest in the standardised measurement of the FPS achieved by different aircraft models. It would be very useful if airplane reviewers could also provide such measurements in their reviews. If they follow the procedure below, they can produce a figure which should be relatively independent of their own hardware and software configuration.1. Set up the stock Microsoft CRJ700 at the end of any runway where the overall FPS is high (so that the absolute number is large and the results will be more 'granular'. Not La Guardia or Kennedy where the figure may be only 2 or 3!) Go for somewhere well into double figures. Sit in the VC looking forward, switch the avionics on and set the engine running. Set Traffic to zero and Weather to constant, to remove possible sources of variability. Leave for five minutes, then measure the average FPS.2. Ditto, but with the aircraft under review.Its standardised performance will be Av. FPS (2) / Av. FPS (1). This value can then be compared against the table that we'll be building up in the FSX Forum.
I might consider doing that in future reviews if someone would be so kind as to point me to a program or plugin that would create a graph from the FPS readout in fsx, or an external FPS readout... I don't see myself being able to take accurate measurements for that unless I can get something like that...

Share this post


Link to post

This is an excellent idea. Please consider, however, that frame rates in Flight Simulator are extremely variable because certain elements change from one flight to another, and even over time.For example, as weather changes, so will performance. Also, autogen displays differently from one flight to another, even if the flight has been saved. And there are other factors as well that cause fluctuations in frame rates, including hardware and the level of display settings that should be standardized if this sort of measurement is to be meaningful.So, it would be important to consider the overall flight situation and the means of eliminating those variable elements so as to allow a consistent method of measuring frame rate changes only due to the use of different aircraft.Best regards.Luis

Share this post


Link to post
If you've been following this thread http://forums1.avsim.net/index.php?showtopic=239078, you'll see there's quite a bit of interest in the standardised measurement of the FPS achieved by different aircraft models. It would be very useful if airplane reviewers could also provide such measurements in their reviews. If they follow the procedure below, they can produce a figure which should be relatively independent of their own hardware and software configuration.1. Set up the stock Microsoft CRJ700 at the end of any runway where the overall FPS is high (so that the absolute number is large and the results will be more 'granular'. Not La Guardia or Kennedy where the figure may be only 2 or 3!) Go for somewhere well into double figures. Sit in the VC looking forward, switch the avionics on and set the engine running. Set Traffic to zero and Weather to constant, to remove possible sources of variability. Leave for five minutes, then measure the average FPS.2. Ditto, but with the aircraft under review.Its standardised performance will be Av. FPS (2) / Av. FPS (1). This value can then be compared against the table that we'll be building up in the FSX Forum.
We look forward to this FPS table.

Share this post


Link to post
I might consider doing that in future reviews if someone would be so kind as to point me to a program or plugin that would create a graph from the FPS readout in fsx, or an external FPS readout... I don't see myself being able to take accurate measurements for that unless I can get something like that...
Replacing TextInfo2 in fsx.cfg with...[TextInfo.2]FrameRate=1,1LockedFrameRate=1,2AverageFrameRate=1,3GForce=1,4FuelPercentage=1,5...will display an average figure which settles down in 2-3 minutes. Alternatively FRAPS (Payware) gives a good figure.

Share this post


Link to post
This is an excellent idea. Please consider, however, that frame rates in Flight Simulator are extremely variable because certain elements change from one flight to another, and even over time.For example, as weather changes, so will performance. Also, autogen displays differently from one flight to another, even if the flight has been saved. And there are other factors as well that cause fluctuations in frame rates, including hardware and the level of display settings that should be standardized if this sort of measurement is to be meaningful.So, it would be important to consider the overall flight situation and the means of eliminating those variable elements so as to allow a consistent method of measuring frame rate changes only due to the use of different aircraft.Best regards.Luis
I think what Petreaus was trying to illustrate was a method to isolate all extraneous variables on an FPS readout. By normalising all figures to a well known default aircraft (in this case the Microsoft CRJ700) in a static situation (engines running in VC view at the end of the runway with clear skies and no AI) on the SAME machine, one can get a standardised FPS measurement (FPSnew/FPScrj700) that can hopefully be "transferred" to another given situation or machine. I don't think one can be too ambitious with such figures, but they do give a useful starting point, within statistical error. Of course, such standardised measures will also be variable across different platforms but the more samples collected the more reliable the results (hopefully!). Differences will arise due to the use of Intel/AMD, ATI/NVidia, DirectX9/10, etc but I don't believe the overall ranking of payware and freeware aircraft will deviate much from a general trend. If a payware aircraft performs poorly in a static sense using the approach Petraeus just mentioned, then there is probably not much hope that it will be stellar performer when actually flown in the sim, be it my or your PC. The gist here is to compare standardised measures and not raw FPS readings, i.e. standardised to YOUR version of the Microsoft CRJ700, for instance.

Share this post


Link to post

Hi guys,It could be that I'm missing something or I really don't understand it.What I miss is the influence of your PC hardware in relation to the producted FPS.Standardizing FPS for readers is great and I would like to add this also in my reviews but what I wrote already; the actual FPS counter depends to my personal opinion also on your hardware unless I'm wrong and somebody can explain me why this is not true. Since our hardware is in most of the cases different, tweaking is done or not done by the flight simmer and there are so many other things which could influence the FPS, or not :( ?Standardizing settings or pre conditions where to put the aircraft are I believe not enough!

Share this post


Link to post
Hi guys,It could be that I'm missing something or I really don't understand it.What I miss is the influence of your PC hardware in relation to the producted FPS.Standardizing FPS for readers is great and I would like to add this also in my reviews but what I wrote already; the actual FPS counter depends to my personal opinion also on your hardware unless I'm wrong and somebody can explain me why this is not true. Since our hardware is in most of the cases different, tweaking is done or not done by the flight simmer and there are so many other things which could influence the FPS, or not :( ?Standardizing settings or pre conditions where to put the aircraft are I believe not enough!
The measurement is not an absolute figure of frames per second, because obviously that depends on everyone's particular system. That's the problem we have at the moment - people say they are getting so-and-so FPS with a particular plane on their system, but that doesn't help me because my system is not the same; it's like comparing apples and oranges. So the Index is a relative measurement of the FPS with one aircraft compared to the FPS of a standard aircraft (in this case the stock CRJ700, because everyone has that). Because we run the two tests, for the CRJ700 and the plane under test, on the identical hardware, then it becomes more independent of the hardware; it removes the difference between powerful systems and less powerful systems.Suppose a Boeing 797 model runs 50% better than the stock CRJ700, on any system. Then on my old system it may give 24 FPS compared to the CRJ's 16. On your more powerful system the corresponding figures may be 45 and 30. But the Index will be (24/16)*100 or (45/30)*100, in other words 150, on both systems. We have taken out the variability due to system power, and are using a more 'level playing field'.So, I have a middle-range system, and I read a review for the 797 which says its Index is 150. How does that help me? Because I know that on my system at my favourite airport, the stock CRJ runs at 20 FPS. So now I know that the 797 should run at 30 in the same situation, which helps me with my buying decision.Now it's quite possible that even on 'equivalent' machines, with the same sort of power, a particular plane may run better on one than the other. Perhaps they run better on Intel and ATI or on AMD and nVidia, or whatever. So there may still be variation. But at least we've removed the raw horsepower of each system from the comparison. So we are now comparing apples and apples, although one may be Cox's Orange Pippin and one Golden Delicious. :(

Share this post


Link to post

There's now an Avsim Wiki entry for what's been dubbed the 'Petraeus Index', and that also has the link to the most recent set of figures.I would urge all reviewers to supply a figure for aircraft they are reviewing, so there's a relatively objective measurement of how well (or otherwise) they perform.

Share this post


Link to post

I'll gladly help out on this index, and I hope you don't mind if I supply graphs based on this in future reviews.

Share this post


Link to post
I'll gladly help out on this index, and I hope you don't mind if I supply graphs based on this in future reviews.
Thanks Peter, that would be great.

Share this post


Link to post
Thanks Peter, that would be great.
No worries, I'll gladly do what I can to help the community.

Share this post


Link to post

I simply put mine at 20 and leave it. Uses less RAM and processor that way. Good enough!

Share this post


Link to post