Sign in to follow this  
ryanbatcund

Addon Performance: What to Expect on your System

Recommended Posts

This is an update and addition of variables comparing stock C172 VC to other addons, with respect to FPS (represented by PI), VAS used in mb, and FPS variance percentage

 

This is purely in FSX, the setup involves me restarting FSX each time and comparing addons to the default C172 VC.  

 

Vsync off, unlimited fps, no clouds, no ai traffic.  Upon loading I go to the external view, spin around once and go to the VC, spin around once, all avionics are on.

 

I then measure default 172 first, I measure the current VAS, and fps (but the PI is derived from FPS of addon divided by FPS of default c172 - so you're not seeing fps, you're seeing how addons perform relative to c172) along with the fps variance.   I did this part based on the Petraeus Index so a user might see how an addon would compare on THEIR system to their default 172.  Obviously all I have is one source... if more people sent results I could include an average.  The FPS Variance is important because it's a mathematical way to describe stutters. 

 

I'm not excellent with Excel so while I'd like to display all the data on one chart I've done it with two for now.  But I've kept the order of addons so it's easier to look at.

 

It's also pretty clear that lower performing addons use more VAS (logic).  What surprised me is that lower performance doesn't necessarily mean more fps variance.   Higher the variance percentage the more chance of stutters.  But what I have found is that glass cockpit addons have the most cases of stutters.  Which I think we knew already as well....

 

PI_VAR.jpg

 

VAS.jpg

Share this post


Link to post
Share on other sites
Help AVSIM continue to serve you!
Please donate today!

Ryan, thanks for your excellent work.

 

It would be superb if there were some sort of standardized reporting method for individuals to submit their results.  As much as peoples systems vary, there's likely a lot of merit in not simply describing a specific FPS, but in describing it as relative to a standard (in this case the default 172).

 

I will admit though, I looked at the fancy chart before reading your explanation, and it took me a minute or two to figure out what the heck was going on! ^_^

Share this post


Link to post
Share on other sites

Brilliant work Ryan!

I'd also like to see a step by step guide so others could submit their results.

Share this post


Link to post
Share on other sites

1)      Setup FSX with unlimited FPS, vysnc off (make sure off at driver level too).  Shift+Z to determine avg fps (you’ll need the min max avg mod – goes in your FSX.cfg):

[TextInfo.X] <<<-----whichever window you want to put it in
FrameRate=1,1
AverageFrameRate=1,2

LockedFrameRate=1,3

 

2)    No weather, no ai traffic will help with a more accurate value.  Pick a default airport in the middle of nowhere…close to no addon scenery if possible.  Load default C172.  Make sure aircraft state is ready for takeoff etc (avionics/engines/systems on).   Go to external view – spin around aircraft to load up the GPU buffer… do the same inside the VC. 

 

3)    Note avg fps after 1 min, note virtual address space from any monitoring program, note the variance % (V: xx.x%).  This is your baseline.  Whatever FPS you get here = 100%. 

 

4)    Restart FSX and load the C172 again, do items 2 and 3 (a default flight with the 172 would be helpful from now on – engines running).  Now load another addon.  Take note of avg FPS.  Divide that by the avg fps you found with the 172 to get your PI.  Example:  you found 200 fps with 172.  You loaded XYZ addon, it got 100 fps.  Your PI is 100/200 = .5 or 50%.  Meaning your addon lost 50% of fps compared to the c172.  Also note the change of VAS from step 3 to loading a new addon.  You might have gone from 1400MB of VAS in 172 usage to 2000 MB VAS with XYZ addon.  Enter 600 MB for XYZ addon VAS usage.  Finally note the FPS variance.

 

5)    Keep repeating for all your addons and remember to restart FSX in between the 172 and addon pair.

 

6)    Record values preferably in excel or other spreadsheet software.

 

7)    Share if you want 

Share this post


Link to post
Share on other sites

Could I suggest that a specific baseline location might be useful for step # 2?

 

 

Yeah I didn't label the charts very well!

Bad boy!

 

Nope, you labeled them fine, it just presumes a familiarity with the Petraeus Index.  Once you realize that, then it's fine!

 

What would be perfect is a community site / effort where such results could be submitted, tallied and displayed.  However, who has time for that!!  Meanwhile, we simply can just thank you for your efforts!  ^_^ 

Share this post


Link to post
Share on other sites

I barely had time for this to be honest lol...  but I like to see pretty charts and graphs of stats and stuff... stats have always been interesting to me.

 

Picking a baseline location could be good but assuming you use it each time and you're testing against your default c172 it shouldn't really matter as long as its default-ish

Share this post


Link to post
Share on other sites

Thanks Ryan! It's amazing how the graphs match general consensus on the forums and personal experience. The ones known for having more stutters show themselves pretty clearly, as do the memory hogs. I'm starting to think that the frame rate variance is even more important than frame rate itself. Nothing ruins an immersive cockpit experience or smooth approach more than stutters.  

Share this post


Link to post
Share on other sites

The Milviz 350i looks troubling, I hope it gets better before release. I really want this to be the definitive King Air addon.

Share this post


Link to post
Share on other sites

Thanks Ryan! It's amazing how the graphs match general consensus on the forums and personal experience. The ones known for having more stutters show themselves pretty clearly, as do the memory hogs. I'm starting to think that the frame rate variance is even more important than frame rate itself. Nothing ruins an immersive cockpit experience or smooth approach more than stutters.  

 

Agreed.  Stutters (free) and VAS are more important than FPS.

The Milviz 350i looks troubling, I hope it gets better before release. I really want this to be the definitive King Air addon.

 

They usually include lower-res textures which should help... plus its still a WIP.  I probably should exclude it anyway, it was more or less for personal reasons that I included it.

Share this post


Link to post
Share on other sites

F

 

Agreed.  Stutters (free) and VAS are more important than FPS.


 

They usually include lower-res textures which should help... plus its still a WIP.  I probably should exclude it anyway, it was more or less for personal reasons that I included it.

Agreed FRAME rates are not really a problem as the brain can  adapt to 12 to 40 FPS

But stutters and VAs are the problems that all have workable solution depending on your specific sytem.

 

Be good if you could put all your computer parameters ie: graphic card and hardware , operating system and also what sim youare flying into a software program which thenchurns out the Graphics settings and or the FSX.cfg for each specific individual system.

 

Be nice - be a lot of happy folks what ever sim your using.....

Share this post


Link to post
Share on other sites

Didn't have much time so I just tested a few addons quickly. Tested the addons I have that Ryan didn't test/doesn't have so we can get data on more aircraft. All tests had "fair weather" because I forgot to change it to "clear all" (whoops).

 

FPS:

FPS.png

VAS:

VAS.png

Share this post


Link to post
Share on other sites

Q400 isn't as nice on the system as people say hehe

 

A large issue with measuring fps only is that a lot of people have theirs locked and if it gets 30 they say it's good.

 

This PI method is pretty good.

Share this post


Link to post
Share on other sites

This is really outstanding work. Maybe this will become something of a standardized method for the community to compare models.

 

Thank you very much.

Share this post


Link to post
Share on other sites

This is pretty cool. Thanks! Already referenced the charts to help make some purchase decisions, since I found that the results for the aircraft I do have line up with my experiences pretty well.

Share this post


Link to post
Share on other sites

Here's mine.

 

This is for Prepar3D v2.5 (I know this is the FSX forum, but I didn't want to start another topic unless this one gets bigger!)

 

I performed the steps slightly different.

1) I used ZKPY in North Korea (!) Runway 01.

2) Changed plane, setup 'takeoff' state. saved as default flight. restarted P3D

3) 360 degree spin outside, 360 degree spin in VC and recorded the values after 1 minute in VC view.

 

I used the Mooney Bravo as the default baseline as P3D doesn't have a default C172.

 

For the FPS, I got on average for the Mooney Bravo 86.3 FPS (Variance 4%)

the lowest was the PMDG 737-900 with 37.7 FPS (Variance 9%)

 

Ssd5bfA.png

 

for VAS.

Mooney Bravo remaining VAS = 2190MB

PMDG 777-200 = 1295MB

uSKMUI3.png

Share this post


Link to post
Share on other sites

Cool.  For your VAS remaining is that a percent of remaining VAS?

 

I guess that's a little different than mine I'm just trying to make sure I get it right.  On mine I'm literally adding the amount the VAS increases after I load a new aircraft.  With a restart in between.

 

Also I noticed the variance is still the same on the P3D F1 KA...  tis a shame... the glass just kills my smoothness on FSX version...  an otherwise fantastic addon.

 

Then again you mentioned this is P3D.

Share this post


Link to post
Share on other sites

Ryan, I'm using the VAS offset from FSUIPC, hopefully I'm reading it properly!

I explained just above the VAS graph that when the Mooney Bravo is loaded, FSUIPC is showing 2190MB remaining.

With the 777, I'm getting 1295MB remaining, which looks about right to me!

Share this post


Link to post
Share on other sites

Good thread! I find it pretty interesting how A2A 182 Skylane is on par with NGX in the last table, and on my system is completely different situation. In NGX between 30-40fps, in 182 between 45-60. Another clue that these tests are just valid on paper but in practice it's another story. Also I have better performance with Milviz B732 than in Ryan tests, it is somewhere around 50% in the first table.

Share this post


Link to post
Share on other sites

That's why it's important to not measure fps - and measure the % loss from c172

 

Also more user input would help a lot. IOW we have a tiny sample size.

Share this post


Link to post
Share on other sites

 

 


In NGX between 30-40fps, in 182 between 45-60.

 

Pe11e, just as a reminder I'm using P3D, so Optimize_Parts is on, which means I get about 10 FPS more in the 737 and the 777.

As the table states, I'm getting about 45 FPS with the A2A 182 and around the same with the NGX, turn off Optimize_Parts and I get roughly 35 FPS in the NGX which matches what you see.

Share this post


Link to post
Share on other sites

Pe11e, just as a reminder I'm using P3D, so Optimize_Parts is on, which means I get about 10 FPS more in the 737 and the 777.

As the table states, I'm getting about 45 FPS with the A2A 182 and around the same with the NGX, turn off Optimize_Parts and I get roughly 35 FPS in the NGX which matches what you see.

 

That's fine, but it can be confusing for new members since this is FSX/SE forum. :) Alslo there are certainly differences in performance in various addon planes in FSX vs P3D.

Share this post


Link to post
Share on other sites

Has anyone ever figured out what exactly causes the sim to stutter. Is it access to information, non related software robbing access to the CPU, Antivirus, cup runneth over or what?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this