Jump to content
Sign in to follow this  
Petraeus

A Benchmark Comparison

Recommended Posts

As much out of personal interest as anything, I decided to do a comparative benchmark of the FSX passenger planes in my hangar, to compare the FPS they would give under standardised conditions. I'm sharing the results because others may find them interesting.I've described my system configuration, and the FSX situation that I set up, at the end of this post. Obviously the absolute numbers for FPS that I achieved were very specific to this setup. However it is the relative performance of one aircraft versus another that is of most general interest, and should apply to many, if not all, FSX setups. I've summarised the result as a graph below.Perhaps not surprisingly, the four Microsoft stock aircraft perform the best. They don't have complex avionics such as an FMS, and one would expect MS to know how to build something that performs well in the environment it designed.However the two Flight 1 / Coolsky MD80's perform very well in comparison, especially as they both have flight management systems, albeit of different generations. The original MD80 is particularly impressive, given that it is ported across from FS9 rather than being a native FSX build.Then there comes "the pack" of reasonable performers from Level D, PMDG and feelThere / Wilco, although interestingly they are led by another port to FSX, the Level D 767."The pack" is followed by the feelThere / Wilco CRJ 700 and the iFly 747-400, both starting to struggle a bit, although bearing in mind that the iFly 747 is freeware.Finally we have the Captain Sim 757. This is actually the demo version with no avionics. It should be fairly obvious why I didn't go on to buy the full version, even with the recent deal at $9.99. For the sake of an open/close perspex bookcase cover in the cockpit and similar novelties, this thing just sucks the lifeblood out of my system like some software vampire. It needs 40+ FPS to feed its habit, leaving a mere 7.9 to drive scenery rendering, AI traffic, weather etc. Which is a great shame, because the 757 is my personal favourite real-life aircraft.I don't have figures for the PMDG 747 or Ariane 737, for the very simple reason that I just haven't bought them to date.It may be that there are components in my system that tend to favour one aircraft over another, so the precise ratios between the aircraft may differ on your system. However, I would be very surprised if the rankings or overall picture is vastly different.[System: Intel Core 2 Duo 8600; 4Gb RAM; GeForce GTX 280; nothing overclocked. Windows Vista 64.FSX Situation; FTX Australia scenery; Autogen dense; No AI (to remove a possible source of variability in readings); static weather (ditto); aircraft is lined up on YMML runway 27, all engines running and avionics active.Measurement: Situation and aircraft set up and left for 5 minutes, then average FPS reading taken.]


Petraeus

 

Share this post


Link to post
Share on other sites

Nice work Petraeus. This will be very useful for others. All a/c are not created equal.Pierre


Pierre

Share this post


Link to post
Share on other sites

Very cool!However, I find it odd that the 757 demo performs so poorly on your machine. With the demo I got default plane-like FPS. The full version gets FPS similar to the LDS-767. FSX is so odd...To make your graph completely accurate, I think you should comment on which planes are true FSX models. By default, they will give better FPS. Perhaps use the same colored bars or darker shades for FS9 portovers, and use lighter colors for the true FSX planes.This is a real neat idea though. It would be cool to see you do some graphs for other types of products too ;)


13900K | MSI RTX 4090 | 64 GB 3600 MHz | 4x SSD + 1x HDD | ASUS 42" 3840x2160 120Hz OLED
VirtualFly TQ6+ | Virpil WarBRD + Constellation Alpha | MFG Crosswind V2 | RealSimGear GNS530/430

spacer.png

Share this post


Link to post
Share on other sites

Thanks for the project.Very informative.How about some GA prop planes.I love Premierair,but they seem to take a performance hit.Especially if I go to locked spot view often.I tend to get the dreaded OOM.Thanks,Ron


Bring back Chief Illiniwek!University of Illinois.

Share this post


Link to post
Share on other sites
This is a real neat idea though. It would be cool to see you do some graphs for other types of products too ;)
I'll do that when I have a bit of spare time. Although my hangar is a bit of a confused mixture... :(

Petraeus

 

Share this post


Link to post
Share on other sites
Guest kwilson

I've also wished reviewers would do the same thing when reviewing an aircraft. Reviewers telling me they have such and such a computer and that they get so many fps at some airport is pretty much useless. However, if they were to provide some comparison fps with some default aircraft and other addon aircraft I feel this would allow readers to better judge how the product would work on their own computer.Plus I don't have the CS 757 but I can't believe even CS would release such a poor performing aircraft so that me be an anomaly peculiar to your system. The rest of the numbers seem to back up what I have felt when using those aircraft that I have (eg the LevelD 767, Wilco CRJ and Wilco Airbus)

Share this post


Link to post
Share on other sites
Guest simmer_x

Great comparison. Super80 performs excellent on my poor machine, and actually is only airliner addon that I can fly in FSX without problems. :(

However the two Flight 1 / Coolsky MD80's perform very well in comparison, especially as they both have flight management systems, albeit of different generations. The original MD80 is particularly impressive, given that it is ported across from FS9 rather than being a native FSX build.
Original Super80 FSX is not a port over. It was actually rebuild for FSX using FSX SDK and is 100% native FSX model. On the other side Wilco 737 is a cheap port over and it shows on frames.

Share this post


Link to post
Share on other sites

Great post Petraeus, congratulations! :( Best,


Pedro Lima
"The sky is the home of birds... we are just guests... guests of honor" Peter Besenyei
"...redundancy is very important in aviation; that is why airplanes have 2 wings instead of 1!" M.S.A.Q.
My videos

Share this post


Link to post
Share on other sites
Guest SkullxBones

I don't have a problem flying planes like the CS 757. The trick is to save different display settings in FSX and load them each time you fly to match the aircraft and environment you are going to fly in. With the CS 757, I load a file that has all the sliders turned down just enough to make it flyable around the biggest airports. You lose the eye candy, but I didn't pay for complex airliners to fly low and slow looking at scenery.

Share this post


Link to post
Share on other sites
Good job and very useful information.
Thanks a lot. Where the measurements taken inside the virtual cockpit or an exterior view? If you have the time and the inclination a chart comparing both for each aircraft would be very interesting. RegardsSimon

Share this post


Link to post
Share on other sites
Thanks a lot. Where the measurements taken inside the virtual cockpit or an exterior view? If you have the time and the inclination a chart comparing both for each aircraft would be very interesting.
In the VC, looking forward, zoom level 0.70.And yes, I will, if time permits.

Petraeus

 

Share this post


Link to post
Share on other sites
Guest frontendrob

Good post!Wow I always knew Captain Sim 757 was a pile of crap (after I bought it, unfortumately), but this comparison really shows how incapable the Captain Sim team is. It is an insult to flight simmers to even release a product in this state. Everybody should be warned that this product is unusable.

Share this post


Link to post
Share on other sites

This kind of information is very useful indeed. I suggested to one of the after market folks that it would be nice if they would give the consumer a default ac reference number for their product. It didn't go over very well. But I still think it would be nice if those who offer add-on products for MSFS could all agree on some standard of measure regarding performance. For example, using pre-defined mid level settings in FSX (no traffic and no weather) and using the VC view in the default Baron at a pre-defined airport parking spot, product XYZ is yielding 60% of what the Baron does. Then we could get a pretty good idea of whether our systems would run their product by applying the same settings and observing the outcome. Granted it would not be fool proof due to all the variables involved in equipment. And there would be no guarantees. But at least we could get some idea regarding predicted performance with their product.Best,Bill


"A good landing is one you can walk away from. An excellent landing is one you can taxi away from."

 

Bill in Colorado:

Retired

Comm: ASEL/AMEL/Instrument

CFI: ASEL/AMEL/Instrument

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Tom Allensworth,
    Founder of AVSIM Online


  • Flight Simulation's Premier Resource!

    AVSIM is a free service to the flight simulation community. AVSIM is staffed completely by volunteers and all funds donated to AVSIM go directly back to supporting the community. Your donation here helps to pay our bandwidth costs, emergency funding, and other general costs that crop up from time to time. Thank you for your support!

    Click here for more information and to see all donations year to date.
×
×
  • Create New...