Flight model comparison - will it ever happen?

[Noel 4,501]

25 minutes ago, BusheFlyer said:

 New pilots are very much aware of their limitations and lack of experience and tend to be pretty switched on. The danger zone is something like 600 - 1000 hour pilots, who are experienced enough to feel confident which naturally can lead to complacency

And so it is w/ motorcycle operation.  Those first 45 days are peak, the next period is better, and right about the time the rider has gotten comfy w/ the basics of operation crash risk goes up so the data shows.  (Proficient Motorcycling, David Hough)

[bobcat999 8,482]

Well I was flying the G36 last night, and I found it satisfyingly difficult.  Getting the approach / glideslope speed correct wasn't easy, and dealing with all of the systems and displays kept me busy - far from arcade!  Laughing at the youtube click-bait warriors who say this.  And for everyone that does, you can find an actual pilot that says the flight model isn't too bad actually.

I have P3D, XPlane, FSX, and now MSFS.  I am not a GA pilot and so I don't actually know what the C172 or G36 feel like in real life, but I tell you what I do know - in all of these sims, the flight model feels almost the same.  For all the talk of propwash effects etc.  The Xplane C172 feels very, very similar to the MSFS one.

By the way, when I got ice in cloud when flying the G36 last night, the flying characteristics changed, I struggled to see out of any window, and I struggled to hold altitude. Never felt that before in any sim.  It genuinely concerned me as I don't like to crash, even in an 'arcade game' - pretty decent arcade game though! 🙂

Edited October 5, 2020 by bobcat999

[Tierborn 188]

3 hours ago, Mark VII said:

Within you tube there are already several real pilots who have expressed their own opinion on FS.... of course they appreciate the graphic but they also state it’s far from a simulation, that’s closer to an arcade 

  That is the same said by all real world pilots on any of the current mass market sims.

[eslader 1,783]

14 minutes ago, bobcat999 said:

far from arcade! 

Yeah, the "it's only a game" and "it's an arcade flying game" comments are really amusing me. No, an arcade flying game is something like Afterburner, or the planes in Grand Theft Auto. 

Those people also seem to be getting trapped into automatically accepting whatever the maker of something calls it as reality. P3d is a true sim because Lockheed says it is! Yeah, and Cuervo is a good tequila because Jose says it is. Sure. 😉

Never mind that under the hood P3d uses a huge chunk of code directly ripped from FSX which was "only a game." Never mind that Lockheed is only emphasizing that P3d is a professional flight training system because they're contractually obligated to do so lest they get sued by Microsoft. Never mind that anyone who seriously thinks a "professional flight training system" that includes default aircraft such as the F-22 which no real pilot will train on in P3d, as well as weird stuff like a "flyable" nuclear missile submarine is actually a professional flight training system is drinking way too much of the Kool-Aid. 

Oh and while we're at it, those P3d guys who love to come in here and spam the forum about how much more realistic it is than that stupid "game" MSFS - y'all might want to hush, because if Microsoft decides that P3d is siphoning too much business away from MSFS they'll yank Lockheed's license as soon as they legally can (or sooner, because Microsoft is not always terribly worried about strict adherence to laws), and then you won't have P3d anymore. 

 

 

[lambourne 211]

I'd like to see a comparison. The goal of a good flight simulator should be an engine that you can plug real world numbers into and which outputs real world performance, or something close enough to it. The FS9/FSX/P3D engine is well documented and understood whereas MSFS seems to be a closed book so far judging by the amount of conversation about it.

I'm disappointed so far. Asobo's pre launch publicity talking about the fidelity of the new sim engine gave me high hopes and I think we were all hoping for at least one default aircraft that showcased what the new engine was capable of. Instead it's all a bit "well, maybe, not sure, perhaps".

[BusheFlyer 87]

38 minutes ago, lambourne said:

I'd like to see a comparison. The goal of a good flight simulator should be an engine that you can plug real world numbers into and which outputs real world performance, or something close enough to it. The FS9/FSX/P3D engine is well documented and understood whereas MSFS seems to be a closed book so far judging by the amount of conversation about it.

I'm disappointed so far. Asobo's pre launch publicity talking about the fidelity of the new sim engine gave me high hopes and I think we were all hoping for at least one default aircraft that showcased what the new engine was capable of. Instead it's all a bit "well, maybe, not sure, perhaps".

This right here is the problem.. you simply can not compare real world flying to simulated flying. It's not a case of numbers.. In the real world flying is those numbers are variable. Let's suppose my Cessna 182 POH states my performance, what it is not stating is my 182 is a 20 year old airframe with an engine that is almost at TBO. It is also not accounting for that the airframe is a bit dirty after a summers flying adventures. I might hit those performance numbers or I might not..  probably not. Different 182's may well exceed those performance numbers on a regular basis. Mine might stall at 1-2kt's slower than that other 182. 

Real world flying is not exact, it's all an approximation with some sensible margins. From my real world flying experience and my simulator experience, then the stated goal of "something close enough to it" has been achieved.

What I will say from buzzing around in the DV20 in MSFS.. that plane is extremely well modelled, I have over 100 hours in the real one and it's so close to be amazingly real in the sim, if it had wing flex then it would be perfect. I am comparing just the aircraft here. The actual sensation of being in a real DV20 to the sim one is of course incomparable. 

Edited October 5, 2020 by BusheFlyer

[marsman2020 1,310]

As a disclaimer: I am not a pilot in real life.  I did double major in Mechanical & Aeronautical Engineering, so I'm relatively familiar with the math behind how aircraft fly.  

Comparing flight models across sims is difficult, because each sim engine offers a different way of defining the aerodynamic surfaces and engine performance, so what do you do compare?  Stock aircraft only?  Carefully tuned 3rd party aircraft?  A theoretical aircraft that can be represented consistently across the different engines? 

My biggest concern this the sim right now is that the dynamic stability of the aircraft seems a bit off.   Aircraft tend to display long period phugoid oscillations that take a really long time to damp out to a consistent pitch angle and speed at a given trim setting.  I don't think I've seen this behavior in quite this way in other sims (although I have spent way more time in MSFS than any other sim).  Not having flown GA aircraft in real life I can't definitively say this is "wrong", but it certainly feels..fussier then it should be.

I think the POH for a given aircraft should be the source of truth for the sim, since that is the reference information that sim users will find on Google in the absence of any official documentation being provided by Asobo.  If they want to add an 'engine wear' slider and an 'extra drag' slider to the sim so people can represent their tired out engines that are ready for a major overhaul and bug-covered aircraft, that should come after the flight models are tuned to match the POH.  

I can say this - I spent over 12 hours preparing for my around the world flight doing testing many of the GA piston aircraft (anything with a range > 500nm) on a "clear skies" day (ISA conditions) at 8000 ft and 12000 ft and 100%/75%/55% power and measuring cruise speed and fuel consumption.  Most of the aircraft in the sim cruise slower than they should.  A couple show much longer range than reality, a few have much worse range than reality.  It's clear these models are NOT tuned to the POH in many cases right now.  Hence the proliferation of "improvement" mods by fans of many aircraft to dial in the performance.  

I am nervous about Asobo's comments on "everyone is right" that they are intentionally tuning the stock aircraft to perform worse than the POH based on internet feedback.  This would be a mistake in my opinion.  They should be working to the manufacturer data and then applying any de-rating as an optional add-on.

Edited October 5, 2020 by marsman2020
Fixed typos, added last paragraph

[odourboy 1,289]

26 minutes ago, marsman2020 said:

My biggest concern this the sim right now is that the dynamic stability of the aircraft seems a bit off.   Aircraft tend to display long period phugoid oscillations that take a really long time to damp out to a consistent pitch angle and speed at a given trim setting.  I don't think I've seen this behavior in quite this way in other sims (although I have spent way more time in MSFS than any other sim)

I think this is the result of inappropriate moments of inertia values in aircraft configuration data rather than the engine. Of the few aircraft I examined in detail, all had MOIs that were more then twice what I thought they 'should' be. Adjusting these improved the 'wallowing' they demonstrated. There are numerous other flight parameters available for adjustment that are not currently bring used in the default aircraft.

[eslader 1,783]

31 minutes ago, marsman2020 said:

what do you do compare?  Stock aircraft only?

And even that gets weird. If you fly the Savage Cub (the one from the Yosemite bush trip) you'll notice that it yaws slightly back and forth fairly rapidly all by itself when at cruise speed. This started after the first patch when I seem to remember reading something about them modeling cockpit shake from air currents more. They might have gone a bit overboard, because the plane looks like it has palsy. I have verified that the rudder is not wiggling, so it isn't a joystick issue.

But the other planes don't do this, so I would think it's not a problem with the actual physics engine, but rather some instability they have improperly coded into that specific plane. 

 

[marsman2020 1,310]

The aircraft definitely do something weird with respect to the wind.  If you watch from the exterior view, it's as if the aircraft pivots about the center point of the plane with a relatively high frequency.  That's not how disturbances on an aircraft work.  It has static stability, the aero surfaces keep it pointing in one direction.  The entire plane might get shoved around by wind, but it doesn't cause a magical pivot about some center point. 

Honestly I try not to think about this stuff too much and enjoy the sim, because it just becomes really frustrating....but it is the one area where I could bust out the textbooks and do some math to prove its wrong if I wanted to....

[L3m0n 56]

14 minutes ago, odourboy said:

I think this is the result of inappropriate moments of inertia values in aircraft configuration data rather than the engine. Of the few aircraft I examined in detail, all had MOIs that were more then twice what I thought they 'should' be. Adjusting these improved the 'wallowing' they demonstrated. There are numerous other flight parameters available for adjustment that are not currently bring used in the default aircraft.

Yeah changing those values to what this gives; 

 Produces much more plausible handling. There was a developer interview where it was mentioned that the MoI had a big overhaul from what was done in FSX. I don't know if that is why everything is ~2x what the above calculations give. Maybe with their new model they think a doubling is what is needed.

 

3 minutes ago, marsman2020 said:

The aircraft definitely do something weird with respect to the wind.  If you watch from the exterior view, it's as if the aircraft pivots about the center point of the plane with a relatively high frequency.  That's not how disturbances on an aircraft work.  It has static stability, the aero surfaces keep it pointing in one direction.  The entire plane might get shoved around by wind, but it doesn't cause a magical pivot about some center point. 

Honestly I try not to think about this stuff too much and enjoy the sim, because it just becomes really frustrating....but it is the one area where I could bust out the textbooks and do some math to prove its wrong if I wanted to....

How the camera responds to movement needs some real overhauling as well.

[BusheFlyer 87]

49 minutes ago, marsman2020 said:

I think the POH for a given aircraft should be the source of truth for the sim, since that is the reference information that sim users will find on Google in the absence of any official documentation being provided by Asobo.  If they want to add an 'engine wear' slider and an 'extra drag' slider to the sim so people can represent their tired out engines that are ready for a major overhaul and bug-covered aircraft, that should come after the flight models are tuned to match the POH.  

I do understand what you are saying, the thing is POH's (Flight Manual's for the bigger boys) can be far from totally accurate. This doesn't just apply to the light aircraft types, I can assure you from years of experience with real Citations and Learjets and the 737 that the given data is often very coarse. Aircraft performance data is derived from actual real world flight tests in order to build a picture of approximately the performance you should expect. This data therefore was valid for that particular airframe with those engines and on that particular day. Margins and tolerances are used to ensure that the AFM data will be good for all aircraft of that type.

I have a 737 AFM sitting in my garage and I am sure you would be surprised how coarse that data is. 

[KenG 492]

Suffices to say lacking manufacturer's certified flight test data the POH/AFM data is the best to go on. 

Flight Simulation Training Devices for the U.S. is IAW FAR Part 60. In the certification of FTDs and FFS there are both objective data which must meet FAR 60.13 requirements and subjective tests which are covered in the Appendices. 

FAR Part 60.13 certified manufacturers test data often costs hundreds of thousands of dollars for a light airplane to more than a million dollars for a transport category airplane. 

As a simulator instructor who has worked with the certification side of Full Flight Simulators, I would find it highly suspect that Asobo, a video game company, would have the same fidelity test data on an aircraft as a simulator manufacturer such as Flight Safety International. The cost alone would be astronomical not to mention it could throw the manufacturer's data into question. Also, for newer airplanes some aircraft manufacturers have launch training partners. Most likely any data Asobo received was magnitudes less then was is purchased by training device manufactures.

The data in the POH/AFMs are in fact based on test data using average piloting skill and aircraft and engines in good working order. During the flight tests performance gathering runs are repeated and IAW various FAA guidance additional factors are usually added for things such as reaction time. So when you look are runway distance charts you can anticipate that following the conditions and applying the notes to the charts that an average qualified pilot with a good working airplane should generally meet the minimum distances. In fact in many cases a well trained crew will do better than the POH/AFM. That is because of the  additional requirements that the FAA places on that data. When looking at charts such and climb and cruise performance again the conditions and notes have to be applied. Generally a well maintained, clean airplane with excellent performing engines will do a little better than the charts predict. 

As a note on engines, one of my jobs when I flew maintenance was to spec turbine engines. The engine manufacturer guarantees a certain level of performance and we wanted to make certain the engine met that level. Running very specific charts in flight we would pull the trend monitoring and compare it to the engine manufacturer's charts. A spec 1.0 engine meant that we were getting 100% power and efficiency out of the engine. Engine were allowed to go as low as spec 0.85 which means they only produced 85% of the power or were 15% less efficient. We were told the performance charts in the AFM were based on this spec 0.85 engine. Most of the engines we tested actually speced out at 1.2 to 1.5 meaning they were 120% to 150% of a spec engine. Yep 20% to 50% free performance. Placed on a conditional maintenance program we could expect a spec 1.5 engine to last 10,000 to 15,000 hours between overhauls.

So yes there is a lot of slop in the POH/AFM charts when compared to real world performance taken off the airplane. However, it is often the same test data that goes into making the flight training device that get run through the manufacturer's and FAA's conversions before being spit out into the performance section of a POH/AFM. Understanding that there is no reason for a low fidelity simulation such as P3D, Xplane, or even MSFS to use that data as a minimum acceptable performance goal.

[Tierborn 188]

20 hours ago, marsman2020 said:

I am nervous about Asobo's comments on "everyone is right" that they are intentionally tuning the stock aircraft to perform worse than the POH based on internet feedback.  This would be a mistake in my opinion.  They should be working to the manufacturer data and then applying any de-rating as an optional add-on.

 

100% concur with your perspective ......

Edited October 6, 2020 by Tierborn

[bonchie 2,305]

On 10/5/2020 at 7:09 AM, Mark VII said:

Within you tube there are already several real pilots who have expressed their own opinion on FS.... of course they appreciate the graphic but they also state it’s far from a simulation, that’s closer to an arcade 

Nonsense. I'm a real pilot and this sim is not "closer to an arcade" that an simulation with it's flight model. Not even close. Having problems doesn't mean hyperbole is helpful. 

Edited October 6, 2020 by bonchie