Moving Our Sim Forward

[G550flyer 1,010]

Greetings All,

After being in the extensive A320 mod thread, I thought I would share how I go about comparing products to realism. Maybe some of you could use these tips when addressing developers of products. I lean more on the flight dynamics, but I do jump in to the general stuff which at times require some reading and research. First, let's talk about the general which includes everything other than flight dynamics.

General: From my Air Force days, I learned that everything is about documentation and CYA. If you ever learned anything by god you better know where it was written. If if you can quote the verbiage, you better know where it's written. Once I became a flight examiner, you learn right away that Air Force aviators are self made lawyers and use documents to back up things they say or do. Keep this in mind in regard to developers. They have worked producing this product and will look at you with full disdain when you bring up issues with their baby. They may even come off as dismissive, so approach them with supporting documents or well versed information. When someone explains things in detail, they also may appear to be reliable. So always be clear about the issue and detail/support your claim.

When it comes to items such as FMS/FMC behavior or any other system characteristic, you might have your work cut out for you. Myself, I have a good working knowledge of nav systems, but I still give them detailed information specific to that aircraft. You may have to find documents or even buy manuals, but it keeps you from miss-speaking and it makes you accurate. Here are a couple of examples. 1. I was working with a developer on a DC10. The flaps and the slats didn't go to the proper detents or positions. In general, it has the dial a flap system, but the detents were up, slats/0,  5, 15, 22, 35 and 50. The slats he modeled had one position. The slats had a takeoff position and went to landing when the handle moved past the 25 area. Though I was an expert on the type being a pilot, I supported the documents. 2. I was working with a developer on the MD80. He modeled the auto slat system. Though the DC10 had auto slats, it worked different because the 80 has a self test. When they are brought out for takeoff, they continue past the commanded position turning on the the test and disagree light and then returns killing the lights. It's more detailed than that, but any who. The test was happening very quickly not illuminating the lights properly. I had to find and dive deep into a maintenance manual to grab the total seconds and detailed positions during the test. Once I provided those, he modeled it correctly. I bring this up to demonstrate that whether you know it or not, it's still best to provide the documentation or explanation when bringing forth issues. Unfortunately for developers, they may not have the resources or may not be able to interpret documents they have as the documents can be confusing. Next, lets look at flight dynamics.

Flight Dynamics: As you know, flight dynamics are at the heart of a simulation. Not only should it look right, it should act and feel right. Again, this is one of those areas a developer may not have access to. Early on in my career I became fascinated with aircraft performance. I found it amazing that I could run through some charts and the aircraft would behave as such. Then I went searching for answers to questions I had and ended up becoming an aircraft performance guru. It also made it easier to explain things to new pilots and help them under stand why they were floating or why they keep over shooting power inputs. I also started doing functional check flights and acceptance flights for our aircraft that were coming out of heavy maintenance. We did it all from testing all numbers, stalling the aircraft to test the stall warning, shut down and restart engines, over speed, flight control rigging tests and even battery only operations. You learn a lot about an aircraft and it's behavior during those flights. It's one thing to calculate data for an aircraft and it's another to understand what that data means. When you get good at it, you can look at a weight and airport conditions to judge how the jet will behave. Here are some things you can do to test sim aircraft.

My number one item is the stall test. Before I would test the system on the real jet, we would calculate the stall speeds for different configurations. First thing we did was calibrate the AOA system. We start in the approach configuration and slow to VREF. We then adjust the AOA system to read green at VREF. From there we clean up and perform the stall series clean and dirty. We note when the shaker comes on and when the pusher activates. A plane will typically stall at around 15 AOA and may be adjusted at high altitudes. Warnings/shakers may sound at around 80% of the stall and she should start departing at about 100%. Stall should be the bases of the model and you then start working flap and slat performance on lift from there. A clean stall test will tell you exactly where the base of the model sits. You may have to work hard to get the documents that will give you these numbers. Most aircraft Vref speeds are based on 1.3Vstall, but you will find 1.25 or even 1.4. Just divide the approach speed by 1.3 and there is the stall speed for that configuration. Here is how I conduct my tests.

Takeoff: If you are lucky, you can get your hands on the AOM that will have a host of information for you. Look into the takeoff procedures section. It will give you the target pitch and rotation rate all engines. So if the engines are tuned properly, the pitch you attain to keep V2+10 will give you an idea of the drag on the frame and flaps. If lower than target, too much drag or weak engines. If higher than target, too much thrust and too little drag. If you have a performance manual, you might be able to gather the takeoff ground run all engines which will give you an idea of how it's performing in regard to engine thrust. Ground run normally ends at 35ft FAA. This is also a good time to check initial climb performance. Also check the manual for unreliable airspeed charts. This will give good pitch and power data.

Climb: If you have climb charts, another good time to check engine performance. Those pitch and power charts are good here as well. Check for time and distance to climb also. I must admit that Ifly 747 watered my eyes on climb performance. They were spot on.

Cruise: Here is a good spot to check cruise pitch, fuel flow and engine power required to hold cruise speed. Too much power required, then drag or engine curves may be off. vice versa.

Descent: Pitch and power charts come in handy here also. A good reference below 10,000ft with jets, 1500 fpm is the most tou can do at 250 clean. I also use the 13 at 13 or 14 at 14 rule to test slow down. At 13,000 or 14,000, set 1300 or 1400 feet per minute and the plane will be pretty close to 250 by 10,000 clean. Worked well in all of the jets I've flown. 

Approach: As usual, those pitch and power charts are pretty handy when vectoring around with flaps/slats out. Even in GA aircraft, you might find these charts. In the AOM you will find the approach procedures and details of the landing and flare. Read these to capture the text book landing. They will also give you pilots eyepoint and no flare touch down points. Look at the approach diagram that shows the body angle at VREF based on glide path angle. Your approach body angle will be about 1 degree below the VREF number as the pitch changes 1 degree per 5 knots. This will give you an idea of the lift in the landing configuration. Be careful not to use the pitch and power charts in this case because they are carrying about 10 knots above VREF. When you don't know or trust your airspeed, that chart has a safety margin. Read the notes in that chart, they will tell you VREF+10. Many a developer have made the mistake as taking this pitch as the VREF pitch. If pitch is too high on approach, lift is lacking. If it's too low, then there is too much lift.

Landing: Usually in the landing geometry section they will detail the flare and touch down procedure and behavior. The aircraft should continue on it's trajectory just like the no flare touch down point. Even though the aircraft enters ground effect, the nose has a tendency to drop which will be added with your 5 knot reduction to cross the thresh. The landing geometry section will tell you the speed loss and the pitch change in the flare. It's normally a 2 to 3 degree pitch change. Slippery aircraft will be on the 2 degree change side. As you round out, the aircraft should continue sinking and minimize to a general slightly firm touch down. You are basically flying it onto the runway. Your aimpoint should be about 1000ft with the touch down coming about 1000 to 1200ft about VREF-5. In long bodied aircraft, you will aim slightly past 1000ft touching at 1200 to 1500 nominal. Just aim at the captain's bars. Also, perform these tests on standard day conditions and no wind. If you find that the aircraft suddenly stops its descent in the flare or it acts as if it's flaring on it's own, too much ground effect. You will also notice that you are way past the aimpoint touching down way under speed. In the real world, it's easy to get within 200ft of your aimpoint without trying.  

Stall series: I do these separate and first as it will tell me how the rest of the flight will go. Start them at the aircraft's certified max airport altitude and at 5000ft. If it's certified to operate at an airport at 15,000 FE, start your stall tests at 15,000 STD. Start in the clean configuration and write down when the warning starts and the plane departs. If it stalls below target, too much lift. If it too early, too little lift. Then do different configurations until you reach full landing config. This is valuable data as it will tell where the issues may be.

This is a lot, but gives you an idea of how to approach developers and what data is needed to support your claim. The funny thing, some developers provide you with the required documentation.              

 

Edited September 29, 2020 by G550flyer
incomplete

[hs118 707]

Argh @G550flyer Rick, where is the rest of your post! 

[KenG 492]

He must have flew outside of sat coverage for onboard wifi. 🤣

 

The problem with avionics is that even when you say I have a manual for a G1000 equipped Aeromecha Supra and on page 4-13 it says "xxxxxx" there is no guarentee that the Cessna 172R tail number N12345 or the Beech Baron Tail number N54321 are the same. Yes, Garmin logic is very logical, but there are numerous part numbers for G1000 displays as well as software revisions. Far too often you get people who have flown something close, instructed in something close, says it works this way in XPlane, or someone saw something on a Youtube video come forward with a bug. But, the reality is that it may be or may not be a bug as you get deeper into the avionics. No to mention Operators Manuals, which are often the last written by a team of technical writers who get the least amount of time with the engineers. Do you know how many times I have found errors in Operators Manuals? I am not talking punctuation or capitalization, but incorrect images or diagrams, to things described incorrectly. Much less how many times I have found pilots referencing technical manuals that were multiple revisions out of date. 

 

Then there are the thousands of things not documented in the manuals such as the how the logic is run inside of the software or the error capture routines when a pilot fat fingers something. 

 

Anyone who has been around aviation for a while has learned to never speak in definitive terms. As soon as you do someone will find that odd-ball configuration that is just different enough and works so slightly different. 

[ryanbatc 12,798]

True, but the current Gxxx products don't allow the user to navigate very well.  Sure I can use a VOR - but I'd love to be able to reliability navigate with the GPS.

[G550flyer 1,010]

lol, my keyboard set me up when I hit enter. Grab some coffee, this is a rather long post😬

[flyingpauls 66]

While you are absolutely right that the performance numbers must be met as close as possible, I whished for a start that the planes would not do things that are physically impossible, like allowing aileron control in stall situations, not losing speed when pitching up, missing roll or pitch inertia and so on. It seems to be difficult for even X-Plane to model ground contact at landings correctly.

[G550flyer 1,010]

2 hours ago, flyingpauls said:

While you are absolutely right that the performance numbers must be met as close as possible, I whished for a start that the planes would not do things that are physically impossible, like allowing aileron control in stall situations, not losing speed when pitching up, missing roll or pitch inertia and so on. It seems to be difficult for even X-Plane to model ground contact at landings correctly.

You are right, this is an area where it's hard for developers to grasp. The guys at LES had good control inertia in their S340. I had a good discussion with Jan at IXEG and he was able to capture landing behavior in their 737 along with the control inertia. I was very impressed with it. As you crossed the threshold and entered ground effect, you found yourself having to add some back pressure to maintain aimpoint and as your flared per the book, the plane touched down right on the money. Unfortunately they are only available in x-plane, which I don't fly often. I worked very close with rotate simulations on their MD80, they were very open minded and were able to capture the behavior of the 80 and control inertia. I've seen control inertia in microsoft sims, but it's rare. Aircraft are either too sensitive or too sluggish. They don't have that loose feeling and tendency to continue moving when you put in inputs.

[flyingpauls 66]

So, now I know why the IXEG feels so good.

[airlinejets 709]

On 9/29/2020 at 8:31 AM, G550flyer said:

You are right, this is an area where it's hard for developers to grasp. The guys at LES had good control inertia in their S340. I had a good discussion with Jan at IXEG and he was able to capture landing behavior in their 737 along with the control inertia. I was very impressed with it. As you crossed the threshold and entered ground effect, you found yourself having to add some back pressure to maintain aimpoint and as your flared per the book, the plane touched down right on the money. Unfortunately they are only available in x-plane, which I don't fly often. I worked very close with rotate simulations on their MD80, they were very open minded and were able to capture the behavior of the 80 and control inertia. I've seen control inertia in microsoft sims, but it's rare. Aircraft are either too sensitive or too sluggish. They don't have that loose feeling and tendency to continue moving when you put in inputs.

Thank you for the write-up.

I'm wondering if Jan would be available to help the FlybyWireSim guys out with the flight model of the A320? We can always dream.

Edited October 3, 2020 by airlinejets