Programmers do more unit testing, QA does more integration testing, simply for the fact that most large teams have variances in version synchronization that would be hard to test and take too much time for the independent programmers. It is more difficult for general QA to do unit testing, unless they have written test cases, because most integration testing is just testing stability or core functionality, and how would an integration tester know what needs to be tested in a unit test if they didn't write the code. Most programmers dislike writing documentation (not all), and many don't like passing their testing off as "test cases". As a programmer, you have to be a tester as well, if you don't test your own code, how would you know when it is done, you'd be playing back an forth and wasting time.

As far as programmers not being in charge, that is often somewhat true, but in this case (case of Asobo), I'd say the programmers are 100% in charge, and that's the issue. There doesn't seem to be a central governing reasonably experienced "think tank" directing their design decisions, as the decisions are random as if each programmer is making their own decisions, and in which case I do believe most are and that the core issue here is a poor project management and a lack of directional standards. It is literally the "wild west" of development, just look at version 0.8 in the SDK, some of the changes make zero sense.

 

Edited December 3, 20205 yr by SceneryFX

5 minutes ago, bdwhittle64 said:

They are all in this for the money in the end, although I see a lot  of passion for the graphics and flying -- from Asobo.  From the Microsoft motive, why a whole world flight simulator?  Bing, Bing Maps, and Microsoft Cloud (Azure) services, anyone?

Watch this video, if you have not already..

 

31 minutes ago, bdwhittle64 said:

So while you're absolutely right that the AP needs a whole lot of work, for many of us, as long as we have workable heading and altitude hold logic, and two nav radios and DME, we can competently fly IFR without ever looking at a GPS.  And it's a whole lot of fun, too! 

The FAA is in the process of decommissioning many radio navids down to a 'minimum viable' state and there are 6000+ GPS approaches in the US, and many airports are adding only GPS approaches and no ILS.

I agree that heading and altitude hold is the most important thing, but the national airspace is changing and in order to simulate modern IFR flight these items need to be cleaned up (or the SDK needs to add the features that RealityXP needs to bring us RXP Garmins)

2 minutes ago, SceneryFX said:

Programmers do more unit testing, QA does more integration testing, simply for the fact that most large teams have variances in version synchronization that would be hard to test and take too much time for the independent programmers. It is more difficult for general QA to do unit testing, unless they have written test cases, because most integration testing is just testing stability or core functionality, and how would an integration tester know what needs to be tested in a unit test if they didn't write the code. Most programmers dislike writing documentation (not all), and many don't like passing their testing off as "test cases". As a programmer, you have to be a tester as well, if you don't test your own code, how would you know when it is done, you'd be playing back an forth and wasting time.

As far as programmers not being in charge, that is often somewhat true, but in this case (case of Asobo), I'd say the programmers are 100% in charge, and that's the issue. There doesn't seem to be a central governing reasonably experienced "think tank" directing their design decisions, as the decisions are random as if each programmer is making their own decisions, and in which case I do believe most are and that the core issue here is a poor project management and a lack of directional standards.

 

Good sir, I sense that you have spent some time doing formal software development.  

You are absolutely correct.  Coders are typically given a module to code -- or to modify -- along with a design (interface and behavior definitions), and a set of unit test cases the module must meet.  At that level it's usually not tested in the main branch (system test).  That's usually done by a system group against the design test cases.  If passed, then it goes to the QA group.  I'm speaking in general, because depending on the size of the project and the critical nature of it (space shuttle software?), people, groups, and roles change.

Even so, on any size project, all of these roles and functions must exist.  I've read another poster use the term "spaghetti code" and another one use "side effect".  I believe these terms describe the current MSFS software state at present.  FSX was what we call "organically grown", which means that in the end, rather than being designed as modules with independent behavior and clear interfaces, it was hacked and hacked and hacked until it became impossible to maintain.  It developed "solidified lava flows", which means that the last guys who could understand the code are gone, it's now "spaghetti code" that snake it's way throughout the whole code base, and if you change any piece of it, "side effect" breaks something else.  Rather than clean interfaces, global variables are used across subroutines, making it dangerous to change anything.

Sound familiar?

I believe this is the challenge facing Asobo -- taking an FSX code base and architecture, and creating a new product from it.  In the case of the lava flows, you either live with it or in parallel, write a replacement and swap over when the replacement is ready.  From my investigations, I've seen lots of evidence of that going on -- and I believe the "steps back" are actually good symptoms.  They are breaking old code that worked, in favor of new code that doesn't quite work well yet.

I've had some projects where we took some C code and refactored it in C++ (actual classes, objects, and stuff).  While a lot of the behavior was in the C subroutines, it was spaghetti code with lots of side effect.  Putting it into classes, methods, and properties forced clear interfaces, eliminated side effect, and allow the creation of subclasses to generalize common behavior (across all planes), and subclass specific behavior (only the TBM).

I hope I'm right and that explains the symptoms we're seeing.

4 hours ago, bdwhittle64 said:

As to the lateral nav bug, Robert and others are right, but their analysis and proposed solutions are incomplete:

1.      The AP death dive, the wing rocking, the lack of a stable, reasonable bank angle, and the slow acquisition of a new heading are all symptoms of the same problem.

Disagree here. They are different issues, some of which were always poor PID/ai.cfg value settings, but some of them lie in the individual flight_model settings for some individual aircraft.  Before the update, the PID controls largely emulated the legacy FSX autopilot settings, as did the core autopilot controls (maybe written in some dll or embedded in Flightsimulator.exe?). Fire up FSX and try a heading change in the bog standard Cessna 172. It is magnificently accurate, with the roll off to level at exactly the point where the heading is captured. No overshoot - no undershoot. Perfect every time.

The pre-update core a/p code in MSFS didn't approach this but it was adequate. My Bonanza Turbo mod took a lot of tweaking but just using PID values, before the last Asobo update it was pretty rock solid and was consistently good at heading, VS, LOC and APP. Post update it is utterly ruined, because slowing down the heading capture affects everything else that is laterally-based. Actually I have discovered it is not the heading control. It is the ROLL control and WINGS LEVEL control which command when the wings begin to level during a heading/ILS LOC capture. And that is why ILS captures are now as bad as heading capture, because no matter what the PID values, it is slow to adjust if the heading is even slightly off during ILS capture and never properly banks unless you are outside of the 30-40 degree area where the proper roll is resumed.

I agree on this however:

2.      The solution cannot be achieved through PID (proportional error) control only.

I'm hoping you mean that "since the update" the solution cannot be achieved through PID control only. Assuming we do agree on this, the offer of help to just one developer might be kind and helpful but it is not addressing the fundamental issue, which is that heading capture is still too slow, whatever the PID tweak was. I know this because I have spent many, many hours painstakingly trying every possible combination of PID values for each function, and none of them....zero....solve the update to the heading function by Asobo.

See below:

 

Slowing down the heading capture was clearly done without any thought about consequences for other a/p functions such as the ones I discussed above. I'm sure the update was done with sincerity, but it was utterly wrong. But what is more important is that the update simply does not address ANY of the reported problems with Spiral dives, out-of-control approaches etc etc. It might alleviate some of them, but it is not really solving the problem. Many people have now come to the belief that all of the pre-update issues were because of the core a/p. That is almost entirely wrong. There was very little wrong with the core a/p before the update. It wasn't perfect, but it was useable.

The reason for the reported spirals of death etc are in my view NOTHING whatever to do with the core a/p before the update. I think the majority of them were either due to faults in the flight_model.cfg file, or a badly tuned ai.cfg, or a combination of both, for each INDIVIDUAL aircraft. I know this because privately I tweaked some of the aircraft that were reported to be unuseable because of out-of-control a/p functions, and nearly all of them were ENTIRELY owing to the wrong PID settings in the first place, and NONE of the them bore any relationship to the core autopilot.

In other words, the update that slowed down the heading function was completely unnecessary and did not, nor will it ever, address the other problems. But what it will do is permanently ruin any possibility of heading capture or ILS capture, both of which are sluggish, unreliable, and PAINFULLY slow.

You simply cannot have an autopilot that takes more than a minute to capture the last 30 degrees of a heading change. It is just untenable. As marsman2020 has clearly demonstrated, it is just unacceptable. I invite anyone who still has it installed, to start up FSX and do a heading capture, NAV, VS and ILS in almost any of the default FSX aircraft and they will all perform each of these tasks very well indeed, with just a very few exceptions.

In other words, Asobo would be well-advised to scrap what they have and simply go back to the legacy FSX autopilot, which is not perfect but a 100 times better than what we have now. Again, I am not saying this in any intentionally derogative way. I am saying it because it is true.

Edited December 3, 20205 yr by robert young

21 minutes ago, marsman2020 said:

The FAA is in the process of decommissioning many radio navids down to a 'minimum viable' state and there are 6000+ GPS approaches in the US, and many airports are adding only GPS approaches and no ILS.

I agree that heading and altitude hold is the most important thing, but the national airspace is changing and in order to simulate modern IFR flight these items need to be cleaned up (or the SDK needs to add the features that RealityXP needs to bring us RXP Garmins)

Understood, and agreed, especially the way you state a very good software requirement for the SDK GPS requirement.  I would maybe clarify it a bit because the RXP Garmin in X-plane has a few parts to it:

1.  The visual 3D cockpit component. 

2.  A 2D panel that can be shown on a separate monitor.

3.  Control of the GPS buttons and knobs,  etc.

4.  Complete control of the flight plan?

4.  And this is the one I'm not sure about -- the ability and logic to replace the AP logic?

I've used X-Plane since X-Plane 8, bought Reality XP, and watched gazillions of X-Plane videos.  Love those guys.

My impression is that X-Plane has basic flight plan management and AP behavior built in, but provides ways for third parties to override anything they want.  They also provide all the facilities for interacting or replacing the visual models and the AP behavior.  

If I'm Reality XP, I would use the provided behaviors and features where I could, and focus on extending or replacing it where I needed to.  Garmin and an FMS work a little differently, but the underlying flight plan and AP behavior are very similar.

For now, RNAV lateral navigation will work, VOR navigation, and LOC approaches will work -- if they fix the lateral navigation bug in general.  VNAV, LDA / Offset ILS, and those other issues you listed are like a nagging toothache.  As long as you have aspirin (lateral navigation), you can make it through the weekend (months more likely) until you can get to a dentist (Asobo fixes the whole AP and navigation system).

By the way, I'm loving these subjective, but adult, debates.  I usually argue one side until we're stuck, then I argue the other side.  When we're done, there's usually no doubt that we've looked at all sides of the thing.

 

19 minutes ago, robert young said:

Many people have now come to the belief that all of the pre-update issues were because of the core a/p. That is almost entirely wrong. There was very little wrong with the core a/p before the update. It wasn't perfect, but it was useable.

^ This ^

Not to mention the fact they didn't abandon the mesh update (and they should have stopped it from getting into final production), and now there are 3PD's descrambling the mesh creation process, appears someone figured it out at the lower level. Total mesh replacement may now be possible, though the docs haven't been updated and it likely is going to take some work. I don't think I'm going to get into total reworking mesh stuff, because let's say I spend a couple hundred hours working with the SRTM and DEM data and Asobo makes another change breaking my work, I'm planning to stick with scenery design and simple mesh updates where the mesh is easily repairable.

When 3PD or volunteers have to get into the HEX and watch jumps and blob writes to fix issues the devs could have addressed in the first place, it's obviously a lack of organization. It's just the growing pains of game design vs. technically correct sim. It's two different styles of programming, game devs are often encouraged to be "wild west" programmers to induce creativity, but we don't need creativity right now, we need stuff fixed first.

if someone can take someone's hex output and fix their mesh, then the original dev should have been able to fix it much easier, but they aren't making the correct choices on how or where to get some of the data from, seems random.

Edited December 3, 20205 yr by SceneryFX

1 hour ago, bdwhittle64 said:

I want you to know that I believe in aliens, but I'm not a conspiracy theorist.  I do find it highly coincidental, however, that Microsoft releases this game right in the middle of the covid pandemic -- right when everyone's stuck at home with lots of time on their hands.  And lots of forums to fill up with all manner of messages, opinions, and emotion!

You actually may have solved the mystery, the AP was written by aliens, I cannot rule it out. 

 

 

23 minutes ago, robert young said:

Slowing down the heading capture was clearly done without any thought about consequences for other a/p functions such as the ones I discussed above. I'm sure the update was done with sincerity, but it was utterly wrong. But what is more important is that the update simply does not address ANY of the reported problems with Spiral dives, out-of-control approaches etc etc. It might alleviate some of them, but it is not really solving the problem. Many people have now come to the belief that all of the pre-update issues were because of the core a/p. That is almost entirely wrong. There was very little wrong with the core a/p before the update. It wasn't perfect, but it was useable.

The reason for the reported spirals of death etc are in my view NOTHING whatever to do with the core a/p before the update. I think the majority of them were either due to faults in the flight_model.cfg file, or a badly tuned ai.cfg, or a combination of both, for each INDIVIDUAL aircraft. I know this because privately I tweaked some of the aircraft that were reported to be unuseable because of out-of-control a/p functions, and nearly all of them were ENTIRELY owing to the wrong PID settings in the first place, and NONE of the them bore any relationship to the core autopilot.

In other words, the update that slowed down the heading function was completely unnecessary and did not, nor will it ever, address the other problems. But what it will do is permanently ruin any possibility of heading capture or ILS capture, both of which are sluggish, unreliable, and PAINFULLY slow.

You simply cannot have an autopilot that takes more than a minute to capture the last 30 degrees of a heading change. It is just untenable. As marsman2020 has clearly demonstrated, it is just unacceptable. I invite anyone who still has it installed, to start up FSX and do a heading capture, NAV, VS and ILS in almost any of the default FSX aircraft and they will all perform each of these tasks very well indeed, with just a very few exceptions.

In other words, Asobo would be well-advised to scrap what they have and simply go back to the legacy FSX autopilot, which is not perfect but a 100 times better than what we have now. Again, I am not saying this in any intentionally derogative way. I am saying it because it is true.

Actually, this was the post that eventually motivated my finally caving in and contributing to this forum.  Here you were trying to make the Bonanza better with a free mod for everyone, but you got very frustrated with the lack of documentation and the very -- uh -- shall we say "unusual" software practices of Asobo / Microsoft.  I felt for you, brother.

I, too, have been playing around with flight model changes.  In fact, the original planes were so unlike real planes  that I changed a few common things:

1.  I changed the nose wheel steering angle to 45 degrees to eliminate the weird "have-to-be-slow-to-turn" taxi behavior.  (I have actually squeaked the tires of an Arrow turning very tight -- yes, I had full aileron into the turn!)

2. The longitudinal stability bugged me because trimming it properly was impossible, so I increased that.

3. The yaw wiggle bugged me, so I increased yaw stability.

That was all before this latest patch, and it improved things.  I did not even consider the AI.CFG file because I thought that was just about AI aircraft, not the AP.  When I read about this new problem, the first thing I did was to find the old FSX or P3D documentation on it.  Check it out at the link below, but I'll also quote part of it.

http://www.prepar3d.com/SDK/SimObject Creation Kit/SimObject Container SDK/Aircraft Configuration Files.htm#autopilot

     >>>>>> FROM THE P3D LINK - FLIGHT MODELS

     PID Controllers

     PIDs are only used to aid in the handling of AI (computer) controlled aircraft. PID stands for Proportional-Derivative-         Integral controller, and is a feedback controller that takes an error for a controlled state and outputs a correction. For         example, on an airspeed controller, the error would be the desired airspeed minus the current airspeed. The output is       then determined by the sum of three factors using the error and the P, I, and D constants. The "P" factor is simply a           factor proportional to the error. The "I" factor is an accumulated factor scaled by the error. The "D" factor is a factor b          based on the rate of change of the error. 

     <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

In P3D,  and I assume in FSX, the documentation says PID is only for use with AI aircraft.  Implications:

   1.  The normal AP turn and track functions must have been hard-coded and did not use PIDs.

   2.  Pre-patch 7 (11/24), Asobo must have been using the FSX code to handle lateral navigation.

  3.  Since the FSX code did provide a constant-rate initial bank and a smooth -- and timely -- rollout to the correct heading / path, then whatever they did in patch 7 must have been either: 1) a boneheaded coding error that broke the FSX logic and was not caught in QA, or 2) an inadequate attempt at coding replacement logic and a switch to it without fully testing it.

FSX does not prove that PID logic is able to do what we need, because PID was only used for AI aircraft and not the user's aircraft.  I don't have P3D or FSX installed, so I can't test this assertion.  You said you have FSX.  Try this:

Kill any wind or gusts.  Set (bug) a 90 degree heading change.  Monitor the initial bank and ensure that it is constant.  Record the degrees of bank and the time before it begins to roll out.  Watch the rollout bank behavior.  Is it a linear decrease -- does it roll out at a constant rate?  Or is it a high bank at the beginning of the rollout and still banked more than 50% at the halfway point.  I.e. On a graph, is the rollout bank angle decrease a straight line or a curve?

If it's a straight line -- a linear bank, then it's not using PID logic.  If it's a curve, it's either a coded function that will apply at any speed, or it's PID logic.

My instinct is to start a separate PID discussion thread.  This is starting to get very technical and I don't want to divert this thread.  I think a discussion thread may be useful because when they fix this behavior in the future, I'm certainly interested in analyzing what they've done.

Robert, you brought up a very good point about the other PIDs (roll, yaw, pitch, brakes, flight level, steering, etc.).  You are correct about the roll PID and the flight model (specifically pitch, roll, and yaw stability and moments of inertia). Destabilizing inputs into the roll PID "plant" include wind, wind gusts, aileron deflection forces, adverse yaw, and heading error causing commanded roll.

This is very interesting to me, because a lot of coders like to code.  They don't like to think at the system or design level, and they certainly don't like to document anything (including commenting their own code!).  You seem very technically knowledgeable and aware that there is a difference between the unit level (just heading PID) and the system level (sing it with me, "The heading bone's connected to the roll bone, and the roll bone's connected to the ...).

Stay thirsty for knowledge, my friends...

 

48 minutes ago, SceneryFX said:

You actually may have solved the mystery, the AP was written by aliens, I cannot rule it out. 

 

 

Despite the newness of my posts, I believe my analyses has started to gain a foothold.  Do not take it lightly that I, too, have nothing to debunk Scenery's alien AP theory.  Perhaps more evidence can be found...

2 hours ago, leprechaunlive said:

Those cheeky b*stards! 😄

Leprechaun?  Sounds like you might be Irish.  Have you ever met a Frenchman?  They're all cheeky... it's what I like about them!

5 minutes ago, bdwhittle64 said:

 

In P3D,  and I assume in FSX, the documentation says PID is only for use with AI aircraft.  Implications:

   1.  The normal AP turn and track functions must have been hard-coded and did not use PIDs.

   2.  Pre-patch 7 (11/24), Asobo must have been using the FSX code to handle lateral navigation.

  3.  Since the FSX code did provide a constant-rate initial bank and a smooth -- and timely -- rollout to the correct heading / path, then whatever they did in patch 7 must have been either: 1) a boneheaded coding error that broke the FSX logic and was not caught in QA, or 2) an inadequate attempt at coding replacement logic and a switch to it without fully testing it.

FSX does not prove that PID logic is able to do what we need, because PID was only used for AI aircraft and not the user's aircraft.  I don't have P3D or FSX installed, so I can't test this assertion.  You said you have FSX.  Try this:

You seem very technically knowledgeable and aware that there is a difference between the unit level (just heading PID) and the system level (sing it with me, "The heading bone's connected to the roll bone, and the roll bone's connected to the ...).

Thanks for that post. First, I am not a technical person. I have almost no coding ability. I'm a tweaker. I observe how aircraft fly, and I try to use available parameters to improve them. It's a blessing and a curse. 

P3d and FSX Do use PID controllers for user aircraft. You can demonstrate this very easily by altering the standard Nav PID controls which are most definitely listed in the autopilot section of the aircraft.cfg files. I've tuned these for many years and they work.

I've just a minute ago compared an FSX C172 flight at the same airfield, with the same a/p climb out, turns, approach and ILS landing, with a similar Asobo aircraft . It was absolutely flawless in FSX. But the key difference is that when adjusting for Localiser errors, the FSX C172 performs a VIGOROUS roll/bank. It does not rely on a sort of half baked yaw/sideways drift which is what Asobo's defaults now do. Yawing for heading position is just plain wrong and it won't work. All aircraft MUST bank to change heading, unless they are making minute adjustments, but it's bad practice. Watch any YT video of any airbus/boeing on approach. All adjustments are made by positive bank. IN 40 years of piloting I never ever heard a single other pilot say they didn't use assertive banking to change heading.

You cannot do a positive bank in MSFS under autopilot currently because the update all but stops any recognisable rolling at all once you are inside a 20-30 degree offset from the desired heading, the very angle that most ILS captures are performed from. You need to be a full 40-50 degrees offset before any significant bank is implemented under a/p in MSFS. This is what makes the update utterly untenable. No aircraft EVER adjusted for heading or ILS with a neutered roll control.

The identical flight at the same airfield with any Asobo light aircraft is a total disaster. It simply cannot capture any ILS properly unless you are already miles away and within 10-15 degrees offset. This would be declared by any civil aviation authority a reason to withdraw any a/p certification. You simply cannot hope to simulate any aircraft a/p with a roll control that only works when you are 40 degrees offset from a desired heading. 

40 minutes ago, bdwhittle64 said:

   PIDs are only used to aid in the handling of AI (computer) controlled aircraft.

Strange, don't think that is accurate.

17 minutes ago, robert young said:

P3d and FSX Do use PID controllers for user aircraft. You can demonstrate this very easily by altering the standard Nav PID controls which are most definitely listed in the autopilot section of the aircraft.cfg files.

Yes, for example, the below is from a P3D aircraft.cfg file for a default aircraft. Tweaking these values certainly makes a difference.

[autopilot]
autopilot_available=1
......

max_bank=25.0
max_bank_acceleration=1.8
max_bank_velocity=3.00
......
nav_proportional_control=9.00
nav_integrator_control=0.25
nav_derivative_control=0.00
nav_integrator_boundary=2.50
nav_derivative_boundary=0.00
gs_proportional_control=9.52
gs_integrator_control=0.26
gs_derivative_control=0.00
gs_integrator_boundary=0.70
gs_derivative_boundary=0.00

........

Al

Edited December 4, 20205 yr by ark

4 hours ago, marsman2020 said:

Inability to select and load an RNAV approach for an airport in-flight.  It has to be a 'destination' in the world map, or it doesn't work

I was able to do this at CYXT in the TBM yesterday.  I did have the ILS approach selected in the world map, but the ATC assigned the RNAV 15 z approach I think it was so I selected and loaded it without issue.  Maybe because I had selected something in the world map maybe?

13 hours ago, Bert Pieke said:

Watch this video, if you have not already..

I just watched it this morning.  It was very good until the very end when it got a little blurry because I accidentally rubbed hot sauce in my eye or something.

We all bought this sim expecting it to meet the hype, and demanding a minimum set of flight simulation.  I agree with everyone else you thinks we don't yet have we paid for, and that they need to deliver what was promised.

But this video does a good job of explaining where we've been and where we're going.  The enormity of the technical challenges they have already overcome just to get to this point is mind-boggling.  Compared to that, getting the AP to work right seems minor -- it's just not been the highest priority.

On the human side of the equation, you clearly see that this is a labor of love for many members of the Microsoft and Asobo teams.  I loved that they appreciated the escapism value the sim provides in this time of covid, and was moved by the statistic of 75% of simmers first used it to go home.  I know I did -- first my house, then where I grew up, then where I trained for my pilot's license, then eventually recreating some of my real world flights.

Those cuts of older flight simulators sure brought back memories, and reinforced just how amazing the current sim is -- despite all its current warts.  I was a teenager during the Apollo missions, when the original Star Trek and Star Wars first came out, when there were no personal computers, no internet, no cell phones, no civilian GPS, no Google maps and satellite views.  For those of us have real world flying experience, and with the amount of perspective gained in the last 40 years or so, this sim is miraculous as is the technical world we live in.  

LPT: There's an old saying, "Nothing is as good as it seems nor as bad."  Translation: Read all the forums and listen to all hype, then thrown out the top 20% of hype and bottom 20% complaints.  The 60% that remains will be somewhere within arm's reach of the truth.

Thanks for the video recommendation.  Perspective restored.