MSFS Testing & Hubble Space Telescope

[MDFlier 2,031]

I'm perfectly fine with it. MSFS is a huge undertaking with many moving pieces and I'm actually enjoying watching it come together. Life is bumpy. Fasten your seat belts and enjoy the ride! 

I guess I'm a glutton for punishment.  😄

[Matchstick 1,886]

7 minutes ago, MrFuzzy said:

Yeah, the difference between a space telescope and MSFS though, is that to fully test the first you have to bring it into space, for MSFS the same PC used to make it is sufficient. Many bugs could have been detected just by starting a flight... or even in the world map.

I agree on the conclusions.

You are assuming the bugs WEREN'T detected.

[edpatino 1,964]

1 hour ago, hansb57 said:

Maybe this can help us through the holidays:

 

https://forums.flightsimulator.com/t/blog-december-2nd-2021-development-update/478125

Well, it seems they now recognize the several issues introduced with SU7, including of course the weather system that's not reliable. I do hope this waste of time (for them and for the users as well) make they think twice about this crazy updates policy, more likely due to a poor management and marketing interests than anything else.

[MrFuzzy 2,118]

32 minutes ago, Matchstick said:

You are assuming the bugs WEREN'T detected.

I think we can assume that. There were known issues mentioned in the release notes and many SU7 bugs were not there: TrackIR, mouse in VR and world map, weather bugs, assistance options reset... high temperature above FL440 was mentioned. Now it's around FL240.

I cannot think that they knew and pushed the update as it was, omitting the bugs in the release notes. They even edited the release notes 3 times as people reported new bugs.

https://forums.flightsimulator.com/t/game-of-the-year-edition-release-notes-1-21-13-0-sim-update-vii-now-available/472025

Edited December 3, 2021 by MrFuzzy

[Matchstick 1,886]

2 minutes ago, MrFuzzy said:

I think we can assume that. There were known issues mentioned in the release notes and many SU7 bugs were not there: TrackIR, mouse in VR and world map, weather bugs, assistance options reset... high temperature above FL440 was mentioned. Now it's around FL240.

I cannot think that they knew and pushed the update as it was, omitting the bugs in the release notes.

No, you should not assume that.

The NDA gets in the way here but at least some of the issues were not a surprise to everyone who tested SU7.

[Krakin 5,748]

54 minutes ago, edpatino said:

Well, it seems they now recognize the several issues introduced with SU7, including of course the weather system that's not reliable.

There's no way they're only recognizing them now when the fixes have already been implemented on their end. They're now asking for testers to make sure the fixes are good and don't break anything.

Edited December 3, 2021 by Krakin

[March Hare 720]

1 hour ago, MrFuzzy said:

for MSFS the same PC used to make it is sufficient.

That isn't strictly true. I mentioned this somewhere else recently, but my sibling encountered an issue where the code worked as it should in his editor, but didn't work on another machine, for no obvious reason. Across platforms, with varying hardware, software, drivers, connected peripherals... It's better to get code tested on as many different systems as possible -- and it's especially better to do that on live servers, where a large portion of this software is streamed from.

So I don't think it would be wise for any developer to say it works on my machine so just get it out there (that's then as bad as the publisher saying this is the release date, it's set, so just get it out there and we'll fix the known and unknown bugs another time).

Edited December 3, 2021 by March Hare

[edpatino 1,964]

23 minutes ago, Krakin said:

and don't break anything.

Are you sure?

[Twenty6 692]

1 hour ago, March Hare said:

So I don't think it would be wise for any developer to say it works on my machine so just get it out there (that's then as bad as the publisher saying this is the release date, it's set, so just get it out there and we'll fix the known and unknown bugs another time).

Why does this make me think of the Aerosoft forums?

[March Hare 720]

23 minutes ago, Twenty6 said:

Why does this make me think of the Aerosoft forums?

I'm not sure. I've never been on there.

[turbomax 3,284]

20 hours ago, David Mills said:

My summary point is this: Asobo, like the team who build the Hubble Space Telescope, needs to do extensive, end-to-end testing before launch.

like this former NASA engineer Mark Rober:

 

[Bobsk8 12,677]

There is one popular maker of Pre built PCs, ( not one I bought thank goodness), that tests the individual components that they put in their PCs, but never fire up the PC before they shop it to the customer, to see if it actually works.  many of them don't right out of the box. So now the buyers has to go through the nightmare of trying to reach customer service, and possibly RMA the unit if  CS can't tell them over the phone how to fix it. 

[JeffF 26]

20 hours ago, pmb said:

I think the gist of David's analogy isn't just insufficient testing. They may do quite some isolated testing of different modules in - from what I heard - quite a number of different builds in parallel. The point is missing final testing of the complete release with all the modules cooperating (or not).

Or, as some suggested: They should task a couple of their fellows with just normally flying the complete pre-release version for at least one day before release.

Kind regards, Michael

Ok class, can we spell i-n-t-e-g-r-a-t-i-o-n t-e-s-t-i-n-g?  It's only been common IT practice since the punch card days.  Until now, that is.

I somehow survived and a couple years ago retired from a 35 year career in corporate IT, software development.  A common saying/joke among the developers was, and still is, "we never have time to do it right but we always have time to fix it."

 

-Jeff

Edited December 3, 2021 by JeffF
Added name

[pmb 5,137]

2 hours ago, JeffF said:

A common saying/joke among the developers was, and still is, "we never have time to do it right but we always have time to fix it."

This will go to my bookmarks!

Kind regards, Michael

[Twenty6 692]

5 hours ago, March Hare said:

I'm not sure. I've never been on there.

One of their standard, canned replies is "It works on our computers."