4 minutes ago, Car147 said:

Excuse my ignorance, but why can they not just roll the sim back to when it was stable... surely they back up in case of something like this?

Indeed! What annoys me the most is, this issue started yesterday night, which mean, they have deployed something to bing servers that are serving MSFS on a ... Friday! Who deploys to a production on Friday! This is like our no-go in Software releases to deploy right before the weekend. 

However I think the issue is not from Asobo, I think it is from the Bing team since they are responsible for these data.

It is annoying.  I wasted most of the night trying to fix what i thought was an issue on my PC until I posted on the Orbx forums and one of their tech people advised they had tracked it down to a Bing map issue with the sim.  

On the bright side I did not go as far as reinstalling the game trying to fix it 😄

1 minute ago, Glenn Fitzpatrick said:

It is annoying.  I wasted most of the night trying to fix what i thought was an issue on my PC until I posted on the Orbx forums and one of their tech people advised they had tracked it down to a Bing map issue with the sim.  

On the bright side I did not go as far as reinstalling the game trying to fix it 😄

Have they escalated the issue with MFSF team? We need to put pressure on all sides. A link please ?

8 minutes ago, omarsmak30 said:

Have they escalated the issue with MFSF team? We need to put pressure on all sides. A link please ?

There are heaps of Orbx threads on it.

https://orbxsystems.com/forum/topic/205095-uk-orbx-sceneries-crashing-msfs-on-load-screen/page/2/?tab=comments#comment-1745067

42 minutes ago, Glenn Fitzpatrick said:

There are heaps of Orbx threads on it.

https://orbxsystems.com/forum/topic/205095-uk-orbx-sceneries-crashing-msfs-on-load-screen/page/2/?tab=comments#comment-1745067

Not only ORBX, in Aerosoft forums as well. Let them now deal with stupid issue over the weekend from all this fuss, it is indeed irresponsible in all measures to push something to production before the weekend.

Sorry folks, this must all be my fault. I had a spending fit yesterday and bought around eight new airports.

What is it with people making changes on a Friday? I really do believe evolution peaked some time back and we're well on to the downward slope now.

 

13 minutes ago, andy1252 said:

What is it with people making changes on a Friday? I really do believe evolution peaked some time back and we're well on to the downward slope now.

I am still struggling to understand. Okay, pretty sure the Bing team pushed the data to production as part of the bounded context where every team is responsible for their own data and indeed it is their responsibility to fix it even over the weekend. Okay we all do mistakes, I did once push to production on Friday in my previous job (I was stupid), the following Saturday I got few calls from DevOps, my manager, the manager above my manager asking me to react fast and fix it, on a Saturday night. Well I did fix it because it was my stupid mistake and I am responsible for it. I'd expect the same responsibility from Microsoft teams.

Edited February 13, 20215 yr by omarsmak30

I wonder if this delays the U.K. update further then. 

3 minutes ago, Ianrivaldosmith said:

I wonder if this delays the U.K. update further then. 

Well if it will bring with it some stupid issues, then let them delayed 😄. I would rather have a functioning sim rather than broken one. I am starting to get nervous whenever there is an update on the horizon lol. You know folks, we gotta nag in the next Dev Q&A about the urgent need for the beta testing track, not only for the sim but also for the online services as well like Bing, Meteorblue, flightaware .. etc

Edited February 13, 20215 yr by omarsmak30

21 minutes ago, Ianrivaldosmith said:

I wonder if this delays the U.K. update further then. 

I t is probably a fortunate thing the UK update did not drop to compound things even further.

7 minutes ago, Glenn Fitzpatrick said:

I t is probably a fortunate thing the UK update did not drop to compound things even further.

Yes you are right Glenn.  It might have made it harder to trace at least.

While Friday's may not be standard for game releases they are pretty popular for change windows...  Perhaps they tried to push something while there was minimal load on the servers.  Not sure what the issue with dropping code on a Friday.  That's what IT teams do and yes, I am sure they have staff working on the issue.  It appears what they didn't have was a roll back plan once the issue surfaced.  

Change windows on Friday nights aren't stupid though, it's an IT standard for the largest corporations in the world. Give them a day, let's see if they can resolve the issue.  It's not like we haven't seen updates break and turn add-ons into paper weights with other sims, the only difference being it could be months for a hotfix from the Dev to handle what the update broke.  Hopefully this can be resolved much quicker.  

1 minute ago, psolk said:

Change windows on Friday nights aren't stupid though, it's an IT standard for the largest corporations in the world. Give them a day, let's see if they can resolve the issue.  It's not like we haven't seen updates break and turn add-ons into paper weights with other sims, the only difference being it could be months for a hotfix from the Dev to handle what the update broke.  Hopefully this can be resolved much quicker.  

No way, I have been working for several big IT organisations as senior software engineer and in every case we have clear instructions from DevOps to avoid to deploy anything on Friday, that is to reduce the support load on weekend for any unfortunate issues. An exception that, if we really need to deploy something big that introduces a downtime, we do it on the weekend due to the low load since we know from our analytics, the load on the systems is very low on the weekend which is fine. However, MSFS is clearly the opposite, the load at its peak on the weekend comparing to workdays 😄  

I did not use the sim yesterday, that's why I'm really surprised to read all this today.

Turning the Bing data OFF means more or less going back to the old FSX (or even P3D) days. Another solution, as I read, it's to locate all our cgl folders or, even worse, files, and renaming all of them. Neither of both are really a solution.

I think MS should have something like a system restore, to get the Bing data back to the state it was after what they did yesterday. The solution should be in their hands, not ours.

I just remember that we have made solutions for ourselves for all the bugs introduced to the sim after every Sim Update (or World Update), like the terrain spikes bugs, the LOD issues, the silvering coastlines, and so on. Are we going to continue doing that?.

I think it's time for MS/Asobo team to put things in order, before even thinking in releasing another World or Sim Update, and that should be done before Tuesday 16, if that date still stands.

Back to P3D for several days?. I don't know, but it could be an option.

And, btw, thanks to the OP for the information, otherwise I would have seen my sim crashing without knowing why, and that is really scary and frustrating.

Cheers, Ed

Edited February 13, 20215 yr by edpatino

8 minutes ago, omarsmak30 said:

No way, I have been working for several big IT organisations as senior software engineer and in every case we have clear instructions from DevOps to avoid to deploy anything on Friday, that is to reduce the support load on weekend for any unfortunate issues. An exception that, if we really need to deploy something big that introduces a downtime, we do it on the weekend due to the low load since we know from our analytics, the load on the systems is very low on the weekend which is fine. However, MSFS is clearly the opposite, the load at its peak on the weekend comparing to workdays 😄  

So you deploy changes in a production corporate environment mid-week when end users could be impacted?  Sounds like your dev-ops guys are lazy.  I've been doing this for 30 years for the largest companies in the world and have never seen a change applied mid week unless it was an emergency fix.  Otherwise scheduled downtime on a Friday or Saturday so you have a weekend day without end user impact to back out if anything goes wrong.  

As for MSFS being the opposite as you already mentioned, if this push was made by the Bing Team it makes total sense MS has a change window on a Friday night, if it were Asobo perhaps they had to align to MS's change window.  There could be a lot of variables at play, not worth getting all upset I'm sure it will be resolved, just an inconvenience.  

Edited February 13, 20215 yr by psolk