Server Outage, Unplanned - Sorry!

[rsrandazzo 6,353]

Captains-

 

Just wanted to drop a quick apology for the license control outage we had during the day, 19MAR13...

 

During the evening of 18MAR13 (GMT) we conducted some routine maintenance to install various windows updates, check logs, update software suites and generally poke around at the health of our server systems.  This usually requires a reboot of each machine in the process, and while the machine in question came back online without any problems we failed to detect that the update caused the server to cease running the license control software that is it's primary function in life.

 

NORMALLY, one of our techs runs through a detailed checklist of all services and functionality any time a server reboots- with the specific intent to capture stuff like this, but he was out of town and delegated that task to me as he will occasionally do...

 

As a few of you probably could guess, I decided "Hey..."  (because all sentences inside my head start off like a personal conversation) "I've done this a dozen times and there are never problems... So I'll just stick with what I'm doing and not worry about it..."

 

Yeah......................

 

There are two lessons here:

 

1) Never leave me unsupervised with critical equipment.  It can and WILL fail.

 

2) There is a reason for the checklist- and me of ALL people should have no problems adhering to one. 

 

Anyway- Ryan and Paul were both away from tech support until late in the evening 19MAR13, and I had some 777 drivers spending a bunch of time at the office with me today poking around some items on that development process- so NONE OF US noticed that the system was offline until Paul and Ryan started looking over our support tickets for the day.

 

I'm sorry if anyone was inconvenienced.  While the software suite failure wasn't anyone's fault per se, the failure to catch the failure is purely mine...

 

May... uh... Culpa?  Yeah... definitely Culpa....

[SVA1181 56]

May... uh... Culpa?  Yeah... definitely Culpa....

 

MAY.....God be with you all ! 

[Guest Mik75]

[SVA1181 56]

lol! Here we go again!

 

:lol: Nothing meant here, just testing what weapons RSR has in his office these days! 

[EGLL77W 1,508]

We all know you released the 777 into a secret product on the website - you aren't fooling anyone  B)

 

On a serious note: that's fantastic you actually apologise as the CEO. Most wouldn't even care! Nice to know you do. 

[cyclobs 18]

On a serious note: that's fantastic you actually apologise as the CEO. Most wouldn't even care! Nice to know you do. 

 

To be fair though, most CEO's are kept right away from anything to do with servers. :lol:

[tcas_climb 45]

So will you be getting 40 lashes for this or what?

[SVA1181 56]

So will you be getting 40 lashes for this or what?

lol, seems you passed by KSA one day!

[RPRIEVER 52]

Robert.....STEP AWAY FROM THE SERVER!!   ^_^

[AirbusCG 52]

What about a free 777 for all of us as an apology?

 

Honestly, it's good that you're actually writing an apology. As people already said before many wouldn't care... Good to know you do!

[InTheAir 44]

How many here are versed in Murphy's Law? Rob, you have to know he'll get ya!

 

"What ever can go wrong, will go wrong!"

 

 

I think it shows class to start a thread like this. Now, if your just being nice so we'll buy the T7 you'll have to do better cause I'm still not sold! :lol:

[Swofford 155]

My convorsations with myself usually start with "You idiot!"

[squerble 300]

Reading this thread as I arrived at work this morning meant I double checked all of our core services  B)

[rjmcgill 15]

Some people were meant to have computers and some were not. Thanks for the update. Cpt out.

[InTheAir 44]

Some people were meant to have computers and some were not. Thanks for the update. Cpt out.

In the Captains defense, he's an integral part of the development process for these amazing virtual aircraft we get to fly. He's definitely meant to have a computer, lol just not be responsible for their health!!