Avsim down time

[n4gix 4,947]

I remember many years ago that Tom posted several pictures of "The Rack" at the NOC. It's a shame I cannot seem to find them now...

 

...as it was truly impressive seeing that robust and expensive server rack!

[Chuck_Jodry-VJPL 663]

I found a pic from a recent visit to add the new disk array, the beating heart of Avsim . 

 

 

[WarpD 824]

AVSIM does an amazing job of maintaining this website. Complaining about it just doesn't make sense to me. No other website I can think of has the sheer volume of traffic that this one does.

 

I'd say the AVSIM team is doing better than most and deserve kudos not complaint. That's my opinion.

[Christopher Low 5,341]

I think we all probably get a bit frustrated when AVSIM is down, simply because it is a very important part of our lives! Nevertheless, we need to retain perspective where maintenance and running costs are concerned. Like Ed has already stated, the volume of traffic through AVSIM is huge in comparison to any other flightsim website, and is run by enthusiastic volunteers who spend a lot of their free time and effort keeping it operational for the benefit of the entire flightsim community. I am extremely grateful to all of you, and I will be making another donation to the site as soon as I get home from work.

[n4gix 4,947]

I found a pic from a recent visit to add the new disk array, the beating heart of Avsim . 

Thanks for the more recent photo, Chuck. Perhaps the photo should be placed as a sticky somewhere to show to the next person who is inclined to think AVSIM is run on a creaky and ancient 486DX computer from somebody's bedroom...

[Poppet 703]

Great and well said #9 post Chase.   I brought the Subscription thing up last year before Tom passed,  i thought maybe it was a great idea at the time but Tom did explain it different and then it made more sense.  People you can donate do and folks who cant, cant,  everybody gets to share the beautiful hobby.  

 

PS,  the site is flying along now  :smile:

 

 

 

I'd say the AVSIM team is doing better than most and deserve kudos not complaint

Correct 

[TechguyMaxC 516]

AVSIM does an amazing job of maintaining this website. Complaining about it just doesn't make sense to me. No other website I can think of has the sheer volume of traffic that this one does.

 

I'd say the AVSIM team is doing better than most and deserve kudos not complaint. That's my opinion.

 

Complaining and seeking information to offer potentially helpful suggestions based on expertise are two very different things.  I made no criticism of anyone, and even prefaced my remarks to indicate that.  There are lots of sites that have much larger forums than Avsim.  Anandtech, for one.  Check their user count here: http://forums.anandtech.com/index.php (bottom of page).  I've been a member there almost as long as I have here and the number of times that site has gone down unexpectedly can be counted on one hand.  Even this relatively small community also run by volunteers has less down time than Avsim, and their user count isn't too far off from Avsim's: http://www.rage3d.com/board/index.php  It is undeniable that Avsim has high volume, it is also undeniable that they have a high frequency of issues.  Are the two related?  Almost certainly.  Are other large sites able to operate without frequent down time?  You betcha.  So what does that mean?  There is a solution.  The question is, is Avsim willing to do what it takes to solve it?  Time will tell.      

[Luke 782]

AVSIM does an amazing job of maintaining this website. Complaining about it just doesn't make sense to me. No other website I can think of has the sheer volume of traffic that this one does.

 

I'd say the AVSIM team is doing better than most and deserve kudos not complaint. That's my opinion.

 

I can think of one - I used to be responsible for its design and performance. We did around 26 terabytes a day, more on busy days.

 

AVSIM gets a pass on the tech front because they are the one-eyed man in the land of the blind. If they're still buying hardware and (gasp) a SAN in 2015, I'm at a loss for words. There are plenty of multi-million- and billion-dollar businesses that trust their infrastructure to the cloud and wouldn't consider owned infrastructure. I can think of three major ones off the top of my head.

 

Techguy's last posts make sense - the forums aren't particularly large relative to other forums and the file library, while it accounts for much of the bandwidth, is relatively simple and light to process. There are thousands of sites that have more significant loads than AVSIM who can handle it quite well without issues.

 

Cheers!

 

Luke

[Chase 303]

 

 

The question is, is Avsim willing to do what it takes to solve it?  Time will tell.

 

That has never been the question.  I think it's quite clear we are doing everything we can.

[n4gix 4,947]

There are plenty of multi-million- and billion-dollar businesses that trust their infrastructure to the cloud and wouldn't consider owned infrastructure. I can think of three major ones off the top of my head.

Luke, there is a vast difference between "multi-million- and billion-dollar businesses" and a not-for-profit "business" running on a shoestring budget and staffed entirely by volunteers.

 

That should be self-evident. Your comparison is a total non-sequitur.

[WarpD 824]

Cloud.. Ashley Madison... Ashley Madison... all over the internet from the 'cloud'. :grin:

[N400QW 205]

 

[Luke 782]

 

 

Luke, there is a vast difference between "multi-million- and billion-dollar businesses" and a not-for-profit "business" running on a shoestring budget and staffed entirely by volunteers.

 

Bill, I was addressing WarpD's point that he wasn't aware of any site's with Avsim's "sheer volume of traffic" that ran in the cloud, nothing more. The reality is that the cloud hosts digital properties that makes Avsim's "sheer volume of traffic" look like a rounding error. If you now want to shift the goalposts and suggest that these sites are much larger and have larger budgets than AVSIM, you can. It's a legitimate point but I should also point out that by doing so you're admitting that cloud can handle Avsim's traffic, and we're now just talking about cost.

 

Let me tell you a story.

 

I've handled the tech work for a virtual airline for about a decade and a half. Our schtick is being technology- and data-driven, and we've kept a ton of data about flights, examinations, check rides and other stuff for most of that time. It's not huge, but it's pretty similar to AVSIM. On one hand, we have a bunch of database tables that have large numbers of small rows, and then we have the flight data history - it's around 700 million rows and 16 gigabytes. One day over a drink I'd love to have a conversation about all the neat things we can do with the data beyond the eye candy, but let me leave you with some eye candy on what several hundred million position reports look like on a map: https://www.deltava.org/trackmap.do Our bandwidth isn't huge, but it's a couple of terabytes a year.

 

And before you ask, we're staffed entirely by volunteers and our annual budget would power the high-end web sites I've designed for a few minutes, maybe less.

 

With that said, we've not purchased, racked or stacked our own hardware for over a decade. In 2005 we realized that it was a poor idea. "Cloud" wasn't in existence yet, but renting dedicated servers certainly was. We've rented a Linux machine in a third-party data center for a decade. We have access to the same open source software as Weather, Facebook, Netflix and Amazon. And if the hardware goes wonky, we get an alert. We don't wait a week for the tech to go to the data center - we get on the phone and complain - drive replaced, mdadm told to repair the array and we're off to the races with no downtime.

 

In a cloud scenario, it's even easier. You just mount the volumes on a new virtual computer and you don't depend on anyone. Awesome! If cost is no objection, there's no rational argument towards racking and stacking yourself. Do you grow your own food or generate your own electricity? (And if you do - do you depend on it?)

 

As you point out, cost is a concern. We looked at AWS in 2011 and it wasn't there yet. When we did our server upgrade this year, we looked at Amazon and moved our mail, teamspeak and other ancillary services to AWS on a t2.micro. The full site isn't ready yet, but as AWS has dropped their prices significantly and our load has not significantly increased, I expect that our little hobby site is going to move there in 2016 or 2017.

 

The bottom line, Bill, is that it's not a question of if. It's when. AVSIM needs to start planning for the next generation of its infrastructure that goes beyond on-prem. No one (whether they do cloud or not) in the volunteer space does on-prem or colo. It was dumb ten years ago, and if you depend on some weird esoteric hardware configuration then Avsim is doing it wrong.

 

What Techuy and I are trying to point out is that Avsim's data and bandwidth volumes aren't exceptional in 2015. There are plenty of sites that get more traffic and have higher data volumes than Avsim that are able to survive just fine on limited budgets and limited hardware. This organization would do well to stop patting itself on the back and start asking some question as to why this is the case, and what can they do to better utilize the scare resources that they have? We've certainly done that over the years and have been pleasantly surprised. I could probably still run the server in my spare bedroom and be considered a hero by my user base (most of whom thinks what I do is insufficiently distinct from magic). I know better.

 

On a final note, AVSIM and its users are the last people who should be talking about security and the cloud, considering the history this site has with data security and integrity. The reality is that most site attacks are due to poor software or internal threats rather than the hosting provider. Avsim of all communities should know this better than anyone.

 

Cheers!

Luke

[WarpD 824]

Lots of web sites with crappy software have never been hacked because there's no point. It's not because of what they're running for software, it's because of what they represent and what the potential of the access offers. No more, no less.

[Chase 303]

I'm going to consider this topic closed.  We appreciate the feedback, as always, and hope you enjoy our free services.  We also appreciate any donations you can make to our site to help pay the bandwidth bills.  Thank you!