It cant be long right?

[marquesas 5]

@ Mike777: Thanks for your reply.....

 

 

 

Marc

[Boomer 1,619]

IBTL.

[scandinavian13 5,192]

Aurelien is on the mark here. The sim industry fits into quite a niche.

 

It's one thing for a group like Dice doing a wide, very-late-stage, beta across the world because that's exactly what they need: their product is made specifically to have 30-60ish people on a single map, and handle loads of people running through a bunch of servers. They need hundreds if not thousands of people to stress test their product to suss out any bugs. It is a literal necessity for them in order to properly test their product, which exists for and depends on a simultaneous mass of users. If you only had 12 people running around a 60 person map, it would get a bit boring, but it would also mean that your servers aren't getting realistic loads for a session. Again, this sort of thing is also going to be a very-late-stage beta version, where a smaller (perhaps fully internal) beta team has hammered on the software to get the major bugs out.

 

Some developers in our industry might do similar, as more users can help to better identify trends in the data, but you have to weigh that benefit over the cost of managing it. Developing scenery might lend better to having a ton of people look it over, as there's very little 'knowledge' that comes into play. All the same, it might be better to keep that team to a group of people who works at the specific airport being designed to help ensure that you're only having to respond to people who know the airport like the back of their hand. You could also mix those two approaches with little issue, which would allow you to have the people who work there let you know that a texture is off, or a door is missing, or something else that a casual simmer would not, while the casual simmers could help you suss out issues with effects, lighting, general texture corruption and so on. Mass approaches can also work well with weather and other items like camera programs.

 

When you get into aircraft, though, you start introducing more of a knowledge element. It's less of an issue of clear right and wrong. When it comes to complex aircraft, you start getting into topics that require a lot of knowledge. As you can see with a cursory glance around the forum, and the fact that the Intro Manual has an entire section on common misconceptions about how the aircraft should function, there are a number of people who will assume the aircraft should behave one way, but actually behaves in another. People on the beta team carry this trait as well. This is a good thing, because I wouldn't be able to write that "common misconceptions" section if they didn't, but it could also be absolutely dismal if you had thousands of people on a beta team sending you thousands of emails reporting something that isn't actually behaving incorrectly. You'd essentially need someone on the team to manage all of the responses and identify what is, and isn't, actually a bug. Keeping a smaller beta team, however, primarily eliminates this requirement, since the you're only dealing with a handful of reports, and the smaller team also tends to communicate within its members (to correct each other where misconceptions, or previously reported items, come into play). For any of you in or post college, think of the dynamics between a giant lecture-based class, and a smaller class. Similarly, the smaller environment encourages actual involvement. If there are 3000 people, I may not report something because I think one of the other people will do it. On the other hand, if there are only 30, I'm more likely to speak up since I may be the only person in that group of people to notice it.

 

There's also a larger security element. Granted, a fully public beta somewhat eliminates this concern, but a fully public beta also has its own concerns (that I'll discuss in a moment). When you leave on vacation, do you leave your keys with someone in your family, a friend, or some random person? Why? I'm willing to bet that your answer is going to largely center on established trust. People you have a relationship with are less likely to sacrifice that trust. A larger group made up of a higher percentage of a random selection of people is a larger risk in that regard.

 

As somewhat of a final note, there are a number in this industry/hobby who don't quite understand (or don't quite want to understand) the purpose of beta. I see this a lot when people mention being jealous of someone with access to the beta (though I can kind of understand this one), and when I see various posts of other products in beta (though this is usually more in the generic games arena and not so much the sim). While it's true that being on a beta team gets you access to a product that you want, most people want the actual, functional, working, enjoyable copy of the software. If that's what you were getting for being on the beta team, then you wouldn't be on the beta team - it would be released. Being on a beta team means you're testing, writing up reports for issues, running through a test scenario to try and pin down steps to reproduce, running through test scenarios as recommended by the team, and otherwise hammering on the software to try and find out where it breaks. It's kinda neat, but it's not always fun. Looking at various posts from public betas of games, it's obvious that a lot of people just wanted access to the game, and were less than forgiving when reminded that they were there to help test an unfinished product. When you have a smaller team of people that you've invited in because you've seen their passion, it's a lot easier to predict what kind of tester they will be to avoid stuff like this. The last thing I want to do is offer a test position to someone who would see the fact that there are still bugs in an early beta, the software isn't entirely fun to fly yet at that stage, and then have them just give up, not fly, and otherwise "tie up" a beta position that someone with more of a testing mindset could be in.

 

We use somewhat of a two-step approach where we have a Tech Team of actual pilots, mechanics, and other people who have worked specifically with the aircraft we're simulating. They take care of a ton of the "the aircraft should behave this way" stuff. The Beta Team then gets the aircraft and they usually end up getting outside of normal procedures - either due to lack of knowledge, or simply a different mindset - to find out inventive ways to break the plane. The Beta Team is made up of people who are representative of the sim community, but are people who we trust, and know who have a passion about flying/simming. This setup has done well for us over the years. I get that it might not always make sense to everyone, and there are many who will criticize it, but when you break it down for the industry in which we exist, and for our specific purpose, you start to see exactly why we do it the way we do it.

[garrett_frank 150]

 

 

Because that is not proper management.

 

My comment was in jest, in light of may developers who have taken this route, recently (and unfortunately).

[craig_read 43]

Having been on the Beta team at one point (MD-11, I think actually by mistaken identity which I tried to correct, but hey ho..) I can say from experience that it's a lot of effort.  I actually remember writing to RSR towards the end of it simply saying that I didn't have the time to contribute like the others did and consequently should really be left out of it.  I also offered to relinquish the 'perk' as it only seemed right.  He was pretty generous actually, and still gave me the MD-11 despite me saying I'd pay for it.

 

Being on Beta teams may seem like it's a lot of Kudos, we do a lot where I work as well, and it's true that you do get some perks, but there is an element of work to this and effort involved.  To be perfectly honest, at least from my perspective I wouldn't put the time in to be on any Beta team, life for me is just too busy (plus home command thinks I 'fly' too much as it is).  For those that 'wish' to be on it, just be careful what you wish for... personally.. I like to pay my money when it's out.. and have none of the hassle (uninstalling, reinstalling, uninstalling, reinstalling.. sometimes twice a day.. it does become a bind folks).

 

Cheers