Jump to content

Archived

This topic is now archived and is closed to further replies.

MrPlaneDriver

Alpha and Beta Testing

Recommended Posts

In this forum, I've heard a lot of times about alpha and beta testing of an add-on, specially about the NGX. But my doubt is what are the differences between the two stages. What can be interpreted when someone says that something is in alpha stage or beta stage?Any answer (most of) is welcome.Thanks in advance.

Share this post


Link to post
Typically Software goes through 2 stages:1) Alpa Testing2)Beta TestingAlpha Testing: Testing a software product which is not the final version.This software does not have to necessarily contain the full functionality required for an appliation however core functionality to accept input an generate output is required.Beta Testing: Beta Testing is last stage of testing where a product is sent outside the company or offer the product for free trial download.
Credits go to http://www.geekinterview.com/question_details/26313 for a clear answer.

Share this post


Link to post

hello.Well it depends on many thing but basically it oges like this.Alpha testing is susually only made internally, you don't go with a lot of people, and you're reaching alpha when you have implemented all that you wanted to put in the product. Once satisfied in the alpha the product goes in Beta version.Beta testing is done with more people and often it will be externally (while there are of course still internal testing) but usually a Beat test is done on a product that is nearly completed, and is made to check the programm behavior on many combination of hardware and track as much bugs as possible.Cheers.Aurelien

Share this post


Link to post

It depends on the policies of the company, but in general alpha testing is functional testing done by in-house developers to verify that the code is behaving as intended whereas beta testing is done by outsiders or a sample of customers to see how well the code holds up to use by non-developers. This is where the unexpected is supposed to be discovered and fixed. Ideally, everything works exactly as intended before the beta testers get their hands on it and try to do things the developers never expected as well as put the product through its paces on many different machines.

Share this post


Link to post
It depends on the policies of the company, but in general alpha testing is functional testing done by in-house developers to verify that the code is behaving as intended whereas beta testing is done by outsiders or a sample of customers to see how well the code holds up to use by non-developers. This is where the unexpected is supposed to be discovered and fixed. Ideally, everything works exactly as intended before the beta testers get their hands on it and try to do things the developers never expected as well as put the product through its paces on many different machines.
This is a very nice explanation Dan!

Share this post


Link to post
Guest wims

Alpha = Development stage where all features havent been implemented yet. Beta = All features has been implemented, this is the bug squashing stage

Share this post


Link to post

Thanks everybody for the quick answers. No doubts remaining. Very nice explanations. Didn't expect to see so many replies so shortly.

Share this post


Link to post

What time would the beta of ngx take? I remember Ryan saying that it will be a lot faster then in the past, when it would take months. I believe PMDG has such a intensive alpha test that the beta would not give a lot of problems.

Share this post


Link to post
What time would the beta of ngx take? I remember Ryan saying that it will be a lot faster then in the past, when it would take months. I believe PMDG has such a intensive alpha test that the beta would not give a lot of problems.
Maybe they are already in beta testing and forgot to tell us. LOL.gif

Share this post


Link to post
Maybe they are already in beta testing and forgot to tell us. LOL.gif
we will know when the product will be in beta,coz tthe staff will tell us, then about 12 people will quote that staff comment, and the hundreds will quote those comments to other various forums around the world wide web

Share this post


Link to post
we will know when the product will be in beta,
I'm aware that they are supposed to tell us when beta tests start. They've said that looooooooooooooooooooooooooooooooooooooooooooots of times.:(
coz tthe staff will tell us, then about 12 people will quote that staff comment, and the hundreds will quote those comments to other various forums around the world wide web
Agree 100%.

Share this post


Link to post

Answers as they pertain to us:Alpha: Features are still being coded in. Part of our alpha process other developers would probably call beta - near the end of alpha, we send the airplane out to our internal tech team that consists of real life pilots on the aircraft in question, mechanics, design engineers and so on. This is to make sure we haven't missed anything that their expertise would notice before we consider the plane feature complete.Beta: Aircraft is feature complete and essentially ready for release as far as we the developers can tell. We give it to a handpicked team consisting of average simmers, real life GA pilots, and a few partners in the addon industry to basically go and use like they would had they just bought it as a customer. We're also looking to evaluate performance here, as our testers have everything ranging from bleeding edge systems down to machines that are 4 or 5 years old. If problems arise, we address them. If none do, we release!

Share this post


Link to post
Answers as they pertain to us:Alpha: Features are still being coded in. Part of our alpha process other developers would probably call beta - near the end of alpha, we send the airplane out to our internal tech team that consists of real life pilots on the aircraft in question, mechanics, design engineers and so on. This is to make sure we haven't missed anything that their expertise would notice before we consider the plane feature complete.Beta: Aircraft is feature complete and essentially ready for release as far as we the developers can tell. We give it to a handpicked team consisting of average simmers, real life GA pilots, and a few partners in the addon industry to basically go and use like they would had they just bought it as a customer. We're also looking to evaluate performance here, as our testers have everything ranging from bleeding edge systems down to machines that are 4 or 5 years old. If problems arise, we address them. If none do, we release!
Which is why we should expect this thing to be "Coming SOON!"

Share this post


Link to post
Answers as they pertain to us:Alpha: Features are still being coded in. Part of our alpha process other developers would probably call beta - near the end of alpha, we send the airplane out to our internal tech team that consists of real life pilots on the aircraft in question, mechanics, design engineers and so on. This is to make sure we haven't missed anything that their expertise would notice before we consider the plane feature complete.Beta: Aircraft is feature complete and essentially ready for release as far as we the developers can tell. We give it to a handpicked team consisting of average simmers, real life GA pilots, and a few partners in the addon industry to basically go and use like they would had they just bought it as a customer. We're also looking to evaluate performance here, as our testers have everything ranging from bleeding edge systems down to machines that are 4 or 5 years old. If problems arise, we address them. If none do, we release!
Thank you Ryan for explaining how things work at PMDG. It's really good to know how far you take your aircrafts to ensure that they fly just like the real thing, within FS limits of course.I know people tell you this everyday, but I'll say again:YOU ARE DOING AN EXCELLENT JOB WITH YOUR AIRCRAFTS!!! KEEP IT UP! PS.: I'm also an average simmer, so if there's a blank in your list, just call me! LOL.gif

Share this post


Link to post
Guest 413X3

I'm an average simmer who used to do QA testing, can I join the team? :D

Share this post


Link to post

after Alpha and Beta testing, there is then Charlie testing, which is a cocaine party. :( Al

Share this post


Link to post
Beta: Aircraft is feature complete and essentially ready for release as far as we the developers can tell. We give it to a handpicked team consisting of average simmers, real life GA pilots, and a few partners in the addon industry to basically go and use like they would had they just bought it as a customer. We're also looking to evaluate performance here, as our testers have everything ranging from bleeding edge systems down to machines that are 4 or 5 years old. If problems arise, we address them. If none do, we release!
Meanwhile Tabs' email box fills with requests...I wouldn't want to be a beta tester. I can't imagine how frusterating it must be to test an aircraft where you may have to do a whole flight just to find out it crashes when the autobrakes turn on or something. Then start all over to try to reproduce.

Share this post


Link to post
after Alpha and Beta testing, there is then Charlie testing, which is a cocaine party. :( Al
Which is followed by Delta Testing which involves lot's of Liquor. :))

Share this post


Link to post
Meanwhile Tabs' email box fills with requests...I'll be the first to officially say I wouldn't want to be a beta tester. I can't imagine how frusterating it must be to test an aircraft where you may have to do a whole flight just to find out it crashes when the autobrakes turn on or something. Then start all over to try to reproduce.
I second that. I beta tested for a game once and I hated it. Never wanted to do that again.

Share this post


Link to post

I also think I'd hate to be a beta tester. Would be good because you'd be one of the very first people to see the new add-on, but for me this doesn't really matter. You can't really enjoy the add-on, cause your work as a beta tester is to search for malfunctions.

Share this post


Link to post
I also think I'd hate to be a beta tester. Would be good because you'd be one of the very first people to see the new add-on, but for me this doesn't really matter. You can't really enjoy the add-on, cause your work as a beta tester is to search for malfunctions.
Partially true, but you get to use the product as you would normally use it. The best beta testing is the kind that tests the product as it's likely to be used by an end-user. The real frustration as a beta tester is when things don't work and you have to find a workaround. It isn't a treasure hunt for bugs as much as some might think, you sort of just stumble upon them.

Share this post


Link to post

Testing a flight sim product is a bit like being a test pilot in the real world in some respects; it sounds like a cool job but in reality is often rather tedious because it involves very fastidious study of things in a controlled environment. If you ever meet anyone who is a real test pilot, you find they are finicky and meticulous rather than a dashing 'gung ho' type as is portrayed in movies. So with beta testing, you'd typically fly the plane a lot, but generally always on the same test route with the same weather settings so that experiences between several beta testers will be consistent and can be compared to other similar flights to see if a glitch can be repeated. Then you have to accurately report your findings and such.It's not always exactly like that, but most of the time it doesn't involve flying the thing around anywhere you feel like going and just playing with the thing, which is a mistake a lot of people make when volunteering to beta test, thinking it's just a free way to get a product early. You do indeed get it early, but there is certainly work involved in the deal, so you find you are 'singing for your supper'. That's a bit like doing product reviews too in some respects, people think that's a great way to get freebies too, and then they realise how much research and testing of thing you have to do, followed by writing a big long article about it LOL.So as with the army, think carefully before you volunteer for anything like that.Al

Share this post


Link to post
you find they are finicky and meticulous
Considering that I have underwear on the floor from 3 days ago, I kinda figure I'm out of the running.

Share this post


Link to post
It isn't a treasure hunt for bugs as much as some might think, you sort of just stumble upon them.
I used to think that (you may have noticed by what I said in my post).
Testing a flight sim product is a bit like being a test pilot in the real world in some respects; it sounds like a cool job but in reality is often rather tedious because it involves very fastidious study of things in a controlled environment. If you ever meet anyone who is a real test pilot, you find they are finicky and meticulous rather than a dashing 'gung ho' type as is portrayed in movies. So with beta testing, you'd typically fly the plane a lot, but generally always on the same test route with the same weather settings so that experiences between several beta testers will be consistent and can be compared to other similar flights to see if a glitch can be repeated. Then you have to accurately report your findings and such.It's not always exactly like that, but most of the time it doesn't involve flying the thing around anywhere you feel like going and just playing with the thing, which is a mistake a lot of people make when volunteering to beta test, thinking it's just a free way to get a product early. You do indeed get it early, but there is certainly work involved in the deal, so you find you are 'singing for your supper'. That's a bit like doing product reviews too in some respects, people think that's a great way to get freebies too, and then they realise how much research and testing of thing you have to do, followed by writing a big long article about it LOL.
I've recently watched a documentary about a test pilot. He spoke exactly as you said. Many people who came to talk to him usually started a conversation by saying that his job must be awesome, thrilling and exciting, but in the documentary you see that the thing is not so funny. Every flight is meticulously planned to the smallest details. The conference made by the test pilot, the engineers and other staff members before every lasts for hours. The flight itself is not randomly made. There are a bunch of steps that have to be followed, each at a time. If one step is not accomplished, the whole flight has to be interrupted and whatever problem that shows up has to be immediately fixed. Once its done, all the process has to be accomplished again.
So as with the army, think carefully before you volunteer for anything like that.
Will do, thank you Al for the advice. LOL.gif

Share this post


Link to post
Answers as they pertain to us:Alpha: Features are still being coded in. Part of our alpha process other developers would probably call beta - near the end of alpha, we send the airplane out to our internal tech team that consists of real life pilots on the aircraft in question, mechanics, design engineers and so on. This is to make sure we haven't missed anything that their expertise would notice before we consider the plane feature complete.Beta: Aircraft is feature complete and essentially ready for release as far as we the developers can tell. We give it to a handpicked team consisting of average simmers, real life GA pilots, and a few partners in the addon industry to basically go and use like they would had they just bought it as a customer. We're also looking to evaluate performance here, as our testers have everything ranging from bleeding edge systems down to machines that are 4 or 5 years old. If problems arise, we address them. If none do, we release!
That's very clear Ryan,My item being MD11,I have not been part of the Alpfa team as a real pilot that has an MD11 rating, and not been able to indicate imperfections (some call it bugs) in the Alpfa fase.Also for the Beta part of the production I had no way to indicate improvement options.(or it must be the lateral FPV presentation on the PFD which is correct on the MD11 now, compared to the 747)Once the product was released, I signed the license agreement to download and have the 'right' to use the product as it is.Since using the product (including patches) as it is, I admire PMDG for producing a real non-Boeing, but American made product, and see very detailed conformity to the real plane.So Ryan: If I make remarks on the forum after the product is released, I now understand I'm much too late.From the above I understand not to make remarks unless invited. Thanks for making this clear.Business must go on indeed, check the last forum reference on facebook is december 02.Regards,Harry

Share this post


Link to post
×
×
  • Create New...