The FSX Survey is flawed

[Guest cliffie1931]

I believe that the results of the survey will not correctly mirror the feelings of those of us that have completed it.I find it impossible to correctly indicate the answers to the questions about both "...the experience " and the "...frame rates" when faced with the stark choice of either "Good" or "Poor".Had I been given the choice of "Satisfactory" or "Acceptable" I would have opted for that middle ground. Suerly there is a grey, undefined area between good and poor............Cliff

[Guest mariosunshine]

I agree on that. Would be nice to have something like "satisfactory"

[FTD1949 0]

RE: The FSX Survey is flawed Some folks just aren't happy with anything. Craig

[Hornit 3]

Actually I think the survey is perfect. If you are HONEST, it forces you to make a very macro type choice. If you nitpick each item with too many choices you dont get a real picture of what is going on. Personally after taking it and viewing the results, it appears very accurate to me. I might change one or two questions but remember this is a survey taken inside a tight community of EXPERT users. It does make a difference. We should be able to make these types of choices If your being honest.Hornit

[Guest FxF3]

I find nothing wrong with the survey. I'm glad we got one.

[Guest tdragger]

I think it would be more accurate to simply say it's "unscientific". Having studied statistics and been involved in survey and test design I can tell you it is very difficult to design any survey that eliminates bias. Therefore this survey (like any otehr informal poll) is interesting but not terribly useful in trying to extrapolate the results to any population greater than those who responded.

[Guest dswo]

>I think it would be more accurate to simply say it's>"unscientific". Having studied statistics and been involved in>survey and test design I can tell you it is very difficult to>design any survey that eliminates bias. Therefore this survey>(like any otehr informal poll) is interesting but not terribly>useful in trying to extrapolate the results to any population>greater than those who responded.The AVSIM survey may be "flawed," but with 3,000+ respondents I don't think it can be dismissed as "informal," either.

[mgh 89]

The survey represents the views of those members of AVSIM who bothered to respond. It can't be extrapolated to give any indication of the general users of FSX. It's also informal because there's no way to ensure that it even fully represents the views of AVSIM members as a whole - it only represents the views of those who responded. A formal survey would ensure that the views of a fully representative cross-section wre obtained.BTW this isn't a critisism of AVSIM - it's merely pointing out the limitations of this type of self-selecting survey which is all that AVSIM can use.

[Guest dswo]

>It's also informal because there's no way to ensure that it>even fully represents the views of AVSIM members as a whole ->it only represents the views of those who responded. A formal>survey would ensure that the views of a fully representative>cross-section wre obtained.If the survey had only 100 responses, sure. But with more than 3,000? Most political polls, including the Gallup polls, are based on a sample only half that size, and that's to estimate something for the whole U.S.** Curious how that can work? Read this: http://media.gallup.com/PDF/FAQ/HowArePolls.pdf

[greengriffon 51]

>Had I been given the choice of "Satisfactory" or "Acceptable">I would have opted for that middle ground. >>Suerly there is a grey, undefined area between good and>poor............Grey, undefined areas, and middle ground don't help anyone get a good overview of anything.Sometimes very black and white answers work better, as they force you to take a firm stance in your response.

[W2DR 1,859]

But, scientific poll sampling techniques are designed to eliminate, as much as can be, bias. Even though the sample is small, the sample population in a properly run poll is very carefully chosen to insure the validity of the findings. A poll such as we have here can in no way be construed as scientific and the results are only meaningful as they reflect the opinions of those AVSIM members who chose to respond. Extrapolating these results to be representative of the universe of FSX users would be a serious misuse of the data and will not yield anything close to a valid finding.Doug

[Guest dswo]

> the results are only meaningful as they reflect> the opinions of those AVSIM members who chose to respond.No, with a sample this big, the sample will be representative of the whole population (in this case, AVSIM users).>Extrapolating these results to be representative of the>universe of FSX users would be a serious misuse of the data>and will not yield anything close to a valid finding.Agreed, in principle. But statistics -- on this scale -- isn't an all-or-nothing game. The AVSIM population is a subset of the general population, not an alien species. Sampling AVSIM members won't tell you everything about the larger population, but it will give you a hint of where the wind's blowing.

[vgbaron 3,496]

I say we insist that Tom provide a patch at once!VicVisit the Virtual Pilot's Centerwww.flightadventures.comhttp://www.hifisim.com/Active Sky V6 Proud SupporterRadar Contact Supporter: http://www.jdtllc.com/

[jbroadwa 0]

David,What you are arguing is that 3000+ self-selected respondents are a good sample. In the Gallop PDF, they have numerous examples where the number of respondents are large, but the results are still biased. This is a passage from the PDF."The first one thousand people streaming out of a Yankees game in the Bronx clearly aren't representative of all Americans. Now consider a group compiled by selecting 1,000 people coming out of a Major League Baseball game in every state in the continental United States -- 48,000 people! We now have a much larger group -- but we are still no closer to representing the views of all Americans than we were in the Bronx. We have a lot of baseball fans, but, depending on the circumstances, these 48,000 people may not even be a good representative sample of all baseball fans in the country -- much less all Americans, baseball fans or not."The sampling must be done randomly as stated in the Gallop PDF to obtain confidence in the poll results. Since the respondents chose themselves for this poll, it is not random and thus may have some bias. Note I used the word "may" in the previous sentence. The poll may actually represent the true opinions of everyone who visits AVSIM and has "flown" FSX, but we cannot know this scientifically becase of the self-selection of respondents. Sorry for the long post. My career is in statistics and I just couldn't help responding (note the bias as I'm a self-respondent). Thanks,Joshua

[Guest dswo]

Joshua, you're quite right about the random vs. self-selected sample. Frankly, I hadn't thought about that. Still, for such a small population (meaning here, "regular AVSIM visitors), 3,000 is a very large sample. Now I'm off to watch election returns!