How can one view the results of the Developers Survey?I've only seen the interim results once, and that was only AFTER completing the entire Survey.Since I cannot complete it again, how does one view the final tally? ;)

Click on the number of responses (nnnn responses).

Try this, Bill: http://www.avsim.com/poll/stats.php?sid=15&st=a&vw=a works for me, but do not know if it is current, I just know the link works.I'm trying to figure out why it is not in alphabetical order as to developer names. It seems to start off that way, then it goes crazy. No wonder Ron initially thought that ESDG was not in the survey!Now, when you consider that one of the answer choices is:"I have never purchased one of their products so cannot rate them", this means that for some developers it can make that bar look far too long in relation to the other bars, and folks just glancing at the bars can get a false opinion. Example: In DF's case, our Excellent bar was about the same length as our "Never purchased" bar, but in ESDG's case, the Excellent bar is about 1/4 the size of the "Never purchased" bar. Thus everything "looks" weighted at the low end.There are just some developers / products that not everyone is going to purchase, often due to the nature of the product. It has nothing to do whether it is good or bad. Example: "Radar Contact". It got a 7.1% for excellent product quality, yet 84.8% under "never purchased". This could lead folks to think there is something wrong with the product, because so many never purchased it when, in reality, not everyone wants/needs an add-on ATC program.Heck, I never purchased Radar Contact, and as such had to answer that I never purchased it, yet even though I do not own it, I would stake my reputation on the fact that it is "excellent"!Also, by eliminating the "never purchased" category you can then make the bars relative in length.Also...Why is there a need to ask if something is "poor" or "very poor"? A bit of overkill, if you ask me. Just have 5 categories. Excellent, Very Good, Good, Fair, Poor. That should do it. Otherwise, why not have "very, very, very, very poor". I am certain you will find someone to vote for that one for any developer.If you are going to have that "very poor" category, then you should have "Very Excellent, Excellent, Very Good, Good, Fair, Poor, Very Poor".Where does it stop? It should stop at 5 categories, not 6, as currently the very low end is weighted greater than the high end, as there is no "Very Excellent".Anyway, when I look at the responses (1524) vs. sales of our products, it is just a survey, not scientific, and I totally discount the "Very Poor" category.I like the idea of this survey, but it should be in alphabetical order as to developer name, and the "very poor" category should be eliminated, unless you include a "very excellent" category.When it comes to "product quality" DF got 0.2% in the "very poor" category, while PMDG doubled that, and they got 0.5% as "very poor", and ESDG got 0.7%. Do I believe any of that, no, and neither should anyone else. That "very poor" category should be eliminated.Or, as a friend of mine from a popular FS development company said (and I agreed with him) when we discussed the survey, the "very poor" category just served for those "competitors" to knock each other. I mean, 3 people voted DF's product quality as "very poor", care for me to name them? ;-)At least when I answered the survey, I answered it honestly, but that meant for many developers folks having to load up the "never purchased" category, as they never purchased the product. What's the point?Take a survey of automobile owners, and look at Ford vs. Mercedes-Benz. Yeah, I guarantee you far more folks have not purchased a Mercedes-Benz than a Ford, but should that mean anything negative / positive toward M-B? Nope! It's irrelevant there just as much as it is in FS. Eliminate the "never purchased" category and "very poor" category. Let's just hear from those who have purchased, and keep it to 5 categories, as we all know we do not sell to everyone, and we are never going to change that.A nice first start for this survey, but certainly some tweaking is in order.Happy New Year, Fr. Bill! :-)Regards,http://www.dreamfleet2000.com/gfx/images/F...R_FORUM_LOU.jpg

The "never purchased" category should be kept but not included in the displayed results.That way people still have a valid option to choose on each question, without it impacting the final score.Whether such fine grained control is even possible in the used polling software I don't know ;)

I'd already twice in these forums raised the point about including the "I have never purchased one of their products so cannot rate them" responses in the calculation of the percentages. This distorts the percentages significantlyI never received any reply.

Thanks to everyone for the kindness. There is no indication on the page stating how to access the results, so I guess it was blind luck that mgh found the hidden link... ;)I fully agree with the comments regarding the weighted biasing of the totals because of the "very poor" and "never purchased" being included.Like yourself Lou, I tried to be honest and objective while answering the questions... I did not use this as an opportunity to artifically skew the results for competitor's products.It's a good start, and in fact does offer "some" useful feedback, but certainly must be flavored with a large dose of sodium chloride... ;)The only useful information provided by the "never purchased" response is that it illustrates one's market penetration. Of course, since the vast majority of AVSIM users are "heavy iron pilots," that too must be taken with a huge dose of salt......especially since the absolute number answering in that category aren't consistent... :-beerchug

I dunno guys,The level of measurement in this survey is ordinal and, although a Likert-style breakdown would have been optimal, the effect of the choices is Likert-like. In Likert you get equal number of increasingly positive, a neutral and equal number of increasingly negative. While some quibble over language is possible I do believe the following does represent equally ascending and descending measures of satisfaction and quality:ExcellentVery GoodGoodFairPoorVery PoorIf anything, the choice of language in this AVSIM poll is skewed towards the positive. Only Poor and Very Poor truly capture negative sentiment when the strictest interpretation and etymology of the words is considered. You see, "Excellent" can be said to mean:"Of the highest or finest quality" (dictionary.com)Thus, there is none higher.Whereas with "Fair:""Moderately good; acceptable or satisfactory" (dictionary.com)"Fair" is not neutral in meaning and can be said to fall on the side of the positive. Thus, we could also argue that the ordinal scale could be increased to a Likert-like dimension of seven items where a true neutral score is offered. The truth is, "Very Poor" is kind as the truest opposite of "Excellent" would have been "Inferior," which really invites a degree of cruelty (not that some developers are not deserving of it).Language is fun isn't it?J-

Interesting comments, but I guess you have never been on a board of directors?You should have an odd number of choices, not an even number. Where's the "middle"? There is none in this survey!Now, let us eliminate the "very poor category" and it looks like this:ExcellentVery GoodGood >>>>> This is the middle, the average.Fair PoorThis is how it should be.Excellent offsets Poor; simple as that. Forget the dictionary definitions. You see, it is as simple as a "1 to 5" rating. 3 (good) is in the middle. 1,2 and 4,5 are on either side.Or, better yet, drop the words, and just use numbers, 1-5. That is probably better!No offense, but I do not agree with what you are saying at all.Look, it's not a bad survey, but it could certainly use some tweaking.Ultimately, it is not working quite right, and I do not blame AVSIM for this, as I am sure they will tweak it in the future. It's their first go at it, and they had to deal with the software they had at hand. That's certainly understandable, and I am not knocking them in any way in this regard. The key word here, my good man, is "relative", and in this survey, folks scanning the "bars" simply do not get a good idea as to overall quality. In essence, if 10 people purchase product "A" and five think it is "excellent", and 1000 people purchase product "B" and 500 think it is excellent, that means the "bar" for both products in the "excellent category should be the same length, meaning 50% thought each product was excellent. In this survey they are not, and that is where the survey fails.As I said in my first post above, the nature of some products means that they will always be more popular (more people will buy them) than others, that does not mean they are better.The key here is the word "relativity". One should be able to scan the survey, look at the length of the bars / percentages, and they should all be relative. With this survey they are not. The "never purchased" choice throws everything off, and some folks might think that "never purchased" means, well, "very, very, poor", as it follows the needless "very poor" category.I still feel that the "never purchased" category should not be included in the total, although it could still otherwise remain, but perhaps at the TOP of the each survey category, and the "very poor" category is needless.Read my first post and look at the survey and you will see what I mean.I think you said it all when in reference to "very poor" that you said "(not that some developers are not deserving of it)." ;-) If that's the case, them we should have a "very, very, very, very, very poor category", as you can be certain that someone will vote for it for each and every developer. The same for "very, very, very, very, excellent". "Very poor" is NOT the opposite of "excellent", "poor" is, and "good" is in the middle. Very poor is just another "stone" to throw.Again, you might think that "fair" should be in the middle, but this is only due to the "never purchased" choice, which has nothing to do with quality and, as such, should not influence the relative rankings in the other categories. Finally, based on how well DF did, I have NOTHING to complain about as to this survey. I LOVE it! I am saying this out of fairness to all developers, and in hopes that the next survey will be structured differently. :-)Regards,http://www.dreamfleet2000.com/gfx/images/F...R_FORUM_LOU.jpg

On a board of directors... yes.I agree on the requirement of an odd number of choices and moreover agree that numbers would have been good, but they'd have to establish a middle.Best would have been a 7-point scale.better765neutral4worse321The numbers are relative, so they'd fit your call for a relative assessment. However, my diatribe centered solely around the words actually used and the fact that there were more positive than negative choices.

It was interesting reading the survey results. Being one who researches a developers product quality and customer support via the forums before making an add-on purchase, the results for the developers that I was familiar with were as expected.I agree that the number of people who have used a product should not factor into the other ratings. To evaluate the consensus on any particular category you have to look at the shape of the curve created by the bars. If it extends to the right as you go up the charts, the vendor was favored better in that category than if the bars peaked lower down in the chart. The bars are very short for the vendors with less market penetration in the sample, making it more difficult to see a curve. I do feel that the number of people who have used the product should still be in the results, just not factored into the others.OK, I'll stir the pot on the rating system. Should the middle selection on the rating system be "good" or "average"? "Good" suggests that you are somewhat pleased in this category, whereas average denotes that it is similar to the others in this category wether you think that is good or bad. I am also a board member....plus president, treasurer, staff, and janitor, but I'll leave the number of points in the scale to you other executives.Thanks for putting the survey together Avsim, I hope it continues.Lou, what do think about putting a similar survey over at Flightsim? Do you think that there would be any difference in readers opinions between the two sites?Ted

Apart from the rating system, one other issue is which developers should be included.Frankly, I had never heard of Calibre and Majestic before, and, judging from the results, they are almost unknown in the community (95% or more 'never purchased').Some well-known developers, such as HifiSim (Activesky series) and FS2Crew (crew simulations) weren't included-even though they do have their support forums right here at AVSIM. Others come to mind, such as SimFlyer's, FlyTampa and more.Clearly, a choice has to be made-the survey would have become too long otherwise. One way to change this is to provide a list of as many developers as possible and allow users to vote only for those that they have experience with, by checkmarking them before starting the survey. This would also eliminate the need for the 'have not purchased' question.Best,Martijn

>I still feel that the "never purchased" category should not be>included in the total, although it could still otherwise>remain, but perhaps at the TOP of the each survey category,>and the "very poor" category is needless.>Finally, based on how well DF did, I have NOTHING to complain>about as to this survey. I LOVE it! I am saying this out of>fairness to all developers, and in hopes that the next survey>will be structured differently. :-)LOL! I was thinking just that while I was reading your comments, Lou... Most people will simply look at the relative "length of the bars" and not take into account the numbers at all. After all, that's the purpose of a graph!The number of "never purchased" responses should only have been used to "normalize the distribution," so that the graphical plots are meaningfuly related to one another: i.e., they are scaled identically!If someone simply glances at the relative lengths of the bars for Vendor A and Vendor B, Vendor B's bars are "longer" therefore they must be MUCH better than Vendor A.However, once the results are "normalized," the results may well be just the opposite. :)

Have to agree about the alphabetical order Lou, it confused a few folks at the time:-)Also agree on the other points as well. The K.I.S.S.Principal would be most helpful:-)