2 hours ago, St Mawgan said:

Yes, of the 97.4 % male, mainly young, mainly American, mainly airline pilot, Navigraph using section of the flight sim population who can be bothered to complete surveys, the following information is probably true ..... 

(I can only tick one of those boxes)  

Mainly European. 

11 minutes ago, hansb57 said:

Mainly European. 

By overall numbers, true. I can tick two boxes then 🙂

55 minutes ago, JBDB-MD80 said:

Sounds like some hurt feelings were made here by this poll. Its a Navigraph poll so what did you expect? It was done by folks who actually use Navigraph or Not or even cared. You had the option to participate or not and it took me less than 5 mins to complete.

BTW you don't even have to use Navi to participate in this poll.

A small group are discounting the Navigraph Survey, because it produced results that are similar to the Stats on Steam.  2024 users are outnumbered  by 2020 users by a significant amount, and the trends are in the negative direction for 2024. 

1 hour ago, ryanbatc said:

No ones hurt, but it's important to let people know about survey bias.  I think NG isn't very clear on that part.

It's basically like medical studies conducted by big pharma lol...

The data is still relevant but it's very biased.

Well if anyone was not bothered about what the results were then why did they even come in this thread in the first place. Do not mistake facts with Bias. People need to accept the fact what the people who took the time and voted for in this poll and the ones who did not should have nothing to say.

Edited April 17, 20251 yr by JBDB-MD80

7 hours ago, ryanbatc said:

Well yeah we already know it's biased survey towards airliner people - since they would be more likely to use navigraph.

Yes, but the concern is that many folks will believe those survey results as broadly representative and it could then become somewhat of a self-reinforcing "myth".

I.e., developers see this and think that they've got to make airliners instead of just very good addons...?

Or simmers look at it and think there's no point (or value) in flying anything BUT airliners, etc.

Extreme examples, for sure, but in some respects, surveys like this (though valuable), can create bias.

An overly simplistic example is that, according to this survey, the Zibo 737 is "the best" (i.e., most popular) airliner addon for X-Plane. But there's no weighting assigned to the fact that it's popularity has much to do with the fact that it's free vs the next "best", the $90 Toliss addons.

And does anyone REALLY believe that the iniBuilds store is outselling all the others...?

And so on.

Anyhow, I'm sure a good percentage of us aren't so easily swayed, but still...

6 hours ago, Bobsk8 said:

It is amazing how you can invent "facts" and post them on this forum, trying to convince others that your made-up "facts" are true. 

Bob, his statement is literally Truth.

Please see Navigraph's own statement on p93:

"4.1. First, a Word on Sampling Bias and Validity

Since respondents were not selected through a random sampling technique but instead chose to participate voluntarily, there is a potential for bias in the collected data. This is a common limitation in surveys where participants are self-selected rather than randomly chosen based on the survey’s intended focus.

It is important to emphasize that this chapter simply summarizes the survey data rather than drawing definitive conclusions. Because we cannot confirm whether the dataset accurately represents the broader flight simulation community, we refrain from making absolute claims. For this reason, confidence intervals and margins of error were not calculated."

4 hours ago, BrammyH said:

The survey got wide enough press I'd expect some of the bias to smooth out. I forget if you needed a Navigraph account to fill it out. Since the CSV file has a comment where the anonymized the data I think I did. That will also account for straight Simbrief users too.

I'd be curious if MS ever posts any data bout this stuff at something like their Expo talk. They've talked usage before but I don't have any details.

I do think airliners are over 50%. Looking through even Reddit it seems skewed on airliners.

IMHO, I doubt it for the reasons linked in my response to Bob above (i.e., Navigraph's own words on unintended bias).

We can also look at things like Xbox users only being 2% of this survey, yet reportedly as much as 50% of the of the overall MSFS user base.

Or the most popular GPU being an RTX 4090...REALLY? 😉

Back in the day, somebody at Laminar Research published v11 user data based on auto-reported data from the installed sims (the opt-in button when installed). What it showed was that a (slight) majority of users flew Default aircraft, and that the Type utilization was almost exactly split between GA and Airliner.

Found it: http://dashboard.x-plane.com/

Just looking at the 3rd Party addons list shows this data is long out of date (like Toliss doesn't even appear at all), but the overall usage breakdown is illuminating, and highlights how vastly different results can be when impartially collected from a very large, representative sample than from a self-selected sample of "super fans" like us.

 

6 minutes ago, UrgentSiesta said:

Or the most popular GPU being an RTX 4090...REALLY? 😉

18.3% of respondents have an RTX 4090 installed? Is that 18.3% of 23K users? That's 4200 users with RTX 4090 cards!! That statistic right there is about as far away from the average simmer as you are likely to get.

4 hours ago, BrammyH said:

I'd be curious if MS ever posts any data bout this stuff at something like their Expo talk. They've talked usage before but I don't have any details.

 

They did report in 2024 on MP sales across all of 2023.

Again, the (unintentional) bias of this survey shows through via examples like the top selling, "Platinum" addons being these . Note that there is little correlation to the Navigraph results other than airliners (even the 2023 survey):

•  737-700 (PMDG Simulations) – $69.99
• 737-800 (PMDG Simulations) – $69.99
• Airbus A321NEO (LVFR) – $29.49
• Airbus A330-900 NEO (LVFR) – $34.99
• Antonov AN-225 (Microsoft / iniBuilds) – $19.99
• ATR 42-600 / 72-600 (Microsoft / S&H Software) – $19.99
• Boeing 777-300ER (Captain Sim) – $19.99
• Concorde (DC Designs) – $40.49
• F-22A Raptor (Top Mach Studios) – $35.99
• F-35 Lightning II (IndiaFoxtEcho) – $29.99
• Transall C160 (AzurPoly) – $32.99
•  

But if you include the "Gold" and "Silver" addons from the list, even the slight airliners correlation practically disappears from the Navigraph survey.

(this particular list is was re-published by msfsaddons.com and is still available on their site).

I was watching an average simmer messing around with MSFS2024 on Twitch a while back. A guy some of you may have heard of who goes by the name T-Pain. He was flying the 172. He knew enough to be able to program a flight plan into the G1000. I would argue that anyone who took enough time to learn the Garmins isn't exactly a casual user. But their flight time in sims is probably much less than most people on these forums. They're not paying for Navigraph, they're not taking surveys, and they're not posting about their sim of choice online. They're just flying when they have free time and the urge to fire up the sim.

Over 90% of the survey respondents fly airlines. PMDG makes the most popular airliners for MSFS. Is anyone surprised by these results? Was anyone delusional enough to think the PMDGs would be compatible with 2024 within the first few months? The survey results are not surprising at all given the demographic, but they also don't apply to the "average simmer".

19 minutes ago, Christopher Low said:

18.3% of respondents have an RTX 4090 installed? Is that 18.3% of 23K users? That's 4200 users with RTX 4090 cards!! That statistic right there is about as far away from the average simmer as you are likely to get.

It's even more nuanced than that:

24,000 people started the survey, but only 16,000 finished it.

And so Navigraph excluded ALL responses from incomplete surveys (which is the technically correct action to take).

So, while the percentages of the 16k are still accurate, the actual "N" is much lower.

There's that "lies...and statistics" quote coming to mind again 😁

5 hours ago, Bobsk8 said:

A small group are discounting the Navigraph Survey, because it produced results that are similar to the Stats on Steam.  2024 users are outnumbered  by 2020 users by a significant amount, and the trends are in the negative direction for 2024. 

How are you getting "trend" out of it?   It's a snapshot in time, isn't it?

31 minutes ago, UrgentSiesta said:

There's that "lies...and statistics" quote coming to mind again

Yeah, and that hard part is the Navigraph set is best data we have, but it's flawed as all get out.

Even thinking about it, I can't really come up with something that collects data that doesn't still favor airliners. Volanta was a maybe, but still, same bias. Volanta might have slightly more GA, but I doubt I'd be a good slice. Same with Vatsim. Then I thought of FSexp doing a survey of attendees, but, yeah small data set.

It'd be fantastic to look at MS's data for sure.

I like to think that I know a little something about conducting research and I can say that a lot of you are reading much more into the survey and therefore coming to all kinds of nonsensical conclusions. First. It is a casual survey. It is not attached to any hypothesis, so it doesn't try to correlate or refute anything. It doesn't attempt to offer a margin of error - therefore has no confidence interval. But there is no need for a CI if Navigraph is not trying to test a hypothesis, so there's that. 

As far as population size - 23,000 is MORE than enough to be a valid sample size of the representative FS user population. As a matter of fact, to conduct a survey that would test a hypothesis to a confidence interval of 95% (5% margin of error) you'd only need a sample size of 385 to accurately reflect a population of 10,000,000. 

I also see people throwing around "bias". This is a simple RSVP survey. You can't have BIAS when you don't have a hypothesis to be biased about. It's like someone putting out a survey asking what kind of cake you like and listing 10 types. Sure, there will be someone whose choice is missing because there are hundreds of types of cakes, but you have to draw the line in the choices of the survey somewhere, but that does not cause / infer bias, or invalidate the survey.

And if you're going to scream bias then please point out to the TYPE of bias? There are lots of them. Researchers go out of their way to filter bias, but in this case, what bias would be filtered. We go back to the original premise - Navigraph is not correlating a hypothesis so what bias would be relevant? I'd like to know!

It's the modern trend - if you don't like the data results presented, scream bias, scream no one asked me, scream not enough people were surveyed, scream that the results don't line up with my anecdotal observations. 

Pretty hilarious actually.

 

 

Edited April 17, 20251 yr by Mike T

16 minutes ago, Mike T said:

You can't have BIAS when you don't have a hypothesis to be biased about. It's like someone putting out a survey asking what kind of cake you like and listing 10 types.

I am one who used bias. What I meant was the collector of the data is the leading charts and flight planner software. So, people who would fill out their survey are likely customers, and that customer base skews toward airliners.

Now, I do feel the impressions people have of 2024 if you surveyed a group of people who lean GA you'd get the same numbers. Here is the definition of bias I was using. (source: https://www.pragmaticinstitute.com/resources/articles/data/5-common-bias-affecting-your-data-analysis/)

Navigraph themselves indicated the selection and collection bias.

Selection bias is an error that occurs when the population samples do not accurately represent the entire target group or represent skewed insights. This means that the data is selected subjectively rather than objectively.

Selection bias can arise due to poor study design if the sample taken was too small, or the sample is simply not randomized.

An example of selection bias is a study on the health effects of alcohol on the general population. If researchers recruit participants exclusively from bars and nightclubs, they are selecting participants whose behavior may not represent the full population