34 minutes ago, BrammyH said:

I am one who used bias. What I meant was the collector of the data is the leading charts and flight planner software. So, people who would fill out their survey are likely customers, and that customer base skews toward airliners.

Now, I do feel the impressions people have of 2024 if you surveyed a group of people who lean GA you'd get the same numbers. Here is the definition of bias I was using. (source: https://www.pragmaticinstitute.com/resources/articles/data/5-common-bias-affecting-your-data-analysis/)

Navigraph themselves indicated the selection and collection bias.

Selection bias is an error that occurs when the population samples do not accurately represent the entire target group or represent skewed insights. This means that the data is selected subjectively rather than objectively.

Selection bias can arise due to poor study design if the sample taken was too small, or the sample is simply not randomized.

An example of selection bias is a study on the health effects of alcohol on the general population. If researchers recruit participants exclusively from bars and nightclubs, they are selecting participants whose behavior may not represent the full population

That's not how research works.

You're insinuating selection bias to correlate an outcome that was never hypothesized in this survey. What you just did was state your own hypothesis thus: "The collector of the data is the leading charts and flight planner software. So, people who would fill out their survey are likely customers, and that customer base skews toward airliners. ..the impressions people have of 2024 if you surveyed a group of people who lean GA you'd get the same numbers." so you in fact have OUTCOME bias because you want the survey data to somehow show more MSFS 2024 users.

As I said, the research is the research - it is not designed to any level of fidelity to deliver accuracy with any stated margin of error. If you want to take the data and design a survey that would filter both your outcome bias and any perceived selection bias and include a margin of error then that would be the way forward. But its not fair to ding Navigraph for something they never set out to do.

The Navigraph survey data only representative of the people who took the survey - and nothing more.

 

 

 

 

Edited April 17, 20251 yr by Mike T

Msobo has failed simmers worldwide. Their employment contracts should be ripped up.  We need new.

26 minutes ago, Mike T said:

he impressions people have of 2024 if you surveyed a group of people who lean GA you'd get the same numbers." so you in fact have OUTCOME bias because you want the survey data to somehow show more MSFS 2024 users.

First, I am not the camp of either sim. I am agnostic on them. I fly both. I am not someone who is trying to convert folks to 2024, nor trying to make 2024 seem better than it is. By the same token, I do not feel like 2020 is a flawless gem of beauty. I've done my own curiosity analysis of the raw data. I do enjoy a good discussion of data. That 2020 beat 2024 is not surprising to me. Good or bad launch, it was too close to release to capture folks who can move over full time. Again, this is why I am eager for next year's results when we have a full year of 2024.

I do data analysis as part of my day job so I am able to look at results for what they are. I don't have an interpretation bias. I am actually quite enjoying digging through it to kinda correlate someone who responded one way on a sim choice with things like airplanes flown.

I may not have been clear, though. The survey showed that a majority of the respondant's flew airlines. Those respondants also prefer 2020 by about a 2:1 margin. I have no opinion on that result. The data is the data. What I meant was, if this survey was done where GA was 90% of the respondents, I would expect the ratio to be close to 2020 vs 2024 responses in the Navigraph survey.

I think you thought I was trying to add votes to 2024 in some manner and that's not the case.

Edit: I do think 23k respondants is a good sample set of people who take simming seriously enough to fill out a survey. I am not dinging Navigraph at all. Instead I think they have done very good work here.

Edited April 17, 20251 yr by BrammyH

2 hours ago, Mace said:

How are you getting "trend" out of it?   It's a snapshot in time, isn't it?

The Steam data  shows 2024  users declining month by month, that is a "trend". 

Never mind.

Edited April 18, 20251 yr by BrammyH

4 hours ago, Christopher Low said:

That statistic right there is about as far away from the average simmer as you are likely to ge

That's right and yet that survey group preferred MSFS 5 or greater to 1 over Xplane.  I have to believe anyone with that hardware has to be a serious simmer, whatever that might mean beyond will to spend that kind of cash on flight simulation.

3 hours ago, Mike T said:

I like to think that I know a little something about conducting research and I can say that a lot of you are reading much more into the survey and therefore coming to all kinds of nonsensical conclusions. First. It is a casual survey. It is not attached to any hypothesis, so it doesn't try to correlate or refute anything. It doesn't attempt to offer a margin of error - therefore has no confidence interval. But there is no need for a CI if Navigraph is not trying to test a hypothesis, so there's that. 

As far as population size - 23,000 is MORE than enough to be a valid sample size of the representative FS user population. As a matter of fact, to conduct a survey that would test a hypothesis to a confidence interval of 95% (5% margin of error) you'd only need a sample size of 385 to accurately reflect a population of 10,000,000. 

I also see people throwing around "bias". This is a simple RSVP survey. You can't have BIAS when you don't have a hypothesis to be biased about. It's like someone putting out a survey asking what kind of cake you like and listing 10 types. Sure, there will be someone whose choice is missing because there are hundreds of types of cakes, but you have to draw the line in the choices of the survey somewhere, but that does not cause / infer bias, or invalidate the survey.

And if you're going to scream bias then please point out to the TYPE of bias? There are lots of them. Researchers go out of their way to filter bias, but in this case, what bias would be filtered. We go back to the original premise - Navigraph is not correlating a hypothesis so what bias would be relevant? I'd like to know!

It's the modern trend - if you don't like the data results presented, scream bias, scream no one asked me, scream not enough people were surveyed, scream that the results don't line up with my anecdotal observations. 

Pretty hilarious actually.

What's hilarious is that you don't seem to have read Navigraph's own disclaimer, nor many other key pieces of data included in the survey.

I'm not sure anyone's taking issue with the survey, but rather with many folks taking the survey as gospel representative of the general flight simulation community.

And you're conflating the word "bias" with "intent". Again, Navigraph themselves have addressed this real issue. So if you want it pointed out, go read.

And you've also missed the part about the N not being 24,000, but rather 16,000.

as to statistical validity of (either) sample, AGAIN, Navigraph themselves have explained how this survey sample likely is not representative, and it has nothing whatsoever to do with (either) N.

Go read up - you're way behind.

 

Edited April 18, 20251 yr by UrgentSiesta

2 hours ago, Mike T said:

That's not how research works.

so you in fact have OUTCOME bias because you want the survey data to somehow show more MSFS 2024 users.

As I said, the research is the research - it is not designed to any level of fidelity to deliver accuracy with any stated margin of error. If you want to take the data and design a survey that would filter both your outcome bias and any perceived selection bias and include a margin of error then that would be the way forward. But its not fair to ding Navigraph for something they never set out to do.

The Navigraph survey data only representative of the people who took the survey - and nothing more.

OMG...

Yet you fail to see any confirmation bias in all these posts...?

Even after Navigraph have pretty much laid out all the reasons for caution for us (as they always have)?

Statistics are not Physics, so please stop pulling the "Science!" Card.

I'd suggest you take your final statement above to heart.

Edited April 18, 20251 yr by UrgentSiesta

6 hours ago, UrgentSiesta said:

What's hilarious is that you don't seem to have read Navigraph's own disclaimer, nor many other key pieces of data included in the survey.

I'm not sure anyone's taking issue with the survey, but rather with many folks taking the survey as gospel representative of the general flight simulation community.

And you're conflating the word "bias" with "intent". Again, Navigraph themselves have addressed this real issue. So if you want it pointed out, go read.

And you've also missed the part about the N not being 24,000, but rather 16,000.

as to statistical validity of (either) sample, AGAIN, Navigraph themselves have explained how this survey sample likely is not representative, and it has nothing whatsoever to do with (either) N.

Go read up - you're way behind.

 

In your haste to ramble with pseudo-intellectual jibberish, you typed all that nonsense to agree with what I said in the first place. The survey has no statistical validity; it's just a survey, and people are reading way too much into it.

There, I helped you across the finish line - you're welcome.

Edited April 18, 20251 yr by Mike T

6 hours ago, UrgentSiesta said:

Statistics are not Physics, so please stop pulling the "Science!" Card.

You're going to upset many tens of thousands of statisticians and Ph.Ds (social scientists), who are all somehow confused that stats is a science. I guess we can always go back to school and learn Physics per your advice. But IF I learn Physics, THEN can I pull the "Science" Card?

 

1 hour ago, Mike T said:

You're going to upset many tens of thousands of statisticians and Ph.Ds (social scientists), who are all somehow confused that stats is a science. I guess we can always go back to school and learn Physics per your advice. But IF I learn Physics, THEN can I pull the "Science" Card?

 

Remind me again what this has to do with flying an imaginary plane around ? 

statistics and survey results ?

29 minutes ago, Ianrivaldosmith said:

Remind me again what this has to do with flying an imaginary plane around ? 

My imaginary plane is a lot bigger than your imaginary plane.

In the same vein; here in SW England it's chucking it down at the moment. Of course, our rain is wetter than your rain up north...😝

2 minutes ago, DD_Arthur said:

My imaginary plane is a lot bigger than your imaginary plane.

In the same vein; here in SW England it's chucking it down at the moment. Of course, our rain is wetter than your rain up north...😝

Haha yes but our population is smaller, so we have more space. And out of our population, 35% voted and of those 35%, 90% of them are using MSFS2020. 🥱 I suspect this is going to rock the boat over at Asobo and laminar. 

Edited April 18, 20251 yr by Ianrivaldosmith

Depending on how one looks at these numbers, one can always find a way to interpret them as bad for MS/Asobo, or good, or anything in-between. But again it's good to look at the results in the grand scheme of things:

1. A good sample size of Navigraph users were polled (23K).. so it's safe to say the results represent the views of non-casual/serious simmers who're bound to invest in the platform and purchase high fidelity add-ons (as seen per the top add-ons results). Yes they're bound to be airliner simmers, but still.
2. ~77% prefer the MSFS platform overall, 52% on 2020, and 25% on 2024 ... next (3rd) platform in the rankings is XP12 with 11.6% (so the main competition for 2020 and 2024 are each other)
3. 2020 is obviously still holding its own, which speaks to how good and mature of a sim it is
4. 2024, at five months old, despite the launch fiasco, despite having being released too early with too many bugs, is at second place with 25% (and of course perhaps that's due in part to the amount of fixes in SU1 and/or SU2).
5. As 2024 matures with more SUs and fixes/enhancements in the coming months, it is obviously going to be the top sim in the MSFS ecosystem, and all add-on devs will transition to only developing for that
6. As 2024 matures and more of the important add-ons become native/compatible for it, most of the current 2020 users are obviously going to transition to that

So looking at above points, I just don't see how there would be any sort of major turmoil over at MS/Asobo 🙂. Yes, 2024 could be doing better given it was their second go at a major sim release and with all the investment poured in so far.. but then given all the launch mess-ups, the bad press, it could be much worse. And for being 5 months old, is this such a doom & gloom result? Despite the many predictions made for us by some around here about how this is going to tank MS/Asobo, they are sitting pretty at 77%, with most of the major 3rd party devs developing exclusively for their platforms. Not a bad position to be in 🤷‍♂️