VR results in the 2019 Navigraph survey

[HiFlyer 9,343]

1 minute ago, vortex681 said:

How are they “self-selecting”? As far as I’m aware, anyone, not just Navigraph customers, can take part in the annual surveys. It was widely advertised through forums and on developers’ web sites so there’s no reason to think that it’s not representative

Its self selecting in the same way that Avsim and the other dedicated sites are not truly representative of the huge number of people who are interested in simulation and use simulators, but are not denizens of flight forums.

For instance, I was simming for several years before I even heard of a navi-whattzis, and for some time after that, had no interest at all in filling out the survey. In fact, as a general rule, I ignore it, as I don't really find most of the questions to be of interest.

(And its boring to fill it out)

There are apparently 2 million people using FSX on Steam. Now add in P3D. Now add in X-plane. Now add in the 7 or 8 million playing War Thunder. Now add in DCS.......

That's several million people using various flight sims, and some percentage of them have even heard of navigraph.

But that percentage that have heard of it and also choose to fill out the survey are a self selected group with a very specific interest that makes navigraph relevant to them. I would posit that the actual vast majority could not care less, and are not at all represented. Not as much as we pretend.

16 minutes ago, vortex681 said:

 It is, after all, the only large-scale flight sim survey so there’s nothing else out there which you could use to dispute the figures.

Only the acceptance of the fact that the niche of forum visiting simmers represents only a relatively small slice of the people using simulations.

War Thunder, which dwarfs civilian simulation in numbers, is, (for MANY reasons) barely represented here at all, yet as gamers, the millions of people using it are much more likely to be VR first-adaptors.

And..... not likely be represented very much in something like a navigraph survey........ (heck, I don't think it was even mentioned in the survey)

As a general rule I think we tend to have a large dose of tunnel vision, and sometimes an exaggerated vision of our own centrality within the world of people interested in Flight.

I take the results of the survey in this area with a huge grain of salt.

[vortex681 806]

@HiFlyer - so on what are you basing your opinion that the Navigraph survey is not representative? I realise that 17,800 respondents seems like just a small percentage of the total of flight sim users out there but unless you’re able to get the opinions of a similar number of users who seem to be at odds with it, it’s difficult to question the results.

4 minutes ago, HiFlyer said:

War Thunder, which dwarfs civilian simulation in numbers, is, (for MANY reasons) barely represented here at all, yet as gamers, the millions of people using it are much more likely to be VR first-adaptors.

And..... not likely be represented very much in something like a navigraph survey........ (heck, I don't think it was even mentioned in the survey)

I have nothing against War Thunder, but to expect it to be in the Navigraph survey shows that you don’t really understand the point of the survey. Remember that the survey wasn’t primarily about VR usage in gaming in general, VR was just a small subset of the flight sim-specific results.

[HiFlyer 9,343]

2 minutes ago, vortex681 said:

@HiFlyer - so on what are you basing your opinion that the Navigraph survey is not representative? I realise that 17,800 respondents seems like just a small percentage of the total of flight sim users out there but unless you’re able to get the opinions of a similar number of users who seem to be at odds with it, it’s difficult to question the results.

I did explain that above.

Also, currently, I am a member of a few groups devoted to vr on Reddit, Facebook, Discord, etc.

One of those groups, which has been around a few months, currently has about 3,000 people and growing, and as far as I can tell, very few of them are interested in the traditional forums at this point. They are quite active groups, but the level of activity is just not represented very well in forums outside of those groups.

Another dedicated sim group, which is also very Vr oriented, has about 4000 members now, with a tiny bit of overlap with other vr groups as I do recognize a handful of names.

There are also many flightsim discords with vr overlap.

From what I saw, the mention of the navigraph survey at those venues was pretty much a non event. It's just not relevant, though I suspect if those people did bother to jump on the survey, it might have moved the bar a bit. I just dont think very many care, especially since after a few of them did mildly flex their numbers here, the affected surveys were either quickly discredited or closed.

Really, it seems that until the tech becomes more mainstream in the wider community, these segments will continue to remain isolated and perhaps unrepresented.

[HiFlyer 9,343]

46 minutes ago, vortex681 said:

I have nothing against War Thunder, but to expect it to be in the Navigraph survey shows that you don’t really understand the point of the survey. Remember that the survey wasn’t primarily about VR usage in gaming in general, VR was just a small subset of the flight sim-specific results.

No I never expected it to be about Vr primarily, and War thunder is obviously not primarily VR. Instead, the survey bills itself as a look at the broad spectrum of people using flightsims, but in fact I believe its focus excludes broad swaths of simmers who don't fit within a fairly narrow subset of interests.

Do I use Vatsim? nope, fly in a virtual airline? nope.....

Nope, nope, nope... etc.

The questions made it clear within the first few pages that the survey was not aimed at me. Nor, probably quite a few other people to whom the whole thing was largely irrelevant.

[vortex681 806]

On 2/10/2020 at 4:26 PM, HiFlyer said:

No I never expected it to be about Vr primarily, and War thunder is obviously not primarily VR. Instead, the survey bills itself as a look at the broad spectrum of people using flightsims, but in fact I believe its focus excludes broad swaths of simmers who don't fit within a fairly narrow subset of interests.

The Navigraph survey wasn't just a general flying game survey. It was specifically aimed at the flight sim subset of flying games. For me, and probably many others, a desktop flight simulator is something which replicates real-world flying as closely as possible within the constraints of the PC ecosystem. I wouldn't consider War Thunder to be any more a flight simulator than something like The Crew 2 - it's more an arcade game with aircraft (amongst other things). They're not worried about having things like a realistic flight model that's noticeably different for every aircraft - it's more about the game play than the accuracy and there's nothing wrong with that..

On 2/10/2020 at 4:26 PM, HiFlyer said:

Do I use Vatsim? nope, fly in a virtual airline? nope.....

But many do. Most people in the flight sim community would say "do I use VR? nope". Almost all surveys include questions which aren't relevant to everyone. The secret is to make sure that as much as possible applies to most people.

On 2/10/2020 at 4:26 PM, HiFlyer said:

The questions made it clear within the first few pages that the survey was not aimed at me. Nor, probably quite a few other people to whom the whole thing was largely irrelevant.

What, specifically, made you think that? I'm not trying to say you should have taken the survey, I'm just genuinely interested. Do you use P3D, XP or FSX? If so, then the survey definitely was aimed at you. The demographic of respondents ranged from 15-85 years with the majority being between 20 and 45. More than 61% of the respondents were new to the survey so it wasn't as if the same people are participating every year. It included stats for many sims other than just the big 3. They really did try to capture a good cross section of the community.

Edited February 11, 2020 by vortex681

[HiFlyer 9,343]

On 2/11/2020 at 12:45 PM, vortex681 said:

The Navigraph survey wasn't just a general flying game survey. It was specifically aimed at the flight sim subset of flying games. For me, and probably many others, a desktop flight simulator is something which replicates real-world flying as closely as possible within the constraints of the PC ecosystem. I wouldn't consider War Thunder to be any more a flight simulator than something like The Crew 2 - it's more an arcade game with aircraft (amongst other things). They're not worried about having things like a realistic flight model that's noticeably different for every aircraft - it's more about the game play than the accuracy and there's nothing wrong with that..

Look up War Thunder Realistic Battle and Simulator Battle modes.

At one point earlier in its history, War Thunder was indeed more arcadey. But what that did was actually allow the program to become popular.

That brought in literally millions of users (about 80,000 online simultaneously on an average weekend) and let copious amounts of money flow. With that money, they expanded, and eventually added a simulation mode that became very popular.

It appears from what I can tell, that despite the fact I have mentioned it and posted videos, that this information is not that widely known. I suspect because the community had already marked it as a "game" and moved on, not interested in revisiting the subject.

As an aside, my friend Jose, known sometimes here as "The Uninstaller" and one of the most...... persnickety people I know regarding the necessity of absolute realism in flight models..... now almost exclusively plays War Thunder Simulation mode because of its realism. Realism that still did not get War thunder even a mention in the "comprehensive" navigraph survey.

On 2/11/2020 at 12:45 PM, vortex681 said:

But many do. Most people in the flight sim community would say "do I use VR? nope". Almost all surveys include questions which aren't relevant to everyone. The secret is to make sure that as much as possible applies to most people.

Hence why I said the survey felt it was not aimed at me; as about 90% of what it asked was about things I do not now and never did bother with. This makes me not likely to participate in the survey, and the only reason I bothered was because both Ipacs/Aerofly and a Youtube Vr forum I frequent recommended that I and others do so.

Based on a previous encounter with the survey, I very reluctantly did, though I can't imagine a reason to do so in the future.

On 2/11/2020 at 12:45 PM, vortex681 said:

What, specifically, made you think that? I'm not trying to say you should have taken the survey, I'm just genuinely interested. Do you use P3D, XP or FSX? If so, then the survey definitely was aimed at you. The demographic of respondents ranged from 15-85 years with the majority being between 20 and 45. More than 61% of the respondents were new to the survey so it wasn't as if the same people are participating every year. It included stats for many sims other than just the big 3. They really did try to capture a good cross section of the community.

They may have tried and had good intentions, but I would point out that I'm far from the first to be a bit skeptical. Navigraph in its follow-up video mentions that others have raised concerns about possible selection bias.

As far as your other question about sims, the better question would be which ones don't I use. I think I fly as many as I can, since I feel much more interested in aviation generally than in titles specifically (game, serious, etc) or in the concept that some specific criteria makes one legitimate, while some lack in another category makes them not worthy of my time.

Moving on: In the navigraph video below, I felt they kind of hand waved at the possibility of selection bias while not really answering concerns regarding their methodology.

https://youtu.be/-ryJCNwR2HU

Edited February 13, 2020 by HiFlyer

[vortex681 806]

3 hours ago, HiFlyer said:

Based on a previous encounter with the survey, I very reluctantly did, though I can't imagine a reason to do so in the future.

Because even if you answer no to most of the questions, that adds important data to the survey. You don't want all of the respondents to be just the people who use the products/hardware/services mentioned in the questions as this would introduce the very selection bias you talk about below.

3 hours ago, HiFlyer said:

Navigraph in its follow-up video mentions that others have raised concerns about possible selection bias.

Only in as much as people commented that respondents hadn't been picked at random. But how could you possibly do that? To pick respondents at random, you'd first need to know that they had an interest in flight simulation and then somehow already have their contact details without them being product customers or people who'd expressed an interest in the products. That's likely to have introduced more selection bias than just issuing an open invitation to the community at large where no selection is involved. This is another excellent reason why people who don't really think the survey is for them should take part. I think it's significant that the majority of respondents this year had not taken part in the survey previously. That's likely to reduce the selection bias as they're not all habitual respondents.

The bottom line is that however you view the survey and its results, it's the only one of it's kind with those sorts of numbers taking part.

[HiFlyer 9,343]

11 minutes ago, vortex681 said:

Because even if you answer no to most of the questions, that adds important data to the survey. You don't want all of the respondents to be just the people who use the products/hardware/services mentioned in the questions as this would introduce the very selection bias you talk about below.

I never liked boring tests in school, and have not developed any greater liking for them since!

In order for me to participate, there has to be some outcome associated with the endeavour that I support, and since I don't feel the survey has much to do with me or how I sim, there's close to zero incentive, especially when I believe strongly that my choices are bound almost inevitably be overwhelmed by the very hardcore/study-level traditionalists that the survey seems truly designed for.

The survey really has nothing to offer me.

1 hour ago, vortex681 said:

Only in as much as people commented that respondents hadn't been picked at random. But how could you possibly do that? To pick respondents at random, you'd first need to know that they had an interest in flight simulation and then somehow already have their contact details without them being product customers or people who'd expressed an interest in the products. That's likely to have introduced more selection bias than just issuing an open invitation to the community at large where no selection is involved.

i believe the very venues chosen to spread the existence of the survey automatically tended to skew the type of participants. Worse, even if the respondents had come from a broader spectrum of people interested in aviation, I suspect the survey would simply have once again have come under question, just from another direction, as people cried that the answers were corrupted by gamers, "not-true-simmers" and other scurrilous riff raff.

Its essentially a no-win scenario.

I should be clear though. Despite the above, I once did believe in such surveys as useful general indicators.

However, after watching some recent shenanigans within the community, my trust is absolutely totally, completely and irrevocably gone.

I will almost certainly never participate in another community survey, and I dont think thats unique, as the phrase "Maybe we should do a survey." has now for many become an inside joke of dark, cynical humor.

[Casualcas 830]

I'm surprised nobody brought up IL2. Throw that in the VR bucket as well. With all the updates to it, I hear its outstanding