Good and Bad, Amazing, Awesome, Lacking

[MarioDonick 752]

Partly due to my professional backgrounds I am thinking a lot lately about how people judge the quality of addons. This is important for me because I do reviews for FS MAGAZIN since 2012, and also work for vFlyteAir since 2015.

 

The recent negative threads in the avsim X-Plane forum have brought this topic to my attention once again. Often people give a statement like "good addons are lacking" without explaining what is "good" for them or what they expect from an addon. Sometimes "good" seems also to be meant as "100% perfect as in FSX PMDG quality".

 

Sometimes, in reviews it's similar. It all depends on the reviewer's preferences and on the group the reviewer is writing for. Often both aspects are not made clear.

 

In my X-Plane blog, I had some months ago a review of the Alabeo DA-42, and the rating system at the end of the review already tried to take into account for what user "type" (beginner, advanced, expert) the aircraft is suitable for.

 

Today I have explained my thoughts behind this a bit more.

 

And I'd like to have some more opinions about this, so I post this self-promotion thread here.

[GoranM 3,764]

Very keen to see how this follows up.

[ryanbatc 12,834]

Very keen to see a citation release LOL

[CFIJose 5,485]

Very keen to see a citation release LOL

:smile:

Patience! Perfection takes time.

 

BTW, how soon will I be able to fly the "very good" looking Arrow v4. I really like the current Arrow and I'm looking forward to the update.

[Jimmy RFR 406]

Mario, that's a very well written explanation explaining the thought process behind your reviews.

 

I'm rather embarrassed to admit that I accepted, with enthusiasm, months ago, a role to do X-Plane reviews here at Avsim, and have since managed to have absolutely zero reviews completed. Although this is partially because summer, vacation, and family have taken most of my time recently, it's also due to the fact that writing a proper review is really, really hard - a lot harder than I thought it would be!

 

I like to write, but trying to filter my personal experience with a product into something useful for other people to read is proving to be very difficult for me. Your process in regards to taking into consideration the different levels of interest of users of varying experience is really interesting, as is the example you've used in the Alabeo G1000, where by applying your personal preferences, it would of received a lower score that might not be as useful to certain segments of user groups.

[MarioDonick 752]

 

 

how soon will I be able to fly the "very good" looking Arrow v4. I really like the current Arrow and I'm looking forward to the update.

 

There are still small things fixed at the moment, for example yesterday we made some more changes to the notification system which (optionally) gives you hints during startup and flight.

 

 

 

 

I accepted, with enthusiasm, months ago, a role to do X-Plane reviews here at Avsim, and have since managed to have absolutely zero reviews completed.

 

Oh! I'd love to read some more unbiased and in-depth reviews. I was hoping that Angelique with her X-Plained.com website would write a bit more, but her last review is also a while back.

 

I agree, though, that writing a review is not an easy task (and on my blog I only wrote one so far...), but hey, you can do it!!

[ryanbatc 12,834]

:smile:

Patience! Perfection takes time.

 

BTW, how soon will I be able to fly the "very good" looking Arrow v4. I really like the current Arrow and I'm looking forward to the update.

 

Bah you're still an XP virgin lol...  The citation has been in the works for "a while."  I blame Goran and his OCD for re-doing basically the whole plane

[CFIJose 5,485]

There are still small things fixed at the moment, for example yesterday we made some more changes to the notification system which (optionally) gives you hints during startup and flight.

 

 

 

 

 

Oh! I'd love to read some more unbiased and in-depth reviews. I was hoping that Angelique with her X-Plained.com website would write a bit more, but her last review is also a while back.

 

I agree, though, that writing a review is not an easy task (and on my blog I only wrote one so far...), but hey, you can do it!!

TKS Mario.

 

I recently found out about a black vc update from the org. This Arrow looks even better with a black VC and updated attitude indicator. Please consider a black vc as an option.

Bah you're still an XP virgin lol...

:smile:

Soooo true!

[MarioDonick 752]

 

 

I recently found out about a black vc update from the org. This Arrow looks even better with a black VC and updated attitude indicator. Please consider a black vc as an option.

 

We like our Arrow with the black-brown cockpit and the old-style attitude indicator :-) This is how the real one we based our's on looked like. I think that other options are best done by user modifications (liveries, so to say).

 

 

But actually I did not want to talk about our aircraft in this thread. Instead I'd like to know what people are looking for in reviews, and related thoughts.

[CFIJose 5,485]

We like our Arrow with the black-brown cockpit and the old-style attitude indicator :-) This is how the real one we based our's on looked like. I think that other options are best done by user modifications (liveries, so to say).

 

 

But actually I did not want to talk about our aircraft in this thread. Instead I'd like to know what people are looking for in reviews, and related thoughts.

Thank you.

 

Your reviews are objective and well written. It's only human to wish the same level of value for addons from one sim to another.

 

A good example is ASN + REX Soft clouds vs SMP/RWC & FSGRW. If a user has has been an ASN/SC user, they will want the same outstanding features or they might be disappointed. Even if it's a totally different sim.

[GoranM 3,764]

Bah you're still an XP virgin lol...  The citation has been in the works for "a while."  I blame Goran and his OCD for re-doing basically the whole plane

You're not far off.  I haven't redone the WHOLE plane.  At least I don't think I have.  

:wink:

At the moment, I'm on day 12 of remodelling the engines.  Kinda ridiculous.

[ryanbatc 12,834]

Well I'm off topic but the modeling is looking splendid

 

And a good idea Mario!

[tonywob 4,768]

For me, I judge addons based on things I already own, and since I've used FSX for many years, lots of payware products fall behind in X-Plane when I compare to what I had and this is something I have come to expect. Some things I really dislike about some payware airports (Not naming any) are:

 

- Missing gates or wrong markings, missing taxiways etc.

- Textures. Poor use of phototextures for buildings and lack of ambient occlusion. Very clinical looking buildings and objects.

- Ortho photos which haven't been edited to remove cars and buildings. It really isn't hard and makes a difference. Even freeware devs do this.

- When using ortho photos, not colour matching with objects and grass above (e.g. Slapping down bright green tree textures on autumn colours).

 

If I see any of the above in the preview pictures or videos, I skip and don't buy. Some products which are exceptional that I have bought recently are Manchester EGCC, and since I now trust the developers, I'll likely pick up their next products automatically.

 

For aircraft, I am generally happy with the quality, but I don't use airliners except the IXEG 737, and again not naming names, some things I judge on are:

 

- The textures. As above, I've seen many plastic looking interiors that kill immersion

- Sounds, a surpisingly overlooked feature.

- FMC. It should simulate the real-one as much as possible, and not be some fake version with very limited functionality. Luckily XP rescued some older products with the GPS, but this shouldn't be an excuse for devs to get lazy.

- Flight dynamics. It should somewhat behave like the real aircraft regarding the numbers. I'm not so much bothered if it stalls slightly different, as it's only a simulation and will never feel real anyway.

 

I like Dreamfoils recent helicopters very much and they really shine with quality. I also like the AirfoilLabs C172, Saab 340 which is fantastic. Again, both companies have an automatic customer from me ;-)

[MarioDonick 752]

Thanks for your statement, Tony!

 

 

FMC. It should simulate the real-one as much as possible, and not be some fake version with very limited functionality. Luckily XP rescued some older products with the GPS, but this shouldn't be an excuse for devs to get lazy.
- Flight dynamics. It should somewhat behave like the real aircraft regarding the numbers. I'm not so much bothered if it stalls slightly different, as it's only a simulation and will never feel real anyway.

 

 

That's interesting -- when I match this with my rating box, I'd say in regards of FMC you'd be in my "expert" category of expectations, whereas in regard of the flight dynamics, you're more on the "advanced" side. So maybe it'd make sense to differentiate further in my upcoming reviews. Instead of having a single "suitable for" category maybe having one "systems depth suitable for" and another "flight dynamics according to POH and what real-world pilots say in reports about the real aircraft" could be better (well, not named in such a long way as I wrote here, of course). Because I think that even users of my "beginner" category are interested in realistic or plausible flight behavior -- aren't they? Or is an EASY flight behavior more important for them instead of realism or plausibility, and the latter comes into play only when you have more experience?

 

 

Another interesting point mentioned by you and José is simulator independent judgement. This is something we even had an article about in FS MAGAZIN a while ago named "Für X-Plane schon ganz gut" (in English I would call it: "Well, for X-Plane, it's okay"). That article was written when the first Reality Expansion Pack for the CT210 was published (REP was way less complex than it is nowadays) and born out of a little frustration we felt over the general addon market that had a lot of aircraft already, but all more on the "beginner" or "advanced" side of the spectrum (the Saab 340 and partly the FF 757 were the only exceptions at that time). The text consisted of two parts. The first, more critical part, was written by Oskar Wagner. He was a real-world airliner pilot for a long time, did quality management for A2A's Comanche (a great aircraft I have bought, too, even though I don't use P3D anymore) and naturally has high demands. He had a lot to criticize about REP back then, starting with the fact that REP enhances an existing aircraft and therefore can not improve some things (such as the autopilot, which would involve significant editing of 3D objects). The second part was written by myself, and I was pointing out that the development of both high quality addons and of a diverse addon ecosystem takes a lot of time, and that A2A had a 12-years headstart, so any comparison from the development and market perspectives seemed difficult.

 

I concluded my part as follows (I freely translate this to English here):

 

Two souls, alas, are dwelling in my breast: One says we shouldn't be overly critical with new developers if their first ambitioned project does not meet the expectations of long-time FSX/P3D users. We should be happy that they exist and we should support them by constructive feedback to get better and better. The other soul, however, does not accept this "apology". Because in the end a user pays a lot of money and the question is what the user gets for this money. And unfortunately it is still the case that virtual pilots can get a more complete experience for the same money in FSX/P3D. The resulting purchase decisions strengthen or weaken the addon ecosystems. In the end, one has to ponder how much idealism and how much pragmatism is necessary -- not easy, for sure.

 

That was summer 2015 and I was working on avionics at that time that would later become the base for the vFlyteAir SR20's avionics. So I was thinking about systems depth also from a developer's perspective already, and that showed in our article, too. (I can tell you it's sometimes a bit exhausting to constantly have to reflect about both sides just to avoid biases. It would be sooo much easier if I simply could adopt either the "Hey, it's all awesome and amazing!!!!" or the "Everything is lacking!!!!" view, but somehow I have the inner desire to balance out both sides.)

 

Side note: The avionics started as a commissoned work we did for a real-world aviation company (I am not allowed to say who it was) -- "experts", so to say, who (a paradox for me at that time) were not interested in a deep simulation of the avionics, but mainly in looks and flight behavior. After we got the permission to publish a SR20, we knew we had to increase depth for the X-Plane market, because most X-Plane users would not tolerate the very simple avionics of the original product. But when you look at our SR20, you will notice a lot of things are missing and a lot of things are different. We were very open about this to customers, as everybody can download our manuals before purchase (in the introduction I clearly state "this is not a study sim" and in the text I always point out differences to the real thing). I'm still working on SIDs and STARs, though... hopefully next year...

 

 

Anyway, a lot has happened since then -- the many significant REP updates (the CT210 is my absolute favorite GA plane in X-Plane at the moment), the IXEG 737, the FF 767, the many updates to the AirFoilLabs C172, etc. But of course there are still many open holes.

 

Now the question is if a review should take all this into account -- or if the review should stay within the X-Plane world, ignoring what exists outside of it. The question is relevant to me, because the avsim forum shows that more and more FSX users are indeed interested in X-Plane, but have a lot of questions and justified worries about the overall package quality they can achieve in the end.

[JimmiG 244]

For me, I judge addons based on things I already own, and since I've used FSX for many years, lots of payware products fall behind in X-Plane when I compare to what I had and this is something I have come to expect. Some things I really dislike about some payware airports (Not naming any) are:

 

- Missing gates or wrong markings, missing taxiways etc.

- Textures. Poor use of phototextures for buildings and lack of ambient occlusion. Very clinical looking buildings and objects.

- Ortho photos which haven't been edited to remove cars and buildings. It really isn't hard and makes a difference. Even freeware devs do this.

- When using ortho photos, not colour matching with objects and grass above (e.g. Slapping down bright green tree textures on autumn colours).

 

If I see any of the above in the preview pictures or videos, I skip and don't buy. Some products which are exceptional that I have bought recently are Manchester EGCC, and since I now trust the developers, I'll likely pick up their next products automatically.

 

For aircraft, I am generally happy with the quality, but I don't use airliners except the IXEG 737, and again not naming names, some things I judge on are:

 

- The textures. As above, I've seen many plastic looking interiors that kill immersion

- Sounds, a surpisingly overlooked feature.

- FMC. It should simulate the real-one as much as possible, and not be some fake version with very limited functionality. Luckily XP rescued some older products with the GPS, but this shouldn't be an excuse for devs to get lazy.

- Flight dynamics. It should somewhat behave like the real aircraft regarding the numbers. I'm not so much bothered if it stalls slightly different, as it's only a simulation and will never feel real anyway.

 

I like Dreamfoils recent helicopters very much and they really shine with quality. I also like the AirfoilLabs C172, Saab 340 which is fantastic. Again, both companies have an automatic customer from me ;-)

 

 

I think the big difference between FSX/P3D addons and X-Plane is "polish", which applies to many of your points above. FSX design teams generally consist of more people, with higher development budgets and more testers. Many X-Plane developers are still one-man shops or very small teams.

 

For me, that's part of the charm of X-Plane addons. It's not unusual to just get a direct download link to a .zip file that you extract and dump into your X-Plane folder, where as FSX addons have installation wizzards, professionally produced PDF manuals, control panels, automatic updates etc. It also means that some of the smaller X-Plane dev studios don't have the resources to perfectly color-match and edit ortho photos or produce that photo-realistic 3D interior. As long as the price reflects this, I'm generally OK with it. However, "Quality" or "polish" is something reviewers should take into account still. When you pay money for a "product", you expect the overall experience to feel professional.