We will not delve into the complex method our reviewers use to arrive at the final rating other than to give you the basics. The reviewer assigns weighted numeric values to six specific criteria for each component to be rated in a flight simulation package. A program then takes these figures, combines all of them involved in the package and correlates them to the AVSIM Five-Star Rating System.
Now we are aware many other systems out there also use a system based on five values, be they stars, planes, propellers or whatever. And most of them associate the current State-of-the-Art with the top rating of five. But this type of system is flawed in our minds. It does not provide for those quantum leaps in thinking and technology that drive our great hobby. Face it, a five in a five value system is as high as you can go and if that is the current state-of-the-art, there is no where else to go.
AVSIM has always felt it was essential to be able to recognize those who are willing to push the envelope and shoot for those quantum leaps in thinking and technology. So we devised a rating system that does just that. But it requires readers to adopt a slightly different mind set.
In our Five-Star system, a rating of 3-Stars represents the current state-of-the-art equal to the best previously released for this type of product. Face it, all of us want newer releases to be better than what is already out there, so here at AVSIM the best of previous releases becomes the new Norm and thus products released subsequently sit in the middle of the systemnow at 3-Stars. If it further advances the artbe it in quality, functionality, or innovationwe increase the rating from there.
So a 3-Star rated review at AVSIM Online is nothing to look down on. It means the product is equal to the best in the genre at that time. But it does not reflect any improvement on this norm. So what does it take to get higher, or lower ratings? The table below spells that out pretty clearly.
||At the Edge of That Even Conceivable Today
||Far Exceeds Natural Progression
||Natural Upward Progression From State-of-the-Art
||Positive Tweaking of Today's State-of-the-Art
||Today's State-of-the-Art - Where to Begin a Rating
||Below The Current Norm or Expectations
||A Step Backwards In the Genre
||Not Worth Your Consideration
In each case, the product is related to the genre and is not compared to other products from other developers. There may be comparison between products by the same developer, but only if the products are directly correlated.
One final comment relates to the AVSIM "Award of Excellence" discussed in depth elsewhere. What could be better than a product earning Five-Stars? Well it is not easy. The product must represent "A Quantum Leap in Thinking and/or Technology Exceeding that Conceivable Today!" Basically, the developer must do something no one else has done before, or ever thought of. This may involve the development of a bold new technology, or the thought needed to take the best existing technology and use it in a bold new way.
So remember when you see a rating of 3-Stars here at AVSIM Online, it is definitely not a bad thing. To the contrary, this is a product equal to the current state-of-the-art in the genre. And most of our review ratings will fall within a statistical range of say 2.5 to 4 stars. That is what one would expect with a system such as ours.
So when you see one of those elusive ratings of say 4.5-Stars and 5-Stars, know two things. First, this is an exciting product you will definitely want to get your hands on. Second, the bar has now been raised for all that will follow. To produce the same product, very unlikely for a myriad of reasons, would only get the new developer 3-Stars.
We are so sensitive about the integrity of the exclusive 4.5-Star and 5-Star ratings that the AVSIM Editorial Review Board must review them before they are published. The reviewer may be questioned about his conclusions and in some cases; the rating will be lowered if all the reviewer cannot justify his rating to the board.
© 2001 - AVSIM Online
All Rights Reserved