Monday, 3 August 2015

Standardizing the star rating system

In this age of information, if something is available then there's probably some way to review it. However, as many of us already know, the ratings attached to these reviews can vary wildly. There are apps on the store that get 5 star reviews that read "it's okay" while the one directly under it reads "works fine" with a 1 star review.

I'm going to outline the model I use when reviewing. If you disagree with my reasoning or want to share your own, feel free to comment below. I don't expect everyone to adopt my way of thinking, but I want us all to put serious thought into the numbers that we assign when evaluating.

When I was firsting thinking about my rating system, I started at 3. Three is literally the average; it performs its expected task, but doesn't do anything particularly good or bad. I'm going to use 2048 as an example for my system.

2048 is a game that I, and many other people I imagine, absolutely love. However, I only rated it a three. The app does exactly what I expected it to do, no more or less. More recently, I found 2048 Plus. This app got a 4 rating because it went beyond the base game with the inclusion of the new grid sizes and auto-play option. I had considered giving it a 5, but reconsidered when I noticed that the goal remained the same on every grid from 3x3 to 8x8 which resulted in absurd variations in difficulty.



A rating of 2 is designated in a similar way. The Windows Phone version (my previous phone was a Nokia Lumia. Good phone, but it's developed a few glitches of late) originally deserved a 3 star rating just like the Android version. But as time went on, the advertising went from a small banner at the bottom to pop ups that broke the flow of the game. At its lowest point, the Windows version got a rating of 1 from me. A new pop up would open the instant that the previous one closed, making the game unplayable.

These are my criterion for reviewing. In summary, they are;

Rating 5
Essentially the perfect app.

Rating 4
Provides features that aren't necessary for it's function, and improve the experience, but it could be iterated further

Rating 3
App works as expected without obstruction

Rating 2
App works, but there are noticeable flaws such as technical glitches or undesirable mechanics (eg. poorly implemented micropayments)

Rating 1
Application does not work or contains an unacceptable flaw. No app that's received this rating has been on my phone long.

It's reasonable to expect people to have differences of opinions and reviews are by definition subjective, but viewers of your review can't make an estimate as to your opinion of a product from it, then that rating is worth nothing.

No comments:

Post a Comment