The Ratings Problem

You slave away creating your android app or game. You test it on multiple devices, polish it up and then whack it on the old Android Market and watch the good times role, yes?

Well, sometimes. But let’s assume for the moment that your app is well received and the downloads (free or paid) start rolling in. Users start posting comments and ratings out of 5 stars. If those ratings are good your downloads will probably snowball since other users will often search for apps by rating. If you’re near the top, then you will probably stay there. It’s the old “rich get richer” mechanic. HAPPY DAYS!

So a month passes and Motorola, HTC, Sony Ericsson, LG and Samsung release a bajillion new Android devices. You start seeing comments for your app on the marketplace like

Does not work on Sony Xperia X10. 1 star out of 5

or

freezes on Motorola Droid X. 1 star out of 5.

FRIKK!!!

Any developer out there knows how hard it is to keep up with the endless device releases combined with Google’s frantic releases of new Android versions. They are just not consistent and it is too expensive to keep buying each new device for testing and porting purposes. The trouble is, those bad reviews and ratings begin to kill your app’s reputation, even for the many devices that it works perfectly on! You watch as day by day those 1 out of 5 star reviews cause your rating, and subsequently your downloads, to plummet.

The Solution

The star rating system on Android Market should be a rating of how good/useful/fun/polished/ingenious an app is. Developers shouldn’t be punished for the fact that their app does not work on specific devices. Android Market should implement two measures of rating for an app:

1. a rating out of 5 with regards to how good/useful/fun/polished/ingenious the app is
2. the percentage of devices that the app works successfully on, broken down by device model. Call it a ‘reliability’ measure.

e.g.

Dave’s Combat Simulator

rating: 4.34 / 5

reliability: 68% (click for more info)

• Motorola Droid (2.1): 98% from 22,408
• HTC Dream (1.6): 92% from 2,433
• HTC Legend (2.0): 89% from 8,500
• LG GT540 (1.6): 78% from 343
• etc…
• etc…
• Motorola Droid Incredible (2.2): 0% from 91
• etc…

Some might argue that a ‘good’ app is also an app that works properly on a wide range of devices. I don’t think this is true. A good app is an app that is good/useful/fun/polished/ingenious on at least some devices and the developer shouldn’t be punished if it takes them some time to support all the newest devices on the market. I mean, the blockbuster console game  “God of War 3″ is a great game for Playstation. The fact that it does not support X-Box does not mean it is worthy of 1 star out of 5.