App search engine that filters out fake reviews
April 19, 2016 12:34 PM Subscribe
App search engine that filters out fake reviews
While developing mobile apps, I realized how rife the App Store is with rating/review manipulation. So, I created apprecs.com, a facade for the store that allows you to filter out many of the fake, coerced, or otherwise manipulated reviews. You can also filter by other criteria such as how recently the app was last updated and age/gender of the typical user. What do you think? I'm getting started on adding Android (Google Play) support today.
Some of the more interesting features it has so far:
While developing mobile apps, I realized how rife the App Store is with rating/review manipulation. So, I created apprecs.com, a facade for the store that allows you to filter out many of the fake, coerced, or otherwise manipulated reviews. You can also filter by other criteria such as how recently the app was last updated and age/gender of the typical user. What do you think? I'm getting started on adding Android (Google Play) support today.
Some of the more interesting features it has so far:
- Fake/coerced review detection
- Reviewer stats (e.g., how do they tend to vote?)
- Lots of filters, e.g., by rating, price, popularity, demographics
- App alternatives built partially on machine learning
Role: Sole developer
Very nice! I'm an Android guy so I'll be watching for that version.
posted by COD at 3:45 PM on April 19, 2016
posted by COD at 3:45 PM on April 19, 2016
This looks really impressive! Apple's app store reviews are so compromised as to be nearly useless, I'm curious to see how well this works. From your blog posts it sounds like your filtering would work reasonably well for other review sites Yelp, TripAdvisor, etc.
I'd love to read a blog post on how you measure your correctness, your confidence in the filtering.
I appreciate your disclosure of how you make money, Apple Affiliate revenue. That seems completely fair and not in conflict with users.
posted by Nelson at 11:00 AM on April 20, 2016 [1 favorite]
I'd love to read a blog post on how you measure your correctness, your confidence in the filtering.
I appreciate your disclosure of how you make money, Apple Affiliate revenue. That seems completely fair and not in conflict with users.
posted by Nelson at 11:00 AM on April 20, 2016 [1 favorite]
Thanks! Yeah, I love talking about this stuff, so I'd like to share (and hear more ideas about) the mechanisms behind the review classification. It's hard to resist revealing too much, but the analysis is primarily heuristic-based for now.
For example, we can classify reviews from reviewers who have, say, left 2000 solely 5-star reviews as untrustworthy. Same thing with reviewers whose matrix of reviewed apps is the same as multiple other reviewers -- those reviews are untrustworthy as well since they likely were paid for or solicited via one of the review-sharing services.
Review content can be another tip-off that something's up with reviews for a given app. E.g., "They bribed me to write this review" certainly raises a red flag and calls into question the other reviews.
Next step after Android will be to migrate to, say, a random forest classifier that'll make better sense of all the facets of an app/review/reviewer than I can by hand.
As for Yelp, TripAdvisor, etc., I agree that those would be great to analyze as well. I started with iOS partially because Apple provides several official APIs for the data. Most others don't, so it's a trickier proposition to get the data needed and stay up-to-date.
posted by hodgebodge at 11:38 AM on April 20, 2016 [1 favorite]
For example, we can classify reviews from reviewers who have, say, left 2000 solely 5-star reviews as untrustworthy. Same thing with reviewers whose matrix of reviewed apps is the same as multiple other reviewers -- those reviews are untrustworthy as well since they likely were paid for or solicited via one of the review-sharing services.
Review content can be another tip-off that something's up with reviews for a given app. E.g., "They bribed me to write this review" certainly raises a red flag and calls into question the other reviews.
Next step after Android will be to migrate to, say, a random forest classifier that'll make better sense of all the facets of an app/review/reviewer than I can by hand.
As for Yelp, TripAdvisor, etc., I agree that those would be great to analyze as well. I started with iOS partially because Apple provides several official APIs for the data. Most others don't, so it's a trickier proposition to get the data needed and stay up-to-date.
posted by hodgebodge at 11:38 AM on April 20, 2016 [1 favorite]
I wouldn’t mind a bit more in-depth breakdown of the technical deets in the blog, but that’s me. Certainly a cool concept, tho'.
posted by Going To Maine at 10:15 AM on April 22, 2016
posted by Going To Maine at 10:15 AM on April 22, 2016
My only grump with your approach is that it seems to discard not only the 5 star goaded reviews, but also the ones I leave which I even have a textexpander shortcut for.
posted by phearlez at 8:44 AM on May 3, 2016 [1 favorite]
Any time an app interrupts my using it to hassle me for a review, I give it one. Here you go.Since I am enough of a crank to leave those reviews I of course want them to count :) But I can accept the case that they shouldn't impact the rating (though I of course indignantly feel like they deserve it :)
posted by phearlez at 8:44 AM on May 3, 2016 [1 favorite]
I'm an iPhone App Developer and this is very helpful! Thanks for That!
posted by 912lab at 11:56 AM on May 3, 2016
posted by 912lab at 11:56 AM on May 3, 2016
Update: The site now has Android apps! Enjoy. :)
Would love to hear additional feedback or questions.
posted by hodgebodge at 10:17 AM on July 8, 2016
Would love to hear additional feedback or questions.
posted by hodgebodge at 10:17 AM on July 8, 2016
I tried it on my Android app, which has 893 ratings and 229 reviews, and all of them are genuine AFAIK and from strangers except for one which is from my wife. When I run your filter on it it comes back with 123 "trusted reviews", however the interesting thing is that your search engine gives my app an extra 0.2 stars compared to what it has in Google Play. Does that mean it's filtering out negative reviews it suspects as being from competitors as well as fake positive reviews? Or is it just basing the rating on reviews only?
posted by L.P. Hatecraft at 6:51 PM on July 8, 2016
posted by L.P. Hatecraft at 6:51 PM on July 8, 2016
« Older “Can’t Feel Your Pulse” ... | The Disability Integration Act... Newer »
posted by hodgebodge at 12:41 PM on April 19, 2016