Amazon is making a significant change to its Community Guidelines, announced today, which will eliminate any incentivized reviews, except for those that emerge from within its own Amazon Vine program. This program allows Amazon – not the seller or vendor – to identify trusted reviewers, and has a number of controls in place in order to keep bias out of the review process.
Amazon has historically prohibited compensation for reviews – even going so far as to sue those businesses who pay for fake reviews, as well as the individuals who write them, in an effort to make its review and rating system fairer and more helpful to online shoppers. However, it has allowed businesses to offer products to customers in exchange for their “honest” review.
The only condition was that those reviewers would have to disclose their affiliation with the business in question in the text of their review. Reviewers were generally offered the product for free or at a discounted price, in exchange for their review.
Although, in theory, these reviewers could write their true opinion on the product – positive or negative – these incentivized reviews have tended to be overwhelmingly biased in favor of the product being rated.
This is due to a combination of factors – the fact that the vendor or seller has likely sought out those reviewers who are less critical, and the fact that reviewers may believe they would no longer have the opportunity to receive these sorts of offers, if they chose to say negative things.
In general, shoppers have begun to distrust these reviews because they believe them to be biased. That’s not just a “feeling,” as it turns out – a recent study of over 7 million reviews indicated that the average rating for products with incentivized reviews was higher than non-incentivized ones. (That is, a 4.74 average rating versus a 4.36 average rating, out of 5 stars).
Even with this 0.38 star difference, the impact was substantial – boosting products from the 54th percentile to the 94th percentile. Effectively, incentivized reviews could create top-rated products.
The study also found that incentivized reviewers were 12 times less likely to give a 1-star rating than non-incentivized reviews, and almost 4 times less likely to leave a critical review in general.
Technically, these reviewers are not compensated with cash, but writing reviews has become a windfall of sorts. Those who participate in incentivized reviews have written an average of 232 reviews, the study indicated, while those who have not, only wrote an average of 31 reviews. That’s a lot of free and discounted products for these ‘non-compensated’ reviewers!
Amazon says that, going forward, the only incentivized reviews will be those from Amazon Vine. These don’t work the same way, however. For starters, Amazon selects who will be allowed to review products, and it does so mainly to boost the review count on new or pre-release products that haven’t yet generated enough sales to have a large number of organic reviews.
Vine reviewers are invited to join the program only after having written a number of reviews voted as “helpful” by other customers, and tend to have expertise in a specific product category.
“We do not incentivize positive star ratings, attempt to influence the content of reviews, or even require a review to be written,” explains Chee Chew, VP, Customer Experience at Amazon in an announcement about how Vine controls for bias. “And we limit the total number of Vine reviews that we display for each product,” he adds.
In addition, vendors don’t have any contact with Vine reviewers, nor do they get to influence which reviewers will receive their products, which are submitted directly to Amazon for distribution.
These changes will apply to all product categories other than books, as Amazon has always allowed advance copies of books to be distributed, the retailer notes. Amazon also says it has other ideas about making Vine more useful in the future, but didn’t go into detail.
An Amazon spokesperson tells us that reviews that were received prior to the policy change are only being retroactively removed if they are excessive, and don’t comply with prior policy. That means you’ll still see a ton of these seemingly biased reviews live on Amazon, unfortunately. The retailer also says that if it finds anyone is attempting to manipulate reviews by tying reviews to discounted products, it will take action against them, starting today.
Image credits, charts: ReviewMeta
Comment