Steam’s user reviews debuted in 2013, and have since become an important component of Valve’s PC games platform. But reviews have also become a big problem – perhaps not for Valve so much as the developers trying to get their games noticed in a catalogue that now includes more than 30,000 titles. Angry users can bombard a game with negative reviews for reasons that have little or nothing to do with the game itself, lowering its discoverability and potentially dooming it to financial failure. So, what if they actually worked?
The first big problem with Steam reviews is the one they share with professional reviews: aggregation. Sure, there’s a certain appeal to the idea of looking at a single number that neatly summarises a huge set of opinions about a new game, but crucial information and context is lost when even one review is reduced to just this, let alone many of them. A critic hasn’t spent 30 or more hours playing and thinking about a game simply to generate a number between one and ten in their head. There are reasons for each score, and each outlet assigns them in its own way. It’s simply not possible to capture those nuances in a number, much less a Metacritic aggregation of dozens of such scores, which may reflect wholly different standards and methods of criticism across sites.
Similarly, while many Steam reviewers also put a lot of time, thought, and effort into the evaluations they post, the only meaningful metric is the one generated by the aggregation of all reviews posted about a game. That can be ‘mostly positive’, ‘overwhelmingly negative’, or whichever descriptor Valve has assigned to the particular ratio of positive to negative reviews the game has received.
In addition to the loss of nuance that inevitably comes with aggregation, Steam reviews are very often not used as reviews at all. The recent launch of New World is a perfect example of the system being used as a bug reporting tool. As I write this, New World has ‘mixed’ reviews on Steam, with 54% of its staggering 50,233 reviews marked as positive. Of the negative reviews, most cite long queue times – a phenomenon that has plagued the launch of even the best MMOs since time immemorial. It’s a sentiment that’s echoed in our own New World review. Those queue times will likely be resolved in a matter of weeks, as has been the case with other MMO launches, but that ‘mixed’ review score will take months or years to budge, if it does at all.
Often, Steam reviewers will say they’ll change their rating to a ‘thumbs up’ once the issue they’re flagging is resolved, but developers I’ve spoken with have found that this typically doesn’t happen. There are countless examples of negative reviews written by players who say they love the game, except for a highly specific technical issue they want the developers to address.
In some cases, these ratings can be left for misinformed or wildly inaccurate reasons, too. While there are certainly legitimate concerns over the anti-piracy DRM Denuvo, certain corners of Reddit tend to blame it for all problems, real or imagined, that a game might have. It’s thus appropriate to punish any developer who uses it, or so the logic goes, even when Denuvo isn’t causing any perceptible performance problems.
For developers, a game earning anything less than a ‘very positive’ rating on Steam can be disastrous, and it’s wildly unfair that their financial fortunes are bound to a reviews system that’s so frequently and easily misused, either as a coercive bug reporting tool or as a way to mete out punishment over perceived slights.
For readers in English-speaking territories who may be grinding their teeth at any suggestion that Steam reviews are ‘unfair’, as opposed to an essential means of expressing their consumer rights, it may perhaps be easier to see the problem when the example used is a flood of negative reviews from Chinese players over what they perceive to be objectionable content: their fury over an image discovered in Red Candle Games’ Devotion led to the game being delisted from Steam and its publisher losing its business licence.
I want to be extremely clear that I am not lobbying Valve to get rid of Steam reviews. This would be futile in any case, but more importantly, developers I’ve spoken with have said that while the shortcomings of Steam’s user reviews system can be frustrating, it can be a tremendously helpful tool as well. Developers closely monitor their Steam discussions forums and reviews, and often use them to identify what issues are causing players the most friction in their game.
No Man’s Sky provides an instructive example, both in what’s right and what’s wrong with Steam reviews. After an infamously rocky launch, Hello Games set out to improve the ambitious space game, and a year later it was widely praised for having pulled off a stunning turnaround. However, its Steam reviews only ticked over into ‘mostly positive’ territory in September this year, five years after launch and with many more updates. Can it really be argued that No Man’s Sky took that long to become good? Does anyone really believe that Konami’s eFootball 2022 is the worst game ever to appear on Steam?
One potential improvement might be to encourage users who continue to play a game after leaving a negative review to return to it and potentially update it after a set number of hours in-game. That could help relieve developers of the negative ramifications that follow an initial wave of negative reviews, provided some of the launch-day issues and lingering bugs are corrected. It would leave users feeling empowered to leave further meaningful feedback.
It would also benefit Steam’s customers, who would see an aggregate review description that more accurately reflects the state of the game at the moment they’re considering a purchase, rather than the state it was in shortly after it launched.
This idea seems to fit into Valve’s own stated priorities when it comes to curation on Steam: it would be both automated and low-impact, not requiring any normative intervention from company employees after being developed and deployed. It’s just a simple nudge – perhaps similar to the one that encourages players to leave a review in the first place.
Obviously, this wouldn’t help curb review bombing campaigns, which are deliberately engineered to sink a game’s financial futures, but it would be a step in the right direction. Other ideas might be considered as well: developers do have a means to appeal reviews that make demonstrably false claims, but it’s not clear how quick Valve is to act on it. The system could be bolstered with additional tools and increased attention to moderation.
Again, none of this is to say that Steam reviews should be abolished – they’re an important part of what makes Steam a valuable resource. Developers recognise, by and large, that even if the review system is vulnerable to exploitation at times, it’s also a major channel for collecting player feedback, which they can then use to help make their games better. Just like those games, however, Steam reviews could still use a few more post-launch updates.