Metacritic: A Blight on the Industry

The recent release of Risen 2: Dark Waters was met with mixed reviews from fans and critics alike. I never expected Risen 2 to do well among mainstream gamers (whose definition of a good RPG consists of Skyrim and Mass Effect), so it's no surprise to me that its metascore is currently sitting at a mediocre 69. Honestly, I would've given it a similar score myself when I first played it. Seemingly broken combat, graphical glitches galore, watered-down gameplay elements, and other issues with design and presentation left me feeling very conflicted about a sequel that should have only improved on the success of the first game.

Less than two weeks after its release, however, Piranha Bytes released a patch which addressed nearly every major complaint I (or anyone else) had with the game. They added roll-dodging and the ability to block monster attacks, which greatly improved the feeling of combat, they adjusted some really distracting graphical issues like low draw distance and growing/shrinking foliage, they fixed a few quest bugs and other glitches, they improved enemy and companion AI a little bit. It's still not a perfect game, but it became instantly more enjoyable with just a single patch.

And yet virtually every single review was published before the patch was even released. Surely these review scores would have been more favorable, maybe averaging in the 75-80 range, had they reviewed it with the patch. To be fair, the game should never have launched without these basic components, and PB / Deep Silver deserve to take the flack for its launch status. At the same time, however, shouldn't they be cut a little slack for addressing all of the major issues in a timely manner? Either way, as far as Metacritic is concerned (and any casual consumer consulting it), Risen 2 is just another mediocre game not worth your time or money. 

So I have a few bones to pick with Metacritic. In the full article, expect a fair bit of ranting about what's wrong with Metacritic and how it's destroying the industry, with other examples besides just Risen 2.

Metacritic is a nice tool for consumers, providing links to a variety of review sources to let you quickly and easily inform yourself about a game, but I'm not fond of attaching such a rigidly objective, universal score to a video game. It's impossible to come up with a perfect quantification of a game's quality because a video game will always leave subjective impressions on the reviewer. Some games are notoriously difficult to score in a review because the subjective experience often transcends the typical conventions of game reviewing.

Take Pathologic for example -- it's not a very good game per se, with many noticeable flaws that detract from the overall experience. You've got the awkward interface, the poor translation from Russian to English, the shoddy combat system, the excruciatingly slow walking speed, the poor graphics and animations. Any reviewer would be obligated to give it a score somewhere in the 5-7/10 range, but the honest truth is that, even in its flawed state, Pathologic is a monument of a game that deserves more praise and recognition than a meager "average" score could ever describe. Yet, seeing a 6/10 on Metacritic often serves to dissuade people's interest.

Metacritic uses a 100-point grading scale, but many review sites instead use letter grades or stars. Most of us grew up in education systems where an average grade, a "C," was about a 75/100. But when Metacritic translates a "C" to their 100-point scale, they deem it a 50/100 (the perfect average of 0 and 100), which falls right at the beginning their "red" category for negative review scores. The stars system tends to work more like a recommendation system than a rating, with 3 stars meaning "average, nothing special, but still a good game." But when Metacritic translates "3 stars" to their 100-point scale, they deem it a 60/100 -- "below average" according to our perception of academic grades. 

By the same token, when reviews give a game an "A" or "5 stars," that translates to a perfect 100 on Metacritic. Most people within the industry believe that perfect scores should be extremely rare, but it's not uncommon for a game to receive an "A" or "5 stars" for being truly exceptional and standing above the crowd -- that's what those scores mean on those scales. Because of Metacritic's screwy conversion rates, however, these "A" and "5 star" reviews wind up inflating a game's metascore, while average "C" or "3 star" reviews artificially deflate the metascore. Reading the text of a given review, you often find commentary contradicting the scores Metacritic translates, which shows that the metascore isn't always very accurate.

As is evident with Risen 2, metascores can also become quickly irrelevant and obsolete with game updates. One of my most-played games of all time, Killing Floor, is three years old and has received constant free updates since its release. It currently holds an "average" metascore of 72, with virtually every review having been written near the time of its launch in 2009. Most of the Metacritic reviews criticize a general lack of content, but less than six months after release, Tripwire added several maps, a bunch of weapons, a new perk, and a new enemy type, and the updates have just kept on coming. Those early reviews no longer reflect the current product, and yet the metascore remains what it is.

Some review sources like X-Play specifically opted for their reviews not to appear on Metacritic, due to conflicts over how their review scores should be translated to Metacritic's 100-point scale. On the other side of the coin, old bastions of "no review scores" like The Escapist, who were concerned for a long time that review scores might compromise the quality of their editorial reviews, have started giving review scores with the rising prevalence of Metacritic, seemingly with the primary intention of increasing site traffic by being linked to on review aggregates. 

Metascores are becoming a more prevalent quantification within the industry, with many people relying on a game's metascore as the sole determinant for whether a game is worth their time and money. If a game doesn't receive generally favorable reviews, many people won't buy it. Metacritic is even affecting video game developers -- after the brilliant success of Fallout: New Vegas, Obsidian failed to receive its bonus from publisher Bethesda, consequently having to lay off employees and cancel a project, all because the metascore was a single point shy of Bethesda's target of 85. Publishers and stockholders pay a lot of attention to metascores. 

This kind of obsession with Metacritic is bad for the industry. With people judging a game's success by its metascore, studios are more likely than ever to make games based on current trends that they know will yield a high metascore. Big studios were always wary of gambling with new IPs, for fear of low sales and not recuperating their investments, but now they may be even less likely to gamble with unconventional, creative IPs if they're concerned of getting a lower metascore, which will detrimentally affect sales anyway. It's a shame, because I usually find better games in the 70-80 range than I do in the 90+ range. 

Let's take a look at the RPG genre; Mass Effect, Fallout 3, Dragon Age: Origins, and Skyrim all scored in the 90+ range on PC. Despite their massive budgets and extensive levels of polish, I found each one of these games deceptively shallow and, in some cases, uninspired. Note how the user rating is lower than the critical rating for each title. By contrast, Risen, Drakensang, The Witcher, Gothic II, and Vampire: The Masquerade - Bloodlines all scored in the 70-80 range, and yet they have generally superior RPG elements than any big budget, AAA RPG. Notice how, with the surprising exception of Gothic II, the user rating for each of these games is higher than the critical rating. 

And it really drives me nuts every time I see a comment on a forum, where someone casually glances over the Metacritic page and writes the entire game off just because the metascore was somewhere in the "mixed reception" category. It's training some consumers to look straight at the final score, disregarding the editorial commentary that reviewers spend a lot of time writing. It's training some publishers that metascores are the sole indication of a game's success. When clearly, there are several fundamental flaws with metascores. Consider me annoyed.