Wedgwood: Metacritic strain "ridiculous" Comments by Robert Purchese

He's right in some ways. Percentages are hyper-rubbish, you have to justify the difference between 56 and 57. What is it? I can accept that some people can't be bothered reading reviews and need a complex opinion shortened into a numerical value, but it's useless really. I can't think of any review I've ever read that I've thought "Wow, good review, but it doesn't make sense without a score."

If you disagree with a score, you don't have much basis to argue without the words of the review. Thniking a game that was given 7 should be an 8 doesn't make sense, they're just numbers. Disagreeing with a 7 because the review stated the controls were unintuitive when they're clearly brilliant (guess the review Resi 4 Wii *cough*) makes more sense, but your still disagreeing with the words rather than the number.

Wow, I went on a bit. I don't like scores, that's all.

Ehhh, personally I liked the old days of the percentage score, provided you knew how to read them.
75% was between decent and very good, 89% was almost-but-not-quite worth a gold star, and anything above 94% was a "go out and buy this right bloody now!"
Perfect at-a-glance representation of an overall opinion.

Ehhh, personally I liked the old days of the percentage score, provided you knew how to read them.
75% was between decent and very good, 89% was almost-but-not-quite worth a gold star, and anything above 94% was a "go out and buy this right bloody now!"
Perfect at-a-glance representation of an overall opinion.

So wouldn't it be easier to change that into one star (decent and good), two (almost-but-not-quite-gold), and three (buy this now) stars? Why the need for percentages?

Actually, I'd dispense with the stars and just go with "Don't buy", "Decent", "Good", "Excellent" ratings.

I agree. Everything being rated from 60%->100% is just dumb, what's the other 60% for?

GamesWorld, four reviewers - take your pick - cool!

Since when do metacritic rate porcelain, anyway?


...perhaps I should actually read the article...

I do agree publishers shouldn't focus on scores so much but all in all I think any rating system is fundamentally flawed. I am a reviewer myself and what I'm trying to accomplish when I write reviews is to give the gamer a good idea of how the game plays and especially how I felt about the game. I think the main goal, at least for me it is, when you're writing a review is trying to translate the FEEL of the game into words. Whether you work with a percentile scale or some other way of rating, doesn't really matter.

When I read reviews to decide whether I'm going to purchase a game or not, I'm not guided by the number at the bottom of the text but by the feelings I get when I read the full text. Unfortunately there are as many gamers out there who make the same mistake as the publishers and concentrate only on the numeric symbol at the end of the line...

The problem I have with reviews and scores in general is that they are subjective, both from the point of the review and from the point of the reader.

I know on several occasions that a game that has been heralded as a masterpiece has let me down, and games that have received average, or below average, scores have surprised me. This is not an experience that is unique to myself, I am sure that most gamers have come across this from time to time. I cannot see an easy way around this, unless every publications adopts a Famitsu style review panel for each game, giving you a broad spectrum of opinions on a single piece of software. To be honest, I kinda like that as it gives a view on the different perspectives that different reviewers have.

Perhaps the best way to review games is to grade them according to similar games. If you are reviewing Dead Space, for example, you could say that fans of Resident Evil 4 would love it and that fans of Fire Emblem probably would not. But then genres of games are not mutually exclusive. A gamer could love both Resident Evil 4 and Fire Emblem. Would they like Dead Space? Probably. What score do you give it then?

Its all a bit complicated or, indeed, pointless.

Maybe a more basic system is needed, based on simple words. Three words. Crap, OK, Purchase.

I would prefer stars 1-5

Fair enough. Say I give a game 98%. Would you assume that game was great? What kind of game is it? Why is it good? What kind of game is it, how are the visuals, sound, gameplay? Is the story good? How are the controls, what stopped it getting 97, 96, 95 etc?

I realise the high score would suggest it'd be of a generally high standard, but if it's a football game I'd like to know what it is that seperates it from the rest. If it's a Zelda game, what's new? The number doesn't summarise a single thing I need to consider a purchase. I suppose the sites that give a breakdown are better, in that they give scores for everything individually, but in the end I need words. Games can be expensive, if I've not got the time to read words I shouldn't be wasting money on it.

I certainly hate percentages, if you're going to over generalise things in an obscure format, do it by degrees. Scores out of 10 are enough.

EDIT: I took so long writing there's been a bunch of posts, but son't get me started on the bloody 5 star system ,which has no place for average games! It's either rubbish, a bit bad, better than average, great, or Shenmue.

>...but that game sales should still be the bottom line for royalties
LOL what?

Um I have a question. Is two and a half stars out of five stars not 50%, or 5 out of 10 stars 50%, what about 3.5 stars out of 5 that's like 75%. Why would this argument change then you put a pretty picture into the equation...
I'm sure reviewers have a bar that they set then compare the games to that bar. A review is an individuals opinion. Ask a xbot to review Killzone 2 and he'll probably give it 30% or 3 out of 10 stars if you prefer it that way.

Read the review that's why it's there form an opinion it's not that hard. You don't like the score look around there is not only one site/publication/person that reviews games. Find someone who has similar tastes. If that's too hard don't read reviews. Buy the freaking game and write your own.

Gaming companies should focus on what the public want not reviewer scores. Reviewers are gamers too, it you make a game that gamers like chances are reviewers will like them too.

My last note is: why do people get so upset with reviewers, these people play allot more games than most people they have a good idea about what makes a good game and what is missing. These people are doing you a favor playing these games and giving you a heads up, if you know better then read the paragraph about reading reviews.

Perhaps the best way to review games is to grade them according to similar games

I always liked the way (years ago when I read paper magazines, can't say whether it's still going on) some PC Magazine (Gamer or Zone, can't remember which) used to put a 'better than/not as good as' bit after the review and score to relate a game to others in the same genre.

As for scores, I do feel they're important. Much is said about movie reviews, but even Siskel and Ebert applied a score at the end (albeit only a number of thumbs up). I think a score out of 10 (or 5 stars with halves permitted) is probably as much resolution as you need though.


On a separate point, I don't like Metacritic. It seems to be full of reviews from American newspapers I've never heard of, which makes it quite difficult to judge the quality of their opinions.

The argument about review scores has gone on for long before this and has produced little change. Sure, with the percentage system it seems hard to justify what makes a point or two difference but the wider scale also allows for a greater range of flexibility. The important thing is that the review which goes along with it is well written and justifies the number that follows it overall.

Star ratings work for movies which are a contained experience and last a set amount of time - it's easier to define how good a film is. Games, which can be a lot more subjective, depend on a lot more factors (hardware, player intelligence, improvements made upon it's predecessor) so you have to have more room in the rating.

Review scores will always be subjective. Anyone for MGS4?

But I agree with Wedgwood's claim that bonuses shouldn't depend on review scores. Art and gameplay should be furthered at every opportunity, but if the finances don't work and the studio can't afford it, it'll ultimately be a lost cause for everyone.

Ignore what I said, I like what schnide said better. Pretend I said that.

I say break it down into five or so categories - playability, graphics, sound etc with marks out of 10 for each then give it a mark out of five stars based on an overall feeling .
A blanket score means littlle when it comes to deciding if it's worth the cash and is just fodder for people to bicker over for days on end in forums.

Don't buy, Buy if you like the genre/franchise, Buy it.

Or out of three stars if you don't like words.

DrizztP: I'm with you buddy but...3.5 stars out of 5 is 70% ... sorry, not usually bothered by honest mistakes but it's kind of relevant in a forum based on the different ratings systems - just goes to show how easily confusable and over-complicated things can be LOL.

Whether it's stars, percentage points or some other rating system is irrelevant. It's about the onus that a publisher puts on review scores. The reason it matters so much is because it actually makes a difference to sales. Metacritic although of course not ideal is the only aggregator of note so it's the one that always gets called out. Judging the quality of your game can be hard. I think it's great and I made it, but does my publisher agree, does he understand? Am I objective? An aggregated review score provides the best 'truth' we can manage (PR massaging aside hehe). It's a necessary evil if you want accountability for quality. It can work with you as well as against you. It's not ridiculous and it's not something to fear either.

Maybe developers simply don't make that many games that deserve a rating below 60 %. I certainly suspect that most of them work hard to create games that people actually want to buy and play. Except for Sonic Team.

I'm so tired of this "critics are the downfall of the business" nonsense. Critics are there to guide you into not buying something you will not like. You have the option not to read reviews, and you also have the power to disagree with them. So where is the problem actually?

As for whether we rate in percentages, out of ten, stars or funny hats - I don't see the difference. These are all scores. The only truly radical change would be to leave out the scores entirely, forcing people to read the actual reviews. Then Metacritic could just collect quotes, Rotten Tomato-style, but even RT converts that into an aggregate score, don't they?

Personally, I find that the best way to know if I want to play a game or not, is to read all the reviews on sites I've come to respect. And EG is always the first Even though I sometimes don't agree with them, EGs articles are always interesting reads.

Numbers are all meaningless IMO they are added by the reviewer to give the expected closure that people crave.

Pick a trusted reviewer/publication, read what they say, discern what you will like / dislike about the game, go and buy / rent / leave well alone depending on the above.

Good example - FO3 got 9 (I think) from EG and 7 from Edge, both of whom are trusted, well informed, well written sources. Did their score influence my decision to buy FO3? No. Did the comments from both reviewer influence me? Yes. Did i agree with both scores after playing the game? Yes.

Horses and courses. Even the most solid gold title won't appeal to everyone. Make your own decisions based on reliable information.

Anyone else remember being mislead by C&VG reviews from the 1980s surely already knows this though!

the metacritic phenomenon is here to stay unfortunately. There's zero incentive for it to change to something else because the game business is a business. They want "quick numbers" indicating a game's success or failure.

OXM listed Metacritic boss one of the most influenceable guys in the industry. And I think sometimes Metacritic provides a disservice. The averages lack some filtering, for example in Halo 3 average score they included the "joke score" from that sdf site.

I challenge Eurogamer to lead the way and drop the score from their reviews

"I challenge Eurogamer to lead the way and drop the score from their reviews"

I'd second that. It's irrelevant anyway. The words are where it's at.

Like him or not, at least Yahtzee of Zero Punctuation fame doesn't score. His games, I mean.

There's much less at stake when going to see a film or renting a DVD. If Empire thinks something is a 3-star movie I and I think it's 2 stars, I've only lost a fiver plus the price of a bag of Revels. If you I think a game is 2 stars, I've potentially blown £35 - £45.

3 stars is SO ambiguous - technically, it could represent anywhere from 41% up to 79% (depending on your interpretation of 1-star and 5-star). Out of 10 is better - at least you've got some way of delineating those slightly-better-than-average games. (or, in the case of the Wii, slightly-worse-than-average.)

metacritic is fine he's just sore his game sucked etc etc.

Removing the whole percentile concept is a great idea.

Shifting to a star rating like film/music reviews would help reviews to be perceived to be what they are: a reviewers opinion on the quality of a game. They are NOT absolute, and quantifiable measures of a game's true quality.

The sooner we get away from the statistics-fetishism prevalent in the game industry, the better.

Personally, I prefer scores out of 10 or out of 5 stars or a grade of A-F over percentages as an individual reviewer giving a game a score of 57% seems more "pulling numbers out of his ass" than the result of a lengthy process than someone giving a game a score of 6. I know that appearances can be deceptive but yeah, what precisely is the difference between 56 and 57?

However in the case of metacritic, if you're averaging a bunch of scores from different sources using different scoring ranges then calculating and displaying it as a mark out of a hundred seems the only way to go to me. That people (especially publishers) place too much credence in these scores (scores that represent the opinions of maybe a couple of dozen people in the world) is hardly metacritics fault; they'd find/compile their own statistics if it wasn't there and either way you'd still not get your bonus.

As for the "no-one gives less than a sixty thing", I beg to differ, as doubtlessly would anyone who's actually checked up a poorly received game on metacritic (Rise of the Argonauts, for example, got 10 out of 23 scores less than 60 including at the bottom of the list Edge Magazine and Destructoid at 30 and 1UP at 25).

@farticusmaximus
"I challenge Eurogamer to lead the way and drop the score from their reviews"

I'd second that. It's irrelevant anyway. The words are where it's at.

Hmmm... I'm surprised. I read your post and I thought it read more like a "third" or a "fourth" than a "second". ;)

I would prefer no score at all and have gamers use their brains for once. But I guess that would seriously reduce page hits so it'll never be implemented.

just get rid of the scores altogether, if people cannot make a decision based on the text of a review or cant be bothered reading it then screw 'em, most people probably just want a score to argue and compare a game on an 'opposing' system with.

"Better than Halo" couldn't have happened without a score. I'll let you all judge whether that's good or bad.

They are NOT absolute, and quantifiable measures of a game's true quality.

This is the de facto argument against review scores and aggregation tools and, quite frankly, it bores me. No one ever said they are absolute.

Metacritic is an incredibly useful tool. It’s a one-stop-shop for the industry and consumers alike to get an overall picture of how a game has been critically received. It’s that simple.

Put it this way: have you ever seen a Metacritic score that you strongly disagree with (i.e. +/- 20%).

GRID. It's at 87, if I was writing a review, and then attempting to aggregate those words into a score, it'd be significantly less than 87. The fact I'd use "below average" at least once in the review text would make it likely I'd score it lower than 50%, 5/10, or 3/5. Obviously I realise it's just my opinion, but I've never read a review that agrees with my practical experience.

I do like metacritic though, you can browse all the reviews and read the good and the bad.

Sorry, forgot
/urnwanks

Metacritic, its like pricecompare.com but with scores - sites like EG also benefit from it as they get mentioned, causing more traffic - so yeah while I 'd also like to see scores ditched, and metacritic ignored, they'll stay on, as the companies and sites who are in a position to do away with it, won't because its not in their best interest.

Agreed with the stars. And no silly half stars either. Percentages are a totally arbitrary system only useful for slackers who don't read the review. Five stars for a clear view of how good it is (dire, bad, average, good, great) then text to see if it's not just good but also your cup of tea.

But why is that really better than scoring out of 10, should you require a score?

Let's strip it down to 3. Don't Buy, Rent, Buy. You could make up all the systems you want, it'll always be just a bit at the end of the review. I can see the logic now, it is nice to give the review a sense of closure, but what form it takes is almost irrelevant. As long as it gives a reasonable idea of how good the game would be, and if it's worth buying, it'll do as an underscore to the review text.

"Splash Damage boss Paul Wedgwood reckons publishers using Metacritic review score averages to pressure developers and even hold back bonuses for negative outcomes is "ridiculous""

I agree utterly and completely, in particular about the references to bonuses. Any publisher that witholds bonuses from a high selling game simply because it got bad reviews, is morally, if not legally, corrupt.

A bonus or royalty is a share of profits. Plain and simple. If the game sold well and made lots of profit, it matters not one high hoot in heaven what it scored in reviews. For a publisher to withold a share of those profits on that basis is abhorrent. I'm not sure I can word it any more strongly.

Grrrrrr. It makes me mad.

"Fair enough. Say I give a game 98%. Would you assume that game was great? What kind of game is it? Why is it good? What kind of game is it, how are the visuals, sound, gameplay? Is the story good? How are the controls, what stopped it getting 97, 96, 95 etc?"

If it got 98% you can safely assume that all the visuals/audio/gameplay are great. That's how rare it was/is to get a mark that high.

"-snip- but in the end I need words. Games can be expensive, if I've not got the time to read words I shouldn't be wasting money on it."

Oh I agree with that too. I didn't mean that with a percentage you ONLY need the score, just that it tells you more about the reviewers overall opinion provided you are familiar with how the site/magazine 'thinks'.

"I certainly hate percentages, if you're going to over generalise things in an obscure format, do it by degrees...."

Degrees are fine too. Your Sinclair used to rate games out of degrees centigrade. :D

Perhaps EG can simply use their own [spolier] tag over the score and allow readers to choose to ignore it and judge the game from the actual ARTICLE and those w*nks who jumps to the score can unspoil the score and vents their flames without reading through the reasonings as usual?

A simple thumbs up or thumbs down would be best ...oooohh there could be the 180 degree of the thumbs being not so up or slanting more to the downside!!

Metacritic wont go away as hardcore gamers/fanboys care too much and treat those as a premier league and wants to see how their fav games/platform compares against rivals.

Still EG stands a bit taller with comments that 10 out of 10 is better system than the percentage!

The point is:

You're a developer.. The publisher has asked you to do a kids game. You both know it'll sell well, you both know that because it's a kids game, it's automatically going to get a score of 7/10 maximum if it truely is the best kids game EVER! Especially on sites like this which are "geared" towards the "hardcore".

.. But the publisher will only pay out bonuses if the game averages 85% metacritic.. Regardless how well the game sales.

Hardly fair system is it?

I kind of like some vague score just so I know if it's worth my bother to read the review. Beyond that I pay no attention

Personally i'd just prefer a simple system like:

Buy it!
Okay if you like this sort of thing
Shit - avoid.



How the hell you decide one game is 1% better than another is beyond me.

"I agree utterly and completely, in particular about the references to bonuses. Any publisher that witholds bonuses from a high selling game simply because it got bad reviews, is morally, if not legally, corrupt.

A bonus or royalty is a share of profits. Plain and simple."

A bonus can be tied to any performance criterion and doesn't necessarily mean a share of the profit. It's often tied to a good performance review or to revenue for example. But if a company holds back a bonus that's rightfully earned based on bad performance on another measure, without agreeing to such a system upfront, it's definitely abhorrent.

I like Metacritic. It made me buy Dead Space and Prince of persia. They both got 6/10 here but their among my all time favourite games and they have a high rating on Metacritic because many reviewers agree that these are actually very good games. If a game get highly rated on metacritic its always a very good game. If a game gets a bad metacritic average its a shitty game no doubth.
This is my experience from using Metacritic.

"Hardly fair system is it? "

What in the world is actually fair? Nothing is.

Electronic Gaming Monthly had one guy do the main review, and then three other staffers would give their own scores and views. The result was that sometimes one would love it and another would have it! I miss Computer Gaming World's old reviews - no score, but a sufficiently long and detailed review (as opposed to edge's two paragraphs)

Well, I'll leave the score debate nicely in the middle, as I DO look at scores and it often determines if I take the time to thoroughly read the review. Works for me.

But from a purely managerial point of view, basing bonuses on absolutely subjective objectives (i.e. the metacritic game scores) is really scandalous. It makes them depend on the whim of a few, which should NEVER happen, even if metacritic knows their stuff. Factual data, like sales, are the key.
However, it is true that the sales will be influenced by the metacritic reviews. Therefore : it is up to the masses to grow brains and an own opinion to choose the games they want to play. Whether they rely on scores to choose is their own choice. And metacritic - like everyone of us - are entitled to an opinion and to the right to express it in whatever way they see fit.
So who is to blame? De gustibus et coloribus non est disputandum....