Wedgwood: Metacritic strain "ridiculous" Related content
-
Exclusive - Quake Wars: Enemy Territory - Comparison video
-
Latest Enemy Territory: Quake Wars screenshots
- Review Enemy Territory: Quake Wars [59]
Advertisement
Wedgwood: Metacritic strain "ridiculous" News
PC
Xbox 360
PlayStation 3
News
by
Robert Purchese
Splash Damage boss Paul Wedgwood reckons publishers using Metacritic review score averages to pressure developers and even hold back bonuses for negative outcomes is "ridiculous".
He's not been on the receiving end himself, but, speaking to GamesIndustry.biz, Wedgwood put across his case for ditching percentile-based scores altogether, as they put too much pressure on journalists to justify exact scores; numbers that developers waste time trying to interpret.
"We know that some websites score quite high and some quite low, but in general, all websites tend to score between 60 and 100. There's never a 37. It's as if that whole section doesn't exist, so zero starts at 60, so three stars, and goes up to five. It's just not really an accurate enough measure," said Wedgwood.
"I think that if anything, the games press should take the pressure off themselves, and just go across to star ratings, which for films is nothing more than a recommendation that you buy it, watch it when you get the chance, or rush out and see it straight away, and it's your personal recommendation. It's not a 'score'. If that was all you did, nobody would hate you guys for it.
"Out of ten is a good start," he added.
"Percentiles put too much pressure on a journalist to justify an exact score. It puts too much pressure on the developer to try and identify these criteria that lead to very specific point increases or decreases, which is not at all what the developer should be focusing on."
Wedgwood said publishers offering additional bonuses for critical acclaim was a tip top idea, but that game sales should still be the bottom line for royalties. After all, he added, some of the biggest money-makers have netted only average scores.
However, those numbers and "the shameless pursuit of critical acclaim" remain a strong focus for Wedgwood and Splash Damage, which signed a long-term development deal with Bethesda (Fallout 3, Oblivion) and owner ZeniMax last year. But exactly what the partnership will produce is so far unknown.
"It's not so much that it's purely the reception that we get from critics alone - we also mean critical acclaim from fans, and feeling like we've made something that they want to play," said Wedgwood, adding that quality was more important that shipping a game on time.
Head over to GamesIndustry.biz for the full interview with Paul Wedgwood.
Advertisement
Are you excited about Enemy Territory: Quake Wars on PC/Xbox 360/PlayStation 3?
View Eurogamer readers most anticipated games
Thanks!
Related Games
Enemy Territory: Quake Wars
Latest Features
-
Enemy Territory: Quake Wars Review [59]
-
Enemy Territory: Quake Wars Preview [32]
-
Enemy Territory: Quake Wars Review [60]
-
Enemy Territory: Quake Wars Interview [21]
-
Enemy Territory: Quake Wars First Impressions [57]
Latest Videos
Exclusive - Quake Wars: Enemy Territory - Comparison video [3] 30 June, 2008
Latest News
-
Console teams doing Quake Wars demos [1]
-
PC gamers moving to consoles - id [46]
-
More stuff pops up on European PSN [21]
-
Quake Wars demo lands on Live [14]
-
Retailers talk Quake Wars console date [4]
Latest Screenshots
30 June, 2008 [2]
Want to comment on this article? Log in, or register!
Comments: 1-50 of 50 in total
Poster
|
0
He's right in some ways. Percentages are hyper-rubbish, you have to justify the difference between 56 and 57. What is it? I can accept that some people can't be bothered reading reviews and need a complex opinion shortened into a numerical value, but it's useless really. I can't think of any review I've ever read that I've thought "Wow, good review, but it doesn't make sense without a score."
If you disagree with a score, you don't have much basis to argue without the words of the review. Thniking a game that was given 7 should be an 8 doesn't make sense, they're just numbers. Disagreeing with a 7 because the review stated the controls were unintuitive when they're clearly brilliant (guess the review Resi 4 Wii *cough*) makes more sense, but your still disagreeing with the words rather than the number. Wow, I went on a bit. I don't like scores, that's all. |
|
0
Ehhh, personally I liked the old days of the percentage score, provided you knew how to read them.
75% was between decent and very good, 89% was almost-but-not-quite worth a gold star, and anything above 94% was a "go out and buy this right bloody now!" Perfect at-a-glance representation of an overall opinion. |
|
0
Ehhh, personally I liked the old days of the percentage score, provided you knew how to read them.
75% was between decent and very good, 89% was almost-but-not-quite worth a gold star, and anything above 94% was a "go out and buy this right bloody now!" Perfect at-a-glance representation of an overall opinion. So wouldn't it be easier to change that into one star (decent and good), two (almost-but-not-quite-gold), and three (buy this now) stars? Why the need for percentages? Actually, I'd dispense with the stars and just go with "Don't buy", "Decent", "Good", "Excellent" ratings. |
|
MrChuckles
20/01/09 @ 10:51
0
I agree. Everything being rated from 60%->100% is just dumb, what's the other 60% for?
|
|
Mentalist(air)
20/01/09 @ 10:55
0
Since when do metacritic rate porcelain, anyway?
...perhaps I should actually read the article... |
|
0
I do agree publishers shouldn't focus on scores so much but all in all I think any rating system is fundamentally flawed. I am a reviewer myself and what I'm trying to accomplish when I write reviews is to give the gamer a good idea of how the game plays and especially how I felt about the game. I think the main goal, at least for me it is, when you're writing a review is trying to translate the FEEL of the game into words. Whether you work with a percentile scale or some other way of rating, doesn't really matter.
When I read reviews to decide whether I'm going to purchase a game or not, I'm not guided by the number at the bottom of the text but by the feelings I get when I read the full text. Unfortunately there are as many gamers out there who make the same mistake as the publishers and concentrate only on the numeric symbol at the end of the line... |
|
JohnnyWashnGo
20/01/09 @ 10:59
0
The problem I have with reviews and scores in general is that they are subjective, both from the point of the review and from the point of the reader.
I know on several occasions that a game that has been heralded as a masterpiece has let me down, and games that have received average, or below average, scores have surprised me. This is not an experience that is unique to myself, I am sure that most gamers have come across this from time to time. I cannot see an easy way around this, unless every publications adopts a Famitsu style review panel for each game, giving you a broad spectrum of opinions on a single piece of software. To be honest, I kinda like that as it gives a view on the different perspectives that different reviewers have. Perhaps the best way to review games is to grade them according to similar games. If you are reviewing Dead Space, for example, you could say that fans of Resident Evil 4 would love it and that fans of Fire Emblem probably would not. But then genres of games are not mutually exclusive. A gamer could love both Resident Evil 4 and Fire Emblem. Would they like Dead Space? Probably. What score do you give it then? Its all a bit complicated or, indeed, pointless. Maybe a more basic system is needed, based on simple words. Three words. Crap, OK, Purchase. |
|
0
Fair enough. Say I give a game 98%. Would you assume that game was great? What kind of game is it? Why is it good? What kind of game is it, how are the visuals, sound, gameplay? Is the story good? How are the controls, what stopped it getting 97, 96, 95 etc?
I realise the high score would suggest it'd be of a generally high standard, but if it's a football game I'd like to know what it is that seperates it from the rest. If it's a Zelda game, what's new? The number doesn't summarise a single thing I need to consider a purchase. I suppose the sites that give a breakdown are better, in that they give scores for everything individually, but in the end I need words. Games can be expensive, if I've not got the time to read words I shouldn't be wasting money on it. I certainly hate percentages, if you're going to over generalise things in an obscure format, do it by degrees. Scores out of 10 are enough. EDIT: I took so long writing there's been a bunch of posts, but son't get me started on the bloody 5 star system ,which has no place for average games! It's either rubbish, a bit bad, better than average, great, or Shenmue.
Edited 1 times,
most recently on 20/01/09 @ 11:01
|
|
0
Um I have a question. Is two and a half stars out of five stars not 50%, or 5 out of 10 stars 50%, what about 3.5 stars out of 5 that's like 75%. Why would this argument change then you put a pretty picture into the equation...
I'm sure reviewers have a bar that they set then compare the games to that bar. A review is an individuals opinion. Ask a xbot to review Killzone 2 and he'll probably give it 30% or 3 out of 10 stars if you prefer it that way. Read the review that's why it's there form an opinion it's not that hard. You don't like the score look around there is not only one site/publication/person that reviews games. Find someone who has similar tastes. If that's too hard don't read reviews. Buy the freaking game and write your own. Gaming companies should focus on what the public want not reviewer scores. Reviewers are gamers too, it you make a game that gamers like chances are reviewers will like them too. My last note is: why do people get so upset with reviewers, these people play allot more games than most people they have a good idea about what makes a good game and what is missing. These people are doing you a favor playing these games and giving you a heads up, if you know better then read the paragraph about reading reviews. |
|
Mentalist(air)
20/01/09 @ 11:13
0
Perhaps the best way to review games is to grade them according to similar games
I always liked the way (years ago when I read paper magazines, can't say whether it's still going on) some PC Magazine (Gamer or Zone, can't remember which) used to put a 'better than/not as good as' bit after the review and score to relate a game to others in the same genre. As for scores, I do feel they're important. Much is said about movie reviews, but even Siskel and Ebert applied a score at the end (albeit only a number of thumbs up). I think a score out of 10 (or 5 stars with halves permitted) is probably as much resolution as you need though. On a separate point, I don't like Metacritic. It seems to be full of reviews from American newspapers I've never heard of, which makes it quite difficult to judge the quality of their opinions. |
|
0
The argument about review scores has gone on for long before this and has produced little change. Sure, with the percentage system it seems hard to justify what makes a point or two difference but the wider scale also allows for a greater range of flexibility. The important thing is that the review which goes along with it is well written and justifies the number that follows it overall.
Star ratings work for movies which are a contained experience and last a set amount of time - it's easier to define how good a film is. Games, which can be a lot more subjective, depend on a lot more factors (hardware, player intelligence, improvements made upon it's predecessor) so you have to have more room in the rating. Review scores will always be subjective. Anyone for MGS4? But I agree with Wedgwood's claim that bonuses shouldn't depend on review scores. Art and gameplay should be furthered at every opportunity, but if the finances don't work and the studio can't afford it, it'll ultimately be a lost cause for everyone. |
|
bodypopper
20/01/09 @ 11:29
0
I say break it down into five or so categories - playability, graphics, sound etc with marks out of 10 for each then give it a mark out of five stars based on an overall feeling .
A blanket score means littlle when it comes to deciding if it's worth the cash and is just fodder for people to bicker over for days on end in forums.
Edited 3 times,
most recently on 20/01/09 @ 11:30
|
|
gav_and_the_gavster
20/01/09 @ 11:51
0
DrizztP: I'm with you buddy but...3.5 stars out of 5 is 70% ... sorry, not usually bothered by honest mistakes but it's kind of relevant in a forum based on the different ratings systems - just goes to show how easily confusable and over-complicated things can be LOL.
|
|
0
Whether it's stars, percentage points or some other rating system is irrelevant. It's about the onus that a publisher puts on review scores. The reason it matters so much is because it actually makes a difference to sales. Metacritic although of course not ideal is the only aggregator of note so it's the one that always gets called out. Judging the quality of your game can be hard. I think it's great and I made it, but does my publisher agree, does he understand? Am I objective? An aggregated review score provides the best 'truth' we can manage (PR massaging aside hehe). It's a necessary evil if you want accountability for quality. It can work with you as well as against you. It's not ridiculous and it's not something to fear either.
|
|
0
Maybe developers simply don't make that many games that deserve a rating below 60 %. I certainly suspect that most of them work hard to create games that people actually want to buy and play. Except for Sonic Team.
I'm so tired of this "critics are the downfall of the business" nonsense. Critics are there to guide you into not buying something you will not like. You have the option not to read reviews, and you also have the power to disagree with them. So where is the problem actually? As for whether we rate in percentages, out of ten, stars or funny hats - I don't see the difference. These are all scores. The only truly radical change would be to leave out the scores entirely, forcing people to read the actual reviews. Then Metacritic could just collect quotes, Rotten Tomato-style, but even RT converts that into an aggregate score, don't they? Personally, I find that the best way to know if I want to play a game or not, is to read all the reviews on sites I've come to respect. And EG is always the first Even though I sometimes don't agree with them, EGs articles are always interesting reads. |
|
Snakehips76
20/01/09 @ 12:10
0
Numbers are all meaningless IMO they are added by the reviewer to give the expected closure that people crave.
Pick a trusted reviewer/publication, read what they say, discern what you will like / dislike about the game, go and buy / rent / leave well alone depending on the above. Good example - FO3 got 9 (I think) from EG and 7 from Edge, both of whom are trusted, well informed, well written sources. Did their score influence my decision to buy FO3? No. Did the comments from both reviewer influence me? Yes. Did i agree with both scores after playing the game? Yes. Horses and courses. Even the most solid gold title won't appeal to everyone. Make your own decisions based on reliable information. Anyone else remember being mislead by C&VG reviews from the 1980s surely already knows this though! |
|
farticusmaximus
20/01/09 @ 12:51
0
"I challenge Eurogamer to lead the way and drop the score from their reviews"
I'd second that. It's irrelevant anyway. The words are where it's at. |
|
0
There's much less at stake when going to see a film or renting a DVD. If Empire thinks something is a 3-star movie I and I think it's 2 stars, I've only lost a fiver plus the price of a bag of Revels. If you I think a game is 2 stars, I've potentially blown £35 - £45.
3 stars is SO ambiguous - technically, it could represent anywhere from 41% up to 79% (depending on your interpretation of 1-star and 5-star). Out of 10 is better - at least you've got some way of delineating those slightly-better-than-average games. (or, in the case of the Wii, slightly-worse-than-average.) |
|
0
Removing the whole percentile concept is a great idea.
Shifting to a star rating like film/music reviews would help reviews to be perceived to be what they are: a reviewers opinion on the quality of a game. They are NOT absolute, and quantifiable measures of a game's true quality. The sooner we get away from the statistics-fetishism prevalent in the game industry, the better. |
|
0
Personally, I prefer scores out of 10 or out of 5 stars or a grade of A-F over percentages as an individual reviewer giving a game a score of 57% seems more "pulling numbers out of his ass" than the result of a lengthy process than someone giving a game a score of 6. I know that appearances can be deceptive but yeah, what precisely is the difference between 56 and 57?
However in the case of metacritic, if you're averaging a bunch of scores from different sources using different scoring ranges then calculating and displaying it as a mark out of a hundred seems the only way to go to me. That people (especially publishers) place too much credence in these scores (scores that represent the opinions of maybe a couple of dozen people in the world) is hardly metacritics fault; they'd find/compile their own statistics if it wasn't there and either way you'd still not get your bonus. As for the "no-one gives less than a sixty thing", I beg to differ, as doubtlessly would anyone who's actually checked up a poorly received game on metacritic (Rise of the Argonauts, for example, got 10 out of 23 scores less than 60 including at the bottom of the list Edge Magazine and Destructoid at 30 and 1UP at 25). |
|
0
They are NOT absolute, and quantifiable measures of a game's true quality.
This is the de facto argument against review scores and aggregation tools and, quite frankly, it bores me. No one ever said they are absolute. Metacritic is an incredibly useful tool. It’s a one-stop-shop for the industry and consumers alike to get an overall picture of how a game has been critically received. It’s that simple. Put it this way: have you ever seen a Metacritic score that you strongly disagree with (i.e. +/- 20%).
Edited 1 times,
most recently on 20/01/09 @ 13:53
|
|
0
GRID. It's at 87, if I was writing a review, and then attempting to aggregate those words into a score, it'd be significantly less than 87. The fact I'd use "below average" at least once in the review text would make it likely I'd score it lower than 50%, 5/10, or 3/5. Obviously I realise it's just my opinion, but I've never read a review that agrees with my practical experience.
I do like metacritic though, you can browse all the reviews and read the good and the bad. Sorry, forgot /urnwanks
Edited 2 times,
most recently on 20/01/09 @ 14:57
|
|
Vertical Stand
20/01/09 @ 14:03
0
Metacritic, its like pricecompare.com but with scores - sites like EG also benefit from it as they get mentioned, causing more traffic - so yeah while I 'd also like to see scores ditched, and metacritic ignored, they'll stay on, as the companies and sites who are in a position to do away with it, won't because its not in their best interest.
|
|
0
But why is that really better than scoring out of 10, should you require a score?
Let's strip it down to 3. Don't Buy, Rent, Buy. You could make up all the systems you want, it'll always be just a bit at the end of the review. I can see the logic now, it is nice to give the review a sense of closure, but what form it takes is almost irrelevant. As long as it gives a reasonable idea of how good the game would be, and if it's worth buying, it'll do as an underscore to the review text. |
|
kangarootoo
20/01/09 @ 17:45
0
"Splash Damage boss Paul Wedgwood reckons publishers using Metacritic review score averages to pressure developers and even hold back bonuses for negative outcomes is "ridiculous""
I agree utterly and completely, in particular about the references to bonuses. Any publisher that witholds bonuses from a high selling game simply because it got bad reviews, is morally, if not legally, corrupt. A bonus or royalty is a share of profits. Plain and simple. If the game sold well and made lots of profit, it matters not one high hoot in heaven what it scored in reviews. For a publisher to withold a share of those profits on that basis is abhorrent. I'm not sure I can word it any more strongly. Grrrrrr. It makes me mad. |
|
0
"Fair enough. Say I give a game 98%. Would you assume that game was great? What kind of game is it? Why is it good? What kind of game is it, how are the visuals, sound, gameplay? Is the story good? How are the controls, what stopped it getting 97, 96, 95 etc?"
If it got 98% you can safely assume that all the visuals/audio/gameplay are great. That's how rare it was/is to get a mark that high. "-snip- but in the end I need words. Games can be expensive, if I've not got the time to read words I shouldn't be wasting money on it." Oh I agree with that too. I didn't mean that with a percentage you ONLY need the score, just that it tells you more about the reviewers overall opinion provided you are familiar with how the site/magazine 'thinks'. "I certainly hate percentages, if you're going to over generalise things in an obscure format, do it by degrees...." Degrees are fine too. Your Sinclair used to rate games out of degrees centigrade. :D |
|
GamesConnoisseur
20/01/09 @ 18:43
0
Perhaps EG can simply use their own [spolier] tag over the score and allow readers to choose to ignore it and judge the game from the actual ARTICLE and those w*nks who jumps to the score can unspoil the score and vents their flames without reading through the reasonings as usual?
A simple thumbs up or thumbs down would be best ...oooohh there could be the 180 degree of the thumbs being not so up or slanting more to the downside!! Metacritic wont go away as hardcore gamers/fanboys care too much and treat those as a premier league and wants to see how their fav games/platform compares against rivals. Still EG stands a bit taller with comments that 10 out of 10 is better system than the percentage! |
|
0
The point is:
You're a developer.. The publisher has asked you to do a kids game. You both know it'll sell well, you both know that because it's a kids game, it's automatically going to get a score of 7/10 maximum if it truely is the best kids game EVER! Especially on sites like this which are "geared" towards the "hardcore". .. But the publisher will only pay out bonuses if the game averages 85% metacritic.. Regardless how well the game sales. Hardly fair system is it? |
|
0
"I agree utterly and completely, in particular about the references to bonuses. Any publisher that witholds bonuses from a high selling game simply because it got bad reviews, is morally, if not legally, corrupt.
A bonus or royalty is a share of profits. Plain and simple." A bonus can be tied to any performance criterion and doesn't necessarily mean a share of the profit. It's often tied to a good performance review or to revenue for example. But if a company holds back a bonus that's rightfully earned based on bad performance on another measure, without agreeing to such a system upfront, it's definitely abhorrent. |
|
muscleblade
21/01/09 @ 09:26
0
I like Metacritic. It made me buy Dead Space and Prince of persia. They both got 6/10 here but their among my all time favourite games and they have a high rating on Metacritic because many reviewers agree that these are actually very good games. If a game get highly rated on metacritic its always a very good game. If a game gets a bad metacritic average its a shitty game no doubth.
This is my experience from using Metacritic. |
|
muscleblade
21/01/09 @ 09:27
0
"Hardly fair system is it? "
What in the world is actually fair? Nothing is. |
|
metalangel
21/01/09 @ 15:45
0
Electronic Gaming Monthly had one guy do the main review, and then three other staffers would give their own scores and views. The result was that sometimes one would love it and another would have it! I miss Computer Gaming World's old reviews - no score, but a sufficiently long and detailed review (as opposed to edge's two paragraphs)
|
|
0
Well, I'll leave the score debate nicely in the middle, as I DO look at scores and it often determines if I take the time to thoroughly read the review. Works for me.
But from a purely managerial point of view, basing bonuses on absolutely subjective objectives (i.e. the metacritic game scores) is really scandalous. It makes them depend on the whim of a few, which should NEVER happen, even if metacritic knows their stuff. Factual data, like sales, are the key. However, it is true that the sales will be influenced by the metacritic reviews. Therefore : it is up to the masses to grow brains and an own opinion to choose the games they want to play. Whether they rely on scores to choose is their own choice. And metacritic - like everyone of us - are entitled to an opinion and to the right to express it in whatever way they see fit. So who is to blame? De gustibus et coloribus non est disputandum....
Edited 1 times,
most recently on 21/01/09 @ 18:47
|
Comments: 1-50 of 50 in total
Want to comment on this article? Log in, or register!
