Braben wants Metacritic for journos

By Fred Dutton Published 10 February, 2011

Frontier boss: review the reviewers.

Game reviewers should be subjected to a Metacritic-like system to judge which of them are the most reliable, Elite creator and Frontier Developments boss David Braben has suggested.

Speaking in a Develop column, Braben hailed the work of the majority of game reviewers, but singled out a few jobsworths who don't take their role seriously.

"In these connected times, Achievements or Trophies have been a curse for the small minority of hurried or irresponsible reviewers if their online ID is known," Braben wrote.

"Occasionally someone is caught out, or accused of being unreasonable. For example, there was the hoo-hah over the reviews of Space Giraffe and Kane and Lynch and the alleged connection to the firing of Jeff Gerstmann."

So, how to sort the wheat from the chaff? Braben suggested that the best reviewers be recognised by a system akin to the Metacritic aggregation site.

"Most reviewers are excellent at what they do, and it is a very hard job with, frankly, little glory. As an industry, there is something we could do to recognise this – effectively a Metacritic for reviewers," he explained.

"The best reviewers give spot-on reviews pretty soon after a game is released. They do not wait to see what others say, but nevertheless consistently come very close to the final average score. There could be a prize for the best each year."

"Don't forget – this is not intended to influence reviews – just to encourage and reward consistency – as it is not a high reviewer that gets the reward, it is the one that gets the best result," Braben clarified.

Braben's suggestion followed musings over the problems that face developers of children's or casual games. He argued it was often difficult for such a title to get a fair, balanced hearing from 'core' orientated reviewers.

"This method could also be used for non-'core' games, too, with the benchmark being either eventual sales, or eventual average user reviews," he added, "as at the moment it is a real lottery for customers buying games for their younger kids – with few trustworthy reviews – which is one of the reasons, I think, so many shovel-ware games sneak under the radar in this sector."

Frontier's last effort, Kinectimals for Xbox 360, launched back in November, winning a respectable 7/10 from Eurogamer's Keza MacDonald.

Picture of Fred.

About Fred Dutton

Fred Dutton is Eurogamer's US news editor, based in Washington DC.

Read more articles from Fred by visiting the Fred Dutton archive page.

You may also like...

Comments (60) Latest comment 1 year ago

Comments threads automatically close after 30 days, but please feel free to continue chatting on the forum!

  • beastmaster #1 1 year ago

    Who would review the reviews? Publishers I bet.

  • Rauha #2 1 year ago

    I'd love this. Just to spite IGN. 3/10.

  • Lusterpurge #3 1 year ago

    Then we'd need reviews for the review reviewers! It would be an endless cycle!

  • Optyk #4 1 year ago

    Isn't the whole point of a review to be the opinion of the reviewer?

  • NewbieZilla #5 1 year ago

    Anyone else rather hear about The Outsider than the type of stuff Braben talks about?

  • FiReTiGeR2K #6 1 year ago

    im scared

  • MattEdWithCheese #7 1 year ago

    With metacriticcritic.com you need never think for yourself again!

  • drhappy #8 1 year ago

    sounds like a great idea to me! video game journos need something to keep them in check. because as of right now, video game journos are not respected because of their low, low quality of work. for most of them anyways, eurogamer does a great job most of the time!

  • Laraman #9 1 year ago

    I don't understand rewarding a reviewer based on the fact they get closest to the 'average' score since that defeats the purpose reviewing - and that is to review a product and give an opinion on it. An opinion which can not be 'correct'. By awarding a reviewer that gets closest to a product's (or multiple product's) average score it essentially forces 'opinions' to become correct or incorrect and may also force reviewers to change their reviews to what they think will become the average score so they may win a prize.

  • bslsimes #10 1 year ago

    This is what you get when you have something like metacritic in the first place. People hanging huge amounts of importance on a number rather than reading the review to find out what the reviewer actually thought.

  • benfresh76 #11 1 year ago

    I think I know where to go for objective reviews, and if all else fails, metacritic is a great resource. What I'd really like to see is a less 'shmoozy' relationship between game publishers and journalists...Its kind of sickening, like extracting information by torture.

  • coolbritannia #12 1 year ago

    Isn't citing Kane and Lynch and the shitstorm that followed that review the exact reason why this is a terrible idea?

  • IOnlyUseSkill #13 1 year ago

    Two things which I consider to be parasites on the awesomeness that is the gaming industry. Activision and metacritic

  • Markusdragon #14 1 year ago

    Who would review the reviewers of the reviewers of the reviewers of the reviewers of the reviewers of the reviewers of the reviewers of the reviewers of the games?

  • Shadman #15 1 year ago

    You wouldn't need someone to review the reviewers, if it was only nearness to the average score that you're after. It could easily be automated, with a bit of statistical analysis to work out who was most consistently on the money.

    I'm not sure it's a good idea, though, it just adds further weight to the argument that the score is all that's important about a review, which i utterly disagree with. In my opinion, scores should be axed, and a summary paragraph that clearly states the reviewers opinion at the end of a review is used instead.

  • L0cky #16 1 year ago

    "which is one of the reasons, I think, so many shovel-ware games sneak under the radar"

    I was going to say 'or people buying games based on the box/idea/marketing/associated franchise/what their kid points at'; then realised that was his point :)

    If metacritic boosted reviews based on the history of 'accuracy' determined by the number of sales, it simply wouldn't work. Popular and good aren't mutually inclusive. Basing it on the average user score might not be too bad though, as presumably users who visit a gaming site and submit a score have a broader experience of the market than the average buyer.

    Still, the idea of EG biasing it's reviews to popular opinion gives me chills; though if metacritic did implement such a thing, what's the motivation for the gaming media to be more status quo?

  • TonyB #17 1 year ago

    What Laraman said. Suggesting that a review is somehow correct because its final score closely matches the average is ludicrous. By that measurement someone could spend an entire review explaining why the hate a particular game and clearly pointing out its flaws, then give it a top score and, if that's the score everyone else is giving it, end up being rated highly by this scale.

    Reviews are and must always be an opinion, which is partly why the publisher focus on Metacritic is dangerous, and the only thing that the larger sites should try to do is ensure the reviewer chosen will give the game a fair chance of positivity - i.e. you don't give the review of a hardcore tactical wargame to a reviewer who's gaming interests lie in fast-paced action games (at least for their main review, counterpoints like the "Why I Love/Hate..." articles are reasonable when separate). Reviews are there to help people make informed decisions on purchases, so one coming from someone with no interest in the game isn't really helpful as the people they represent aren't those who'll be considering buying it anyway.

  • L0cky #18 1 year ago

    Just noticed metacritic is owned by CBS and associated with gamespot oO

    So they certainly couldn't do such a thing themself and be taken seriously.

  • governmentyard #19 1 year ago

    Score, huh? What is it good for? Occasionally something.

    Here's what people actually do - if a game scores below six, they cross reference with other scores to make sure it's a five or below sort of crappy game and don't buy it unless it's got niche interest for them. Six or above, the score is more for internet arguments than purchasing decisions.

    I've got a dumber and better idea than this one - just make the reviewers produce the games. I' sure they'd get shit done quicker than David Braben.

    Really what he ought to be focusing on (other than releasing The Outsider) is giving Elite an HD makeover and getting the bugger out on XBLA, Steam, etc. It would probably take a whole month. Or, y'know, just another knock-off Nintendogs 'tribute'.

  • governmentyard #20 1 year ago

    @Bluedragon - you see, this is the problem with scores, they're still too subjective. You say psycho = 2.4. I say psycho = 7 (with buyer beware caveats) because she'll likely be adventurous in bed. That may not be a worthy investment to you, others prefer to take the risk.

    Ditto games really, but if we're using an awkward games/women metaphor, does this make 'Project Ten Dollar' DLC like virginity? Only the first user gets to play with that feature?

  • StooMonster #21 1 year ago

    Quis custodiet ipsos custodes?

  • Oh-Bollox #22 1 year ago

    I know this one. It's about custard, right?

  • Garfy #23 1 year ago

    Is he slagging off Jeff Gerstmann for telling the truth about kane and lynch?

  • brod #24 1 year ago

    So an opinion, quantified with a subjective score rating, is 'best' if it's close to the average?

    What a load of rubbish.

  • siro #25 1 year ago

    Any reviewer or publication that constantly ends up near the average doesn't seem very useful to me.

  • Augmentation #26 1 year ago

    Wouldn't that just come down to, 'your opinion is wrong, nyah nyah!' though?

  • Genji #27 1 year ago

    Sounds like a crock of shit, tbh. There is no similar system for movie reviews, music reviews, book reviews, etc - and neither should there be.

    If the reviewers don't play the games enough, then fair enough. That business is sloppy and should be exposed. But this? How do you decide whether a review is "spot-on"? There is not a single game ever made, ever, that everybody agrees upon. Sounds like all it would do is reward groupthink.

    And *who* would decide meta-metacritic scores? Developers? Reviewers? Or, god help us, average gamers? What a shambles that would be.

    I think it's fair to reward reviews that are particularly well-written, but that doesn't seem to be what he's talking about here.

  • mrpsb #28 1 year ago

    Elite 4 or GTFO

  • lostlain #29 1 year ago

    Braben, Love you, but really - did you think about these words before they slipped out your mouth?

  • JohnCaboose #30 1 year ago

    "The best reviewers give spot-on reviews pretty soon after a game is released. They do not wait to see what others say, but nevertheless consistently come very close to the final average score. There could be a prize for the best each year."

    Wow, it's like they're celebrating the AVERAGE. The scores shouldn't even matter to people (The actual reviews should) but somehow it's all that counts.

  • Murton #31 1 year ago

    If there's even the chance that this will make reviewers more honest and stop them from giving scores based on branding then I'm all for it. Who knows, reviewers might actually tell their readers if a game is broken and doesn't work, I'm looking at you Fallout.

    My main concern is the same Beastmaster's, who would review the reviewers? If the publishers are involved then we're just goinig to see more abuse of the system, "you give our game a 9 and we'll give your review a 9 in return"

    Then there's the issues of reviewer willy waving and fanboys. EG and CVG and IGN and everyone else (and their users) will be comparing their stats and claiming to have "won" the COD review or whatever and there's always the chance that journos might begin to mirror the games industry and sack reviewers for getting a poor score.

    Probably best to just leave it alone to be honest.

  • SteelPriest #32 1 year ago

    This might not be used to push up review scores, but might it not cause a gradual regression to the mean?

  • secombe #33 1 year ago

    Utterly bonkers, this proposed system essentially penalises any reviewer who sticks to what they believe even if that doesn't fit the general concensus. If that's the case, why have more than one reviewer?

    Mainstream movie critics are already basically one voice, and that voice is often massively out of touch with general public opinion (comedy and action often being penalised for being comedy and action, and established directors often given the benefit of the doubt even if they are churning out rubbish)

  • redcrayon #34 1 year ago

    Don't different reviewers attached different values to the scores though? Metacritic might conflate them, but a 7 from some mags with some well-written constructive criticism is better than an 'OMG 10!' from others.

    Also, the games industry itself rewards it's staff when they get good review scores. I don't think a magazine's publisher is going to reward it's journo for reaching the most average score, and a worthless prize that says 'congratulations for abandoning your values! You forgot that being a journo is about giving your honest opinion rather than guessing what the rest of the herd think'' isn't going to cut it.

    The problem here is games publishers attaching too much importance to metacritic rather than sales as it's a quick (but flawed) indicator that can't possibly take into account those games that sold loads over time through word-of-mouth, like Dead Space or Valkyria Chronicles.

  • Ferral #35 1 year ago

    It could potentially be a good idea, remember the Gamespot Bourne review, it was a joke to say the least.

    IGN arnt to bad at times, Gamespy or pretty good also, not forgetting Eurogamer (had to get that one in!). There are a few sites though that do have bad reviewers and are a joke to read. i stumbled across a ME2 PS3 review last week and the user was a XBox fanboy the way it read, everything about the game was bad in his eyes. Guess he didnt play it properly as the review suggested loads of stuff that didnt really make sense.

    However, a game might be more for a niche audience and score lower and individuals may regard it higher than the review, its down to personal preference. Reviews are great for reference points and to help make purchase decisions.

  • DiamondIce #36 1 year ago

    User reviews are always the best for me and give me far more indication as to whether a game will be any good. I find too many reviewers are trying to show off their degrees with needless padding.

  • Jay-ITFC #37 1 year ago

    Eurogamer is the only review I need. 95% of the time pretty much in line with Metacritic anyway.

  • waynenot #38 1 year ago

    Probably best to just try and get your development hell game out, David.

    Make it good and the majority of the excellent reviewers will get the score pretty much spot on, eh?

  • Redeye #39 1 year ago

    Braben seems to be coming out with so much bollocks lately, it makes my brain bleed. First all of the BS regarding second-hand sales, now this tripe. Forcing reviewers to confine their opinions to some sort of herd mentality isn't going to do anyone any favours, sunshine.

    Personally, having suffered a number of seemingly-shonky reviews for titles I've worked my balls off on, I very soon came to realise that the bottom line is this: MAN UP, TAKE IT ON THE CHIN AND GET ON WITH THE NEXT PROJECT. If a title doesn't shift numbers, it's not entirely down to reviewers, is it? There are two other major factors to take into account: firstly, is it being marketed properly (insofar as do people even know the game exists)? And secondly, is it actually any good?

    *has worked on both sides of the dev/journo fence, so knows the score from hard - and occasionally painful - experience*
    Edited by 1 at 10/02/11 @ 09:06

  • Beek4257 #40 1 year ago

    Ellie Gibson: 9/10

    Better than HALO ;-)

    /submit

  • djed #41 1 year ago

    If you weren't so bloody stupid, you'd already have an own, personal metacritic inside your head.

  • levitate #42 1 year ago

    I don't get. I always thought Metacritic was a site where all relevant reviews were collected under one roof for you to browse at your leisure. The idea that Metacritic should be some be-all-end-all review site is nothing short of idiocy. Reviewers have different opinions and sometimes a game is reviewed by more than one person and it's given two different sets of scores.

    At the end of the day it's up to the magazine/reviewer site to make sure their reviewers aren't taken straight from the gutter into the room where they are supposed to test the product, but to ensure the staff is reliable.

  • DanWhitehead #43 1 year ago

    This belief in scores being "right" does seem to be fairly unique to games. Maybe it's because film, music and literary criticism have been around for so long, we have a natural cultural understanding of how to parse the language of the reviews. I've never seen comments under a movie review furiously debating the difference between three and four stars. We just know that the rating is a broad summation of the text, a sort of intangible signpost.

    Games criticism is an infant in comparison, and based on a media where technical excellence is often as important as artistic merit, so there's a tendency to expect review scores to be "correct". The weird thing is that a system like Braben suggests would actually make lazy reviewers flourish - the ones who don't really play the games, but just wait to see what everyone else is saying and then say something similar. They'd soon be ranked among the best reviewers for being "right" all the time, while those with more distinctive takes on games would be deemed "wrong".

  • butler` #44 1 year ago

    Good games backed up with good marketing teams sell. Don't worry about the rest. Reviews are irrelevant to all but a select few, and have little to no effect on sales.

  • FireMonkey #45 1 year ago

    I love the idea. Especially if it could somehow link into Metacritic game scores, so that the score the reviewers give modify the Metacritic game score more or less dependent on their own reviewers score.

    Doing it on an average is not the right way to go though. If the public could vote how well they believe each review was done and if they agree with the score, then that would work.

    For those saying that the publishers would just vote against bad reviews, it doesn't 'really' matter. If they all do it it would average out in the end, and I believe they already have an influence over quite a lot of game reviews and so the meta-critic scores for games are already screwed.
    Edited by 1 at 10/02/11 @ 09:54

  • space_ace #46 1 year ago

    the kaiser chiefs said it best... everything is average nowadays

  • mcreddie #47 1 year ago

    But then you'd need to review the reviewers that are reviewing those reviewers!

    Can't we just rely on past experience? ie see if you agree with an earlier review?

  • arcam #48 1 year ago

    There should be a +/- button for EG's stories, not just the comments.

  • Skurmedel #49 1 year ago

    What he suggests is basically the rating comments system we have here on EG. Let people put a score on somebodies opinion.

  • PixelPirate #50 1 year ago

    oh god he should work for the government what does he want offjourno or something similar?

    Computer developers are turning in to a bunch of whining children.

    Unless the publisher releases it in the silly Christmas period, the fact they're games fail are always they're fault.

    1) Stop blaming retail (where all you cash ACTUALLY comes from) for pre-owned, we paid for it we can do what we like with it.
    2) Stop blaming journos for poor reviews, if the game is crap its going to get a bad review, live with it or do better.

    etc...

    //rage.

  • Ryze #51 1 year ago

    It's an EXCELLENT idea. For example, EG has Ellie who's reviews are normally more suited to people who have a more 'casual' perspective.

    I'm a grumpy twat when it comes to repetition, console FPSs and respecting the heritage of a game series, so - even though I don't review games, there'd be certain types of game that I'd be overly harsh on.

    This metacritic for reviewers shouldn't have just one score, but should rate a reviewer across different categories of reviews.

    Some reviewers are harsh on the Wii or PS3 due to the fact that they hate the controller, install or load times. Some people will slate Wii games due to the lack of online functions, even though there are gamers out there who never care about online multiplayer.

    Sounds interesting, this does.

  • Caimbeul #52 1 year ago

    Dont worry about things like that, just give us a next gen Elite for PC.

  • Gaol #53 1 year ago

    What utter bollocks.

    The idea that a reviewer scoring close to the average is somehow more valid than another who particularly likes/hates a game. Stating the fucking obvious: They are all individual opinions. If a particular review is clearly ill-informed then the developer should exclude it from any bonus measurements, but there's little point trying to police 'poor' reviews when anyone with a blog can spout what they like.

  • actionfitz #54 1 year ago

    Who Watches the Watchmen?

    :D

  • actionfitz #55 1 year ago

    Who Watches the Watchmen?

    :D

  • neonxaos #56 1 year ago

    Wipe your tears, Braben.

  • dunbain #57 1 year ago

    Kid games get slammed all the freaking time. It beseeches me how a "hardcore" site that focuses on the mature-side of the market picks up a poppy little-kid's licensed game and deems it a bit short on the fun.

    Eurogamer gets major props on this with their megaton sister-site partnership. So much more sensible.

    I think this proposal's solution can be as simple as rottentomatoes.com's "cream of the crop" feature, which basically only accounts for renown reviewers/publications. Thus, you have your overall score and "critic" score, the difference being, the so-called critic is bound by either reader or publication standards. As in, you expect quality from Eurogamer, 1UP, Joystiq, etc., and not "ZTGameDomain" or wtf.

  • Normski #58 1 year ago

    What's great about the internet is that if you have an idea like this, rather than just talk about it, you can do something about it. But I imagine that David Braben isn't too bothered about actually putting any work in and would rather just bait games journalists and get his face around.

  • parablax #59 1 year ago

    Sounds like Frontier is in serious trouble - a company with massive workforce and overheads the only games to come out recently are kinectimals and lostwinds.

    - outsider cancelled/on-hold
    - job lay offs
    - DB blaming every man and his dog for poor sales

    Looks like those of us waiting for Elite 4 are in for major disappointment.

  • hiddenranbir #60 1 year ago

    I am a great reviewer.

    Reviews can have things like "I found this review helpful"

    and then you get stats like 57/58 people found this review helpful.