Levine warns against early focus testing Related content
BioShock game info
£17.85
RRP: £19.99
Last few copies! Hurry!
Buy Online
Order before 5.30pm and receive your order the next working day!
Advertisement
Levine warns against early focus testing News
PC
Xbox 360
PlayStation 3
News
by
Robert Purchese
2K Boston creative boss Ken Levine has warned developers of the dangers of focus-testing games too early - saying that nobody would have wanted to play Bioshock if the idea had been focus-tested years before release.
Speaking at the Develop conference this morning, Levine said that if Irrational games (as the studio was then called) had gone to a focus group and said, "how would you like to play a first-person shooter set in an underwater objectivist utopia", "nobody would have wanted to play that".
He said there had been a similar experience with the development of Thief, a game which deliberately flaunted the conventions of the top FPS games of the time like Quake and Unreal. "Here was a game where you had a sword and a bow and arrow, and you had to hide under a dresser for half an hour... It's not exactly saying, rev up the focus group, they're going to love this.
"We had people going to focus test groups and asking, 'Would you like more weapons or less weapons?'" explained Levine - the answer, of course, being obvious. "Marketing people have really improved their methods, and it's much more sophisticated now, but you can still get some really bad data out of these things."
Later on in development, when the game is coming together and the scope of potential changes are reduced, Levine does believe that focus testing has a place.
"You have to be the author," he said. "You can't move with the wind. But if everyone tells you they hate something... They can't tell you how to fix your game, but you have to be receptive to them saying there's a problem."
(This is the last story we'll do from that session, we promise - although we'll be chatting to Levine in person later on, and will let you know if he says anything interesting.)
Advertisement
Are you excited about BioShock on PC/Xbox 360/PlayStation 3?
View Eurogamer readers most anticipated games
Thanks!
Related Games
BioShock
Latest Features
-
BioShock Interview [48]
-
BioShock Review [1112]
-
BioShock First Impressions [74]
-
BioShock Interview [45]
-
BioShock Interview [37]
Latest Videos
Exclusive: BioShock comparison video [15] 4 November, 2008
Latest News
-
BioShock Big Daddy celebrated in plush [15]
-
Irrational reveals cut BioShock ideas [18]
-
Pre-order BioShock 2 PC, get original free [22]
-
Levine talks scrapped Irrational game [35]
-
BioShock 1 due for Mac next month [32]
Latest Screenshots
20 August, 2007 [30]
Release Date Out now!
Want to comment on this article? Log in, or register!
Comments: 1-26 of 26 in total
Poster
|
Greychapter
30/07/08 @ 11:19
0
I totally agree. I've worked on several (good) titles in the past where the concepts were focus tested and the titles got canned. The problem with it is that most of the public know what they like and are unable to imagine what the final game will be like when summed up by a couple of paragraphs of text. This system is in some way responsible for a lot of the games which feel the same - the public likes game "X" so in focus testing they lean towards concepts that are similar to "X".
|
|
0
I kind of agree with him on this one but he's purposely demonising the focus-testing process.
"how would you like to play a first-person shooter set in an underwater objectivist utopia" - In games development you don't structure focus-testing like this. You show the group the game (in some form) and gather their reactions to it. You don't leave it up to them to visualise what they think the game is like. I'm sure Bioshock was sigend and in development before they got the focus groups in. |
|
0
One of the biggest problems with basically any media industry today is that it's run by marketing types. Marketing and business people are in it to make money the cheapest, easiest way possible, and the cheapest easiest way to make media(games, films, music) to successfully sell to the masses is to recycle the last successful product endlessly until nobody buys it.
|
|
Mentalist(air)
30/07/08 @ 12:23
0
Oh for fuck sake, how many more "Bioshock should have failed" articles do we need?
|
|
0
Bollocks to focus groups. Bioshock wasn't a flop, and the main reason it wasn't was because of the demo.
Still amazed to see this same tired old news story being diced up and rehashed in different ways. Makes me actually wish that it'd bombed at times, even though I enjoyed it immensely - just so idiots like Levine had to go out and do some work instead of endlessly spouting crap like this... |
|
kangarootoo
30/07/08 @ 12:35
0
I don't think you should ever focus test your concept in the way he is describing. That kind of hands-on playtesting is good for testing mechanics, but there are much better ways to gauge how a concept will be received and marketing people are really the ones with the knowhow to do that. I suppose that is in essence what he is saying.
"One of the biggest problems with basically any media industry today is that it's run by marketing types. Marketing and business people are in it to make money the cheapest, easiest way possible, and the cheapest easiest way to make media(games, films, music) to successfully sell to the masses is to recycle the last successful product endlessly until nobody buys it." I know that is quite a popular point of view, but as often as not it is a distortion of the truth. A lot of talented marketing teams are very aware of what will sell and what won't, and that knowledge is vital is making games is your business rather than your hobby. You say "make money the cheapest, easiest way possible" as if that is a bad thing. What company in its right mind would want to make less money, or take a harder more expensive route to the same money? Selling games is absolutely crucial if you want to stay in business long enough to continue to make games, and knowing what will sell and what won't is a mandatory part of that. I know some people believe that making games should be about the art first and foremost, but that belief just isn't based in reality. The way I see it, I like making games and I want to continue doing it for a long time, and if the games I work on don't sell I won't get to do that.
Edited 1 times,
most recently on 30/07/08 @ 13:35
|
|
0
so kangarootoo - would you be happy working on shovelware if it sold even though it wasnt at all challenging for you to make? obviously theres a balance here games do need to sell to keep people in jobs, but the medium wont progress if we keep on churning out the same old stuff. I suppose if all depends on how much influence the focus groups / marketeers have. Any industry needs people willing to take risks in order to survive, but they are few and far between
|
|
kangarootoo
30/07/08 @ 12:58
0
"would you be happy working on shovelware if it sold even though it wasnt at all challenging for you to make?"
That is a big old strawman right there. "obviously theres a balance here games do need to sell to keep people in jobs, but the medium wont progress if we keep on churning out the same old stuff." The part where you say "obviously there is a balance" is the crucial part. I didn't say that making sales is the ONLY thing of importance, but it is extremely important in its own right as without sales nothing else will allow you to continue to make games of any kind, works of art OR shovelware. The industry does need to take risks, but they should never be needless risks. Working on a new concept is a fantastic thing to be involved in, but that doesn't mean that no mind should be given to finding out whether anyone out there might be interested in buying the thing. If someone conducted no market research at all about a new concept, I would say they were unwise. If they conducted market research (proper market research, which isn't the same as focus testing as previously discussed), found that their new concept wouldn't sell, but made it anyway... I would say they were mad. I absolutely want to work on an exciting product, but I also want to be able to work on the exciting product after that, and the one after that, and pay my bills. |
|
0
kangarootoo,
Actually I think I'd happily work on the odd bit of shovelware if it meant that I made enough money to enable me make the game that people said wouldn't sell and you say I'd be mad to make. That way if everybody was right and nobody buys it then it doesn't matter because the shovelware has made me enough money to continue, but if I was right then something genuinely exciting may have made it to the market and opened doors for others. You need marketing and business analysis to generate enough revenue from safe projects, to give people the chance to occasionally ignore what they say and do something completely original and off the wall. |
|
kangarootoo
30/07/08 @ 13:58
0
I absolutely see where you are coming from, and if it were your own dosh you were knowingly chucking into the fire then I guess that is cool. I'm just not sure it makes good business sense (maybe it doesn't and that is exactly your point :) )
There is nothing wrong with a successful company spending money on stuff that is fun for its staff to be involved with, and I guess making an interesting game that you know won't sell is really just like paying for a big company picnic, but in a slightly different guise. I think you have convinced me actually (at least in spirit), so long as the costs are known and you actually HAVE the big selling shovelware, maybe its ok to spend a bit of dosh on just making fun stuff for its own sake. The games dev business being the unstable beast that it is though, I'm not sure just how cheap a non-seller and how huge a cash cow you would need to have before I felt it was a reasonable thing to chuck money away on :) |
|
kangarootoo
30/07/08 @ 14:46
0
Oh hey man, I didn't take any offense whatsoever. And I certainly didn't take any of it personally. You were perfectly polite, and I certainly don't mind the odd pointed or provocative comment (I write enough of them myself) :)
This has been a good thread, an actual proper discussion and exchange of ideas. Rare enough on the internet at the best of times. |
|
kangarootoo
30/07/08 @ 14:53
0
P.s. you are quite right about the recent boom in what is essentially sharware on consoles that has taken off this generation. The process is still not cheap compared to the traditional PC shareware scene, as certification and dev kits are bloody pricey. I would also say that just because the relative costs seem small, a small dev can still not afford to chuck money anyway. If anything, a small dev may be a greater risk in spite of the savings stuff like XNA affords them.
That said, a dev that works on several XBLA titles (for example) is able to spread the investment, and thus spread the risk, which makes good sense. One thing that does really help (and allows devs to take greater risks in these areas) is that XBLM and the PSN Store are exactly the places where gamers are happy to take chances with their purchases. Gamers tend to expect slightly more zany stuff via these channels, AND the cost of the average XBLA or PSN game is such that gamers are also more willing to take a chance on something a bit left field. I think both these store fronts (I'm not excluding Wii Ware from my comment 'cos I don't rate it, I just don't know anything about it) are really news for everyone, devs and gamers. Consoles have been crying out for a proper budget line of software for years. I don't mean platinum ranges, which are really just price cuts on old games that aren't selling anymore. I mean the equivalent of PC shareware or the £1.99 range on the Amiga. Cheap to make, cheap to sell, driving variety and innovation in a relatively safe environment. |
|
0
I didn't say it was bad business, I was explaining the reason why the media industries basically churn out the same stuff year after year with little or no innovation taking place.
Granted, if someone finds that a given original idea will probably not sell, and then don't bother producing it, that obviously makes sense. These things are not free, after all. As long as their marketing research methods are sound, anyway. I personally would like there to be a lot more original ideas in such industries, especially in TV, movies and games, however, if that would result in the total collapse of the industry in question then obviously... It aint such a good idea, is it. I wasn't suggesting some kind of new status quo, I was just explaining why we get so much shovelware and cloning going on. |
|
0
"One of the biggest problems with basically any media industry today is that it's run by marketing types."
Well, you mean business really. Eg people who employ people and make anything at all. Marketing, the way Levine and others are using it in this context, is about the entire market facing aspect of a business. To figure out what you should make so you sell some product and your employees get paid. It's genuinely absurd to think that this whole approach is wrong. Far from it, a decent marketing/strategy department ought to be able to shed some valuable light on what kind of game people want while allowing creatives the freedom to make something unique. Interesting I would have said a key problem with Bioshock was the fact they turned what could have been the best action RPG of all time into essentially a slightly smarter shooter. That's a critical decision though. As far as a marketing decision, what they did was genius and they made a fuckton of money. You'll have a hard time convincing anyone of the developers that it was wrong because it was *good* for them all, personally and professionally, to sell that game to the mass market. There ends your business 101. |
|
stoopidgreg
31/07/08 @ 08:26
0
"how would you like to play a first-person shooter set in an underwater objectivist utopia", "nobody would have wanted to play that".
i disagree. maybe if you just asked randoms off the street they would give it 5 seconds thought and say "nah give me mario" but if you actually asked gamers that and described a bit more about the game they would love the idea. oh and it was a dystopia, not utopia. |
|
kangarootoo
31/07/08 @ 08:44
0
@stoopidgreg
/pedantic hat on Isn't the point that all utopia are impossible to maintain, and although they start with the best of intentions, they inevitably devolve into the dystopia you describe. /pedant hat off /pedant hat off /CTRL-P pedant hat OFF /help, its got jammed |
|
stoopidgreg
31/07/08 @ 10:02
0
but the fact remains the game is set in a dystopia, even if it started as utopia :p
|
Comments: 1-26 of 26 in total
Want to comment on this article? Log in, or register!
