OnLive Latency: The Reckoning
Can Cloud gaming really compete with PC and console response?
OnLive latency has finally been measured, and the results are pleasantly surprising. In Digital Foundry's independent tests, we achieved an optimum response of 150ms - similar to playing Killzone 2 locally, and in line with Rare's claims for lag when using the new Kinect camera controller.
The tests form part of our complete breakdown of the OnLive platform, based on extensive testing of the launch titles in the USA. This article is set to be published on Digital Foundry this Saturday and includes an extended look at the front end and the OnLive feature-set, along with detailed analysis of game performance and video encoding quality.
The crucial factor in judging whether the platform is viable has always been the lag. Just how playable OnLive is all comes down to how quickly it responds to your commands, and here is where the company's claims have been met with the most incredulity.
There are many different elements to the overall make-up of the latency of course: the lag of controller input, OnLive's processing latency, the time it takes for data to be sent and received from the server, and of course the display latency. Previous Digital Foundry features have also proven that games themselves all operate with their own individual levels of response.
In testing the system we used our Ben Heck latency controller monitor board and made use of OnLive's support for the Xbox 360 joypad. The board works by lighting up an LED whenever a button is pressed. By filming the board and the display in a single shot using a 60FPS camera, the process of counting the frames between button-press and resultant action on-screen gives us our end-to-end lag measurement.
We chose a fast TN-style LCD panel for minimal latency and compared it with CRT measurements to ascertain that the display adds an additional frame of lag which is then factored out to give us the final figure.
So let's take a look at the highlights of the latency tests we conducted.
In these tests we judge response time for shooting in Unreal Tournament III (look for the muzzle flash), punching in Assassin's Creed II and braking in DiRT 2 (signified by the brake lights on the vehicle). We count the number of frames between the button being pressed and the action occuring on-screen. Each frame accounts for 16.67ms of latency.
In a best-case scenario, we counted 10 frames delay between button and response on-screen, giving a 150ms latency once the display's contribution to the measurement was removed. Unreal Tournament III worked pretty well in sustaining that response during gameplay. However, other tests were not so consistent, with DiRT 2 weighing in at 167ms-200ms while Assassin's Creed II operated at a wide range of between 150ms-216ms. We'll be discussing the reasons behind this variance in our upcoming feature.
Our figure of an optimum 150ms end-to-end latency (166ms including our chosen display) comes with a heap of caveats, however: the nature of the Internet means that your mileage inevitably varies, as connection performance varies from ISP to ISP. Indeed, even population density of your home town can have a big impact on the quality of your line.
OnLive says that the system works within 1000 miles of its datacentres on any broadband connection and recommends 5mbps or better. We gave OnLive the best possible ISP service we could find: Verizon FiOS, offering a direct fibre optic connection to the home. Latency was also reduced still further simply due to the masses of bandwidth FiOS offers compared to bog standard ADSL: in our case, 25mbps.
Even with this mammoth connection, in our tests OnLive never meets anything like the claims made for it by company front-man Steve Perlman, on the record as describing end-to-end lag as being under 80ms and "usually... between 35-40ms".
That said, OnLive's recently announced partnership with BT, which will see datacentres integrated into the ISP's infrastructure directly, means that the service will have the best possible chance of succeeding over here, as all data should be siphoned through BT's internal systems without the need to traverse the Internet.
While latency is clearly important, the quality of the overall service is paramount. We'll be posting our complete analysis on Saturday and the results are surprising. Look out for it.
You may also like...
-
Digital Foundry: PS3 Skyrim Lag Fixed?
-
Face-Off: The Darkness 2
-
Metal Gear Solid: The "Lost" HD Remasters
-
Face-Off: SoulCalibur 5
-
Eurogamer.net Podcast #99: FF13-2 and Amalur RPG Special
-
Why Devs Owe You Nothing
-
EA evaluating FIFA Street features for FIFA 13
-
Who Killed Rare?
-
Gotham City Impostors Review
-
App of the Day: Sir Benfro's Brilliant Balloon
-
Kingdoms of Amalur: Reckoning Review
-
Sony admits "dropping the ball" with Demon's Souls
-
Ultimate Marvel vs. Capcom 3 Vita Review
-
The Darkness 2 Review
-
Grand Slam Tennis 2 Review
-
Skyrim patch 1.4 now live for Xbox 360
-
CD Projekt: Witcher 2 intro cinematic "the most expensive asset we ever created"
-
One Piece: Unlimited Cruise SP Review
-
King Arthur 2 Review
-
Skyrim patch 1.4 performance tip: make a new manual save
-
Epic's Sweeney on graphics tech: "the limit really is in sight"
-
Samsung Galaxy Note Review
-
Next Xbox has tablet-like touch-screen controller - rumour
-
Mass Effect 3 FemShep trailer debuts
-
App of the Day: Superman









Comments (72) Latest comment 2 years ago
Comments threads automatically close after 30 days, but please feel free to continue chatting on the forum!
Comment below viewing threshold Show
Comment below viewing threshold Show
the real issue is that you have to buy a subscription and still buy games
you can expect the games to be full rrp and the price of the subscription soon adds up to the same as a gaming rig after a few years
not value for money
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
...sorry but that just made me laugh.
Comment below viewing threshold Show
Are you sure you don't mean 25mbps upload?
I have FIOS and it's 25 both ways.
Comment below viewing threshold Show
I have FIOS and it's 25 both ways."
It depends on the area you live in.
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
I can't believe they are still actually doing that in some countries - and supposedly it's only getting worse?
I think it's been ten years or more since I last heard about an ISP with bandwidh caps here in Denmark (and even back then there were several alternatives that didn't). I don't even think there's much "traffic shaping" or throttling going on - I definitely haven't noticed any slowdown of any types of traffic (including torrent) with any of the two ISPs I've had while living on this address.
I'm currently on a 50/2 Mbit cable connection. My bandwidth monitor shows I typically have between 600 and 800 GB of traffic (down/up) a month.
Comment below viewing threshold Show
I would hate to see what some with an avergae display and average connection... would get 300ms+
Comment below viewing threshold Show
Have there been lag testing on the games running locally? If so, maybe take these numbers into account...
Even so, OnLive is gonna be difficult to sell unless games and the system for the average user is cheaper than just buying a rig/console.
Finally, it's likely to be destroyed by the monopolism (in the UK) and oligopolism (in the US) of internet speeds which are in general awful.
Comment below viewing threshold Show
Comment below viewing threshold Show
I appreciate that its pretty much the fundamental make or break point of OnLive but there'is soooooo much being made of LAG nowadays that I think it's getting a bit silly. I mean, who the fuck can tell the difference between 100ms and 150ms with the naked eye?
Not exactly on topic but just a thought.
Comment below viewing threshold Show
I can't see it getting much traction in the core gamer market, but might be do well in the more casual sector. I don't think we should be any more concerned than we are about Farmville, for instance. Unless you like to rage about things of course,
Comment below viewing threshold Show
http://ww w.eurogamer.net/articles/gdc-wh...
Comment below viewing threshold Show
Anyway, 150ms is eurgh!
Comment below viewing threshold Show
Comment below viewing threshold Show
It can....just not yet
Comment below viewing threshold Show
I appreciate that its pretty much the fundamental make or break point of OnLive but there'is soooooo much being made of LAG nowadays that I think it's getting a bit silly. I mean, who the fuck can tell the difference between 100ms and 150ms with the naked eye?
Not exactly on topic but just a thought.
Yep, but when you were playing MW2 MP you weren't subscribing to play the game as well as having to 'buy' it from the service too.
You can forgive lag when it happens on a random server and there's no real cost involved. Just drop out and try for a different server. At the point where you're paying for a service though, anything that interrupts gameplay will without doubt irritate.
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
As well as an objective test, though, I'd also like to see a systematically tested subjective test -- get a group of people to play various kinds of games, and analyse their subjective views as to how satisfying an experience it was. Partition the data according to game type (FPS, RTS, platform, driving, etc.) and gamer type (hardcore FPSer, casual, etc.).
Comment below viewing threshold Show
I don't see how bandwidth would decrease latency. Bandwidth does obviously not mean speed, it means width. I guess there would be congestion if you had less than the required 5 mbps, but it's still a strange sentence (as you can easily get 5mpbs ADSL).
Comment below viewing threshold Show
I don't see how bandwidth would decrease latency. Bandwidth does obviously not mean speed, it means width. I guess there would be congestion if you had less than the required 5 mbps, but it's still a strange sentence (as you can easily get 5mpbs ADSL). "
True bandwidth makes no real difference beyond the point of 0 congestion, but the use of fiber directly into the house means this is the fastest possible connection, limited only by the speed of light.
So essentially 150ms is the best possible time, and anyone that doesn't use fiber will automatically have more latency.
Comment below viewing threshold Show
the question now is will this sort of gaming take off, can it eat into the pc or console markets, will it be user friendly enough to bring more people into gaming etc etc.
Comment below viewing threshold Show
both famed for being rubbish.
is DF glad because it never truly believed the tech could work? I will proceed to read on now.
Comment below viewing threshold Show
Personally I see no other reason for the service. Works as Steam in a manner ( not as good neither same way cause one streams and the other doesnt ) with the difference that you also get lag on top of it with no graphical edge for visual and high end gaming rig junkies like myself.
Comment below viewing threshold Show
Comment below viewing threshold Show
I'm not sure that we should be encouraging tech like this at such a protozoic stage of its development - not to the hardcore and not to the casual mainstream either, if we want games to be taken seriously.
I mean, my mother was playing Carcassonne the other week and she complained that her opponent was 'a bit laggy', and she cared enough about it to stop playing. That from someone who mostly plays Zuma and Tetris and hasn't been near an FPS since I tried to get her to play Halo a long time ago.
Comment below viewing threshold Show
Comment below viewing threshold Show
What was the ping to the server itself?
Comment below viewing threshold Show
As I understand it, latency is the time to send a packet with no data to the other end of the connection.
I guess that you need to receive an entire frame of video data before it can be displayed, so the actual latency you see in game is dependent on how quickly you can download that frame, which depends on bandwidth.
Comment below viewing threshold Show
Also correctly noted above, latency is not bandwidth past a certain point is it peering. Putting servers in datacenters is a nice step because the ISP can sort their routing so everyone gets great peering to the game servers, but it's still a far more unreliable situation considering they will no doubt be under some kind of SLA and not a priority for engineers working for BT.
Comment below viewing threshold Show
Indeed, there's a huge area of game programming that exists to hide the effects of ping in games. And, as you say, none of it is relevant here because the delay is effectively between the controller and the screen, not the client computer and the server.
Comment below viewing threshold Show
http://en .wikipedia.org/wiki/The_Phantom...
Comment below viewing threshold Show
Then there's the broken pricing system, and the issue of how they'll make enough money to expand for top end PC titles as they're released over the next few years.
This is an interesting concept for the future.
If BT integrate the tech into their BT vision boxes, then they could be onto a winner if they bundle the service with their broadband at a decent price. The BT deal is the best thing that Onlive have announced so far, although I'll not be choosing BT as my ISP anytime soon...
Comment below viewing threshold Show
Have they clarified if the games you subscribe to will be available beyond 2012? I know that EA had put a clause that their availability would be reappraised then. So you might end up paying a subscription fee, buying a game full price and only having access to it for a limited time.
Yeah, i'll be waiting for the limitations and real world performance & quality info to come out before touching this service.
Comment below viewing threshold Show
Say the game is running at 30fps, and each frame is 0.1Mbit in size. If you have a 3Mbit/s connection, it takes 33ms to download the frame before you can begin to decode and display it. If you have a 30Mbit/s connection, it only takes 3ms (and then your connection is idle for another 30ms before receiving the next frame). That means you save 30ms off the total latency.
Comment below viewing threshold Show
I'm sure I read recently that the image quality isn't consistant either. I can't quite see how less responsive games with worse image quality is better. The whole concept seems like a backwards step to me, or at best an idea too far before it's time.
I think the only way this product can possibly sell is if they hand it over to Apple, put an "i" in front of the name, make it white and shinier, get Steve Jobs to do a reveal, tie it into iTunes and the Apps Store, and put out a series of sickeningly pretentious adverts to promote it.
Comment below viewing threshold Show
but regardless of whether or not it works, the current pricing structure is just a joke.
pay a sub, and then buy or rent games on top.
no way in hell, if it was a flat rate to access anything, maybe it would be interesting, assuming it works well enough.
but buying games and then having onlive hold them hostage.
no, no, no, NO.
Comment below viewing threshold Show
Its possible to envision that in a few years time we could all be potentially playing top end games on cheap/small devices (Netbooks, iPads, Smartphones,etc) in our local Starbuck or on a basic desktop, MAC, TV at home.
As more services move into this space costs will come down and it's not impossible to imagine the big console developers taking this route for the next gen systems (just look how far MS, Sony and Nintendo have come with there online services already).
The only real issue holding this back is our connectivity levels and surely these can only get better.
Comment below viewing threshold Show
Comment below viewing threshold Show
Im hoping they can get it down though. i personally game on a ps3 and 360 but the laptop im using whilst not that powerful (NFS MW is close to its limit, runs CS well though) its got a fairly decent 720p screen so the latest games would look nice through onlive.
And for me going to unviersity, this would be a godsend. University accomidation isnt really designed for a 26" hd tv and a ps3 in there it would make studying in there impossible. Onlive would be a nice alternative.
Comment below viewing threshold Show
Time and time again I have viewed as sacred cows of gaming have been swept away, replaced by new norms that were always worse and to the detrement of the gamer. Who can forget the ridicule with which horse armor was greeted - now we have a culture of DLC as the expected mod of operation on all games. If we accept things like this onlive we will re-train another generation to accept 200+ latency speeds and locked down games being bought, to then double charge you as you then pay the service charge of onlive and your internet.
Who knows, if this washes, maybe we will also get things like a E.A and acti sub to play online games, on top of the other 2 charges. In effect you are paying for the right to be a captive customer and get access to other things you can then buy as well...its all went wrong somewhere. Are we now the lesser sons of greater fathers? A bastard generation who fast forget our glory days of free maps, mods and real expansions. We are redefined and monitarized at the behest of our masters like the redoubtable Mr Kotik, a new era of high margin gaming waits to plunder any who are stupid enough to let it happen.
Will we go quiet in the night of this type of future, or has it already happend?
I won't be getting onlive, If anyone needs this soapbox after me its in the garden shed, next to my companion cube and stack of PC gamer mags from the glory days
Comment below viewing threshold Show
I'm also unclear on the target market, with the games it has it certainly isn't all casual. Surely the few people who can't afford the arcade aren't going to be tempted by a subscription service.
The only thing that makes me worry it will stick around is the Bt partnership thing.
Comment below viewing threshold Show
I'm not holding my breath though.
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
I live in the centre of London and I can tell you we aren't even close to getting fibre optic, its still years off.
Comment below viewing threshold Show
Indeed. It seems a bit odd that part of their sales pitch is "you don't need to maintain a high-end PC to play games" when most of the games have been ported to the PC from cheap, easy-to-use consoles in the first place.
Comment below viewing threshold Show
I still think it's a viable service, but I just don't understand who they're hoping to sell it to. They should really be hooking up with the WoW or SW:TOR type of consumer, rather than thinking they'll attract the CoD brigade. They'll not make enough money to compete with the Kotick creature, but at least they'd make something.
Ach - I know: They'll flog it to hotel chains.
Comment below viewing threshold Show
Comment below viewing threshold Show
How is it at all positive? Aside from the ridiculous subscription costs, poorer image quality and latency issues, this is a bad thing for gamers because you don't own the games. You're simply renting them for limited time, and the publishers have total control of pricing so there'll be no deals or second hand purchases.
Comment below viewing threshold Show
Comment below viewing threshold Show
Comment below viewing threshold Show
Yep, but you're only paying a subscription as a replacement for your PC or console. If you take the amount of money you spend on a new gaming PC, and divide it by how often you replace your gaming PC (e.g. every 3 years), then how does that compare to the OnLive subscription cost? That's what you're paying for. It's not just a subscription for the sake of it.
Comment below viewing threshold Show
Comment below viewing threshold Show
Electrical pulses/data travel over copper wires at the same speed as optical pulses/data travel over fiber optics. Therefore having a non-optical connection does not "automatically" introduce added latency whatsoever. Optical data transmission allows for a much greater bandwidth than electrical, with less need for amplification over given distances, but thats about it...
Comment below viewing threshold Show
Comment below viewing threshold Show
Sor..r.y, i'm l.a.....gg.i.n....g!
Comment below viewing threshold Show
Comment below viewing threshold Show
With all this cloud-computing nonsense, I'm always wondering what people will do without any connection to the internet?
- Mails are unavailable, because they're using Google Mail
- Your pictures are unavailable, because you saved them on your Live Skydrive or Picasa, Flickr, Whatever
- Your music is unavailable, because you stored it on your Skydrive or in your Dropbox
- Your office documents are unavailable, as your used Google Docs or fell into Microsoft's Hype-trap
- You can't play games, because you didn't bother to buy actual physical copies
- You can't play games, because your game has some crappy DRM (and is not distributed via Steam OR is distributed via Steam, but has Ubisoft's DRM or Windows Live integrated as well..
It's just the tip of the iceberg - the sheep/people are swayed so easily by the whole cloud-nonsense, that they throw reason and some foresight right out of the window. If we make ourselves so dependent on the web, we'll suffer as soon as there are problems with it - and there are constantly problems with it.
Maybe I just got lucky and grew up with computers in a still analog world, learning the limits of computers and their wonders and knowing when to trust your own local hard-drive and backups.
Comment below viewing threshold Show
The *only* ISP in the UK who guarantees this is Virgin Media, and only if you're on cable. ADSL in this country, despite industry claims of "up to 20mbps" is averaging around 3.5mbps with many people reporting speeds significantly lower than this.
I'd also move to say that any "study" of this service in the US is completely inapplicable here in the UK due to the whole net neutrality issue. Here in the UK just about every major ISP throttles video streaming back to the stone age, the complaints BT has made about streaming services and the way they handle them is very well documented, imagine what they'll be like if this actually proves popular.
Comment below viewing threshold Show
I guess it will have a place and maybe even a bright future, but it's certainly not for the hardcore gamer. Maybe it's convenient, but in the end you're renting games at lower quality for a higher price.
Comment below viewing threshold Show
OnLive = s*** imitation copy of Xbox Live without all the plus's.
Comment below viewing threshold Show
Comment below viewing threshold Show
?????