Source: MS quadrupling Kinect accuracy

By Wesley Yin-Poole Published 17 December, 2010

Finger movement and hand rotation possible.

Microsoft is working to improve the accuracy of Xbox 360 motion-sensing add-on Kinect so that it could detect finger movement and hand rotation, Eurogamer understands.

Microsoft's Kinect team is said to be working "very hard" on a switching or compression technology that will allow a greater amount of data to pass through Kinect to the Xbox 360 console.

Kinect features are dictated by firmware so that they can be added and upgraded over time.

The depth sensor used by Kinect is also dictated by firmware – it is currently set at a 30 frames per second limit and a 320x240 resolution limit.

At a 640x480 resolution, however, Kinect could begin to detect fingers and hand rotation – an effective quadrupling of its accuracy.

The issue relates to the USB controller interface, Eurogamer was told. It is capable of around 35MB/s, but it only uses around 15/16MB/s.

This artificial limit is in place because multiple USB devices can be used at once on an Xbox 360. But Microsoft is working on a technology to allow greater throughput in this regard, Eurogamer understands.

If Microsoft achieves its goal it could double the spec of Kinect's depth camera with a simple dashboard update.

Microsoft had not responded to Eurogamer's request for comment before publication, but Digital Foundry's Rich Leadbetter described the potential accuracy improvement as "eminently doable".

Microsoft would need to "disable or lower throughput of game installs running from USB flash drives to free up additional bandwidth," Leadbetter said.

"All eminently doable though bearing in mind that Kinect 'only' needs 20MB/s for full res from both cameras.

"The resolution coming out of the depth camera via PC is indeed 640x480, but it is uncertain just how accurate the camera's sensor is.

"Additionally, processing four times as many depth pixels could slow things down more."

Last month Anton Mikhailov, a software engineer at Sony Computer Entertainment America's research and development department, told Eurogamer Sony turned down Kinect's 3D camera because of the limitations of the tech.

"In reality, the 3D cameras we surveyed and what Kinect ended up using, they're 320x240 resolution, so when you're talking about tracking fingers, or even tracking things like the rotations of your hand, you're working with 10x10 pixels," Mikhailov said.

"It's very hard to get anything useful out of it."

Mikhailov doubted Kinect's capacity to create a decent Star Wars game "because there are so many ambiguities, and it's nearly impossible to track the angles of your wrists".

If Microsoft achieves its goal of improving Kinect's accuracy, however, Mikhailov could be proved wrong.

Picture of Wesley.

About Wesley Yin-Poole

Wesley is Eurogamer's news editor. He likes news, interviews, and more news. He also likes Street Fighter more than anyone can get him to shut up about it.

Read more articles from Wesley by visiting the Wesley Yin-Poole archive page.

You may also like...

Comments (58) Latest comment 4 months ago

Comments threads automatically close after 30 days, but please feel free to continue chatting on the forum!

  • Phishfood #1 1 year ago

    I thought it could already detect finger movements

  • Shikasama #2 1 year ago

    Wasn't this said in a Digital Foundry article a little while ago?

    Either way it is rather embarrasing for the Sony engineer to be saying things that DF could prove incorrect. Unless during the process of making and developing Kinect Microsoft worked on a new and better camera than the one Sonyman saw.

  • spazmo #3 1 year ago

    I feel sorry for all the kids out there when on xmas morning they find a Kinect under the tree when all they wanted was CoD.

  • Psychotext #4 1 year ago

    "I thought it could already detect finger movements"

    Depends entirely how close you are to the camera.

  • chrisjm #5 1 year ago

    i feel sorry for the people that still havent had the chance to play this so continue to diss it.

  • feistycheese #6 1 year ago

    Getting Kinect to detect finger movements and minor body details would really open up the scope and depth of any possible 'core' games that we might one day see using the device, but if it slows what is already a laggy device down by 4 times the amount, then id rather they stick to the current resolution and concentrate on improving response times.

  • Grievous1976 #7 1 year ago

    Roll on Kinect 2.

  • jtodroc #8 1 year ago

    I doubt the Sony Engineer would have opened his mouth if he thought he was going to be proven wrong in a couple of weeks. Nintendo also recently claimed that they rejected the Kinect-style camera.

    We'll probably eventually see some decent games for Kinect that don't need the extra resolution anyway.

  • DarkSeptember #9 1 year ago

    After some consideration , I finally purchased a Kinect today .I now own all three motion control systems. I've only been playing Kinect for about an hour , but I am impressed with it .After reading about the amount of space needed to play it , I'm surprised that it works so well considering that I have a bloody great big double bed in the way , and it completely ignores it !

  • captain_Carl #10 1 year ago

    And i shall continue to not give a fuck about kinect

  • vizzini #11 1 year ago

    Well, it will be interesting to see if the “can” (not “could”) get it working; and working consistently well, without draining away too much of the deterministic processing resources of the Xenos GPU(which they use for the stream processing aspect).

    There is a communication bandwidth issue as one problem, but there is also a memory bandwidth issue if this additional data will be processed in main memory by the Xenon cores rather than on the GPU.
    But if they do decided to use more GPU resources then the games will suffer in other ways, so I'm inclined to believe they won't be able to do it well with games; even if they could achieve it in the future.

  • FogHeart #12 1 year ago

    So, it seems so far Kinect's capabilities have been pared back on the off-chance that someone might complain that while Kinect does intensive work, their USB-connected doohickey doesn't work as well. Well I'm glad to see MS' attitude changing and allowing Kinect to have more oomph. I mean, if you ran a lawnmower and a lamp off a generator (work with me!) would you write to the lawnmower company to complain if the light dimmed when you started up the lawnmower?

  • Kill_Crazy #13 1 year ago

    the real question is why wasn't this done before launch?

  • makattack #14 1 year ago

    Ahh... great... cue all the wanking and two finger salute games.

  • thedgam #15 1 year ago

    Pew! Pew! Pew! (Kevin Butler)

  • patch #16 1 year ago

    Just look at some of the amazing things they can do with Kinect on a PC!!! (NSFW?!?)

  • des #17 1 year ago

    but but but
    Kinect haters must be feeling pretty embarrassed right now,first sales,now this
    lulz

  • Gastrian #18 1 year ago

    Regarding the bandwidth surely adding a second USB controller to the mainboard purely for kinect would alleviate the problem as nothing else is going through the USB bus or are they specifically talking about CPU and ram drain?

  • altitude2k #19 1 year ago

    I find the motion detection to be pretty accurate already. But fine-tuning to see finer extremities would be great.

  • Skooch #20 1 year ago

    "And i shall continue to not give a fuck about kinect"

    @ captain_Carl - So you'll continue to read articles about Kinect, and post how you don't give a fuck, all whilst not caring at all about it? Riiiggghhht.

  • LetsGo #21 1 year ago

    Maybe they will release a 'Kinect Motion Plus' like the Wii so it actually makes it work as designed.

  • StooMonster #22 1 year ago

    spazmo: @I feel sorry for all the kids out there when on xmas morning they find a Kinect under the tree when all they wanted was CoD."

    You feel sorry that parents won't buy 18-certificate Call of Duty for their kids for Xmas?

  • layleeloo #23 1 year ago

    @ spazmo. Yeh thats right - cos all kids want is the same boring shite they've played 5 or 6 times a year for the last 5 years. What an exciting Christmas. Go play with your 5 year old american trigger happy clones.

  • SeesThroughAll #24 1 year ago

    Hardly a surprise that quadrupling the video resolution (from 320X240 to 640X480) will quadruply the accuracy...

  • Stop-gap #25 1 year ago

    "Yeh thats right - cos all kids want is the same boring shite they've played 5 or 6 times a year for the last 5 years."

    I bet good money that is *exactly* what they want. :p

  • Exarch #26 1 year ago

    Right so, seesthruall. Oddly phrased as well - as if detecting fingers was necessary to quadruple anything.

  • vizzini #27 1 year ago

    SeesThroughAll: Hardly a surprise that quadrupling the video resolution (from 320X240 to 640X480) will quadruply the accuracy...

    Sounds correct; and will most likely double or quadrupling the processing requirements, just like it does if you texture map with a texture that is double in both dimensions; but people don't seem to like the harsh reality, that a regression analysis with 4 times the samples, requires four times the computational work.

  • Phantom_Dynamite #28 1 year ago

    MS you now you have split your consumer base if you want to do Kinect things fine but your forgetting the rest of us who have been with u since the original xbox who just want amazing games.

  • Machiavellian #29 1 year ago

    I doubt the Sony Engineer would have opened his mouth if he thought he was going to be proven wrong in a couple of weeks. Nintendo also recently claimed that they rejected the Kinect-style camera.

    Just because one company could not figure out a solution to a problem doesn't mean another will not. A lot of companies have rejected ideals because they could not see the solution only to have another company come to market (Wii). Sony and MS definitely understand this concept pretty well.

    On another note, isn't the original 360 holding back Kinect. The 360 s has the power supply and USB cable in one. I am sure that USB port is dedicated to Kinect and thus do not have a bandwidth issue.

  • DoctorFraud #30 1 year ago

    Dance central, Yourshape and Kinect Sports are very high quality titles that work consistently, accurately and with little to no percievable lag.

    Your lulz are hollow and pathetic.

  • des #31 1 year ago

    @cheekybyname

    Not really.
    Right now there is enough material to laugh for months,but I'm not a mean person.
    lulz

  • DoctorFraud #32 1 year ago

    Hollow and pathetic.

  • Osahi #33 1 year ago

    So then it would finally become a bit like what they promised and if they can add up some neat games too they finally could have something worthwhile... But then again, maybe not. I still can't figure out how they can develop fun, non-casual/minigame-esque stuff for Kinect that isn't on rails... (But I'll be glad if they prove me wrong one day.)

  • Machiavellian #34 1 year ago

    wow, they sound amazing. For a teenage girl ! Forgive me whilst i carry on lulzing, lulling, ehm lolzzing?

    You know something. If MS can really tap this market, I am sure you can lolz all you want. This is a huge and very untap market. This could also be an entry point in Japan that MS is always looking for (too bad about the space issue). Teen/ girl gaming market is huge in Japan and even the US. It's very much untap and the company that could leverage their device to hit that market will run away with sells out the ying yang (opps looks like Nintendo already have).

  • Dizzy #35 1 year ago

    Pc Kinect stuff can already do this, I expect the 360 to follow in the next wave of Kinect titles.

  • Machiavellian #36 1 year ago

    On the tear down of Kinect, wasn't it proven that the device comes with the co-processor. If so it appears that MS is still working out bandwidth issue and some other issues which may result in turning on co-processor and upping the camera depth sensor.

  • spazmo #37 1 year ago

    "Hollow and pathetic."

    I thought you liked Kinect?

  • dirtyvu #38 1 year ago

    there are plenty of videos that already show finger detection. it's just that MS's skeletal system doesn't do fingers.

    but some developers already said that internally, the IR sensor of the Kinect is higher than the projected IR image.

    if you check out the Anandtech review, the guy stated that too:

    "The projected image doesn’t change in time - it’s fixed this way. The IR CMOS sensor images this pattern projected onto the room and scene, and given the camera’s displacement a few inches from the projector, from the displacements in the semi-random projected pattern is able to back out the corresponding depth image. That computation is done onboard the Kinect itself, and it’s entirely possible (read: likely) that the IR sensor inside the Kinect is higher than the 640x480 resolution of the resulting image."

  • Collymilad #39 1 year ago

    @cheekybyname

    Way to miss the point, dumbass.

    The point is that it's capable of accuracy already, the titles used are irrelevant as the capability can be transferred into better games or games more suited to core gamers. If MS are indeed improving it that only adds to that point.

  • dirtyvu #40 1 year ago

    Anton Mikhailov is a smart guy. But he's trying to spread FUD in order to downplay the abilities of the Kinect. He even goes so far in his FUD as to conject how Dance Central functions by claiming it works the same way as Harmonix handled their Eyetoy game. He has no basis for claiming that Dance Central works by background subtraction.

    the PSEye and PS Eyetoy are just RGB cams. they can only work by detecting pixel changes. because it works by merely comparing pixels, the conditions needed for it to work are very stringent. if you actually played eye games back in the day, you would know this. the lighting has to be spectacular, the clothing has to be a clear contrast, etc.
    also, with only an rgb cam, it's nearly impossible to detect depth. that's why eye games only had motion in a 2d plane. you reach up to pop a bubble. you jump up and down. to detect depth in a sea of pixels is a logistical nightmare.

    that is why the move uses a colored ball. that ball is a clear contrast to the environment. very easy for an rgb cam to pick up. they see a pink pixel coming off the ball and chances are the surrounding is not pink. that ball is also a fixed sized so the system can detect if the move is moving toward the screen or away from the screen based on the measured size. if the ball is smaller than expected, it's moving away. if it's larger, it's moving closer. that's the depth work. then you add in the sensors inside the move to detect orientation of the move controller.

    to do what kinect is doing needs more than what an rgb cam can do. in controlled environments (which you never get outside of a lab), you can do minority report stuff with only an rgb cam. but the thing is you can't expect that in a typical home environment. people come in all different shapes and colors. no two people motion exactly the same way. there are so many different types of clothing. worrying about shutters or blinds in the background, etc.

    kinect does none of this because it's motion tracking does not depend at all on an rgb camera. you can play dance central in complete darkness. if Dance Central was working the way Harmonix games worked on the Eyetoy, it would not work at all. Go back and play any eyetoy games in the dark. They don't work.

  • TwitchyMcTwitch #41 1 year ago

    Hey that's cool, and with usb 3.0 and some good software; eventually I might actually pick up kinect for my pc.
    Edited by 1 at 18/12/10 @ 00:27

  • orangpelupa #42 1 year ago

    hmm maybe just allow game developer to completely disable the RGB video feed, audio feed? Basically disable all features except the Depth data stream. Then use the USB should have enough bandwidth. Also maybe less processing overhead.

    btw why in the artcle many "Eurogamer understands" ?
    english is not my native language, i kind of confused with that....

  • jenguin #43 1 year ago

    Re 'eurogamer understands' = eurogamer believes, eurogamer has been told, eurogamer is under the impression.

  • vizzini #44 1 year ago

    Orangpelupa: btw why in the artcle many "Eurogamer understands" ?
    english is not my native language, i kind of confused with that....

    Non-cynically, Eurogamer have used these words because Microsoft are still hard at work trying to make this happen (believing they will) from their own R&D & adapting any open kinect techniques (running on PC hardware); so getting a firm quote to say they will achieve it on 360 hardware and the USB bandwidth for Kinect will be changed hasn't been given corporate approval yet, but it will shortly.

    I'm actually really interested in the technical problem of how I might interpret this data for motion recognition in real-time, and would gladly take a free kinect to play with on PC for development fun. But as yet, I still can't quite generate one of those, “wouldn't it be great if you could ...” ideas for games with a 3D camera; probably because I think accelerometer wands will always be faster to communicate a user's broader motions and can provide feedback with rumble and sound.

  • coolbritannia #45 1 year ago

    Kinect is awesome, it keeps getting better, and is sellling like hot cakes. And the haters have no response to that.
    Edited by 1 at 18/12/10 @ 19:25

  • phosphor112 #46 1 year ago

    Dirtyvu: Anton Mikhailov is a smart guy. But he's trying to spread FUD in order to downplay the abilities of the Kinect. He even goes so far in his FUD as to conject how Dance Central functions by claiming it works the same way as Harmonix handled their Eyetoy game. He has no basis for claiming that Dance Central works by background subtraction.

    Keep in mind both Sony and Nintendo have turned down Kinect. Also, there is video footage of of Sony working with the Zcam (which was it's name back then). Also, what makes you think that Dance Central isn't using the technique brought up by Mikhailov? Dance Central has ZERO need for actual motion tracking, so why not cut down the overhead and latency by just matching key frames of the skeletal figure with the ones in the game? It's so much easier and is more logical. That's why even Dance Central is (in some ways) more accurate than games like Kinect Sports.

    Also, what does bringing up Move have to do with anything? They (Kinect and Move) are two different technologies. If you're trying to bring this (the fact of it being only an RGB camera) up as a point to why Mikhailov doesn't know what he's talking about, you're mistaken, both Richard Marks (the creator of the Eye Toy) and Anton Mikhailov have both worked in both RGB and RGB + IR tech. But that's besides the point.

  • vizzini #47 1 year ago

    Trebell: t's business, even clever guys like them will talk a bit of smack if they have to.

    I think Sony and Nintendo's engineers are too professional to get into smack talk, were their professional reputations are concerned; Nintendo tend to focus on making choices that are “correct” to compliment their family and child friendly outlook, and Sony tend to focus on making “correct” choices from a science and lifestyle perspective; ie “correct” for educated people.

    They've both said that 3D cameras are good, but professionally substantiated reasons why they weren't correct for them to use in a variety of gaming experiences.

    Microsoft on the other have been known to use the expression Embrace, extend and extinguish and have mostly been successful from being closely tied to an architecture that people believed could result in Electronic Armageddon in 2000 with the Millennium Bug. So, taking the correct choices from a science perspective has probably been less important to them than market share.

  • mgillespie #48 1 year ago

    This sounds like a lame Microsoft excuse in prime buying season. Why know Kinect might be a bit rubbish today, but please buy it anyway and we might upgrade it....

    Save your cash... Kinect's problems are not just the lag and poor recognition problems, it's far bigger problems are that it's VERY limited in the type of game it can work with. How many shovelware jump and wave your arms about can you put up with....

  • 43n1m4 #49 1 year ago

    So... the possibility for Kinect to be even more precise and possibly open up for more game ideas is a bad thing? Some of you should take off your very small shoes.

  • vizzini #50 1 year ago

    43n1m4 :So... the possibility for Kinect to be even more precise and possibly open up for more game ideas is a bad thing? Some of you should take off your very small shoes.

    Maybe, but the bandwidth calculations above, “needing 20MB/s for full res” are surely wrong, if we are meaning 24bit RBG camera, 16bit IR depth (40bits total) @ 60fps and 640x480.

    As that actually needs:

    40*60*640*480/(8*1024*1024) = 87.89 MB/sec bandwidth, which probably exceeds the 35MB/sec they said the 360 USB controller provides even if they can get a real-time lossless compression working; as that is a big compression ratio needed.

  • ThePissartist #51 1 year ago

    @vizzini

    You can probably knock that number in half, I don't remember 60fps being part of the equation. Equally, I'm not sure that all games are going to need both the RGB and the IR feeds.

  • TheGoose126 #52 1 year ago

    Got a kinect on Friday, not played much as my two and half yr old runs round the living room constantly getting in the way. I can however say that this is one of the best bits of tech out there at the moment. Really clever stuff and works really well. Pity I now have to play with a virtual lepoard called Dunbar as lil girl can't really get to grips with what she has to do!
    Who am I kidding Kinectimals is awesome!!!
    Haters keep hating.

  • nolim #53 1 year ago

    @vizzini
    A small point but your 87.89 mb/sec calculation assumes no compression what so ever and, if they use 16 bit colour at 30fps, then the calculation comes out at 35.16 mb/s, again assuming no compression. Now use the depth data as a mask for the colour data and lossless compression becomes unnecessary.

  • Cronan #54 1 year ago

    Why fuck up a nice article about the tech with a quote from some Sony cunt-rag?

  • vizzini #55 1 year ago

    DF: @vizzini
    A small point but your 87.89 mb/sec calculation assumes no compression what so ever and, if they use 16 bit colour at 30fps, then the calculation comes out at 35.16 mb/s, again assuming no compression. Now use the depth data as a mask for the colour data and lossless compression becomes unnecessary.

    Having study signal processing years ago, I don't think 30fps is enough samples per second to accurately capture finger movements(with 33ms between samples); so just quadrupling the quantization levels doesn't fix the second problem of sample-rate.

    Adding resolution would certainly help them, if they try to extrapolate more frames with subframe interpolation, but then the processing demands to do this and delay from detecting the user motion to on-screen action will most likely get even bigger, using general purpose CPU core architectures(that max-out at 60% processing efficiency due to cache misses)

    There is a big difference between people saying why they believe a product doesn't work fully as intended (technologically), and just hating on a product, but it seems like people who get fulfilment from Kinect games and are wishing for it succeed don't seem to differentiate the two types of comments.

  • BelstaffOutlet #56 4 months ago

    It makes me feel so surprise.I never know there is such a place that I can find designer clothing wholesale.The site offers different kinds cheap designer clothing.So great clothes stores with clothes wholesale.Join it now!

  • cheapclothesonline #57 4 months ago

    Do you like motorcycling and have your own motor equiments?You can come here Belstaff Outlet to find your Belstaff Jacket.Belstaff Coat can not little.If your foot is cold,Belstaff Boots is the right choice.You can buy a Belstaff Bag to hold those equipments.

  • kengriffshoes #58 4 months ago

    This article is impressive,I hope that you will continue doing nice article like this.And do you like ken griff shoes,I have collected many beautiful griff shoes these years, the griff are so popular, I think you also can shop oneken griff shoes online store.