Hacked Kinect running on PC?

By Richard Leadbetter Published 8 November, 2010

Open source drivers in development.

Gizmodo reported over the weekend that Kinect for Xbox 360 has been hacked, or rather reverse-engineered, with the motion-sensing hardware operating on an ordinary PC.

Two videos, uploaded to YouTube by NUIGroup admin AlexP, show the compromised motion sensor in action, demonstrating the progress made in the few days since Kinect arrived at US retail. While some may might consider this is an elaborate fake, AlexP is something of a renowned reverse-engineer, having produced open source drivers for PlayStation Eye, so the chances are this is very much the real deal.

Alex's first video demonstrates how the Kinect motor is controlled via PC while a second reveals the RGB and depth camera feeds running on PC. The latter video is especially intriguing as, according to Alex, the depth feed can be captured at 640x480 - just like the PrimeSense reference camera from which Kinect is adapted - while the official Kinect spec lists depth resolution at a quarter of that - 320x240.

The popular conception is that Kinect has been cut down in several ways from the original PrimeSense tech, and enhanced in others (the multi-array mic, for example). However, iFixit's brilliant teardown of the sensor reveals that the "brain" of Kinect - the PrimeSense processor - has indeed made its way through into the final shipping product.

The PrimeSense website shows a PS1080-A1 processor at the centre of the design, whereas iFixit's article shows that a PS1080-A2 chip is contained within Kinect's innards. This is curious bearing in mind comments from Alex Kipman, director of incubation at Xbox, who recently confirmed reports that the processor was removed. Perhaps the A2 version of the PrimeSense chip is a cut-down version of the original, or maybe there is more to the Kinect's guts than we're aware of? The chances are that the truth will out sooner rather than later.

In the meantime, amateur robotics enthusiasts around the world will be watching AlexP's progress very closely. Kinect's ability to see in 3D and "hear" so precisely while consuming a relatively miniscule 18 watts of power makes it the perfect device to add to a homemade robot, and it's likely that many other innovative homebrew applications for the sensor could be conjured up once fully refined PC drivers are available.

Picture of Richard.

About Richard Leadbetter

Rich has been a games journalist since the days of 16-bit and specialises in technical analysis. Commonly known around Eurogamer as the Blacksmith of the Future.

Read more articles from Richard by visiting the Richard Leadbetter archive page.

You may also like...

Comments (32) Latest comment 2 years ago

Comments for this article are now closed, but please feel free to continue chatting on the forum!

  • Spekingur #1 2 years ago

    This is very interesting. It means that we could soon see a Kinect enabled house with a butler AI! I wonder if we could get John Cleese's voice for that?

  • arcam #2 2 years ago

    Sweet :)

    Hoping there will be some nice HTPC uses for this. Still haven't found a completely satisfying way to control the PC from the sofa. Ironically I hope this can end up replacing the Xbox controller that I currently use for navigating XBMC ;)

  • sink257 #3 2 years ago

    Didn't take too long... hopefully we could be seeing some ingenious projects created with the use of Kinect in the near future. Not sure how Microsoft would feel though :|

  • bad09 #4 2 years ago

    No surprise really seeing as MS were planning on it coming to PC as well down the line. Personally I'm happy for it to stay on the console.

  • FogHeart #5 2 years ago

    OK, but what could you do on a PC with it? That question isn't as daft as it sounds...

    Sure, maybe you have a motorised webcam straight away. But the depth sensing camera? A driver just lets you read the data from it. A hell of a lot of software - not present on the device is needed to make sense of that, it's not going to be part of any driver. So with a new driver, new software is needed, complex software to analyse the position and movement of the human body which took years for MS to make from scratch.

    And then what? Existing software in Windows - OS or otherwise - is designed to work with specific devices. Mice, drawing tablets etc. The Kinect input is analogous to none of them. Can software be written to 'pretend' a hand is a mouse pointer, to act between driver and Windows software? No - you can't 'remove' a mouse pointer from the screen like an indicating hand can be withdrawn, and programs do not expect a hand held steady to be analogous to a mouse click.

    No, I can't see this getting any practical use.


    Edit1: You can state your case with common sense and logic and get a kicking...
    Edited by FogHeart at 08/11/10 @ 16:55

  • charming_fox #6 2 years ago

    Of course all that stuff can be done, that's what PCs are for.... for screwing around with things.

  • Retro_ #7 2 years ago

    Robots.....Hooray, Kinect has a useful purpose!

  • telboy007 #8 2 years ago

    Johnny 5s alive!!!

  • MENTAL1ST Verified Senior Software Engineer, Picsel UK Ltd. #9 2 years ago

    Can software be written to 'pretend' a hand is a mouse pointer, to act between driver and Windows software? No - you can't 'remove' a mouse pointer from the screen like an indicating hand can be withdrawn

    Windows can work with touchscreens and trackpads, which have exactly those properties, you divot.

    Use as a handsfree mouse is one obvious option, another is recognising a set of gestures and interpreting that as keypresses or system messages, equivalent to media control keys on the likes of keyboards, laptops and Media Center remotes.

  • sfp_noodle #10 2 years ago

    Wow that was quick! It's been on sale what, 4 days in the US?

  • drxym #11 2 years ago

    Hacking the kinect hardware is like hacking winmodem hardware. It's a milestone of sorts but it doesn't mean a huge a amount when a substantial part of functionality is implemented in software.

  • FogHeart #12 2 years ago

    I did consider the touchscreen idea, but remember that on a touchscreen you are either 'clicking in a certain place (or two places)' or 'not clicking anywhere'. The Kinect has 'hand up but not clicking' which has no analogy on a touchscreen.

    Trackpad is used to shove a pointer around, where you touch it has nothing to do with the pointer position, which is not like Kinect at all.

    Either way, software that sees a depth map, converts it to a skeletal frame, sees the hand(s), tracks/predicts their motion...if all those highly paid MS people had so much trouble with it, we can't expect it to be done by hobbyists!

    Divot? I didn't neg you for that, but as you can see there's a valid debate to be had, I'm not bone-headed in my reasoning here.

    Edit2:...and later restate your case and get a plus. Madness.
    Edited by FogHeart at 08/11/10 @ 16:56

  • orangpelupa #13 2 years ago

    hmm atleast doing "hand to controll mouse" will be not too hard, i think.
    but for doing full skeletal mapping will be hard...

  • orangpelupa #14 2 years ago

    there even already have "hand to control mouse" without the need of kinect.
    look at this first few sec of the video. The bare hand control the "audiosurf" car just like mouse.

    <a href="http://www.youtube.com/watch?v=I15fp4iG5NM
    ">http://www.youtube.com/watch?v=I15fp4iG5NM
    </a>

    also used to do the clicking by doing "On_rollover, do click" thing.

    this also only use bare head
    <a href="http://www.youtube.com/watch?v=yMJDoXXdnOU
    ">http://www.youtube.com/watch?v=yMJDoXXdnOU
    </a>
    Edited by orangpelupa at 08/11/10 @ 15:02

  • arcam #15 2 years ago

    The best thing about these sort of 'hacks' is seeing what other uses and functions it ends up getting used for. Just trying to recreate Kinect's software isn't the point (would we really want to recreate MS's hold-for-two-seconds-to-select anyway?).

    Converting all movement to a skeletal frame with minimal lag is obviously a big ask (although I've been surprised by hobbyists before), but it might not be necessary to do something cool with the device. Just having an affordable depth-mapping sensor and the 4-mic array will get people thinking.

  • charming_fox #16 2 years ago

    I'd like to see some homebrew software that captures your skeleton and drops the data into a 3d studio max compatible file so anyone that feels like it can have a fairly sweet motion-capture device for £130... f*cking bargain!

  • Dolly #17 2 years ago

    Anonymous porn-vids are GO!

  • TheJuriel #18 2 years ago

    This IS interesting.

  • lavalant #19 2 years ago

    MS really shouldnt complain, it can only increase sales.

  • Bander #20 2 years ago

    Practical use for Kinect on a PC?

    Easy. 3D modeling. Either use your arms to sculpt any shape you like, as though there's a big block of plasticine between you and the screen. Or have it scan real objects.

    For game development this could be a massive time and money saver.

  • dingo75 #21 2 years ago

    Anonymous porn-vids are GO!

    Actually I always wondered why there is no adult only stuff for the Wii (I mean it's motion-sensoring).
    As probably Nintendo wants to keep their console free from this stuff Kinect might be the answer.
    Yourself 3d in some porn game or something. Maybe a couple living in different cities and only seeing each other rarely.
    I see a potential market in that (stuff like Virtual Girl sell quite nicely as far as I know).

  • Haloboy #22 2 years ago

    Kinect coming to PC courtesy of GFWL. Now that would be a crowd pleaser! xD

  • Seehuusen #23 2 years ago

    Why microsoft didnt release this for the PC as well in the first place....i have no clue. Hook up the kinect and laptop to the pc and there ya go.

  • monkfishjoe #24 2 years ago

    If the specs on the camera and chip are true then this is really interesting. Seems odd that MS would be downplaying its potential.

    I second being able to feed the data direct into a 3D program. Would make my life tons easier

  • Farzlepot #25 2 years ago

    The application of this in robotics is not to be understated. Various people across the globe are working on the problem of getting a robot to 'see' properly, and Microsoft's gaming device may have the answer. Were Microsoft to be a farsighted company they would already be supporting this line of research. With their resources anything is possible.

    I doubt they are, though.

  • Bander #26 2 years ago

    "I doubt they are, though."

    Indeed. MS invest so much money in new technologies and research, but then this is the same company that cancels great ideas like Courier and decides Windows Phone 7 users don't want copy and paste or memory cards!

  • xboxrob #27 2 years ago

    Echoing some comments above. Although this is potentially useful the key to Kinect really is the software. I am currently studying for a PhD in Machine Learning which is at the heart of how Kinect perceives you.

    In order for skeletal tracking to work you would need several examples (the more the better) of different poses which have the correct labelling of the different points of which you wish to track. These need to be passed through an algorithm in order for it to learn how to label based on the information given it.

    A good article can be found at wired (here) which explains some of the required process.

    The people who are leaders in this field all work at the Microsoft Reasearch Centre so it is unlikely anything that comes out of homebrew applications will be as good as what Kinect can do now.

    This also relates to the array microphone. They use several algorithms to calibrate the microphone which allows it to cancel sound and even combine it with the skeletal tracking which gives even more information to the software to allow it to "hear" you. This does have more of a chance though of being done well as it is easier.

    One other thought I had about the downplay in specs. These algorithms are probabilistic which means they assign values to different skeletal points. The less points the more likely you are to hit the right area I would think. Therefore there is probably no need for a higher resolution to be used with the algorithms at the moment hence it probably feeds over USB a lower resolution than it captures for quicker analysis.

    If Microsoft were clever they would design a free game which gave rewards (small monetary value) and made you do certain poses. Then this information of the depth camera and the pose given to you would be sent back and used in the algorithm. The key here really is the more data the more reliable the tracking would be and hence perhaps more precise??

  • Bigglesworth #28 2 years ago

    One of the camera lenses really needs to glow red.

  • davisorle #29 2 years ago

    Post deleted at 20:44:35 16-04-2012

  • CrispyXUK64 #30 2 years ago

    Move drivers please!!

  • toa_boa #31 2 years ago

    Microsoft need to get this device onto Windows - throw drivers and SDK into the wild, get homebrewers, indies etc on the bandwagon in droves. Get the coders and designers experiments with improved UIs, alternative game experiences etc. Learn from every possible angle and feed it back into even better dashboard and Xbox 360 kinect titles
    Edited by toa_boa at 11/11/10 @ 06:05

  • Zeydlitz #32 2 years ago

    I would like to made a small note: there is no way to use Kinect with "ordinary" Windows-PC right now. Only linux-specific drivers was made. And main goal was to use Kinect in robot's projects as sensor.
    And then what? Existing software in Windows - OS or otherwise - is designed to work with specific devices. Mice, drawing tablets etc. The Kinect input is analogous to none of them. Can software be written to 'pretend' a hand is a mouse pointer, to act between driver and Windows software? No - you can't 'remove' a mouse pointer from the screen like an indicating hand can be withdrawn, and programs do not expect a hand held steady to be analogous to a mouse click.
    You wrong about this. Every device is only push some data to computer. And if we know data format, we could use it to implement, for example, a "device" abstraction. It's a file, and data in it is 3-d coordinates of viewed points, and some other info. It's raw device is not a mouse at all, but we could then wrote another mouse device, that would use it to move a mouse cursor, just like any additional mouse do. You could have a lot of pointer device with cursor, such as touchpad, touchscreen, tracball, and all of this could exists simultaneously. For example, in HP tablet notebooks touchscreen is 3 different devices (and one of this is a magnet surface, you could use stilo without touching).