Kinect Hacking: The Story So Far

These particular utilisations of the Kinect tech are particularly interesting bearing in mind that one of the most under-developed and even disappointing elements of the Xbox 360 Kinect experience is the front-end menu control, where movement of the cursor is slow and ponderous, while the simple notion of activating a button is overly fussy and time-consuming, involving hovering over buttons until the OS is sure you actually mean to press it.

However, as interesting and functionally rich as the demos seem, perhaps they aren't the fairest comparison. When putting together a simple piece of code to test functionality, the process begins and ends there, whereas Microsoft and Kinect game-makers need to create mechanisms that work at various distances, in various settings when scanning people of all ages (with corresponding differences in the size of their hands and fingers). However, the "hand over icon" activation approach still seems rather long-winded when an exaggerated button-push gesture would surely be more intuitive and much faster.

The Windows 7 interaction demo also highlights rather well another element that Microsoft needs to factor into its game-making: camera positioning. The field of view in the demo is firmly on the upper-half of the body only, and the closeness of the subject to the camera obviously increases the amount of resolution available to measure things like hands and fingers - this is a luxury holder the platform holder and game developers simply don't have.

Even considering all these advantages, there is one further factor that demonstrates ably that Kinect will always have interface issues - latency. Coders can work around the lag element to a certain degree, and it's also fair to say that as players we have now been conditioned to a certain degree of latency. However, all bets are off when it comes to the user interface. If you play OnLive with a controller, it appears to function OK with an acceptable lag. But try any kind of streaming video system with a mouse pointer and it simply doesn't work very well.

Due to the amount of time it takes for the depth map to be generated and beamed across USB, this is always going to be an issue with any similar control scheme conjured up for Kinect. However, as a birthplace for some intriguing tech, the Kinect demo community continues to surprise and delight.

As we speak, Sony's 3D team working at Evolution Studios is combining stereoscopic 3D with head-tracking to create what it calls "holographics", where the on-screen 3D is adjusted to factor in where you are looking. A similar effect can be generated with head-tracking alone, as this demo shows and perspective-based Kinect head-tracking has also been incorporated into a prototype version of Tetris.

In addition to the functionality and concept demos, it's fair to say that the homebrew community has come up with Kinect implementations that range from the sublime to the ridiculous to the downright sinister. There are the obligatory camera tricks, such as the Predator cloaking demo (which basically takes a shot of the background and uses the depth data to distort parts of the image), and the augmented reality demo that maps skeletal data to a medical CT scan to give a kind of real-time x-ray effect. Neat.

Other camera-based tricks include fan favourites such as real-time lightsaber rotoscoping along with the augmented reality Japanese superhero demo.

Worth a watch from a technical perspective is the bizarre volumetric fluid occlusion demo, which looks a hell of a lot cooler than it sounds, but is an interesting exercise in what Microsoft calls "registration". This is where the depth data is aligned with the image from the RGB camera to a decent degree of precision to the point where the PC (or 360) is able nicely to "cut out" people and objects from the backgrounds.

We've not seen much in the way of registration in 360 titles thus far, but this is perhaps not surprising - Kinect is constrained by a lack of bandwidth at the moment across the USB port, and the process of aligning the images does require some additional CPU power.

This isn't really a problem on PC however, and another example of RGB image and depth data being combined comes in the form of this extremely cool tech demo where Kinect's output is recreated in Minecraft form. Other uses of the depth data can be equally cool and cute - this exercise in Kinect-fuelled shadow puppetry is well worth a look.

And finally, to round off, the basic human fascination with breasts has also seen its fair share of exposure in the various Kinect demos we've seen. From a light-hearted perspective, the Kinect titty-tracker raises a few chuckles while the extreme boobie jiggling demo (featuring Ivy from Soul Calibur) is of course a "physics simulation of two capsules on dual-axis hinges, with high bounce and barely-existent gravity factors" which is "not intended to model the behaviour of real boobies". Fap at your own risk, advises the author.

And finally, there's the infamous Kinect sex game - a sinister, but intercourse-free experience which seems to boil down to groping the breasts and bottoms of badly animated, completely unresponsive CG women. At the time of writing, the extent of Kinect's penis-tracking capabilities remain unknown - which is perhaps for the best...