Secrets of the Wii U GamePad
Digital Foundry on how the controller was reverse-engineered - and exactly how it works.
The Wii U GamePad has been reverse-engineered by the developers behind the Dolphin GameCube/Wii emulator, with the controller's functions completely simulated on PC. When we heard the news on Twitter, we contacted developer Pierre Bourdon to find out more. In hacking the GamePad, we reasoned that the developers would have extensive information on how Nintendo's innovative controller actually works.
"We started working on the Wii U GamePad as soon as we (me and two other hackers) got our hands on it," Bourdon tells us. "The GamePad is actually not a very secure device (compared to the Wii U). The device firmware is stored in an unencrypted Flash, which allowed us to reverse engineer the binary code pretty easily. It is also using almost standard 802.11n, which made things easy to experiment on a PC."
The GamePad itself isn't anywhere near as complex as even the most basic Android tablet - essentially it's a standard game controller, with a touch-screen and a WiFi card, paired up with a decoder chip for the incoming video stream. Controller inputs themselves are beamed back to the Wii U via the same WiFi channel (180 times per second no less), in contrast to the Bluetooth employed by the Wiimote.
"We started our work by sniffing a Wii U/GamePad pairing, assuming WiFi encryption keys would be transferred during pairing. It turns out the pairing is (almost) standard WPS [a standard WiFi encryption protocol], with just a little obfuscation added to the crypto," Bourdon continues. "We modified wpa_supplicant/hostapd to have it work with the non-standard things, and were able to pair a PC with a Wii U that way. This took us less than a week - our time since then has been spent reverse engineering the custom communication protocol used between the Wii U and the GamePad."
"The GamePad is actually not a very secure device (compared to the Wii U). The device firmware is stored in an unencrypted Flash, which allowed us to reverse engineer the binary code pretty easily."
Custom communication protocols means we can discount previous theories that Nintendo employed the use of Broadcom's streaming video Miracast technology to get the Wii U GamePad working, although there are similarities.
"Video is compressed using h.264 (baseline profile, so no B frames)," Bourdon shares. "Audio is usually uncompressed, but we've found mentions of compressed audio formats in the firmware... We found mentions of [Miracast] when we started working on the GamePad, but it turned out to be false. There is no Miracast anywhere in that GamePad. Audio, video and input streaming is done over custom protocols."
Baseline profile h.264 rules out many of the more advanced compression techniques employed by the codec, but Nintendo makes up for it via sheer, raw bandwidth. A sample capture from the Wii U WiFi stream offers up 33MB of data captured across 87 seconds - this gives us an average of around 3mbps. This is fairly lavish for an 858x480 stream at 60 frames per second, but the video captured here is only displaying the Wii U's front-end menus. Pierre Bourdon tells us that the Wii U uses variable bitrate, meaning that bandwidth scales up according to the complexity of the image it has to encode.
"This measurement does not include audio. Here is a graph of frame size over time in these 33MB," he says.
"Despite the 3mbps average, we're seeing spikes of anything between 25-40mbps, and a massive variation in bandwidth that can only be down to variable bitrate encoding."
Despite the 3mbps average, we're seeing spikes of anything between 25-40mbps, and a massive variation in bandwidth that can only be down to variable bitrate h.264 video encoding. The more complex the image, the more information is required to maintain image quality - something the Wii U seems more than capable of successfully transmitting over its 802.11n wireless link.
"I haven't checked but I think the spikes are just the Wii U sending a large I-frame (full picture/key frame)," Bourdon explains. "If you average the bandwidth over something like 10 frames these spikes mostly disappear. In normal operation mode the Wii U sends one I-frame and then only P frames, unless the application requests to send an I-frame or a frame was not received properly by the GamePad (because of packet loss)."
The confirmation of an h.264 encoder built-in to the Wii U (it's believed to be a part of the graphics chip) is similar to elements of the hardware found in both the next-gen Xbox and the PlayStation 4, so there is the slight possibility that cloud functions could be grafted on to the Wii U, especially since Bourdon confirmed that functionality for supporting two Wii U GamePads simultaneously is built into the firmware.
"The GamePad's flash is upgradeable, meaning Nintendo can add to the functionality of the controller over time."
"I don't see any reason why it could not send GamePad video/audio to the internet (both internet and GamePad might be difficult/impossible)," offers Bourdon. "The firmware of the chip handling communication with the GamePad (called DRH) can be upgraded, so this might be in Nintendo's future plans. Only speculation though, we didn't see anything in the firmware that would indicate they are planning to do this."
But what about the chances of full Wii U emulation, bearing in mind the hacking team's pedigree? The work carried out by Pierre Bourdon and his colleagues on the Dolphin emulator is remarkable [Update: a quick clarification - Bourdon's team-mates on the GamePad project are not involved with Dolphin but Pierre is] and we were curious as to whether the shared CPU heritage between Wii and Wii U could offer any kind of leg-up in running Nintendo's 'next-gen' software on PC.
"PowerPC is 'meh' for emulation - not hard, not easy. Having more cores to emulate might actually make things easier for emulation in my opinion: it means games will usually rely less on precise timing," he says. "The biggest problem will be the GPU: emulating a complicated programmable GPU is something that I think hasn't been done yet, and it might be very difficult with APIs like DX and GL hiding so many details from the developers."