Sony's answer to the problem of PS2 backwards compatibility for PS3 appears to be a case of repackaging the vast majority of the original hardware and supplying it as an all-new add-on.
The recently unearthed patent application, first revealed in the West by Siliconera and currently being dissected on Japanese gaming forums, describes how a "removable adaptor" is connected to a "next generation games machine". The not-so subtle codenames ("200" and "300") are clearly supposed to represent PS2 and PS3, though the patent could apply to any games machines going forward.
English-translated versions of the diagrams show us how Sony intends to make it work. The "200" box appears to be an almost entirely complete rendition of the original hardware: CPU, memory, graphics chip and RAM are in there, along with a bespoke module for DVD emulation (note that there is no actual optical drive). Inputs and outputs are routed through a hub that connects both to the "300" unit and a router.
The real challenge here is bandwidth. The PS3's USB ports are capable of transferring around 35 megabytes per second at their maximum throughput level - no way is this fast enough to host an entire console. The fact that the hub appears to be handling Ethernet traffic suggests that Sony's solution to the bandwidth issue is to use the gigabit network port on the rear of the unit.
This offers a 125 megabyte per second connection between the host console and the "removable adaptor". The theory is that the PS2 game disc is inserted into the PS3 with data from the drive combined with input from the controller(s) being beamed over the LAN port. The adaptor then decodes the data and processes it exactly as a PS2 would. The output data is then transmitted back to the PS3.
Quite what form that data takes remains unknown, but an educated guess would be uncompressed video and audio which is then displayed (and perhaps upscaled) by the PS3: a 480p video signal at 24-bit RGB running at 60Hz would probably entail around 72MB of bandwidth with a minimal amount of overhead for audio. That's way beyond USB 2.0, but should be manageable via the gigabit Ethernet port.
It's possible that the encoder mentioned at point 212 could be compressing the image using something like MJPEG (as used by the PlayStation Eye's 60Hz throughput) and this would work over USB 2.0. However, any form of additional compression and decompression will badly impact image quality and it would not explain why the "removable adaptor" would require its own connection to the router at all. Logically speaking, the only reason it is there is because the PS3's own LAN port will be occupied by the adaptor itself, and the PS3 still needs support for a wired internet connection.
All told, it's a bit of a crazy scheme. This notion of effectively cramming an entire PS2 into a box you attach to your PlayStation 3 might seem like rampant overkill. However, we live in a world where electronics are radically shrinking year on year. If Microsoft can shrink its CPU and GPU onto a single, relatively low-power chip, then the notion of combining an entire PS2 into a single package isn't particularly far-fetched at all.
If there's any disadvantage to this idea, it'll almost certainly be the case that the games will be laggier than playing them on original hardware: all this traffic across LAN ports, perhaps combined with upscaling on the PS3 end must surely be adding latency you wouldn't get on the original PS2 itself.
However, it is worth remembering that this is merely a patent application, a method of protecting an idea and not necessarily any indication of any kind of final end product. However, it does provide a fascinating insight into the thoughts and processes going on at Sony Computer Entertainment.